Failure to resolve Brexit tensions could place the Union at risk, says Institute for Government