You can remove most things from a land rover without it having any structural impact whilst the vehicle is stationary. The front or gear box cross member is just bolted into place. Jack up your gearbox to provide some additional support and then unbolt the bugger. You'll probably have to hit it out from above or lever it some how. you can lift the seat bottoms up in the cab and remove the panels to get access (to some extent). I wouldn't entertain removing it as a permanent fix though. It's there for a reason. The more 'runs' there are in the ladder chassis, the stronger it is. If you can't get the down pipe around the cross member, chop it with an angle grinder and buy a length of stainless steel flexi pipe to join the bits you want to connect. It works well!
Exhausts are one of the most frustrating jobs, so don't let it get to you!
-Pos