I wouldnt do it. Lots of vibration coming from the GB end, and lots from the engine. RTV won't kill as much vibration as the gasket, which is not very thin. RTV might become "exhausted" or thin out as it's a soft plastic. And going in there again is a pain...
Get the gasket, and use very little (carefully) RTV on the engine side.