When the opposite bank is next to fire the low pressure behind the gas you theorised on is lost. The v6 (leaving aside the turbo) and v8 don't fire in a series of opposing banks.
Increase exhaust size / flow too much and valve overlap can result in some intake charge being lost out of the exhaust valve.
The last cylinder to fire on that bank is doing the scavenging. With valve overlap some of the inlet mixture is always lost out of the exhaust valve until the engine reaches it's cam sweet spot. Basically comes on cam. This can vary subject to cam design and torque needs, can be low range, mid range or high range. Most engines are a compromise with power around mid range. Some engines now have variable valve timing to give a more linear engine power output throughout the rev range.