Cooler air is more dense, therefore there is more energy in it to burn.
A Turbocharger heats the charge air quite considerably and the intercooler cools it down and makes it more dense and better to burn whilst maintaining the pressure (the higher the pressure the more dense the air for a given volume) so with a larger more efficent intercooler, you will have charge air at pressue AND be cooler and therefore more dense and as such when it burns will be more energetic, giving you increased performance.
A larger impeller will not increase the charge pressue, only the amount it can compress in one go, but given the volume within the system is set, this oversized impeller will spin slower (due to the inertia within it, and also due to the back pressure created by trying to force more air into a fixed volume - think of an air compressor slowing down when the pressure increases), this will cause a restriction in the exhaust, causing a back pressure in the cylinders, reducing the efficency of the engine further, unless you increase the inlet port size to handle the increased charge air volume....
Everything in Thermodynamics is a balancing act across the equation...increase one thing, and you have to balance that out somewhere else....
A larger intercooler will assist in cooling the charge air and increasing density...a bigger turbo will do little.