Had exactly this problem on my TD4, by ensuring the small spade connector on the front of the starter motor is fully home, the problem went away!
Still very occasional refuses to start, as you say, all lights on but when key turned nothing happens. So push this connector fully home and hey...