I had the same problem with my TD5. Turned out it was a contact in the starter motor. No need to replace the entire motor. I managed to buy a contact on the net... But don't ask where from because I can't remember... Sorry. This is tho worn contact...

I decided on a new shiny starter motor because I didn't relish going through swapping it over again anytime soon. It means I have the old one to recycle now though. I may yet dismantle it over the winter. There are plenty of other jobs to get on with in the meantime...