It's happened before so I know what to listen for. My car seems to make a very distinctive sound when the pulley damper starts to go- it sounds like a badly worn, knocking engine, only very faint, as if the engine is trying to be polite as it knocks itself to oblivion.
I think it's quite early...