Sounds like the switch isn't quite in the right place or the little plastic levers that operate the self cancel are worn. Pop the cowling off and have a look - the mechanism is fairly simple when you see it, a lump welded onto the column acts as a cam on two levers which reset the indicator switches. A new switch isn't expensive and they're fairly easy to fit.
Or it might need a spot of grease, especially if someone has swamped it in switchcleaner in the past.