I also had the same problem ... so I swopped it with the passenger one ... don't tell the wife!..
Only joking ... make sure there isn't anything stuck down it ... more of a problem on the rear seat belts (I found 2p firmly lodged in one) failing that ... get the WD40 out...