From what i have read (and I have a very tiny wobble even in the mornings :)) it can be the sum of a little wear in all parts of the suspension and steering.
Not much help, is it?
Again, from reading, the single biggest contributors are the swivel hub joints.
Possibly followed by worn...