As above for the mirror head on the ball joint, and the arm itself but...
It may also be that the mirror arm bracket is loose on the hinge. Remove the hinge bolts through the door and you can then access the bolts that hold the mirror arm to the hinge. When I had this on mine however the movement had caused the threads to wear so I needed to fit longer bolts to bite into clean threads to solve the problem.