Have you taken it apart to have a look?
It could be:
Loose connection to the motor
Loose connection to the BECM
Loose connection to the switch
Broken actuator
Broken switch
There are your starting points...
Just be careful when removing the door panel, if you take your time none of the clips will break