If it's also the axle on the offside moving, causing the bar to move, then it could be offside tyre different pressure, damper fecked, spring fecked, suspension bushes fecked .. might be any number of issues that the sway/anti-roll bar is having to cope with.
If it's just the bar end moving without the axle then maybe remove it and see what differences there are with the bushes from the nearside.
Maybe remove it completely and see what effect it has on handling ..