Really sympathise mate.
I only recently found out that one of my feet is a full size bigger than the other and that is probably the reason why I have such trouble with them now. And stupidly they are half sizes... 9 1/2 and 8 1/2 so finding foot wear is very tricky.
They are also wide which never helps.
Sooooooooo.
I think having trainers to wear or even something softer than that is the way to go.
It took a podiatrist to sort me out, have you seen one?
Insoles, cream, a buffing/scoriating machine, you name it, I've got them!
The very best of luck with sorting it out, seriously, mate.