I had exactly the same problem a few years back; unless there is a large length of the shaft left in the diff that extends beyond the splines, it will come out fine with the diff. Normally they seem to break right at the end of the splines, so you shouldn't have any trouble getting the diff out.
I tried a magnet and that got stuck too, I wouldn't try the magnet again, in my experience it is likely to cause you more problems than it solves.
Do you know the cause of the breakage? If it is related to excess torque then it is likely to have twisted in the diff and you will have to dismantle the diff to get it out.
Good luck!