Military FFR pulleys also take a wider belt than the standard civvie pulleys. Don't try and bodge things with grinding 'a bit off here, packing out with washers, reversing thus or that'. Just fit the proper parts and it will all line up nicely. The work you have done so far is very good.