Of course you do. If a fault on a vehicle could not have conceivably developed within the time scale from the MOT to the fault developing or becoming apparent then you have some grounds to fight a bit.
For example a friend of mine bought a car last year which made a horrible clunk one day, on inspection discovered the front suspension mounting had ripped off the chassis. Rust was so severe there is no chance it developed in the 2 months since it's last MOT.
Your issue is that with brakes there is a chance it was a seal which wasn't good but was still working. An MOT brake test should be pretty serious, well the guys I know make it so, applying a lot of pressure, basically the MOTer is trying to make anything about to go fail. Even a seal which is working under normal driving may show as being faulty under seriously hard braking.
I actually find it difficult to brake really hard for testing purposes on my own vehicles because I just know it's not really good. MOTer doesn't have this problem, he gives it everything.
If it turns out to be a knackered caliper, or rotten pipe you might be able to go and have a word with the test station, they have been known to put issues right very quickly as the VOSA inspector could be called upon to inspect your vehicle and if found not to be MOT worthy, the test station is in trouble. Might be other things too.