Re replying to this as have only just noticed you said "one" wall unit.
Still trying to unnerstand why you had to fit the fiddly bits on then put it back up.
Specially as you often have to scribe the side panel to the wall anyway.
AHA!
Are you avoiding having screwheads showing inside the cabinets?
Now that would make sense!
You glutton for punishment you!
In the past I have either screwed them on from inside the cupboad, or in the case of modern rectangular section cornice and pelmet, stuck them on wiv a strong version of "No-Nails".
Member of the Odger family here, me being Brian Odger!!!
No visible screws on my watch. Its all about the finish yer it takes longer but worth it I suppose.