Hmmm Fair Point....
After reading back, whilst the OP doesn't want the potential needless effort of taking the heads off, it may be the only resort....
Even just removing the Plenum, Trumpets and Inlet Manifold, remove the valley gasket to look at the rods, lifters and Cam lobes would be a start, as he indicates it is coming from under the Inlet Manifold....
My option would be to whip those items off and take a goosey....could be a duff follower bouncing around intermittently causing the slight engine rocking due to in-efficent valve opening times, but then it behaves and not ticking sound...then it looses internal hydraulic pressure, and bounces around again causing the ticking, until it pumps itself back up again.
This would be worse when the oil is cold and thicker, but getting better once it is warm.
When it happens under load my thoughts would be piston gudgeon pin bearings, the big ends or Crankshaft Bearings as the extra load would cause the knocking to become worse due to the extra force these components will have to stand up to. Usually (I could be wrong) the camshaft, lifters and valves don't have anywhere as much force applied during load conditions, so if it is worse under load, it is time to look at what is stressed during this time, which would be Pistons, Con Rods, Gudgeons, Big Ends, Crankshaft Bearings, Liners etc....
But the OP seems to not want to strip a duff engine due to it being a waste of time (who would blame him - I sure wouldn't want to do that much work if it is only going to go to the scrapper in the sky anyway), but unless he has a pot of spare cash for a new engine/vehicle, the only option I can see is to get into the engine internals and see what is a miss, unfortunalty