Maybe best to buy a decent one to start with….
I suspect you've hit the nail squarely on the head there. My first car was a 1961 Mini, with a MG1100 engine, twin choke 28/36 Weber on waterheated manifold, 3 branch exhaust manifold and a Cooper S differential.
It still had drum brakes all round, there wasn't really any options for discs back then, the only option being Cooper 7" & 7.5 discs IIRC. Which were expensive hens teeth and meant a lot of changes.
To improve the braking I fitted a Lockheed remote servo ( on the inner wing above the clutch housing) It worked really well no real difference to the standard brake feel other than it stopped much much quicker with way less effort. The only downside was that I had to adjust the brakes every month as it ate linings. Although that may in part have been due to the way I drove it
Definitely no sticking or delayed operation, just progressive, push harder, stop quicker. I could easily out brake my friends 850 minis and it was at least as good if not better than the one Cooper S in our group.
I suspect manufacturing quality has dropped from back then along with the price, it probably cost me around £50 for the remote servo back in 1970/71, can't remember exactly other than it being expensive but worthwhile