It’s not always true that longer reach brakes have less mechanical advantage. Sometimes it is (Weinmann centrepulls), but with the Shimano dual pivots the arms got wider as well as longer so the result was about the same re power - prior to 7900 at least.
Still, until carbon rims effectively killed rim brakes, and we got wider tyres in part to compensate for stiffer forks, 25mm tyres were seen as wide by road racers, and it’s not that long ago that journos were writing about “extra wide” 27mm tyres at Paris-Roubaix. If you were a pro using ~23mm maximum tyres 99% of the time, there was no reason to have anything except short reach brakes, and thus longer reach brakes were looked down on as only for tourists.
Unfortunately, cycling has always been largely about cosplaying for many, and imitating the pros doesn’t always make sense for those not averaging 45+km/h on closed roads.
I think there’s also a sort of feedback loop. Disc brakes didn’t enable fatter tires, but rim-brake design needed to be taken into account when contemplating fatter tires. Disc brakes removed that consideration entirely; once that consideration was removed, bike companies could go nuts, and it would be hard to go back.
There’s also just a matter of timing. Roadies started changing their minds about fatter tires at about the same time that bike companies were experimenting with disc brakes. Also at the same time, the new-ish category of gravel bikes mostly used disc brakes from the start, and that probably also played into adoption on road bikes.
I don’t think there’s much actual feedback loop there, because with both rim and disc brakes, the frame and fork are the end-all of tire clearance. Pros riding mid-reach (“standard” reach) brakes over 23mm tubulars in 1975 just had lots more gap between tires and frame/forkcrown/brakes than needed. Over time those got tightened up, whether because shorter-reach brakes were lighter and had less flex, or because some notion of aerodynamics.
If discs had become popular when roadies believed 23mm tires were the fastest, those frames would have had clearance for 25mm or 28mm max.
We’re on the same page with timing being largely coincident.
I believe the balding guy is Joop Zootemelk. Won the TdF once in 1980 and finished second 5 times. When I rode L’Eroica a couple of years ago I was struck how riders used to be on long slack bikes with wide tires, then geometries switched to upright bikes with skinny tires and now we are back to long / slack / wide.
Over time those got tightened up, whether because shorter-reach brakes were lighter and had less flex…
The mid reach brakes had enough clearance that one could fit wider tyres for cobbles or (gasp!) touring, but the shorter reach brakes (i.e., Campag “Piccolo”) had noticeably more mechanical advantage = power.
Back around 1984 I had a frame built that was supposed to fit the Piccolo calipers, but the builder made it for medium reach, and I ended up using drop bolts front and rear; braking improved noticeably after the switch from medium to short reach Record sidepulls. (Annoyingly, the builder made the chainstays short enough that tyre clearance was limited to 23mm by the seat tube, regardless of the calipers used.)
Re “longer, slacker, wider”: Seat tube angles and seat positions have been consistently getting steeper and more forward in pro racing lately. Along with shorter cranks and longer stems this helps with the “aero hoods” position commonly seen in breakaways and at the head of the peloton. Gravel bikes have got longer and slacker (at the front), and tyres have got wider, but STAs are on the increase there too.