I think, you somewhat answered your own question, albeit indirectly and ex negativo. Widely ignoring what we already know or could have learnt, and pushing the next vanity-boost feature seems more attractive than learning from our experiences and drawing the consequences. And the people doing it always find rosy language that makes it seem like a heroic feat of progress, and an audience that falls for the BS.
Except if there was a method that actually provably worked in producing lower cost, more reliable software on schedule without requiring your entire staff to be composed of demigods, then it would have already been adopted by EVERYONE.
There is no such thing as practical state of the art in our field for the problem being discussed (which is the software requirements capture/design/engineering side of things, not the software construction side of things), most of the so called "state of the art" focuses on a problem of very limited utility:
problem a) if I have a complete, consistent and fixed requirements, can you write code that provably implements it it? Sure. But very little software is developed under those conditions (avionics, safety critical stuff, some crypto stuff, etc)
problem b) code/methodologies to make writing easier/more concise/more predictably once as the specification is discovered. Nice, but it is optimizing the 10% of the task, whereas the 90%, the requirements discovery and requirements consistency management between requirements discovered amongst different branches of the problem is where the bulk of the time is spent.
Those are the "assume spherical cow" of our space. Interesting, but limited.
Or going with the bridge analogy that seems to be popular here, you're asked to build a bridge. Nobody tells you over what, what load it has to carry, under what weather conditions. So you don't know whether they want a rope bridge over a tiny stream or a multilane car/train combined suspension bridge. And as you ask for the requirements it is now a bridge over the Atlantic with a stop in Australia. And when you query the Australia part they confirm and casually mention that they might be planning on a stop in Pluto too and can you fit that in the schedule.
In my current project we have a manager on customer side querying about how we can fit various celestial bodies in the existing solution and discussing paint colours, while my usual contact keeps reminding them we really need to drive that truck to Australia ASAP...
Anyway, you're absolutely right about "state of the art" in requirements capture / design / engineering, and this is the larger context of discussion. But I admit, my original post was an assertion that there's no sensible, practical "state of the art" on the construction side either - it's mostly either fads or working with lowest-common-denominator tools (basic Java). The situation seems to look better in embedded in particular, but that's either because you're really, really limited in tools there, or because I have very little experience there and only think it makes sense...
I think, you somewhat answered your own question, albeit indirectly and ex negativo. Widely ignoring what we already know or could have learnt, and pushing the next vanity-boost feature seems more attractive than learning from our experiences and drawing the consequences. And the people doing it always find rosy language that makes it seem like a heroic feat of progress, and an audience that falls for the BS.