On the off chance Dan sees this: one of the footnotes ("Some Zitron predictions" > July 2024) is broken. Right now it's showing a little 0 that doesn't actually link to anything if you click it :(
I'm biased, being a software technical writer and not a dev, but at least from my side of the fence I think LLMs are no better at generating docs than they are at generating code, and might actually be way worse :)
My devs keep throwing Claude-generated writeups at me that look okay at first glance but fall apart when you actually try to follow the instructions they lay out (which makes them useless for anyone who's not already familiar with the product or feature they're describing), and/or have major structural and logical gaps, and/or gloss over messy details in a way that makes the product or feature sound better but doesn't reflect the reality of what we actually built.
Which turns into an ironic ouroboros where I then have to punch my own queries into an LLM and ask it to read our codebase, compare those details against the original writeup, try to decipher what the hell it was trying to say, etc. And even that turns into an extended back-and-forth where my LLM is like "The original writeup is inaccurate; it should say X" and I'm like "Are you sure? That contradicts what I know about related concept Y", and then the LLM is like "Good point, I dug into it more and the real answer is Z." (And even then, god only knows if Z is correct. I still have to get real developer eyeballs on anything before it goes live.)
Documentation was difficult to keep in sync with code because the tools didn't operate at a natural language level. Things like refactoring names, or terminology, or concept changes (foo is now fooGroup that contains bar) were tedious.
The granularity at which documentation parity is kept makes a big difference.
For example, brekaign things down into explicit assertion blocks in the code, and a link to the documentaiton that covers that assertion. So block-to-block tracking works well. Tracking completely separate documentation to independent codebase... idk I haven't tried it but my guess it that the context windows are too small for that.
But minor things like making sure there are unique error codes, that the codes somewhat make sense, etc. are all helpful. If nothing else, the model being stupid and acting as a rubber duck for pair-programming is useful for knowing what to document given a different perspective.
> But, and maybe I'm just a curmudgeon, but I kind of wonder why we write articles like this. Like if a culture is bad, is it likely a CEO is going to be looking for random blog posts for constructive criticism?
I think the odds of any particular article making it onto a CEO's radar are low, and the odds of them actually taking good advice to heart (let alone implementing it) are even lower. That said... I also get the impression that the shittest kinds of C-levels and VPs are disproportionately swayed by whatever opinion they last read in their daily email newsletter or heard on a five-minute podcast snippet. So at least in theory, if you were able to inject a seed of a good idea into the zeitgeist, that idea might reach the level of saturation where it turns into a form of white-collar peer pressure.
Unfortunately I also think the ideas that reach the peer pressure saturation point tend to be less "be nice to your employees" and more "put shock collars on them and make them beg for their jobs," but I guess it can't hurt to try?
I think sharing such ideas openly allows people to converge on similar feelings and experience which eventually leads to people experimenting these for real. We need to take into account that a lot of people reading and writing these kind of thoughts seriously may not be CEO at present but that does not prevents the possibility of them becoming the CEO's in future
> The present understanding is that the tale's moral supports team effort and recognition of the vital part that all members play in it. In more authoritarian times, however, the fable was taken to affirm direction from the centre.
IIRC that's what happened with the Boar's Head listeria outbreak. Inspectors had repeatedly documented that the facilities were nasty, but BH was allowed to continue operating for years because <reasons>. Things only came to a tipping point after multiple people died.
The way I like to describe it is that it feels slippery, like it's been polished down to oblivion and my eyes slide right off of it. There's no place to find a mental foothold, and if you zoom in there aren't any details.
But I also like your candy analogy because I think it's spot-on for how LLM text superficially looks informational/nutritious, even though it's actually just junk.
To this day we have no idea why or how, but years ago someone at a chain of walk-in clinics once put a note in my wife's file indicating that she was a habitual cocaine user. She had no idea until she went in one time and they were like "So, are you still using cocaine?"
(I do think this is extraordinarily rare compared to how often AI hallucinations happen, but it's such a bizarre story that I have to mention it.)
reply