That’s very intriguing. I guess it would involve specifying a mental model in clear enough terms for it to be comparable to an already verified model, or something.
Although I write very little code myself anymore, I don't trust AI code at all. My default assumption: every line is the most mid possible implementation, every important architecture constraint violated wantonly. Your typical junior programmer.
So I run specialized compliance agents regularly. I watch the AI code and interrupt frequently to put it back on track. I occasionally write snippets as few-shot examples. Verification without reading every line, but not "vibe checking" either.
I like this. The few-shot example snippet method is something I’d like to incorporate in my workflow, to better align generated code with my preferences.
I have written a research paper on another interesting prompting technique that I call axiomatic prompting. On objectively measurable tasks, when an AI scores below 70%, including clear axioms in the prompt systematically increases success.
In coding this would convert to: when trying to impose a pattern or architecture that is different enough from the "mid" programming approach that the AI is compelled to use, including axioms about the approach (in a IF this THEN than style, as opposed to few shot examples) will improve success.
The key is the 70% threshold: if the model already has enough training data, axioms hurt. If the model is underperforming because the training set did -not- have enough examples (for example hyperscript), axioms helps.
Compilers have a finite set of inputs and outputs that should generate reproducible results. There's a larger amount of possible outputs for the same question with AI and very little reproducbility.
Yes but once the code is written it’s not going to magically change. I am going to test the code just like I would test something I wrote - again like I’ve been doing for 40 years when writing my code by hand.
But your thought process during coding influences your testing. At least for most of us, we find edge cases or point of concern during coding that we place extra focus on in test.
This is different than what you've done for the past 40 years becuase you're not testing your code. This would be analogous to you testing someone else's code. The vast majority of people and places have not followed that paradigm until AI showed up.
My thought process during my architecture influences my testing.
Since AI has been a thing, I’ve been in a customer facing cloud consulting role - working full time at consulting departments (AWS ProServe) and now a third party company - specializing in app dev.
Before my hands actually write a line of code or infrastructure as code, I’ve already spoken to sales to get a high level idea of what the customer wants, read over the contract (SoW) to see what questions I have, done discovery sessions/requirements analysis, created architecture diagrams, done a design review, created detailed stories/workstreams (epics), thought about all the way things can go wrong etc.
I very much keep my hands on the wheel and treat AI as a junior coder that might not follow my instructions. I can answer any question about architectural decisions, repo structure, what any Lambda does the naming conventions etc.
I’ve also intuited “these are the things that I need to think about and test for from my 30 years of professional experience as a developer and 8 years of experience across literally dozens of AWS implementations”.
In the before times, if I were doing this without AI, I would have to have two or three more junior people doing the work just because I couldn’t physically do it in 40 hours a week. Even then I would be focused on how it works and look for corner cases.
I don’t have to think about what I need to test for. I did specifically call out concurrency because there are subtle bugs.
Ironically, what I am working on now had a subtle concurrent locking bug that Codex wrote. I threw the code into ChatGPT thinking mode and it found it immediately and suggested better alternatives. I also have Claude and Codex cross check each other.
"I don’t have to think about what I need to test for."
Good luck then. The business process flow including edge cases should arguably be top of mind for what to test. Testing shouldn't be an afterthought but rather an integral thought when writing the code that needs to be tested.
"I would have to have two or three more junior people doing the work"
Yeah, and they're the ones thinking about testing the code they write. Architects (which it sounds like you are an architect and not a dev) don't get into thay much detail.
If I’m starting off from sales -> reading the contract -> discovery -> design -> project plan -> implementation -> implementation review -> handover, how am I not involved with the business case?
I would never trust a junior developer who is just an experienced ticket taker (and most don’t get their first job after 10 years of being hobbyist) to look in that level of detail. Honestly the code is the least important. What it does is. If I’m 50 years old and still just a “human LLM ticket taker”, I’ve done something horrible wrong in life.
By definition, this is the worse AI coding will ever be, anyone hoping to stay in this game long term by being able to “codez real gud” is going to be in for a rude awakening.
Enterprise development where most developers work was becoming a commodity in 2016 where it was easy to become “good enough” and comp still looks like it did on the high end a decade ago. Now it’s even harder to stand out from the crowd.
Now we are seeing that even BigTech jobs where “I can reverse a b tree on the whiteboard” developers are becoming a disposable commodity with all of the layoffs. There is a reason I’ve been moving up the stack and closer to “the business” over the last decade
I input x and I expect y behavior and check for corner cases - just like I have checked for correctness for 40 years. Why do I care how the code was generated as long as it has the correct behavior?
Of course multithreaded code is the exception unless the LLM is putting a bunch of rnd() calls in the code to make it behave differently.
Yes, and I like the points you are making. I feel like the mental models we make are exercises in a purer form of knowledge building than the code artifacts we produce. A kind of understanding that is liberated from the confines of languages.
Are we sure that unrestricted free-form Markdown content is the best configuration format for this kind of thing? I know there is a YAML frontmatter component to this, but doesn't the free-form nature of the "body" part of these configuration files lead to an inevitably unverifiable process?
I would like my agents to be inherently evaluable, and free-text instructions do not lend themselves easily to systematic evaluation.
>doesn't the free-form nature of the "body" part of these configuration files lead to an inevitably unverifiable process?
The non-deterministic statistical nature of LLMs means it's inherently an "inevitably unverifiable process" to begin with, even if you pass it some type-checked, linted, skills file or prompt format.
Besides, YAML or JSON or XML or free-form text, for the LLM it's just tokens.
At best you could parse the more structured docs with external tools more easily, but that's about it, not much difference when it comes to their LLM consumption.
The modern state of the art is inherently not verifiable. Which way you give it input is really secondary to that fact. When you don't see weights or know anything else about the system, any idea of verifiability is an illusion.
Sure. Verifiability is far-fetched. But say I want to produce a statistically significant evaluation result from this – essentially testing a piece of prose. How do I go about this, short of relying on a vague LLM-as-a-judge metric? What are the parameters?
You 100% need to test work done by AI, if it's code it needs to pass extensive tests, if it's just a question answered, it needs to be the common conclusion of multiple independent agents. You can trust a single AI as much as a HN or reddit comment, but you can trust a committee of 4 as a real expert.
More generally I think testing AI by using its web search, code execution and ensembling is the missing ingredient to increased usage. We need to define the opposite of AI work - what validates it. This is hard, but once done you can trust the system and it becomes cheaper to change.
How would you evaluate it if the agent were not a fuzzy logic machine?
The issue isnt the LLM, its that verification is actually the hard part. In any case, its typically called “evals” and you can probably craft a test harness to evaluate these if you think about it hard enough
"if the input contents were parameterized and normalized to some agreed-upon structure"
Just the format would be. There's no rigid structure that gets any preferrential treatment by the LLM, even if it did accept. In the end it's just instructions that are no different in any way from the prompt text.
And nothing stops you from making a "parameterized and normalized to some agreed-upon structure" and passing it directly to the LLM as skills content, or parsing it and dumping it as skills regular text content.
The DSPy + GEPA idea for this mentioned above[1] seems like it could be a reasonable approach for systematic evaluation of skills (not agents as a whole though). I'm going to give this a bit of a play over the holiday break to sort out a really good jj-vcs skill.
I love this.
I tried doing sub-pixel simulation for a tool I created (screenstab.com if anyone’s interested – yeah I know, shameless plug, etc.). I ended up abandoning the sub-pixel aspect in my shader because of the distracting patterns caused by the Moire effect.
Neat. Would be nice if there were some examples of what a beads rendition looks like. Maybe it's obvious for people in the game. I assume they are hexagonal?
While Figma can be a useful tool for aligning design with code, I think it's unrealistic to expect it to accommodate all the constraints of the web platform. Relying solely on a 1-to-1 mapping between Figma components and code components can be problematic and may not accurately reflect the complexities and nuances of web development.
