Scenario testing is the new word for it and I think this is a game changer.
Two of the reasons I never liked writing tests is
- they didn’t seem to usually assert much internal logic
- they would have to be maintained along with the original code
I think scenario testing is much better instead because the actual way a person uses a feature hardly changes but the internals might change a lot.
So imagine I’m making an e-commerce website. There are lots of internal mechanisms. I’ll have an agent testing all the functionalities as if it were a customer. This gives me much much more confidence while writing code because it is more uncorellated with the code.
Tomorrow I can change a lot of internals but the testing agent stays the same.
There’s something to note though: not all code is possible to be scenario tested. Like data engineering and other things where the feedback time is huge.
Also the merits of documentation and specs. It’s been eye-opening to see the subset of developers who were almost disdainful about writing documentation for their colleagues but are now tripping over themselves to do so for their clanker.
People falling all over themselves to write docs for their pile-of-linear-algebra-with-a-smiley-face-painted-on-it [0] don't read the docs, no. People who give a shit about writing solid software that doesn't get them paged at three in the damn morning do.
[0] The face is there to provide social-trustworthiness signals to engage the human pack-bonding instinct, natch.
Your sarcasm is unwarranted, because what I said is true and reflects the experience of a lot of people.
A decade ago I left a job and spent the last week thoroughly documenting every flow and code section of an app that I worked with, which was the core value proposition of the company. A couple years later I ask around and nobody even took a look at that.
People just don't read, and there are actually good reasons for that, one of them being that documentation is outdated in most orgs and the effort to keep it up to date is greater than reading the code.
That’s a rather stunning comparison: racism is a problem because it’s unfairly treating sentient beings but a pile of linear algebra is not even sentient, much less your peer. That’s part of why I used the term: “agent” isn’t current because agents have, well, agency and can be held accountable.
People are rediscovering everything. Some people have proposed using a more formal language to tell the AI precisely what code to write. That's a compiler.
Ťhose are all technology variations of “automated web ui tests”, which is a subset of “automated ui tests”, which is itself almost (but not quite exactly) a subset of “automated user acceptance tests”, none of which are new categories.
So, throw out the traditional test pyramid, shift right, and rely more on persona testing than fine-grained atomic tests? I would hope teams don't need to re-learn that lesson for themselves, but...
My clankers are instructed to use "Outside In development" with "red/green TDD" at all times.
They write really good Gherkin features and then work inwards writing unit tests as they go - checking that they fail before implementation so it's actually testing something worthwhile.
And the code they ship is decent quality (not as good as me most of the time - but a LOT better than me when I'm tired or I'm pissed off about something or the work is really boring).
This already exists. You mean capturing user flows which should already be supplied by product to the developer. A decent system is Behavior Driven Development (though honestly a poor acronym for it’s use).
Are you an engineer ? You must test your "creation". Or would you expect that the microwave owen you just bougth will be tested by your child while getting burned ?
Two of the reasons I never liked writing tests is
- they didn’t seem to usually assert much internal logic
- they would have to be maintained along with the original code
I think scenario testing is much better instead because the actual way a person uses a feature hardly changes but the internals might change a lot.
So imagine I’m making an e-commerce website. There are lots of internal mechanisms. I’ll have an agent testing all the functionalities as if it were a customer. This gives me much much more confidence while writing code because it is more uncorellated with the code.
Tomorrow I can change a lot of internals but the testing agent stays the same.
There’s something to note though: not all code is possible to be scenario tested. Like data engineering and other things where the feedback time is huge.