Part
Scenario
You might say: end-to-end user story, acceptance scenario, scenario test
A user story, written down, that a harness runs from start to finish against the built software. It checks what a user would see, and a model can judge results that are hard to assert exactly.
- What it does
- Tests the whole change the way a user meets it, as close to the real environment as the factory can get.
- Ask
- Does each scenario run end to end against the real app or a faithful twin of its services?
What to look for
- A folder of scenario files written as user stories
- Scenario runs listed beside the unit test results
- A model's judgement recorded for each scenario run
Connections
- Made from: Spec
Sources
- Justin McCarthy (StrongDM), Software Factories And The Agentic Moment, Software Factories And The Agentic Moment
We repurposed the word scenario to represent an end-to-end "user story", often stored outside the codebase (similar to a "holdout" set in model training), which could be intuitively understood and flexibly validated by an LLM.
- Justin McCarthy (StrongDM), Software Factories And The Agentic Moment, The Principles
Your validation harness must be end-to-end, as close to the real environment as possible
- Simon Willison, How StrongDM’s AI team build serious software without even looking at the code
Their scenario tests became scripts for agents to constantly execute against the new systems as they were being built.
Use it on your project
teardown-apply: hold "Scenario" (scenario) from the teardown at https://teardownatlas.com/software-factory/ against my project
Paste it into a coding agent that has the teardown skills.
An AI agent wrote this map with the teardown skill. It can contain errors: check the sources before you rely on a part. About and legal