{"title":"Software factory","prompt":"I want agents to turn tickets into merged code while I'm away","summary":"A software factory turns events, such as a new ticket, a pull request, a failed check or a clock, into coding-agent runs, and turns their output into merged code. Inside each run a harness drives a model through tool calls, the context window fills as it works, and its permissions are set in advance because nobody is watching. Each run works on a disposable runner and remembers nothing; the ticket tracker, the git host and the orchestrator hold the state between runs. The factory succeeds by giving agents a space where they can do the work and prove it: an agent-ready codebase with a short map, one verify command and a tool that drives the app, plus verification against scenarios the author never saw. Practitioners start with one throttled trigger and human review on every pull request, turn what review catches into checks, and let evidence earn trust. At level 4, low-risk work merges on checks and evidence, and people spend their review on specs, one-way doors and high-risk changes.","root":"software-factory","nodes":[{"term":"Software factory","aka":["agent factory","autopilot","software kitchen","Michelin kitchen"],"kind":"product","plain":"A system that turns tickets, pull requests, failed checks and timers into coding-agent runs, and turns the runs' output into merged code. It works when agents have a space where they can do the work and prove it, so that human review leaves the critical path for low-risk changes.","ask":"What starts the work, what evidence proves it, and who may merge it?","cues":["Tickets move to Done without a person starting each run","Bot-authored pull requests carry check results and the run's own evidence","Low-risk pull requests merge with no human approval","A run page records each ticket from trigger to finish"],"children":["ticket-tracker","git-host","orchestrator","runner","coding-agent","agent-ready-codebase","task","verification","brakes","observability","factory-design"],"pattern":"pattern-kitchen-brigade","variants":[{"label":"Setup","items":[{"id":"all-in-one-git-platform","name":"All on one git platform","plain":"The ticket board, pull requests, checks and workflow engine live in one hosted git platform.","why":"The shared platform joins the work item, the source change and its checks in one place.","sources":[{"label":"GitLab, Platform overview","url":"https://about.gitlab.com/","kind":"maker","page":"Platform","quote":"Planning, source control, and CI/CD in one platform. Projects, releases, and code share one data plane, so your team and agents work from a single source of truth.","work":"gitlab-platform"}]},{"id":"dedicated-tracker-own-orchestrator","name":"Dedicated tracker and own orchestrator","plain":"A separate tracker sends issue context to a team-owned service when an event fires.","why":"The tracker and dispatcher meet at an explicit event boundary.","sources":[{"label":"Linear, Getting Started with Agents","url":"https://linear.app/developers/agents","kind":"maker","page":"Receiving your first webhook","quote":"This triggers a `created` AgentSessionEvent webhook containing an `agentSession` object with the relevant issue, comment, and context.","work":"linear-agents"}]},{"id":"own-ticket-run-app","name":"Own app for tickets and runs","plain":"A purpose-built application holds project context, work orders, agent connections, and feedback.","why":"The application is the durable interface for work context, work orders, and feedback.","sources":[{"label":"8090 Software Factory, Quickstart","url":"https://docs.8090.ai/general/quickstart","kind":"maker","page":"Create your first project","quote":"A project holds the context that defines your software, from requirements and blueprints to work orders and feedback.","work":"software-factory-8090"}]},{"id":"vendor-hosted-agent-platform","name":"Vendor hosted agent platform","plain":"A vendor hosts the agent's ephemeral development environment while repository instructions shape its work.","why":"The platform supplies the run environment while repository instructions shape the task.","sources":[{"label":"GitHub, Best practices for using GitHub Copilot to work on tasks","url":"https://docs.github.com/en/copilot/tutorials/cloud-agent/get-the-best-results","kind":"maker","page":"Pre-installing dependencies in GitHub Copilot's environment","quote":"While working on a task, Copilot has access to its own ephemeral development environment, powered by GitHub Actions, where it can explore your code, make changes, execute automated tests and linters and more.","work":"github-copilot-task-practices"},{"label":"GitHub, Best practices for using GitHub Copilot to work on tasks","url":"https://docs.github.com/en/copilot/tutorials/cloud-agent/get-the-best-results","kind":"maker","page":"Adding custom instructions to your repository","quote":"By adding custom instructions to your repository, you can guide Copilot on how to understand your project and how to build, test and validate its changes.","work":"github-copilot-task-practices"}]}]}],"sources":[{"label":"Simon Willison, How StrongDM’s AI team build serious software without even looking at the code","url":"https://simonwillison.net/2026/Feb/7/software-factory/","kind":"practitioner","quote":"We built a Software Factory: non-interactive development where specs + scenarios drive agents that write code, run harnesses, and converge without human review.","work":"simon-willison-software-factory"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Custom Automations","quote":"Each automation starts a Droid session when its trigger fires: on a schedule, when a message lands in a Slack channel, on a GitHub event, or when another service sends a webhook.","work":"factory-software-factory"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Events feed","quote":"Runs Each run's outcome and duration, with a link to its session.","work":"factory-software-factory"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Custom Automations","quote":"Automations turn Droid into a teammate for work that keeps coming back.","work":"factory-software-factory"},{"label":"Matt Pocock, Software Factories for Real Engineers","url":"https://www.aihero.dev/cohorts/software-factories-for-real-engineers-xdy1m","kind":"practitioner","page":"Classic Concepts + Coding Agents","quote":"Assign your coding agent to chew through tickets, and come back to clean code and informative PRs for your review","work":"matt-pocock-software-factories"},{"label":"Matt Pocock, software-factory skill (pull request #1316)","url":"https://github.com/mattpocock/skills/blob/b59cab42bf1206e1363eafc37f71b21c3bcaa12f/skills/in-progress/software-factory/SKILL.md","kind":"practitioner","page":"Software factory","quote":"A software factory is a system of agents that turns tickets, PRs and schedules into merged code, deployed on remote infrastructure the user chooses.","work":"matt-pocock-software-factory-skill"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/software-factory","kind":"practitioner","page":"Software factory","quote":"A factory takes the human out of starting sessions, and not necessarily out of anything else.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"Audit any repository and set it up so coding agents ship changes safely with little human review.","work":"kent-c-dodds-kcd-skills"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/README.md","kind":"practitioner","page":"pstack","quote":"throughput without quality is not a goal i aspire to.","work":"lauren-tan-pstack"}],"id":"software-factory","look":{"parts":[{"slot":"floor","shape":"box","size":[6.6,0.12,2.2],"pos":[0,0.06,0],"color":"#d9dbe0","material":"matte","radius":0.06},{"slot":"belt","shape":"box","size":[5.2,0.16,0.7],"pos":[0.2,0.2,0.3],"color":"#3a3f47","material":"matte"},{"slot":"tickets","shape":"box","size":[0.3,0.03,0.22],"pos":[-1.9,0.3,0.3],"color":"#f7f4ec","material":"matte"},{"slot":"tickets-2","shape":"box","size":[0.3,0.03,0.22],"pos":[-1.45,0.3,0.3],"color":"#f7f4ec","material":"matte"},{"slot":"tracker","shape":"box","size":[1,1.4,0.08],"pos":[-2.75,0.95,-0.2],"color":"#6b5bd2","material":"matte","radius":0.04},{"slot":"tracker-face","shape":"plane","size":[0.9,1.26],"pos":[-2.75,0.95,-0.155],"svg":"InboxReadyDone"},{"slot":"task-card","shape":"plane","size":[0.5,0.62],"pos":[-1.5,1.05,0.3],"svg":"TASKTrigger: labelDoes: implementNext: reviewHuman: on PRMax tries: 3"},{"slot":"runners","shape":"box","size":[0.6,0.6,0.6],"pos":[-0.8,0.6,0.3],"color":"#3b6ea8","material":"gloss","radius":0.05},{"slot":"runners-2","shape":"box","size":[0.6,0.6,0.6],"pos":[0,0.6,0.3],"color":"#3b6ea8","material":"gloss","radius":0.05},{"slot":"runners-3","shape":"box","size":[0.6,0.6,0.6],"pos":[0.8,0.6,0.3],"color":"#3b6ea8","material":"gloss","radius":0.05},{"slot":"sparks","shape":"particles","count":40,"box":[2.2,0.3,0.5],"pos":[0,1.05,0.3],"size":0.025,"color":"#9cc4f0","motion":"rise"},{"slot":"agent","shape":"sphere","radius":0.09,"pos":[-0.8,1.02,0.3],"color":"#8b6bd6","material":"glow","motion":"bob"},{"slot":"agent-2","shape":"sphere","radius":0.09,"pos":[0,1.02,0.3],"color":"#8b6bd6","material":"glow","motion":"bob"},{"slot":"agent-3","shape":"sphere","radius":0.09,"pos":[0.8,1.02,0.3],"color":"#8b6bd6","material":"glow","motion":"bob"},{"slot":"cage","shape":"box","size":[2.5,1,1],"pos":[0,0.73,0.3],"color":"#8a8f98","material":"glass","opacity":0.16},{"slot":"control","shape":"box","size":[1.7,0.75,0.1],"pos":[0,1.8,-0.6],"color":"#1d2a2a","material":"matte","radius":0.04},{"slot":"control-screen","shape":"plane","size":[1.56,0.62],"pos":[0,1.8,-0.545],"material":"screen","svg":"RUNSQueued 3In progress 2Done 41"},{"slot":"board","shape":"box","size":[0.95,0.72,0.06],"pos":[-1.75,1.65,-0.58],"color":"#c9a46a","material":"matte","radius":0.02},{"slot":"board-face","shape":"plane","size":[0.88,0.64],"pos":[-1.75,1.65,-0.545],"svg":"AGENTS.mdAa>_"},{"slot":"monitor","shape":"box","size":[0.9,0.6,0.08],"pos":[1.55,1.65,-0.55],"color":"#2a2a2e","material":"matte","radius":0.03},{"slot":"monitor-screen","shape":"plane","size":[0.82,0.52],"pos":[1.55,1.65,-0.505],"material":"screen","svg":""},{"slot":"scanner","shape":"box","size":[0.22,0.14,0.9],"pos":[1.5,1.12,0.3],"color":"#2a2a2e","material":"gloss","radius":0.02},{"slot":"scan-beam","shape":"box","size":[0.03,0.75,0.8],"pos":[1.5,0.66,0.3],"color":"#7fe08a","material":"glow","opacity":0.5},{"slot":"scanner-post","shape":"cylinder","radiusTop":0.04,"radiusBottom":0.04,"height":0.93,"pos":[1.5,0.585,-0.12],"color":"#5a5f68","material":"metal"},{"slot":"scanner-post-2","shape":"cylinder","radiusTop":0.04,"radiusBottom":0.04,"height":0.93,"pos":[1.5,0.585,0.72],"color":"#5a5f68","material":"metal"},{"slot":"gate-post","shape":"cylinder","radiusTop":0.05,"radiusBottom":0.05,"height":1.1,"pos":[1.85,0.7,-0.15],"color":"#5a5f68","material":"metal"},{"slot":"gate-post-2","shape":"cylinder","radiusTop":0.05,"radiusBottom":0.05,"height":1.1,"pos":[1.85,0.7,0.75],"color":"#5a5f68","material":"metal"},{"slot":"gate-bar","shape":"box","size":[0.12,0.1,1],"pos":[1.85,1.25,0.3],"color":"#c8432f","material":"gloss"},{"slot":"gate-light","shape":"sphere","radius":0.08,"pos":[1.85,1.36,0.3],"color":"#2e9b4f","material":"glow"},{"slot":"pr-stand","shape":"box","size":[0.95,1.3,0.08],"pos":[2.75,0.9,-0.2],"color":"#e8743b","material":"matte","radius":0.04},{"slot":"pr-face","shape":"plane","size":[0.85,1.18],"pos":[2.75,0.9,-0.155],"svg":"Pull requestFix: retry on timeoutChecks passedAgent reviewScenarios 50/50Low riskAuto-merge"},{"slot":"blueprint","shape":"cylinder","radiusTop":0.09,"radiusBottom":0.09,"height":0.9,"pos":[2.55,0.21,0.85],"rot":[0,0,90],"color":"#2f5d9e","material":"matte"}]}},{"term":"Ticket tracker","aka":["issue tracker","work board","backlog","project board"],"kind":"part","plain":"A service that records work items and their state. It is the factory's durable queue and its human-facing record.","does":"Holds the task request, ownership, routing state, and links to discussion so runs can return to the same work.","ask":"Can a run find one clear task, update its state, and leave a useful record for the next run?","cues":["Board columns or status pills","A ticket detail page with title and body","Assignee, labels, and activity controls"],"range":"Built into a git host, a dedicated tracker, a self-hosted tracker, a general work tool, files in a repository, or a custom service.","trouble":"Tickets are duplicated, stale, unowned, or routed to a task that cannot act on them.","children":["ticket","ticket-source","routing-label","comment-thread"],"at":[-2.75,0.95,-0.15],"links":[{"to":"git-host","rel":"see also"},{"to":"comment-thread","rel":"see also"}],"variants":[{"label":"Tracker type","items":[{"id":"tracker-git-host-built-in","name":"Git-host built-in","plain":"Issues live beside the repository, pull requests, labels, and checks on one host.","why":"The tracker shares the code host's identity and event stream.","sources":[{"label":"GitHub, REST API endpoints for pull requests","url":"https://docs.github.com/en/rest/pulls/pulls","kind":"maker","page":"About pull requests","quote":"Pull requests are a type of issue. Any actions that are available in both pull requests and issues, like managing assignees, labels, and milestones, are handled by the REST API to manage issues.","work":"github-pull-requests-rest"},{"label":"GitHub, About webhooks","url":"https://docs.github.com/en/webhooks/about-webhooks","kind":"maker","page":"About webhooks on GitHub","quote":"When an event that your webhook is subscribed to occurs, GitHub will send an HTTP request with data about the event to the URL that you specified.","work":"github-webhooks-about"}]},{"id":"tracker-agent-api","name":"Developer tracker with agent API","plain":"A dedicated development tracker exposes agents as workspace actors that can be assigned issues and write comments.","why":"The tracker gives an agent a first-class identity and delegation path.","sources":[{"label":"Linear, Getting Started with Agents","url":"https://linear.app/developers/agents","kind":"maker","page":"Overview","quote":"Agents behave similar to other users in a workspace. They can be @mentioned, delegated issues through assignment, create and reply to comments, collaborate on projects and documents, etc.","work":"linear-agents"}]},{"id":"tracker-self-hosted","name":"Self-hosted open source","plain":"A tracker is run and operated by the team that owns the factory.","why":"The team controls its deployment, storage, and integrations.","sources":[{"label":"Forgejo, Installation with Docker","url":"https://forgejo.org/docs/latest/admin/installation/docker/","kind":"maker","page":"Installation with Docker","quote":"Forgejo provides container images for use with Docker or other containerization tools.","work":"forgejo-installation-docker"}]},{"id":"tracker-general-work-tool","name":"General work tool","plain":"A broad work-management product stores work items and can react to their changes.","why":"The product exposes task events that automation can consume.","sources":[{"label":"Atlassian, Jira automation triggers","url":"https://support.atlassian.com/cloud-automation/docs/jira-automation-triggers/","kind":"maker","page":"Jira automation triggers","quote":"Triggers will listen for events in Jira, such as when a work item is created or when a field value is changed.","work":"jira-automation-triggers"}]},{"id":"tracker-repo-files","name":"Tickets as files in the repository","plain":"Each request is a versioned file or record next to the code.","why":"The repository itself becomes the durable task store.","sources":[{"label":"Backlog.md, README","url":"https://raw.githubusercontent.com/MrLesk/Backlog.md/6f988de052b8d8ef2eac2828ade7d8690ac7213f/README.md","kind":"practitioner","page":"README","quote":"Backlog.md turns any folder into a self-contained project board powered by plain Markdown files and a zero-config CLI.","work":"backlog-md-readme"}]},{"id":"tracker-build-your-own","name":"Build your own","plain":"A small service owns its records and emits change events.","why":"Its records can emit events that downstream automation consumes.","sources":[{"label":"Airtable, Webhooks API overview","url":"https://support.airtable.com/articles/7385030819-airtable-webhooks-api-overview","kind":"maker","page":"Webhooks API overview","quote":"The Webhooks API allows developers to be notified in real-time about changes in their base that they care about, from newly created records to a field being updated in an existing record.","work":"airtable-webhooks"}]}]}],"sources":[{"label":"GitLab, Issues","url":"https://docs.gitlab.com/user/project/issues/","kind":"maker","page":"Issues","quote":"Issues help you collaborate with your team to plan, track, and deliver work in GitLab.","work":"gitlab-issues"},{"label":"Linear, Getting Started with Agents","url":"https://linear.app/developers/agents","kind":"maker","page":"Agent session lifecycle","quote":"Session state is visible to users, and updated automatically based on the agent's emitted activities.","work":"linear-agents"}],"id":"ticket-tracker","look":{"parts":[{"slot":"housing","shape":"box","size":[1.15,1.55,0.09],"pos":[0,0.78,0],"radius":0.05,"color":"#6b5bd2","material":"matte"},{"slot":"screen","shape":"plane","size":[1.03,1.39],"pos":[0,0.8,0.055],"svg":"WORK QUEUEINBOXREADYDONE"},{"slot":"ticket","shape":"plane","size":[0.45,0.32],"pos":[-0.28,0.42,0.09],"face":"camera","svg":"TASK"},{"slot":"source","shape":"plane","size":[0.36,0.2],"pos":[-0.34,0.72,0.09],"face":"camera","svg":""},{"slot":"label","shape":"plane","size":[0.32,0.12],"pos":[0.1,0.66,0.1],"face":"camera","svg":"READY"},{"slot":"thread","shape":"plane","size":[0.34,0.35],"pos":[0.31,0.2,0.09],"face":"camera","svg":""}]}},{"term":"Ticket","aka":["issue","work item","card","spec"],"kind":"part","plain":"A work item that records a request, its state, its owner and its discussion, and scopes one session of work.","does":"Provides a durable place to assign and update work.","ask":"Is the request specific enough to implement, review, or route without guessing?","cues":["Title and body text","State, labels, and assignee shown together","Links to a pull request, commit, or report"],"range":"Bug, feature request, support request, investigation, maintenance task, or decomposition item.","trouble":"The ticket has no acceptance signal, mixes several problems, or has no owner or next state.","at":[-0.28,0.42,0.06],"links":[{"to":"ticket-source","rel":"made by"},{"to":"routing-label","rel":"see also"}],"sources":[{"label":"GitLab, Issues","url":"https://docs.gitlab.com/user/project/issues/","kind":"maker","page":"Issues","quote":"Issues: Track feature proposals, tasks, support requests, and bug reports. Organize and prioritize work with assignees, due dates, and health status.","work":"gitlab-issues"},{"label":"Atlassian, Jira automation triggers","url":"https://support.atlassian.com/cloud-automation/docs/jira-automation-triggers/","kind":"maker","page":"Work item created","quote":"You can use this trigger with actions to customize the new work item, including populating fields, assigning to users, and adding sub-tasks.","work":"jira-automation-triggers"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/ticket","kind":"practitioner","page":"Ticket","quote":"A handoff artifact scoping one session of work. Stands alone, or hangs off a spec as one of its children. Tickets can block or be blocked by sibling tickets, so the order of work falls out of their dependency graph rather than a linear plan.","work":"matt-pocock-ai-coding-dictionary"}],"deeper":"Ticket fields, acceptance checks, state transitions, deduplication keys, and links to commits and pull requests.","id":"ticket","look":{"parts":[{"shape":"box","size":[1.1,0.72,0.05],"pos":[0,0,0],"radius":0.05,"color":"#d8cfbf","material":"matte"},{"slot":"card","shape":"plane","size":[1.03,0.65],"pos":[0,0.02,0.035],"face":"camera","svg":"TICKETREADYowner · links · state"}]}},{"term":"Ticket source","aka":["intake","feedback button","monitor","error tracker"],"kind":"part","plain":"Something outside the tracker that supplies a request or observation. It can create a ticket directly or send evidence for a human or task to file.","does":"Converts outside signals into durable work while preserving enough context to trace the signal back.","ask":"What creates this signal, and what prevents the same signal from filing the same ticket twice?","cues":["An incoming event, alert, or form entry","A source URL or external event id on the ticket","An intake rule that maps signals to a project or label"],"range":"Feedback button, monitor, error tracker, scanner, inbox, or another webhook producer. Deduplicate by external event id or fingerprint grouping before opening another ticket.","trouble":"A noisy source floods the tracker, loses context, or creates duplicate tickets.","at":[-0.34,0.72,0.05],"links":[{"to":"ticket","rel":"raises"},{"to":"webhook","rel":"see also"}],"variants":[{"label":"Source kind","items":[{"id":"source-feedback","name":"Feedback button","plain":"A user submits a short report from the product or a support surface.","sources":[{"label":"GitHub, Syntax for issue forms","url":"https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms","kind":"maker","page":"Syntax for issue forms","quote":"You can define different input types, validations, default assignees, and default labels for your issue forms.","work":"github-issue-forms"},{"label":"Linear, Asks","url":"https://linear.app/docs/linear-asks","kind":"maker","page":"Asks","quote":"Asks turns requests like bug reports, questions, and IT needs into actionable issues in Linear.","work":"linear-asks"},{"label":"Atlassian, Jira Service Management help centers","url":"https://support.atlassian.com/jira-service-management-cloud/docs/how-do-customers-send-requests-to-your-service-project/","kind":"maker","page":"Customer requests","quote":"Customers can raise a request online through help centers on your site.","work":"atlassian-jsm-help-center"}]},{"id":"source-monitor","name":"Monitor alert","plain":"A service-health monitor emits an alert when a measured condition crosses a rule.","sources":[{"label":"Datadog, Jira integration","url":"https://docs.datadoghq.com/integrations/jira/","kind":"maker","page":"Jira integration","quote":"Issues are created when the monitor triggers.","work":"datadog-jira"},{"label":"Grafana, Jira alerting integration","url":"https://grafana.com/docs/grafana/latest/alerting/configure-notifications/manage-contact-points/integrations/configure-jira/","kind":"maker","page":"Jira alerting integration","quote":"Use the Jira integration in a contact point to create issues in your Jira instance when alerts fire.","work":"grafana-jira-alerting"}]},{"id":"source-error-tracker","name":"Error tracker","plain":"An error collection service groups failures and sends a report for investigation.","sources":[{"label":"Sentry, Event fingerprinting","url":"https://docs.sentry.io/platforms/javascript/guides/astro/enriching-events/fingerprinting/","kind":"maker","page":"Event fingerprinting","quote":"All events have a fingerprint. Events with the same fingerprint are grouped together into an issue.","work":"sentry-event-fingerprinting"},{"label":"Sentry, Create an alert for an organization","url":"https://docs.sentry.io/api/monitors/create-an-alert-for-an-organization/","kind":"maker","page":"Create an alert for an organization","quote":"**Create a Jira ticket**\n\n* `integrationId`: The stringified ID of the integration.\n* `data` - A list of any fields you want to include in the ticket as objects.\n","work":"sentry-alerts"},{"label":"Sentry, Create an alert for an organization","url":"https://docs.sentry.io/api/monitors/create-an-alert-for-an-organization/","kind":"maker","page":"Create an alert for an organization","quote":"**Create a GitHub issue**\n\n* `integrationId`: The stringified ID of the integration.\n* `data` - A list of any fields you want to include in the ticket as objects.\n","work":"sentry-alerts"}]},{"id":"source-scanner","name":"Scanner","plain":"A security, dependency, or quality scanner reports a finding that needs work.","sources":[{"label":"GitHub, Code scanning alerts","url":"https://docs.github.com/en/code-security/concepts/code-scanning/code-scanning-alerts","kind":"maker","page":"Code scanning alerts","quote":"Each alert highlights a problem with the code and the name of the tool that identified it.","work":"github-code-scanning-alerts"},{"label":"GitHub, Dependabot alerts","url":"https://docs.github.com/en/code-security/concepts/supply-chain-security/dependabot-alerts","kind":"maker","page":"Dependabot alerts","quote":"Dependabot alerts help you find and fix vulnerable dependencies before they become security risks.","work":"github-dependabot-alerts"},{"label":"Snyk, Partner integrations","url":"https://docs.snyk.io/developer-tools/integrations/partner-integrations","kind":"maker","page":"Partner integrations","quote":"Snyk Jira integration allows you to manually raise Jira issues in the Snyk UI for vulnerabilities or license issues.","work":"snyk-partner-integrations"}]},{"id":"source-inbox","name":"Inbox","plain":"Email or a message queue collects requests until an intake task files them.","sources":[{"label":"Linear, Asks","url":"https://linear.app/docs/linear-asks","kind":"maker","page":"Asks","quote":"Once enabled, asks can be submitted through Slack, email, or web forms.","work":"linear-asks"},{"label":"Atlassian, Jira Service Management requests by email","url":"https://support.atlassian.com/jira-service-management-cloud/docs/how-customers-create-requests-by-email/","kind":"maker","page":"Requests by email","quote":"A customer emails a request to your service space email address. The request becomes a work item in your service space and is added to a queue.","work":"atlassian-jsm-customer-requests"},{"label":"GitLab, Service Desk","url":"https://docs.gitlab.com/user/project/service_desk/","kind":"maker","page":"Service Desk","quote":"Service Desk emails are created in your GitLab project as new tickets.","work":"gitlab-service-desk"}]}]}],"sources":[{"label":"GitHub, About webhooks","url":"https://docs.github.com/en/webhooks/about-webhooks","kind":"maker","page":"About webhooks","quote":"Webhooks let you subscribe to events happening in a software system and automatically receive a delivery of data to your server whenever those events occur.","work":"github-webhooks-about"},{"label":"Sentry, Event fingerprinting","url":"https://docs.sentry.io/platforms/javascript/guides/astro/enriching-events/fingerprinting/","kind":"maker","page":"Event fingerprinting","quote":"All events have a fingerprint. Events with the same fingerprint are grouped together into an issue.","work":"sentry-event-fingerprinting"}],"id":"ticket-source","look":{"parts":[{"shape":"box","size":[1.15,0.72,0.1],"pos":[0,0.1,0],"radius":0.06,"color":"#8a8f98","material":"matte"},{"slot":"inbox","shape":"plane","size":[1.06,0.58],"pos":[0,0.14,0.06],"face":"camera","svg":"INTAKEfeedback · alert · report"},{"slot":"signal","shape":"particles","count":32,"box":[0.7,0.22,0.05],"pos":[-0.7,0.48,0.08],"size":0.025,"color":"#e8743b","motion":"drift"}]}},{"term":"Routing label","aka":["ready label","ready-for-agent","queue label","status column"],"kind":"knob","plain":"A label or status value that tells the factory which task may handle a ticket.","does":"Selects work for a task without hiding the ticket's human state.","ask":"Which exact label or state means this task may claim the ticket now?","cues":["A coloured label chip on a ticket","A board column such as Ready or In progress","A workflow rule matching the label or state"],"range":"One explicit ready label, a small set of task labels, or a status transition with a task filter.","trouble":"Broad labels start the wrong task, a missing label strands work, or an agent adds its own trigger label repeatedly.","at":[0.1,0.66,0.07],"links":[{"to":"trigger","rel":"controls"},{"to":"work-claim","rel":"see also"}],"sources":[{"label":"GitHub, Managing labels","url":"https://docs.github.com/en/issues/using-labels-and-milestones-to-track-work/managing-labels","kind":"maker","page":"About labels","quote":"You can manage your work on GitHub by creating labels to categorize issues, pull requests, and discussions.","work":"github-labels"},{"label":"GitHub, Triggering a workflow","url":"https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow","kind":"maker","page":"Using event activity types","quote":"For example, the following workflow triggers when an issue is opened or labeled.","work":"github-actions-triggering-workflow"}],"id":"routing-label","look":{"parts":[{"shape":"box","size":[1.1,0.48,0.04],"pos":[0,0,0],"radius":0.08,"color":"#d9dbe0","material":"matte"},{"slot":"chip","shape":"plane","size":[0.82,0.27],"pos":[0,0.04,0.03],"face":"camera","svg":"READY FOR AGENT"},{"slot":"pointer","shape":"cone","radius":0.1,"height":0.22,"pos":[0.48,-0.18,0.02],"rot":[0,0,-90],"color":"#e8743b","material":"gloss"}]}},{"term":"Comment thread","aka":["comments","discussion","conversation"],"kind":"part","plain":"The ordered conversation attached to a ticket or pull request. People and runs leave context, questions, decisions, and links there.","does":"Keeps human re-entry and run results beside the work item instead of in transient runner output.","ask":"Can a new run tell what was tried, what failed, and what a human decided?","cues":["A chronological list of comments","Replies or threaded discussions under a ticket","Mentions, links, and author identities"],"trouble":"Results are only in logs, comments are ambiguous, or an automated reply triggers itself forever.","at":[0.31,0.2,0.06],"links":[{"to":"bot-identity","rel":"see also"},{"to":"ticket","rel":"see also"}],"sources":[{"label":"GitLab, Issues","url":"https://docs.gitlab.com/user/project/issues/","kind":"maker","page":"Issues","quote":"Facilitate team discussion and decision-making through comments and threaded discussions.","work":"gitlab-issues"},{"label":"Linear, Getting Started with Agents","url":"https://linear.app/developers/agents","kind":"maker","page":"Overview","quote":"They can be @mentioned, delegated issues through assignment, create and reply to comments, collaborate on projects and documents, etc.","work":"linear-agents"}],"id":"comment-thread","look":{"parts":[{"shape":"box","size":[1.05,1,0.05],"pos":[0,0.5,0],"radius":0.04,"color":"#d8cfbf","material":"matte"},{"slot":"thread","shape":"plane","size":[0.96,0.9],"pos":[0,0.52,0.035],"face":"camera","svg":"DISCUSSION"}]}},{"term":"Git host","aka":["code host","repository host","forge","GitHub"],"kind":"part","plain":"A service that stores Git repositories and hosts their review objects, such as pull requests, checks and merge controls.","does":"Turns workspace changes into a reviewable change and records the events and gates that govern merging.","ask":"Where are the branch, diff, checks, review, identity, and merge decision recorded?","cues":["Repository, branch, and commit pages","A pull request with a diff and checks","A merge control beside required gates"],"range":"All-in-one platform, suite module, self-hosted forge, review-first system, agent-native host, or Git storage used by a separate run service.","trouble":"Runs cannot push, reviews are detached from the diff, checks are missing, or an identity can bypass merge controls.","children":["pull-request","bot-identity","branch-protection","webhook"],"at":[2.75,0.9,-0.15],"links":[{"to":"ticket-tracker","rel":"see also"},{"to":"orchestrator","rel":"see also"}],"variants":[{"label":"Git host type","items":[{"id":"git-host-all-in-one","name":"All-in-one platform","plain":"Repository storage, code review, checks, and merge controls share one product surface.","why":"Pull requests and issues share the host's repository resources and controls.","sources":[{"label":"GitHub, REST API endpoints for pull requests","url":"https://docs.github.com/en/rest/pulls/pulls","kind":"maker","page":"About pull requests","quote":"Pull requests are a type of issue. Any actions that are available in both pull requests and issues, like managing assignees, labels, and milestones, are handled by the REST API to manage issues.","work":"github-pull-requests-rest"},{"label":"GitHub, About protected branches","url":"https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches","kind":"maker","page":"Require status checks before merging","quote":"After enabling required status checks, all required status checks must pass before collaborators can merge changes into the protected branch.","work":"github-protected-branches"}]},{"id":"git-host-suite-module","name":"Suite module","plain":"Git hosting is one module in a larger development or work-management suite.","why":"The project surface combines files and tasks in one project.","sources":[{"label":"GitLab, Get started organizing work with projects","url":"https://docs.gitlab.com/user/get_started/get_started_projects/","kind":"maker","page":"Get started organizing work with projects","quote":"Projects in GitLab organize all the data for a specific development project. A project is where you work with your team, store your files, and manage your tasks.","work":"gitlab-projects"}]},{"id":"git-host-self-hosted","name":"Self-hosted forge","plain":"The organization runs the repository and review service on its own infrastructure.","why":"Hosting, storage, and network policy stay under the organization's control.","sources":[{"label":"Forgejo, Installation with Docker","url":"https://forgejo.org/docs/latest/admin/installation/docker/","kind":"maker","page":"Installation with Docker","quote":"Forgejo provides container images for use with Docker or other containerization tools.","work":"forgejo-installation-docker"}]},{"id":"git-host-review-first","name":"Review-first host","plain":"Review records organize changes through patch sets, labels, and approvals.","why":"The host makes review state central to contribution.","sources":[{"label":"Gerrit, Patch sets","url":"https://gerrit-review.googlesource.com/Documentation/concept-patch-sets.html","kind":"maker","page":"Patch Sets","quote":"When a change is approved, only the latest version of a commit is submitted to the repository.","work":"gerrit-patch-sets"},{"label":"Gerrit, Review labels","url":"https://gerrit-review.googlesource.com/Documentation/config-labels.html","kind":"maker","page":"Review labels","quote":"Gerrit comes pre-configured with the Code-Review label that can be granted to groups within projects.","work":"gerrit-review-labels"}]},{"id":"git-host-agent-native","name":"Agent-native host","plain":"The host gives automated actors bot identities, event triggers, and native review operations.","why":"Agents can act in the same review and merge surfaces as people.","sources":[{"label":"Gitea Enterprise, Agents","url":"https://docs.gitea.com/enterprise/features/agents/","kind":"maker","page":"Agents","quote":"Each agent is bound to its own bot account, and the agent name is the username of that bot. The bot can therefore be mentioned (@name), assigned to an issue or pull request, or requested as a reviewer like any other user.","work":"gitea-enterprise-agents"},{"label":"Gitea Enterprise, Agents","url":"https://docs.gitea.com/enterprise/features/agents/","kind":"maker","page":"How agents work","quote":"The runner changes the code and pushes the branch to the target repository, while Gitea records the result and performs the follow-up actions, such as creating or updating pull requests, commenting, reviewing and merging.","work":"gitea-enterprise-agents"},{"label":"Gitea Enterprise, Agents","url":"https://docs.gitea.com/enterprise/features/agents/","kind":"maker","page":"Agents","quote":"You create an agent once, pick the action it performs and the events that trigger it, and it then analyzes issues and pull requests, writes code, opens and maintains pull requests, reviews changes, or scans repositories for security problems.","work":"gitea-enterprise-agents"}]},{"id":"git-host-run-storage","name":"Git storage for runs","plain":"Git repositories hold workflow definitions while another service executes their jobs.","why":"The host remains the source of code and workflow history.","sources":[{"label":"GitHub, Workflow syntax for GitHub Actions","url":"https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax","kind":"maker","page":"About YAML syntax for workflows","quote":"You must store workflow files in the `.github/workflows` directory of your repository.","work":"github-actions-workflow-syntax"}]}]}],"sources":[{"label":"GitHub, REST API endpoints for pull requests","url":"https://docs.github.com/en/rest/pulls/pulls","kind":"maker","page":"About pull requests","quote":"Pull requests are a type of issue. Any actions that are available in both pull requests and issues, like managing assignees, labels, and milestones, are handled by the REST API to manage issues.","work":"github-pull-requests-rest"},{"label":"GitHub, About webhooks","url":"https://docs.github.com/en/webhooks/about-webhooks","kind":"maker","page":"About webhooks on GitHub","quote":"When an event that your webhook is subscribed to occurs, GitHub will send an HTTP request with data about the event to the URL that you specified.","work":"github-webhooks-about"},{"label":"GitHub, About protected branches","url":"https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches","kind":"maker","page":"Require status checks before merging","quote":"After enabling required status checks, all required status checks must pass before collaborators can merge changes into the protected branch.","work":"github-protected-branches"}],"id":"git-host","look":{"parts":[{"slot":"housing","shape":"box","size":[1.18,1.55,0.1],"pos":[0,0.78,0],"radius":0.05,"color":"#e8743b","material":"matte"},{"slot":"screen","shape":"plane","size":[1.06,1.4],"pos":[0,0.8,0.06],"svg":"REPOSITORYbranch · diff · checks · merge"},{"slot":"pr","shape":"plane","size":[0.42,0.32],"pos":[-0.3,0.55,0.1],"face":"camera","svg":"PR"},{"slot":"identity","shape":"plane","size":[0.32,0.22],"pos":[0.24,1.02,0.1],"face":"camera","svg":""},{"slot":"gate","shape":"box","size":[0.1,0.46,0.1],"pos":[0.32,0.42,0.1],"color":"#c8432f","material":"gloss"},{"slot":"hook","shape":"torus","radius":0.16,"tube":0.025,"pos":[0.34,0.82,0.1],"rot":[90,0,0],"color":"#f3efe4","material":"metal"}]}},{"term":"Pull request","aka":["PR","merge request","MR","change proposal"],"kind":"part","plain":"A proposed change from one branch or repository into another, with a diff, discussion, checks, and a merge decision.","does":"Gives automated and human reviewers one stable object to inspect before code reaches the protected branch.","ask":"What changed, which checks and reviews passed, and who may merge it?","cues":["Source and target branch names","A file diff with review comments","Check results and a Merge button"],"range":"Draft or ready, one commit or many, same repository or fork, mergeable or blocked.","trouble":"The diff changes after approval, checks are stale, review comments are lost, or the request has no clear base branch.","at":[-0.3,0.55,0.06],"links":[{"to":"branch-protection","rel":"controls"},{"to":"bot-identity","rel":"see also"},{"to":"comment-thread","rel":"see also"}],"sources":[{"label":"GitHub, REST API endpoints for pull requests","url":"https://docs.github.com/en/rest/pulls/pulls","kind":"maker","page":"About pull requests","quote":"You can list, view, edit, create, and merge pull requests using the REST API.","work":"github-pull-requests-rest"},{"label":"GitHub, About protected branches","url":"https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches","kind":"maker","page":"Require status checks before merging","quote":"After enabling required status checks, all required status checks must pass before collaborators can merge changes into the protected branch.","work":"github-protected-branches"}],"id":"pull-request","look":{"parts":[{"shape":"box","size":[1.18,1.3,0.05],"pos":[0,0.6,0],"radius":0.05,"color":"#e8d1c2","material":"matte"},{"slot":"card","shape":"plane","size":[1.08,1.2],"pos":[0,0.63,0.04],"face":"camera","svg":"PULL REQUESTfix: retry timeoutchecks passedreviewedhuman gateMERGE"},{"slot":"diff","shape":"particles","count":36,"box":[0.7,0.42,0.06],"pos":[0,0.02,0.08],"size":0.02,"color":"#2e9b4f","motion":"rise"},{"slot":"merge","shape":"box","size":[0.8,0.14,0.08],"pos":[0,-0.12,0.08],"radius":0.04,"color":"#2e9b4f","material":"gloss"}]}},{"term":"Bot identity","aka":["bot account","GitHub App","service account","automation token"],"kind":"part","plain":"The app, bot account, or token under which a run reads, writes, comments, and opens changes.","does":"Makes actions attributable and gives the host a place to apply permissions and event-recursion rules.","ask":"Which actor made this change, which permissions does it have, and can its events start another run?","cues":["An author avatar or bot badge","An app name in an event or check","Token permission settings or an installation record"],"range":"Repository token, app installation token, service account, or personal token used for automation.","trouble":"The actor has broad write access, actions are anonymous, or a token silently suppresses or causes recursive events.","at":[0.24,1.02,0.06],"links":[{"to":"least-privilege","rel":"see also"},{"to":"webhook","rel":"see also"}],"sources":[{"label":"GitHub, Triggering a workflow","url":"https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow","kind":"maker","page":"Triggering a workflow from a workflow","quote":"When you use the repository's GITHUB_TOKEN to perform tasks, events triggered by the GITHUB_TOKEN will not create a new workflow run, with the following exceptions.","work":"github-actions-triggering-workflow"},{"label":"GitHub, Use GITHUB_TOKEN for authentication in workflows","url":"https://docs.github.com/en/actions/tutorials/authenticate-with-github_token","kind":"maker","page":"Modifying the permissions for the GITHUB_TOKEN","quote":"As a good security practice, you should grant the GITHUB_TOKEN the least required access.","work":"github-actions-token"}],"id":"bot-identity","look":{"parts":[{"shape":"box","size":[0.9,0.9,0.08],"pos":[0,0.45,0],"radius":0.08,"color":"#3a3f47","material":"matte"},{"slot":"badge","shape":"plane","size":[0.78,0.78],"pos":[0,0.48,0.055],"face":"camera","svg":"BOT ACTORscoped token"},{"slot":"key","shape":"cylinder","radiusTop":0.07,"radiusBottom":0.07,"height":0.34,"pos":[0.5,0.25,0.05],"rot":[0,90,0],"color":"#e0a526","material":"metal"}]}},{"term":"Branch protection","aka":["merge gate","protected branch","ruleset","required checks"],"kind":"knob","plain":"Rules that constrain updates to important branches and require selected checks, reviews, or other conditions before merge.","does":"Stops a passing-looking run or actor from changing the shared branch without the required evidence.","ask":"Which branch is protected, which checks and approvals are required, and who can bypass them?","cues":["A protected-branch or ruleset settings page","Required checks and approval counts in a merge box","A blocked merge message"],"range":"Required checks, required approvals, code-owner review, signed commits, linear history, merge queue, deployment gates, and bypass lists.","trouble":"A check name is ambiguous, rules overlap, approvals go stale, or an administrator can bypass a gate without a recorded reason.","at":[0.32,0.42,0.06],"links":[{"to":"pull-request","rel":"controls"},{"to":"bot-identity","rel":"see also"}],"sources":[{"label":"GitHub, About protected branches","url":"https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches","kind":"maker","page":"About branch protection rules","quote":"You can protect important branches by setting branch protection rules, which define whether collaborators can delete or force push to the branch and set requirements for any pushes to the branch, such as passing status checks or a linear commit history.","work":"github-protected-branches"},{"label":"GitHub, About protected branches","url":"https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches","kind":"maker","page":"Require status checks before merging","quote":"Required status checks must have a `successful`, `skipped`, or `neutral` status before collaborators can make changes to a protected branch.","work":"github-protected-branches"},{"label":"GitHub, About protected branches","url":"https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches","kind":"maker","page":"About branch protection rules","quote":"Only a single branch protection rule can apply at a time, which means it can be difficult to know which rule will apply when multiple versions of a rule target the same branch. This restriction does not apply to rulesets","work":"github-protected-branches"}],"id":"branch-protection","look":{"parts":[{"slot":"post","shape":"cylinder","radiusTop":0.06,"radiusBottom":0.06,"height":1.1,"pos":[-0.38,0.55,0],"color":"#5a5f68","material":"metal"},{"slot":"post2","shape":"cylinder","radiusTop":0.06,"radiusBottom":0.06,"height":1.1,"pos":[0.38,0.55,0],"color":"#5a5f68","material":"metal"},{"slot":"bar","shape":"box","size":[0.16,0.14,0.9],"pos":[0,1.02,0],"color":"#c8432f","material":"gloss"},{"slot":"checks","shape":"plane","size":[0.72,0.38],"pos":[0,0.48,0.06],"face":"camera","svg":""}]}},{"term":"Webhook","aka":["event delivery","callback","signed HTTP event"],"kind":"part","plain":"An HTTP delivery that carries an event from one service to another so the receiver can act without polling.","does":"Carries ticket, repository, check, review, or run events into the orchestrator or another service.","ask":"Is this delivery authentic, acknowledged quickly, and safe to process more than once?","cues":["An endpoint URL and subscribed event list","Delivery id, event header, and response status","X-Hub-Signature-256 or another signature header"],"range":"One-way event delivery, signed payload, retrying delivery, or a queue-backed receiver.","trouble":"The receiver accepts forged payloads, times out, drops events, or repeats side effects on redelivery.","at":[0.34,0.82,0.06],"links":[{"to":"trigger","rel":"raises"},{"to":"bot-identity","rel":"see also"}],"sources":[{"label":"GitHub, About webhooks","url":"https://docs.github.com/en/webhooks/about-webhooks","kind":"maker","page":"About webhooks","quote":"Webhooks provide a way for notifications to be delivered to an external web server whenever certain events occur on GitHub.","work":"github-webhooks-about"},{"label":"GitHub, Validating webhook deliveries","url":"https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries","kind":"maker","page":"Validating webhook deliveries","quote":"The hash signature will appear in each delivery as the value of the X-Hub-Signature-256 header.","work":"github-webhooks-validating"}],"id":"webhook","look":{"parts":[{"shape":"cylinder","radiusTop":0.42,"radiusBottom":0.42,"height":0.32,"pos":[0,0.16,0],"rot":[90,0,0],"color":"#3a3f47","material":"metal"},{"slot":"receiver","shape":"box","size":[0.62,0.7,0.42],"pos":[0,0.55,0],"radius":0.08,"color":"#8a8f98","material":"matte"},{"slot":"header","shape":"plane","size":[0.52,0.32],"pos":[0,0.57,0.23],"face":"camera","svg":"X-HUB-SIGNATUREsha256=..."},{"slot":"packet","shape":"particles","count":28,"box":[0.4,0.24,0.25],"pos":[0,0.95,0.28],"size":0.025,"color":"#e8743b","motion":"drift"}]}},{"term":"Orchestrator","aka":["workflow engine","dispatcher","scheduler","control plane"],"kind":"part","plain":"The service that receives trigger events, starts workflow runs, and records their lifecycle.","does":"Turns an event into a bounded run and exposes enough state for later triggers, humans, and metrics.","ask":"What starts this run, which task does it select, how is duplicate work prevented, and what event comes next?","cues":["A run queue or dispatch panel","Queued, in-progress, and finished run records","Trigger rules and concurrency settings"],"range":"Hosted CI, workflow engine, agent platform, queue-backed service, a small custom controller, or a coordinator agent that directs subagents and writes no code.","trouble":"Events disappear, runs start twice, state goes stale, or a completion event starts an uncontrolled chain.","children":["trigger","run-state","concurrency-limit","work-claim","throttle","try-cap","budget-cap","parallel-runs"],"at":[0,1.8,-0.55],"links":[{"to":"runner","rel":"controls"},{"to":"run","rel":"see also"},{"to":"trigger","rel":"see also"}],"sources":[{"label":"GitHub, Events that trigger workflows","url":"https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows","kind":"maker","page":"About events that trigger workflows","quote":"Workflow triggers are events that cause a workflow to run.","work":"github-actions-events"},{"label":"GitHub, REST API endpoints for workflow runs","url":"https://docs.github.com/en/rest/actions/workflow-runs","kind":"maker","page":"About workflow runs in GitHub Actions","quote":"You can use the REST API to view, re-run, cancel, and view logs for workflow runs in GitHub Actions.","work":"github-actions-workflow-runs"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/07-overnight.md","kind":"practitioner","page":"When the night holds a queue, not a task","quote":"The coordinator authors briefs, collects what its subagents finish, keeps the lowest unmerged PR green, and never writes code itself.","work":"lauren-tan-pstack"}],"id":"orchestrator","look":{"parts":[{"slot":"console","shape":"box","size":[1.85,1.18,0.1],"pos":[0,0.78,0],"radius":0.05,"color":"#1d2a2a","material":"matte"},{"slot":"screen","shape":"plane","size":[1.7,1.02],"pos":[0,0.8,0.06],"material":"screen","svg":"ORCHESTRATORQUEUED 3RUNNING 2DONE 41"},{"slot":"trigger","shape":"plane","size":[0.34,0.26],"pos":[-0.33,0.78,0.1],"face":"camera","svg":""},{"slot":"state","shape":"plane","size":[0.42,0.3],"pos":[0,0.45,0.1],"face":"camera","svg":""},{"slot":"limit","shape":"plane","size":[0.32,0.32],"pos":[-0.27,0.18,0.1],"face":"camera","svg":""},{"slot":"throttle","shape":"plane","size":[0.32,0.32],"pos":[0.32,0.62,0.1],"face":"camera","svg":""},{"slot":"claim","shape":"plane","size":[0.32,0.26],"pos":[0.28,0.12,0.1],"face":"camera","svg":""},{"slot":"cap","shape":"plane","size":[0.32,0.26],"pos":[0.31,0.92,0.1],"face":"camera","svg":"3/3"},{"slot":"budget","shape":"plane","size":[0.32,0.26],"pos":[0.66,0.36,0.1],"svg":"$ cap"}]}},{"term":"Trigger","aka":["event","start condition","hook"],"kind":"part","plain":"The event that starts a run. It carries enough identity and context for the orchestrator to choose a task.","does":"Converts a change in a tracker, git host, clock, monitor, or prior run into one dispatch decision.","ask":"Which event starts this task, which activity types count, and what stops the event from starting a duplicate run?","cues":["An event name and activity type in a run record","A webhook delivery or schedule rule","A manual Run workflow control"],"range":"Ticket created or labelled, status moved, comment or mention, pull request opened, check failed, review submitted, schedule, run finished, monitor alert, or manual dispatch.","trouble":"A broad event fires too often, a self-written event loops, or a filtered event leaves work silently unhandled.","at":[-0.33,0.78,0.06],"links":[{"to":"run-state","rel":"raises"},{"to":"throttle","rel":"see also"},{"to":"webhook","rel":"made by"}],"variants":[{"label":"Trigger kind","items":[{"id":"trigger-ticket-created","name":"Ticket created","plain":"A new ticket enters the tracker and starts intake or triage.","sources":[{"label":"Atlassian, Jira automation triggers","url":"https://support.atlassian.com/cloud-automation/docs/jira-automation-triggers/","kind":"maker","page":"Work item created","quote":"The flow will run when an work item is created.","work":"jira-automation-triggers"}]},{"id":"trigger-ticket-labelled","name":"Ticket labelled","plain":"A label is added to a ticket and selects a task such as implementation.","sources":[{"label":"GitHub, Triggering a workflow","url":"https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow","kind":"maker","page":"Using event activity types","quote":"For example, the following workflow triggers when an issue is opened or labeled.","work":"github-actions-triggering-workflow"}]},{"id":"trigger-status-moved","name":"Status moved","plain":"A ticket moves from one workflow state to another, such as Ready to In progress.","sources":[{"label":"Atlassian, Jira automation triggers","url":"https://support.atlassian.com/cloud-automation/docs/jira-automation-triggers/","kind":"maker","page":"Work item transitioned","quote":"The flow will run when a work item transitions from one status to another.","work":"jira-automation-triggers"}]},{"id":"trigger-comment-mention","name":"Comment or mention","plain":"A new comment or an explicit mention asks a task to respond.","sources":[{"label":"Linear, Getting Started with Agents","url":"https://linear.app/developers/agents","kind":"maker","page":"Overview","quote":"They can be @mentioned, delegated issues through assignment, create and reply to comments, collaborate on projects and documents, etc.","work":"linear-agents"}]},{"id":"trigger-pr-opened","name":"Pull request opened","plain":"A new pull request starts checks or an automated review.","sources":[{"label":"GitHub, Events that trigger workflows","url":"https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows","kind":"maker","page":"pull_request_target","quote":"Runs your workflow when activity on a pull request in the workflow's repository occurs. For example, if no activity types are specified, the workflow runs when a pull request is opened or reopened or when the head branch of the pull request is updated.","work":"github-actions-events"}]},{"id":"trigger-check-failed","name":"Check failed","plain":"A check or commit status reports failure and starts diagnosis or rework.","sources":[{"label":"GitHub, Events that trigger workflows","url":"https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows","kind":"maker","page":"status","quote":"Runs your workflow when the status of a Git commit changes. For example, commits can be marked as error, failure, pending, or success.","work":"github-actions-events"}]},{"id":"trigger-review-submitted","name":"Review submitted","plain":"A reviewer submits an approval, request for changes, or comment on a pull request.","sources":[{"label":"GitHub, Events that trigger workflows","url":"https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows","kind":"maker","page":"pull_request_review","quote":"To run your workflow when a pull request has been approved, you can trigger your workflow with the `submitted` type of `pull_request_review` event, then check the review state with the `github.event.review.state` property.","work":"github-actions-events"}]},{"id":"trigger-schedule","name":"Schedule","plain":"A clock starts a task at a configured time or interval.","sources":[{"label":"GitHub, Events that trigger workflows","url":"https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows","kind":"maker","page":"schedule","quote":"The `schedule` event allows you to trigger a workflow at a scheduled time.","work":"github-actions-events"}]},{"id":"trigger-run-finished","name":"Run finished","plain":"A completed or requested run starts the next task in a chain.","sources":[{"label":"GitHub, Events that trigger workflows","url":"https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows","kind":"maker","page":"workflow_run","quote":"This event occurs when a workflow run is requested or completed. It allows you to execute a workflow based on execution or completion of another workflow.","work":"github-actions-events"}]},{"id":"trigger-monitor-alert","name":"Monitor alert","plain":"A monitored signal crosses a threshold and asks the factory to investigate.","sources":[{"label":"Datadog, Jira integration","url":"https://docs.datadoghq.com/integrations/jira/","kind":"maker","page":"Jira integration","quote":"To create Jira issues from Datadog alert events, enter the notification handle of one or more issue templates.","work":"datadog-jira"}]},{"id":"trigger-manual-dispatch","name":"Manual dispatch","plain":"A person presses a run control or sends a dispatch API call.","sources":[{"label":"GitHub, Events that trigger workflows","url":"https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows","kind":"maker","page":"workflow_dispatch","quote":"To enable a workflow to be triggered manually, you need to configure the `workflow_dispatch` event.","work":"github-actions-events"}]}]}],"sources":[{"label":"GitHub, Events that trigger workflows","url":"https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows","kind":"maker","page":"About events that trigger workflows","quote":"You can configure your workflows to run when specific activity on GitHub happens, at a scheduled time, or when an event outside of GitHub occurs.","work":"github-actions-events"},{"label":"GitHub, Triggering a workflow","url":"https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow","kind":"maker","page":"Using events to trigger workflows","quote":"If multiple triggering events for your workflow occur at the same time, multiple workflow runs will be triggered.","work":"github-actions-triggering-workflow"},{"label":"GitLab, Trigger pipelines with the API","url":"https://docs.gitlab.com/ci/triggers/","kind":"maker","page":"Trigger a pipeline","quote":"You can use an API call to the pipeline triggers API endpoint to trigger a pipeline for a specific branch or tag.","work":"gitlab-trigger-pipelines"}],"id":"trigger","look":{"parts":[{"slot":"event-card","shape":"box","size":[1.1,0.72,0.05],"pos":[0,0.52,0],"radius":0.05,"color":"#e8743b","material":"matte"},{"slot":"event","shape":"plane","size":[1.02,0.64],"pos":[0,0.55,0.035],"face":"camera","svg":"EVENT"},{"slot":"clock","shape":"torus","radius":0.14,"tube":0.025,"pos":[-0.46,1.02,0.08],"rot":[90,0,0],"color":"#e0a526","material":"metal"},{"slot":"dispatch","shape":"cone","radius":0.12,"height":0.26,"pos":[0.48,0.18,0.04],"rot":[0,0,180],"color":"#2e9b4f","material":"glow"}]}},{"term":"Run state","aka":["run status","queued","in progress","completed"],"kind":"part","plain":"The durable state record for one run from dispatch through completion.","does":"Lets the orchestrator show capacity, decide which transitions are legal, and trigger the next task from a known outcome.","ask":"Is this run queued, in progress, or finished, and what conclusion or failure explains the state?","cues":["Queued, in progress, and completed status pills","Start and finish timestamps","A conclusion such as success, failure, cancelled, or timed out"],"range":"Queued, requested, waiting, in progress, completed, cancelled, failure, skipped, stale, success, or timed out.","trouble":"A run stays queued forever, finishes without a conclusion, or a retry looks like a new piece of work.","at":[0,0.45,0.06],"links":[{"to":"try-cap","rel":"see also"},{"to":"trigger","rel":"see also"}],"sources":[{"label":"GitHub, REST API endpoints for workflow runs","url":"https://docs.github.com/en/rest/actions/workflow-runs","kind":"maker","page":"List workflow runs for a repository","quote":"Can be one of: `completed`, `action_required`, `cancelled`, `failure`, `neutral`, `skipped`, `stale`, `success`, `timed_out`, `in_progress`, `queued`, `requested`, `waiting`, `pending`","work":"github-actions-workflow-runs"},{"label":"GitHub, REST API endpoints for workflow runs","url":"https://docs.github.com/en/rest/actions/workflow-runs","kind":"maker","page":"About workflow runs in GitHub Actions","quote":"You can use the REST API to view, re-run, cancel, and view logs for workflow runs.","work":"github-actions-workflow-runs"}],"id":"run-state","look":{"parts":[{"shape":"box","size":[1.22,0.9,0.05],"pos":[0,0.45,0],"radius":0.04,"color":"#1d2a2a","material":"matte"},{"slot":"lanes","shape":"plane","size":[1.1,0.8],"pos":[0,0.48,0.035],"face":"camera","svg":"QUEUEDIN PROGRESSCOMPLETED"},{"slot":"bead","shape":"sphere","radius":0.09,"pos":[0.46,0.49,0.08],"color":"#7fe0c8","material":"glow","motion":"bob"}]}},{"term":"Concurrency limit","aka":["WIP limit","parallelism cap","concurrency group"],"kind":"knob","plain":"A bound on how many runs or jobs may be active together for one key, repository, branch, or resource.","does":"Prevents conflicting runs and keeps the runner pool and review queue within capacity.","ask":"What is the key, how many runs may be active, and should a new run wait, replace, or cancel an older one?","cues":["A concurrency group name","A counter or queue badge","A cancel-in-progress or pending policy"],"range":"One active run per group, a small fixed worker pool, or a larger queue with explicit cancellation rules.","trouble":"Parallel runs edit the same target, stale runs consume capacity, or the cap hides a slow downstream gate.","at":[-0.27,0.18,0.06],"links":[{"to":"work-claim","rel":"see also"},{"to":"throttle","rel":"see also"}],"pattern":"pattern-accelerator-brake","sources":[{"label":"GitHub, Control the concurrency of workflows and jobs","url":"https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/control-workflow-concurrency","kind":"maker","page":"Using concurrency in different scenarios","quote":"You can use `jobs..concurrency` to ensure that only a single job or workflow using the same concurrency group will run at a time.","work":"github-actions-concurrency"},{"label":"Dan Radigan, Working with WIP limits for kanban","url":"https://www.atlassian.com/agile/kanban/wip-limits","kind":"practitioner","page":"What are WIP limits?","quote":"In agile development, work in progress (WIP) limits set the maximum amount of work that can exist in each status of a workflow.","work":"atlassian-wip-limits"}],"id":"concurrency-limit","look":{"parts":[{"slot":"dial","shape":"cylinder","radiusTop":0.38,"radiusBottom":0.38,"height":0.12,"pos":[0,0.12,0],"rot":[90,0,0],"color":"#d9dbe0","material":"matte"},{"slot":"face","shape":"plane","size":[0.62,0.62],"pos":[0,0.15,0.07],"face":"camera","svg":"MAX 2"},{"slot":"queue","shape":"box","size":[0.82,0.16,0.22],"pos":[0,0.62,0],"radius":0.04,"color":"#e0a526","material":"gloss"},{"shape":"particles","count":24,"box":[0.65,0.18,0.24],"pos":[0,0.9,0],"size":0.025,"color":"#3b6ea8","motion":"settle"}]}},{"term":"Work claim","aka":["claim","lock","lease","dedupe","idempotency key"],"kind":"process","plain":"The act of marking a piece of work as owned by one run or one task before side effects begin.","does":"Stops two runs from implementing, reviewing, or reworking the same ticket at the same time.","ask":"What is the claim key, when does it expire, and how can a later run tell whether the work is still owned?","cues":["An assignee, delegate, lock, or claim label on the ticket","A run id or idempotency key in an event record","A concurrency group derived from the work key"],"range":"Concurrency group, claim label or assignee/delegate set before work starts, lease with expiry, or idempotency key.","trouble":"A claim is made after editing, never expires after a crash, or two identifiers refer to one ticket.","at":[0.28,0.12,0.06],"links":[{"to":"concurrency-limit","rel":"see also"},{"to":"ticket","rel":"see also"},{"to":"bot-identity","rel":"see also"}],"pattern":"pattern-stateless-worker","sources":[{"label":"GitHub, Control the concurrency of workflows and jobs","url":"https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/control-workflow-concurrency","kind":"maker","page":"Example: Concurrency groups","quote":"Concurrency groups provide a way to manage and limit the execution of workflow runs or jobs that share the same concurrency key.","work":"github-actions-concurrency"},{"label":"Linear, Getting Started with Agents","url":"https://linear.app/developers/agents","kind":"maker","page":"Mention + assign scopes","quote":"Assigning an issue to your app now sets it as the delegate, not the assignee","work":"linear-agents"},{"label":"GitHub, Managing labels","url":"https://docs.github.com/en/issues/using-labels-and-milestones-to-track-work/managing-labels","kind":"maker","page":"About labels","quote":"You can use these default labels to help create a standard workflow in a repository.","work":"github-labels"},{"label":"GitHub, Best practices for using webhooks","url":"https://docs.github.com/en/webhooks/using-webhooks/best-practices-for-using-webhooks","kind":"maker","page":"Use the X-GitHub-Delivery header","quote":"You can use the X-GitHub-Delivery header to ensure that each delivery is unique per event.","work":"github-webhooks-best-practices"}],"id":"work-claim","look":{"parts":[{"shape":"box","size":[0.9,0.95,0.06],"pos":[0,0.48,0],"radius":0.05,"color":"#d8cfbf","material":"matte"},{"slot":"clipboard","shape":"plane","size":[0.78,0.82],"pos":[0,0.5,0.04],"face":"camera","svg":""},{"slot":"lock","shape":"torus","radius":0.18,"tube":0.04,"pos":[0.5,0.28,0.08],"rot":[90,0,0],"color":"#3b6ea8","material":"metal"},{"shape":"box","size":[0.22,0.16,0.12],"pos":[0.5,0.16,0.08],"radius":0.03,"color":"#3b6ea8","material":"metal"}]}},{"term":"Throttle","aka":["rate limit","sampling","trigger budget"],"kind":"knob","plain":"A limit on how often an automated trigger may create work over time or across a stream of events.","does":"Keeps noisy sources and fast loops below the review, API, runner, and budget capacity of the factory.","ask":"Is the limit per source, repository, task, hour, or event sample, and what happens when the budget is spent?","cues":["A per-hour or per-minute counter","A sampling percentage or cooldown window","A disabled, delayed, or queued trigger"],"range":"Every event, sampled events, one run per interval, an open-PR cap, or a fixed API request budget.","trouble":"The trigger floods the queue, a critical event is delayed behind noise, or retries multiply the rate.","at":[0.32,0.62,0.06],"links":[{"to":"trigger","rel":"controls"},{"to":"concurrency-limit","rel":"see also"},{"to":"review-queue","rel":"see also"}],"pattern":"pattern-accelerator-brake","sources":[{"label":"Renovate, Configuration Options","url":"https://docs.renovatebot.com/configuration-options/","kind":"maker","page":"prHourlyLimit","quote":"Rate limit PRs to maximum x created per hour. 0 means no limit.","work":"renovate-configuration-options"},{"label":"GitHub, Dependabot options reference","url":"https://docs.github.com/en/code-security/reference/supply-chain-security/dependabot-options-reference","kind":"maker","page":"open-pull-requests-limit","quote":"When either limit is reached, Dependabot reports an error and stops opening additional pull requests until some of the open pull requests are merged or closed.","work":"github-dependabot-options"},{"label":"GitHub, Rate limits for the REST API","url":"https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api","kind":"maker","page":"Primary rate limit for GitHub App installations","quote":"GitHub Apps authenticating with an installation access token use the installation's minimum rate limit of 5,000 requests per hour.","work":"github-rest-rate-limits"},{"label":"Betsy Beyer et al., Site Reliability Engineering","url":"https://sre.google/sre-book/embracing-risk/","kind":"book","isbn":"9781491929124","page":"Chapter 3, Embracing Risk","quote":"Many products use this control loop to manage release velocity: as long as the system’s SLOs are met, releases can continue. If SLO violations occur frequently enough to expend the error budget, releases are temporarily halted","work":"site-reliability-engineering"},{"label":"Gene Kim et al., The Phoenix Project","url":"https://reader.itrevolution.com/api/pub/phoenix-project-extra-excerpt/pdf/file","kind":"book","isbn":"9781950508945","page":"Chapter 7, p. 89 (5th Anniversary Edition excerpt, ISBN 9781942788294)","quote":"Therefore, one of the most critical mechanisms in the management of any plant is job and materials release","work":"phoenix-project-gene-kim"},{"label":"Gene Kim et al., The Phoenix Project","url":"https://reader.itrevolution.com/api/pub/phoenix-project-extra-excerpt/pdf/file","kind":"book","isbn":"9781950508945","page":"Chapter 7, p. 89 (5th Anniversary Edition excerpt, ISBN 9781942788294)","quote":"they all agree on one thing: WIP is the silent killer","work":"phoenix-project-gene-kim"}],"id":"throttle","look":{"parts":[{"slot":"gauge","shape":"cylinder","radiusTop":0.36,"radiusBottom":0.36,"height":0.12,"pos":[0,0.12,0],"rot":[90,0,0],"color":"#d9dbe0","material":"matte"},{"slot":"face","shape":"plane","size":[0.62,0.62],"pos":[0,0.15,0.07],"face":"camera","svg":"2 / HOUR"},{"slot":"stream","shape":"particles","count":48,"box":[1.1,0.26,0.2],"pos":[0,0.82,0],"size":0.025,"color":"#e8743b","motion":"drift"}]}},{"term":"Try cap","aka":["retry budget","max attempts","max turns","loop stop"],"kind":"knob","plain":"The maximum number of attempts a task may make in one automated loop before it stops and asks for human attention.","does":"Converts repeated failure into a visible handoff instead of an endless chain of runs.","ask":"What counts as an attempt, what evidence is carried forward, and where does a person re-enter after the cap?","cues":["An attempt counter on a run or ticket","A retry button with a remaining budget","A stopped loop and a human-review label"],"range":"A few attempts for a code or review loop, with a separate policy for transient infrastructure failures.","trouble":"Each failure starts another run forever, retries hide a deterministic defect, or the cap is reached with no useful diagnosis.","at":[0.31,0.92,0.06],"links":[{"to":"review-rework-loop","rel":"see also"},{"to":"run-state","rel":"see also"}],"pattern":"pattern-accelerator-brake","sources":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents","quote":"It’s also common to include stopping conditions (such as a maximum number of iterations) to maintain control.","work":"anthropic-building-effective-agents"},{"label":"OpenAI, Agents SDK running agents","url":"https://openai.github.io/openai-agents-python/running_agents/","kind":"maker","page":"Running agents","quote":"If we exceed the `max_turns` passed, we raise a `MaxTurnsExceeded` exception.","work":"openai-agents-sdk-running-agents"},{"label":"Matt Pocock, 11 Tips For AI Coding With Ralph Wiggum","url":"https://www.aihero.dev/tips-for-ai-coding-with-ralph-wiggum","kind":"practitioner","page":"2. Start With HITL, Then Go AFK","quote":"For AFK Ralph, always cap your iterations. Infinite loops are dangerous with stochastic systems.","work":"matt-pocock-ralph-tips"}],"id":"try-cap","look":{"parts":[{"shape":"box","size":[0.9,0.82,0.08],"pos":[0,0.42,0],"radius":0.08,"color":"#c8432f","material":"matte"},{"slot":"counter","shape":"plane","size":[0.74,0.66],"pos":[0,0.45,0.05],"face":"camera","svg":"ATTEMPTS3 / 3"},{"slot":"stop","shape":"box","size":[0.62,0.12,0.12],"pos":[0,0.02,0.08],"radius":0.04,"color":"#3a3f47","material":"gloss"}]}},{"term":"Budget cap","aka":["spend limit","token budget","cost cap"],"kind":"knob","plain":"A hard limit on model spend for an organization or a project. When spend reaches it, the provider refuses new requests until someone raises or removes the limit or the next monthly cycle resets it, so a runaway loop stops instead of running up a bill. Set it low while you learn what a run costs, and raise it with evidence.","does":"Puts a ceiling on what a flood of triggers or a looping run can cost.","ask":"What happens to runs in flight when the cap is hit, and who may raise it?","cues":["A hard spend limit in the model provider's project settings","A cost figure for each run on the run page","Runs that fail with a spend-limit error"],"range":"From a small project cap while learning to a large daily budget. StrongDM argues that a mature factory should spend at least $1,000 of tokens per engineer each day.","trouble":"The cap is hit mid-run and leaves half-finished work, or there is no cap and a loop runs all night.","at":[0.66,0.36,0.06],"links":[{"to":"token-cost","rel":"measured by"}],"sources":[{"label":"OpenAI, Spend limits","url":"https://developers.openai.com/api/docs/guides/spend-limits","kind":"maker","page":"Spend limits","quote":"To stop traffic after tracked spend reaches a configured amount, enforce a hard spend limit for your organization or an individual project.","work":"openai-spend-limits"},{"label":"OpenAI, Spend limits","url":"https://developers.openai.com/api/docs/guides/spend-limits","kind":"maker","page":"Spend limits","quote":"Hard spend limits can interrupt production traffic.","work":"openai-spend-limits"},{"label":"OpenAI, Spend limits","url":"https://developers.openai.com/api/docs/guides/spend-limits","kind":"maker","page":"Spend limits","quote":"Reaching a project hard limit returns a 429 error … Raising or removing the reached limit allows traffic to resume after the update propagates. Otherwise, the limit resets with the next monthly cycle.","work":"openai-spend-limits"},{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/","kind":"maker","page":"Software Factories And The Agentic Moment","quote":"If you haven’t spent at least $1,000 on tokens today per human engineer, your software factory has room for improvement","work":"strongdm-software-factory"},{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Watch the outcomes","quote":"Know your metrics and unit economics. \"It works\" isn't the bar. Does it work at a cost that makes sense?","work":"kent-c-dodds-you-own-the-outcome"}],"id":"budget-cap","look":{"parts":[{"slot":"meter","shape":"box","size":[1.2,0.5,0.06],"pos":[0,0.45,0],"color":"#2a2a2e","material":"matte","radius":0.04},{"slot":"meter-face","shape":"plane","size":[1.14,0.44],"pos":[0,0.45,0.031],"material":"screen","svg":"SPEND LIMIT$352 todayhard cap $500"}]}},{"term":"Parallel runs","aka":["parallel agents","background agents","fan-out","coordinator and subagents"],"kind":"concept","plain":"Several runs work at once, each in its own worktree or runner, so they do not fight over files, ports or build output. A coordinator run can split a job, hand the parts to subagents and write no code itself. Each subagent works in its own context window and reports back one result. Merges still happen one at a time. Each extra run adds a pull request to the review queue, so parallel runs pay off only once verification carries most of the review.","does":"Multiplies how much work the factory attempts at once, while each attempt stays isolated.","ask":"Does each run get its own worktree or runner, and who will review everything they produce?","cues":["Several runs in progress at once on the run page","One worktree and branch per run","A coordinator run that hands out briefs and opens no pull request of its own"],"at":[-0.05,0.95,0.06],"links":[{"to":"review-queue","rel":"raises"},{"to":"concurrency-limit","rel":"controls"}],"pattern":"pattern-kitchen-brigade","sources":[{"label":"Anthropic, Run parallel sessions with worktrees","url":"https://code.claude.com/docs/en/worktrees","kind":"maker","page":"Run parallel sessions with worktrees","quote":"A git worktree is a separate working directory with its own files and branch","work":"claude-code-worktrees"},{"label":"Anthropic, Run parallel sessions with worktrees","url":"https://code.claude.com/docs/en/worktrees","kind":"maker","page":"Run parallel sessions with worktrees","quote":"Subagents can run in their own worktrees so parallel edits don't conflict.","work":"claude-code-worktrees"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/02-poteto-mode.md","kind":"practitioner","page":"Give parallel work its own machine","quote":"If you run several agents against one repository on one computer, they will fight over the working tree, the ports, and the build output.","work":"lauren-tan-pstack"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/08-principles.md","kind":"practitioner","page":"Steer with principle names","quote":"separate before serializing shared state. give each attempt its own worktree, no locks.","work":"lauren-tan-pstack"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/07-overnight.md","kind":"practitioner","page":"Run many projects in parallel","quote":"The coordinator doesn’t write code. It directs subagents, which run in the cloud by default","work":"lauren-tan-pstack"},{"label":"Lauren Tan, Noodle","url":"https://raw.githubusercontent.com/poteto/noodle/82d2921c52370f23f29086de81ccfb600939c037/docs/concepts/scheduling.md","kind":"practitioner","page":"docs/concepts/scheduling.md","quote":"Multiple agents run in parallel, each in its own git worktree.","work":"lauren-tan-noodle"},{"label":"Lauren Tan, Noodle","url":"https://raw.githubusercontent.com/poteto/noodle/82d2921c52370f23f29086de81ccfb600939c037/docs/concepts/scheduling.md","kind":"practitioner","page":"docs/concepts/scheduling.md","quote":"Merges are serialized, one at a time, to keep the main branch clean.","work":"lauren-tan-noodle"},{"label":"Lauren Tan, Noodle","url":"https://raw.githubusercontent.com/poteto/noodle/82d2921c52370f23f29086de81ccfb600939c037/INSTALL.md","kind":"practitioner","page":"INSTALL.md","quote":"a head chef (the scheduler) reads the backlog, writes orders, and dispatches line cooks (agents) to do the work in isolated worktrees.","work":"lauren-tan-noodle"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/subagent","kind":"practitioner","page":"Subagent","quote":"Runs in its own session with its own context window, and reports a single tool result back.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/subagent","kind":"practitioner","page":"Subagent","quote":"a parent can fan several out at once over independent pieces of work.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Anthropic, How we built our multi-agent research system","url":"https://www.anthropic.com/research/multi-agent-research-system","kind":"maker","page":"How we built our multi-agent research system","quote":"Our Research system uses a multi-agent architecture with an orchestrator-worker pattern, where a lead agent coordinates the process while delegating to specialized subagents that operate in parallel.","work":"anthropic-multi-agent-research"},{"label":"Mohannad Hammadeh et al. (AWS), Scaling agentic coding past the human bottleneck","url":"https://builder.aws.com/content/3Jzr7Zmo4OxkGxLZ83VofzQqT0W/scaling-agentic-coding-past-the-human-bottleneck","kind":"maker","page":"Scaling agentic coding past the human bottleneck","quote":"each agent you add makes one review queue longer.","work":"aws-scaling-agentic-coding"}],"id":"parallel-runs","look":{"parts":[{"slot":"coordinator","shape":"box","size":[0.9,0.3,0.04],"pos":[0,1.15,0],"color":"#1d2a2a","material":"matte","radius":0.03},{"slot":"coordinator-screen","shape":"plane","size":[0.86,0.26],"pos":[0,1.15,0.021],"material":"screen","svg":"COORDINATOR3 briefs out, writes no code"},{"slot":"handoffs","shape":"plane","size":[1.4,0.5],"pos":[0,0.75,0],"svg":""},{"slot":"run-a","shape":"box","size":[0.42,0.5,0.4],"pos":[-0.55,0.25,0],"color":"#3b6ea8","material":"gloss","radius":0.03},{"slot":"run-b","shape":"box","size":[0.42,0.5,0.4],"pos":[0,0.25,0],"color":"#3b6ea8","material":"gloss","radius":0.03},{"slot":"run-c","shape":"box","size":[0.42,0.5,0.4],"pos":[0.55,0.25,0],"color":"#3b6ea8","material":"gloss","radius":0.03},{"slot":"worktrees","shape":"plane","size":[1.6,0.16],"pos":[0,0.3,0.201],"svg":"wt-awt-bwt-c"}]}},{"term":"Runner","aka":["worker","execution environment","build machine","server","box","sandbox"],"kind":"part","plain":"The disposable machine or container that executes one run. It supplies the filesystem, tools, network path, and credentials that the task can reach.","does":"Turns a queued run into an isolated working copy and returns its files, status, and evidence.","ask":"What can this run reach, and what survives when it ends?","cues":["The runner image, machine size, and status appear on the run page","A fresh workspace path and a run timer","Runner labels or a sandbox provider on the run page"],"range":"One runner per run is the safe default. Choose a CI worker, a sandbox API, a bundled agent platform, a worker you operate, your own container cluster, or a local worktree for a few runs on one machine.","trouble":"Two runs share files or credentials, a worker stays alive after failure, or a task depends on state from an earlier run.","children":["disposable-environment","run-timeout","cache","network-egress","secrets","security-boundary"],"at":[0,0.6,0.3],"variants":[{"label":"Runner type","items":[{"id":"ci-runner","name":"CI runner","plain":"A hosted CI service runs each job on a fresh instance of a runner image.","why":"The CI provider hosts the worker and starts a new one for every job.","sources":[{"label":"GitHub, Choosing the runner for a job","url":"https://docs.github.com/en/actions/how-tos/write-workflows/choose-where-workflows-run/choose-the-runner-for-a-job","kind":"maker","page":"Choosing GitHub-hosted runners","quote":"If you use a GitHub-hosted runner, each job runs in a fresh instance of a runner image specified by `runs-on`.","work":"github-actions-runner"}]},{"id":"microvm-sandbox-api","name":"MicroVM sandbox API","plain":"An API creates a strong isolation boundary for code, then destroys or pauses the sandbox by policy.","why":"The service exposes a sandbox boundary as an API call.","sources":[{"label":"E2B, Security and compliance","url":"https://e2b.dev/security","kind":"maker","page":"Isolation and tenancy","quote":"Every E2B sandbox runs in its own Firecracker microVM with its own kernel. Isolation is at the hypervisor boundary, not the container or process boundary","work":"e2b-security"}]},{"id":"bundled-agent-platform","name":"Bundled agent platform","plain":"The agent service provisions a Linux workspace, runs the task, and returns its artifacts through one product surface.","why":"Agent execution and environment provisioning arrive as one managed service.","sources":[{"label":"OpenAI, OpenAI-hosted sandboxes","url":"https://developers.openai.com/api/docs/guides/agents-api/environments/openai-hosted","kind":"maker","page":"OpenAI-hosted sandboxes","quote":"An OpenAI-hosted sandbox gives your agent a Linux workspace with Python, Node.js, and command-line tools. OpenAI provisions and connects it; your application supplies the task and retrieves the results.","work":"openai-hosted-sandboxes"}]},{"id":"bring-your-own-worker","name":"Bring-your-own worker","plain":"You register a machine you operate as a self-hosted runner and choose its labels, image, and cleanup.","why":"The operator supplies the host while the orchestrator still assigns jobs.","sources":[{"label":"GitHub, Self-hosted runners reference","url":"https://docs.github.com/en/actions/reference/runners/self-hosted-runners","kind":"maker","page":"Requirements for self-hosted runner machines","quote":"You can use a machine as a self-hosted runner as long as it meets these requirements","work":"github-self-hosted-runners"}]},{"id":"own-vps-containers","name":"Own container cluster","plain":"You run ephemeral runners on a container cluster you operate. A controller such as Actions Runner Controller creates a runner for each job and cleans it up afterwards.","why":"The operator owns the cluster that hosts the runners, and automation gives each job a clean environment.","sources":[{"label":"GitHub, Self-hosted runners reference","url":"https://docs.github.com/en/actions/reference/runners/self-hosted-runners","kind":"maker","page":"Actions Runner Controller","quote":"ARC handles the full lifecycle of runners within your cluster, from provisioning to job execution to cleanup.","work":"github-self-hosted-runners"},{"label":"GitHub, Self-hosted runners reference","url":"https://docs.github.com/en/actions/reference/runners/self-hosted-runners","kind":"maker","page":"Ephemeral runners for autoscaling","quote":"This approach allows you to manage your runners as ephemeral systems, since you can use automation to provide a clean environment for each job.","work":"github-self-hosted-runners"}]},{"id":"local-worktree","name":"Local worktree","plain":"The run works in its own git worktree on a developer's machine, with its own files and branch. Files are isolated, while the machine, its ports and its credentials stay shared, so a laptop holds only a few runs at once.","why":"The worktree isolates the working copy, and nothing else.","sources":[{"label":"Anthropic, Run parallel sessions with worktrees","url":"https://code.claude.com/docs/en/worktrees","kind":"maker","page":"Run parallel sessions with worktrees","quote":"A git worktree is a separate working directory with its own files and branch","work":"claude-code-worktrees"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/02-poteto-mode.md","kind":"practitioner","page":"Give parallel work its own machine","quote":"Worktrees cost disk and machine resources, so a laptop runs only a handful at once.","work":"lauren-tan-pstack"}]}]}],"sources":[{"label":"GitHub, Choosing the runner for a job","url":"https://docs.github.com/en/actions/how-tos/write-workflows/choose-where-workflows-run/choose-the-runner-for-a-job","kind":"maker","page":"Choosing GitHub-hosted runners","quote":"If you use a GitHub-hosted runner, each job runs in a fresh instance of a runner image specified by `runs-on`.","work":"github-actions-runner"},{"label":"GitHub, Self-hosted runners reference","url":"https://docs.github.com/en/actions/reference/runners/self-hosted-runners","kind":"maker","page":"Ephemeral runners for autoscaling","quote":"With ephemeral runners, this can be guaranteed because GitHub only assigns one job to a runner.","work":"github-self-hosted-runners"},{"label":"E2B, Security and compliance","url":"https://e2b.dev/security","kind":"maker","page":"Isolation and tenancy","quote":"Sandboxes are destroyed on timeout or shutdown. Filesystem and memory state can be preserved across a pause and restored on resume.","work":"e2b-security"},{"label":"Daytona, Isolation","url":"https://www.daytona.io/docs/en/isolation/","kind":"maker","page":"Isolation","quote":"Daytona sandboxes are isolated by default. Code running in a sandbox cannot read another sandbox’s filesystem or memory, is not on a shared network with other sandboxes, and its credentials and API access are scoped to its own organization.","work":"daytona-isolation"},{"label":"Modal, Sandboxes","url":"https://modal.com/docs/guide/sandboxes","kind":"maker","page":"Sandboxes","quote":"Sandboxes have a default maximum lifetime of 5 minutes. You can change this by passing a `timeout` of up to 24 hours to the `Sandbox.create(...)` function.","work":"modal-sandboxes"},{"label":"Vercel, Vercel Sandbox","url":"https://vercel.com/docs/sandbox","kind":"maker","page":"Run agent-generated code in isolation","quote":"Run untrusted or agent-generated code in isolated Linux microVMs. Use Vercel Sandbox for agent workflows, debugging, and one-off commands.","work":"vercel-sandbox"},{"label":"Matt Pocock, 11 Tips For AI Coding With Ralph Wiggum","url":"https://www.aihero.dev/tips-for-ai-coding-with-ralph-wiggum","kind":"practitioner","page":"9. Use Docker Sandboxes","quote":"This runs Claude Code inside a container. Your current directory is mounted, but nothing else.","work":"matt-pocock-ralph-tips"}],"id":"runner","look":{"parts":[{"slot":"shell","shape":"box","size":[2.2,1.4,1.3],"pos":[0,0.7,0],"radius":0.08,"color":"#3b6ea8","material":"matte"},{"slot":"screen","shape":"plane","size":[1.45,0.75],"pos":[0,0.85,0.66],"svg":"RUNNER / RUN-184clean image · 4 vCPU · 16 GBIN PROGRESSworkspace: /work/run-184"},{"slot":"cache","shape":"box","size":[0.62,0.18,0.22],"pos":[-0.67,0.28,0.69],"radius":0.03,"color":"#e0a526","material":"metal"},{"slot":"timeout","shape":"cylinder","radiusTop":0.16,"radiusBottom":0.16,"height":0.08,"pos":[0.7,0.28,0.69],"rot":[90,0,0],"color":"#f3efe4","material":"metal"},{"slot":"egress","shape":"box","size":[0.28,0.52,0.08],"pos":[0.94,0.9,0.68],"radius":0.03,"color":"#2e9b4f","material":"gloss"},{"slot":"secret","shape":"box","size":[0.28,0.52,0.06],"pos":[-0.94,0.9,0.7],"radius":0.02,"color":"#f3efe4","material":"gloss"},{"slot":"sandbox","shape":"box","size":[2.6,1.75,1.7],"pos":[0,0.875,0],"color":"#9cc4f0","material":"glass","opacity":0.15}]}},{"term":"Disposable environment","aka":["ephemeral runner","clean workspace","one-shot worker"],"kind":"concept","plain":"A fresh environment is created for one run and then wiped or discarded. The run carries its durable state through the ticket tracker, git host, and orchestrator.","does":"Limits cross-run contamination and gives each task a repeatable starting point.","ask":"What evidence shows that this workspace began clean and will be removed?","cues":["A new runner identifier on every run","A workspace creation and teardown event","No previous run files in the checkout"],"range":"One job per ephemeral runner is the usual boundary. A cache or artifact store may outlive the workspace by explicit policy.","trouble":"Credentials, untracked files, or processes from an earlier run appear in a later run.","at":[0,1.2,0.6],"pattern":"pattern-stateless-worker","sources":[{"label":"GitHub, Choosing the runner for a job","url":"https://docs.github.com/en/actions/how-tos/write-workflows/choose-where-workflows-run/choose-the-runner-for-a-job","kind":"maker","page":"Choosing GitHub-hosted runners","quote":"If you use a GitHub-hosted runner, each job runs in a fresh instance of a runner image specified by `runs-on`.","work":"github-actions-runner"},{"label":"GitHub, Self-hosted runners reference","url":"https://docs.github.com/en/actions/reference/runners/self-hosted-runners","kind":"maker","page":"Ephemeral runners for autoscaling","quote":"This approach allows you to manage your runners as ephemeral systems, since you can use automation to provide a clean environment for each job.","work":"github-self-hosted-runners"},{"label":"E2B, Security and compliance","url":"https://e2b.dev/security","kind":"maker","page":"Isolation and tenancy","quote":"Sandboxes are destroyed on timeout or shutdown. Filesystem and memory state can be preserved across a pause and restored on resume.","work":"e2b-security"}],"id":"disposable-environment","feel":{"shape":0.72,"energy":0.35,"weight":0.2,"warmth":0.18,"grain":0.12,"motion":"settle","why":"A clean, sharp shell appears for one task, then settles into the external record and disappears."}},{"term":"Run timeout","aka":["execution deadline","wall-clock limit","time budget"],"kind":"knob","plain":"The maximum time a run or one of its steps may execute before the runner cancels it. The limit covers hangs as well as useful work. A timeout only stops a run. To show the work is done, the run still needs a check that can pass or fail.","does":"Bounds compute cost and gives the orchestrator a clear failure signal for a stuck task.","ask":"What should happen when this run reaches its deadline?","cues":["A timeout-minutes field beside the job settings","A countdown or elapsed-time badge on the run page","A failed or cancelled result marked DeadlineExceeded or timeout"],"range":"Set a positive integer per job and step. GitHub Actions defaults a job to 360 minutes and supports up to 360 minutes for a step. Kubernetes uses activeDeadlineSeconds for a Job.","trouble":"A run burns the whole budget on retries, leaves child processes behind, or times out before it can save its result.","at":[0.75,0.25,0.65],"links":[{"to":"disposable-environment","rel":"controls"}],"sources":[{"label":"GitHub, Workflow syntax for GitHub Actions","url":"https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax","kind":"maker","page":"jobs..timeout-minutes","quote":"The maximum number of minutes to let a job run before GitHub automatically cancels it. Default: 360","work":"github-actions-workflow-syntax"},{"label":"GitHub, Workflow syntax for GitHub Actions","url":"https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax","kind":"maker","page":"jobs..steps[*].timeout-minutes","quote":"The maximum number of minutes to run the step before killing the process. Maximum: 360 for both GitHub-hosted and self-hosted runners.","work":"github-actions-workflow-syntax"},{"label":"Kubernetes, Jobs","url":"https://kubernetes.io/docs/concepts/workloads/controllers/job/","kind":"maker","page":"Active deadline","quote":"The `activeDeadlineSeconds` applies to the duration of the job, no matter how many Pods are created.","work":"kubernetes-jobs"},{"label":"Kubernetes, Jobs","url":"https://kubernetes.io/docs/concepts/workloads/controllers/job/","kind":"maker","page":"Active deadline","quote":"Once a Job reaches `activeDeadlineSeconds`, all of its running Pods are terminated and the Job status will become `type: Failed` with `reason: DeadlineExceeded`.","work":"kubernetes-jobs"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/07-overnight.md","kind":"practitioner","page":"Let loops start themselves","quote":"a duration is not a finish condition. \"work on this for 4 hours\" gives the agent nothing to check","work":"lauren-tan-pstack"}],"id":"run-timeout","look":{"parts":[{"slot":"dial","shape":"cylinder","radiusTop":0.42,"radiusBottom":0.42,"height":0.12,"pos":[0,0.42,0],"rot":[90,0,0],"color":"#3b6ea8","material":"metal"},{"slot":"needle","shape":"box","size":[0.06,0.32,0.06],"pos":[0.14,0.42,0.08],"rot":[0,0,-45],"color":"#e0a526","material":"gloss"},{"slot":"readout","shape":"plane","size":[1.2,0.42],"pos":[0,0.95,0.1],"svg":"TIMEOUT30 min"}]}},{"term":"Cache","aka":["dependency cache","reusable build cache"],"kind":"part","plain":"An optional store for dependencies or intermediate files that a run can regenerate. It speeds later runs without becoming their source of truth.","does":"Cuts repeated downloads and setup time while leaving a clean run able to recover from a miss.","ask":"Can this run recreate the files when the cache is empty or poisoned?","cues":["A cache hit or miss line in the run log","A key based on a lockfile or runner image","A cache retention and eviction setting"],"range":"Cache dependencies and intermediate outputs. Keep secrets and sensitive data out.","trouble":"A stale or poisoned cache changes the result, a cache miss breaks the task, or cache growth evicts useful entries.","at":[-0.7,0.35,0.65],"links":[{"to":"disposable-environment","rel":"see also"}],"sources":[{"label":"GitHub, Dependency caching","url":"https://docs.github.com/en/actions/concepts/workflows-and-actions/dependency-caching","kind":"maker","page":"Artifacts versus dependency caching","quote":"Use caching when you want to reuse files that don't change often between workflow runs, such as dependencies downloaded by a package management system, intermediate build outputs, or other files that are expensive to regenerate.","work":"github-actions-cache"},{"label":"GitHub, Dependency caching","url":"https://docs.github.com/en/actions/concepts/workflows-and-actions/dependency-caching","kind":"maker","page":"Artifacts versus dependency caching","quote":"a job should always be able to re-download or regenerate these files if a cache isn't available.","work":"github-actions-cache"},{"label":"GitHub, Dependency caching","url":"https://docs.github.com/en/actions/concepts/workflows-and-actions/dependency-caching","kind":"maker","page":"Cache security","quote":"Any run that can read a cache restores its contents as-is, so you should treat restored files as untrusted input and never store secrets or other sensitive data in a cache.","work":"github-actions-cache"}],"id":"cache","look":{"parts":[{"slot":"drawer","shape":"box","size":[1.45,0.52,0.72],"pos":[0,0.42,0],"radius":0.05,"color":"#e0a526","material":"metal"},{"slot":"label","shape":"plane","size":[1.05,0.32],"pos":[0,0.44,0.37],"svg":"CACHEnpm-lock · hit"},{"slot":"files","shape":"particles","count":34,"box":[0.95,0.18,0.4],"pos":[0,0.73,0],"size":0.018,"color":"#f3efe4","motion":"settle"}]}},{"term":"Network egress","aka":["outbound access","network allowlist","exit policy"],"kind":"knob","plain":"The destinations and protocols a runner may reach outside its boundary. A policy can block all traffic, allow a short list, or route requests through a controlled proxy.","does":"Reduces the paths a compromised task can use to fetch payloads or send data away.","ask":"Which exact hosts must this task reach, and what happens to every other destination?","cues":["An allowlist of hostnames in the runner or proxy settings","A proxy or firewall decision in the run log","A blocked outbound request with its host named"],"range":"Prefer deny-all or a narrow domain or CIDR allowlist. Add package registries and source hosts only when the task needs them.","trouble":"The task cannot fetch a required dependency, reaches an unexpected host, or uses a broad route to exfiltrate data.","at":[0.95,0.85,0.65],"links":[{"to":"prompt-injection","rel":"see also"},{"to":"secrets","rel":"see also"}],"sources":[{"label":"Anthropic, Configure the sandboxed Bash tool","url":"https://code.claude.com/docs/en/sandboxing","kind":"maker","page":"What the sandbox restricts","quote":"Network | No direct route out. Connections go through a proxy on your machine that checks each host against your allowed domains, which start empty.","work":"claude-code-sandboxing"},{"label":"OpenAI, OpenAI-hosted sandboxes","url":"https://developers.openai.com/api/docs/guides/agents-api/environments/openai-hosted","kind":"maker","page":"Control network access","quote":"restricted | Allow only the hosts listed in `allowed_domains`.","work":"openai-hosted-sandboxes"},{"label":"Daytona, Isolation","url":"https://www.daytona.io/docs/en/isolation/","kind":"maker","page":"Network isolation","quote":"each sandbox can be locked down further with one of three mutually exclusive settings: block all traffic, allow specific CIDR ranges, or allow specific domains.","work":"daytona-isolation"}],"id":"network-egress","look":{"parts":[{"slot":"firewall","shape":"box","size":[1.55,1,0.18],"pos":[0,0.62,0],"radius":0.06,"color":"#8a8f98","material":"metal"},{"slot":"allowlist","shape":"plane","size":[1.1,0.56],"pos":[0,0.72,0.11],"svg":"OUTBOUNDapi.example.comother hosts"},{"slot":"port","shape":"cylinder","radiusTop":0.14,"radiusBottom":0.14,"height":0.2,"pos":[0.73,0.62,0.14],"rot":[0,90,0],"color":"#2e9b4f","material":"gloss"}]}},{"term":"Secrets","aka":["scoped credentials","runtime tokens","protected variables"],"kind":"part","plain":"Credentials made available to a run through a controlled scope. The runner receives only the values and permissions the task needs.","does":"Lets a run authenticate to required services while reducing the value of a stolen workspace or log.","ask":"Which credential reaches this run, for how long, and can its value or authority leave the boundary?","cues":["A named secret reference in the run configuration","A masked value in logs and a permissions scope","An OIDC or short-lived token exchange"],"range":"Use repository, environment, or organization scopes. Prefer short-lived tokens and OIDC where the target supports it. A proxy can add the value to outgoing requests, so the agent uses the credential without seeing it.","trouble":"A secret is available to an untrusted change, appears in output, persists on disk, or grants more API access than the task needs.","at":[-0.95,0.85,0.65],"links":[{"to":"prompt-injection","rel":"see also"},{"to":"network-egress","rel":"see also"}],"sources":[{"label":"GitHub, Using secrets in GitHub Actions","url":"https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/use-secrets","kind":"maker","page":"Using secrets in a workflow","quote":"With the exception of `GITHUB_TOKEN`, secrets are not passed to the runner when a workflow is triggered from a forked repository.","work":"github-actions-secrets"},{"label":"GitHub, Using secrets in GitHub Actions","url":"https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/use-secrets","kind":"maker","page":"Using secrets in a workflow","quote":"To provide an action with a secret as an input or environment variable, you can use the `secrets` context to access secrets you've created in your repository.","work":"github-actions-secrets"},{"label":"OpenAI, OpenAI-hosted sandboxes","url":"https://developers.openai.com/api/docs/guides/agents-api/environments/openai-hosted","kind":"maker","page":"Set environment variables","quote":"For secrets, use vault credentials to keep the real values outside the sandbox.","work":"openai-hosted-sandboxes"},{"label":"Kody, Documentation","url":"https://kody.codes/docs/credentials","kind":"maker","page":"Credentials","quote":"Code names a credential. Kody puts the value into the outgoing request at the network boundary, only for the hosts that credential allows, and only over HTTPS. Agents, packages, logs, and tool output never see it.","work":"kody-docs"},{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Design the playground","quote":"Agents should be able to use credentials without ever seeing them. This is non-negotiable once agents are acting on your behalf.","work":"kent-c-dodds-you-own-the-outcome"}],"id":"secrets","look":{"parts":[{"slot":"card","shape":"box","size":[1.5,0.88,0.07],"pos":[0,0.58,0],"radius":0.06,"color":"#f3efe4","material":"gloss"},{"slot":"stripe","shape":"box","size":[1.5,0.13,0.08],"pos":[0,0.78,0.05],"color":"#e0a526","material":"matte"},{"slot":"lock","shape":"torus","radius":0.2,"tube":0.05,"pos":[0,0.42,0.08],"rot":[90,0,0],"color":"#8a8f98","material":"metal"},{"slot":"label","shape":"plane","size":[0.78,0.2],"pos":[0,0.61,0.08],"svg":"OIDC / 15m"}]}},{"term":"Security boundary","aka":["trust boundary","agent containment","blast-radius control"],"kind":"part","plain":"The set of gates around a run that separate trusted factory services from task input, tools, data, and side effects. It combines isolation, identity, least privilege, and outbound controls.","does":"Limits what hostile text or a compromised task can read, change, or transmit.","ask":"If the task follows an attacker instruction, which layer stops the next harmful action?","cues":["Isolation, token-scope and network settings in the runner configuration","Separate permission scopes for read, write, and publish actions","Blocked actions and approval gates on the run page"],"range":"Use several independent gates: disposable environment, egress policy, scoped secrets, least-privilege identity, and human approval for high-risk actions.","trouble":"A ticket or PR can steer a run into private data, a powerful token, and an external channel in one path.","at":[0,1.62,0.85],"children":["prompt-injection","least-privilege"],"links":[{"to":"network-egress","rel":"see also"},{"to":"secrets","rel":"see also"},{"to":"bot-identity","rel":"see also"}],"pattern":"pattern-defense-in-depth","sources":[{"label":"Simon Willison, The lethal trifecta for AI agents: private data, untrusted content, and external communication","url":"https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/","kind":"practitioner","page":"The lethal trifecta","quote":"If your agent combines these three features, an attacker can **easily trick it** into accessing your private data and sending it to that attacker.","work":"simon-willison-lethal-trifecta"},{"label":"OWASP GenAI Security Project, LLM01:2025 Prompt Injection","url":"https://genai.owasp.org/llmrisk/llm01-prompt-injection/","kind":"standard","page":"Prevention and Mitigation Strategies","quote":"Restrict the model’s access privileges to the minimum necessary for its intended operations.","work":"owasp-llm01-prompt-injection"},{"label":"OWASP GenAI Security Project, LLM01:2025 Prompt Injection","url":"https://genai.owasp.org/llmrisk/llm01-prompt-injection/","kind":"standard","page":"Prevention and Mitigation Strategies","quote":"Implement human-in-the-loop controls for privileged operations to prevent unauthorized actions.","work":"owasp-llm01-prompt-injection"},{"label":"Heather Adkins et al., Building Secure and Reliable Systems","url":"https://google.github.io/building-secure-and-reliable-systems/raw/ch08.html","kind":"book","isbn":"9781492083115","page":"Design for Resilience","quote":"When you use compartmentalization, however, a security breach or traffic overload in one compartment does not jeopardize all of the compartments.","work":"building-secure-reliable-systems"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/sandbox","kind":"practitioner","page":"Sandbox","quote":"Limits the blast radius of agent actions","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/sandbox","kind":"practitioner","page":"Sandbox","quote":"The safety substrate that makes AFK practical.","work":"matt-pocock-ai-coding-dictionary"}],"id":"security-boundary","look":{"parts":[{"slot":"boundary","shape":"box","size":[2.2,1.35,0.16],"pos":[0,0.68,0],"radius":0.08,"color":"#8a8f98","material":"metal"},{"slot":"layers","shape":"plane","size":[1.65,0.9],"pos":[0,0.78,0.1],"svg":"RUNNER ISOLATIONUNTRUSTED INPUTSIDE EFFECTSGATES · IDENTITY · EGRESS"},{"slot":"prompt","shape":"box","size":[0.3,0.5,0.1],"pos":[-0.75,0.35,0.17],"radius":0.03,"color":"#e0a526","material":"gloss"},{"slot":"least","shape":"box","size":[0.3,0.5,0.1],"pos":[0.75,0.35,0.17],"radius":0.03,"color":"#2e9b4f","material":"gloss"}]}},{"term":"Prompt injection","aka":["hostile ticket text","indirect instruction","untrusted content attack"],"kind":"concept","plain":"An attacker places instructions in a ticket, PR, document, web page, or tool result that the model may treat as factory instructions. The attack uses the run's authority rather than changing the model itself.","does":"Turns ordinary input into a path toward data disclosure, unauthorized tool use, or unwanted changes.","ask":"What untrusted text can this run read, and what action could that text cause?","cues":["A ticket or PR body containing instructions aimed at the agent","A tool result that asks the agent to ignore its task","A run that reads private data and has an outbound channel"],"range":"Direct injection arrives in the user request. Indirect injection arrives through external content such as a web page, file, ticket, or pull request.","trouble":"The run follows content as a command, exposes a secret, changes unrelated files, or sends data to an attacker-controlled destination.","at":[-0.75,0.35,0.17],"links":[{"to":"network-egress","rel":"see also"},{"to":"secrets","rel":"see also"}],"sources":[{"label":"Simon Willison, The lethal trifecta for AI agents: private data, untrusted content, and external communication","url":"https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/","kind":"practitioner","page":"The problem is that LLMs follow instructions in content","quote":"The problem is that they don’t just follow *our* instructions. They will happily follow *any* instructions that make it to the model, whether or not they came from their operator or from some other source.","work":"simon-willison-lethal-trifecta"},{"label":"Simon Willison, The lethal trifecta for AI agents: private data, untrusted content, and external communication","url":"https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/","kind":"practitioner","page":"The lethal trifecta","quote":"If your agent combines these three features, an attacker can **easily trick it** into accessing your private data and sending it to that attacker.","work":"simon-willison-lethal-trifecta"},{"label":"OWASP GenAI Security Project, LLM01:2025 Prompt Injection","url":"https://genai.owasp.org/llmrisk/llm01-prompt-injection/","kind":"standard","page":"Indirect Prompt Injections","quote":"Indirect prompt injections occur when an LLM accepts input from external sources, such as websites or files.","work":"owasp-llm01-prompt-injection"},{"label":"OWASP GenAI Security Project, LLM01:2025 Prompt Injection","url":"https://genai.owasp.org/llmrisk/llm01-prompt-injection/","kind":"standard","page":"Prevention and Mitigation Strategies","quote":"Separate and clearly denote untrusted content to limit its influence on user prompts.","work":"owasp-llm01-prompt-injection"},{"label":"OWASP GenAI Security Project, LLM Top 10 for Large Language Model Applications","url":"https://genai.owasp.org/llmrisk/llm01-prompt-injection/","kind":"standard","page":"LLM01:2025 Prompt Injection","quote":"A Prompt Injection Vulnerability occurs when user prompts alter the LLM’s behavior or output in unintended ways.","work":"owasp-llm-top10"}],"id":"prompt-injection","feel":{"shape":0.92,"energy":0.85,"weight":-0.1,"warmth":0.78,"grain":0.7,"motion":"jitter","why":"A sharp, restless signal slips through ordinary text and tries to steer a powerful tool."}},{"term":"Least privilege","aka":["minimum permissions","narrow token scope","permissions block"],"kind":"knob","plain":"The smallest set of identities, API scopes, files, and actions a run needs to finish its task. Every omitted permission is a deliberate barrier.","does":"Shrinks the blast radius when a tool, dependency, or model decision is wrong or compromised.","ask":"Which exact read, write, publish, and cloud actions does this task require?","cues":["A permissions block with read and write scopes named","A token that expires with the job","A denied API call for an omitted scope"],"range":"Start with no permissions or contents: read, then add one job-level scope at a time. Keep publish, deploy, and identity-token access separate.","trouble":"A read-only task can push code, a token reaches unrelated repositories, or a broad permission is inherited by every job.","at":[0.75,0.35,0.17],"links":[{"to":"bot-identity","rel":"controls"},{"to":"secrets","rel":"see also"}],"sources":[{"label":"GitHub, Use GITHUB_TOKEN for authentication in workflows","url":"https://docs.github.com/en/actions/tutorials/authenticate-with-github_token","kind":"maker","page":"Using the GITHUB_TOKEN in a workflow","quote":"As a good security practice, you should always make sure that actions only have the minimum access they require by limiting the permissions granted to the `GITHUB_TOKEN`.","work":"github-actions-token"},{"label":"GitHub, Use GITHUB_TOKEN for authentication in workflows","url":"https://docs.github.com/en/actions/tutorials/authenticate-with-github_token","kind":"maker","page":"Modifying the permissions for the GITHUB_TOKEN","quote":"As a good security practice, you should grant the `GITHUB_TOKEN` the least required access.","work":"github-actions-token"},{"label":"OWASP GenAI Security Project, LLM01:2025 Prompt Injection","url":"https://genai.owasp.org/llmrisk/llm01-prompt-injection/","kind":"standard","page":"Prevention and Mitigation Strategies","quote":"Restrict the model’s access privileges to the minimum necessary for its intended operations.","work":"owasp-llm01-prompt-injection"},{"label":"Heather Adkins et al., Building Secure and Reliable Systems","url":"https://google.github.io/building-secure-and-reliable-systems/raw/ch05.html","kind":"book","isbn":"9781492083115","page":"Design for Least Privilege","quote":"The objective of least privilege should extend through all authentication and authorization layers of the system.","work":"building-secure-reliable-systems"}],"id":"least-privilege","look":{"parts":[{"slot":"track","shape":"box","size":[1.7,0.12,0.14],"pos":[0,0.48,0],"radius":0.05,"color":"#252a32","material":"metal"},{"slot":"slider","shape":"box","size":[0.28,0.36,0.2],"pos":[-0.48,0.48,0.1],"radius":0.06,"color":"#2e9b4f","material":"gloss"},{"slot":"labels","shape":"plane","size":[1.65,0.5],"pos":[0,0.92,0.1],"svg":"PERMISSIONScontents: readdeploy: none"}]}},{"term":"Coding agent","aka":["agent","AI coding agent","model plus harness"],"kind":"part","plain":"A model harnessed with tools, a system prompt, and a context window, configured to carry one coding task through a run.","does":"Turns a task into model decisions, tool calls, and a result by letting the harness repeat the agent loop.","ask":"What model and harness should drive this task, and what must the run leave behind?","cues":["A run page names the model and harness","A tool trace shows model turns and tool results alternating","A final result links to changed files or verification evidence"],"at":[0,1.02,0.39],"children":["agent-harness","model-choice","effort","context-window","permission-mode"],"links":[{"to":"instructions","rel":"see also"},{"to":"tool-access","rel":"see also"},{"to":"security-boundary","rel":"see also"}],"sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/agent","kind":"practitioner","page":"Agent","quote":"A model harnessed with tools, a system prompt, and a context window, that takes turns with a user. The model in motion.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/model","kind":"practitioner","page":"Model","quote":"The parameters. Stateless — does next-token prediction and nothing else. Cannot do anything agentic on its own.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/model","kind":"practitioner","page":"Model","quote":"Everything that feels like an agent working — choosing tools, reading results, looping until the task is done — is the harness orchestrating many of those predictions in a row.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents","quote":"Once the task is clear, agents plan and operate independently, potentially returning to the human for further information or judgement.","work":"anthropic-building-effective-agents"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents","quote":"During execution, it's crucial for the agents to gain “ground truth” from the environment at each step (such as tool call results or code execution) to assess its progress.","work":"anthropic-building-effective-agents"}],"id":"coding-agent","look":{"parts":[{"slot":"shell","shape":"box","size":[1.8,1.12,0.1],"pos":[0,0.58,0],"radius":0.06,"color":"#202a3a","material":"matte"},{"slot":"screen","shape":"plane","size":[1.62,0.86],"pos":[0,0.62,0.06],"material":"screen","svg":"CODING AGENTmodel + harness + toolsplan → call → inspecttask: implement run: active"},{"slot":"model","shape":"sphere","radius":0.16,"pos":[-0.58,0.2,0.14],"color":"#8b6bd6","material":"glow","motion":"bob"},{"slot":"effort-low","shape":"box","size":[0.07,0.08,0.05],"pos":[0.55,0.18,0.14],"color":"#e0a526","material":"glow"},{"slot":"effort-medium","shape":"box","size":[0.07,0.13,0.05],"pos":[0.64,0.205,0.14],"color":"#e0a526","material":"glow"},{"slot":"effort-high","shape":"box","size":[0.07,0.18,0.05],"pos":[0.73,0.23,0.14],"color":"#4a5568","material":"matte"},{"slot":"context","shape":"box","size":[0.9,0.07,0.05],"pos":[0,0.14,0.14],"radius":0.03,"color":"#4fc3a1","material":"glow"},{"slot":"permission","shape":"box","size":[0.3,0.16,0.08],"pos":[0.56,0.84,0.14],"radius":0.04,"color":"#d45a5a","material":"gloss"}]}},{"term":"Agent harness","aka":["headless agent loop","coding-agent runner","harness"],"kind":"part","plain":"The program around a model that drives an agent loop in a run. It supplies tools, a system prompt, context-window management, permissions, and hooks, executes the model's tool calls, and returns results in a non-interactive run.","does":"Makes an agent loop repeatable inside a non-interactive run.","ask":"Which tools can the agent call, what can it change, and how does the harness know the run is finished?","cues":["A headless command with an explicit prompt and allowed tools","Tool calls and their results in a run log","A non-zero exit or structured result when the run fails"],"at":[0,0.62,0.1],"links":[{"to":"model-choice","rel":"controls"},{"to":"instructions","rel":"see also"},{"to":"security-boundary","rel":"see also"}],"sources":[{"label":"Anthropic, Run Claude Code programmatically","url":"https://code.claude.com/docs/en/headless","kind":"maker","page":"Run Claude Code programmatically","quote":"To run Claude Code in non-interactive mode, pass `-p` with your prompt and the CLI options you need:","work":"anthropic-claude-code-headless"},{"label":"Anthropic, Best practices for Claude Code","url":"https://code.claude.com/docs/en/best-practices","kind":"maker","page":"Give Claude a way to verify its work","quote":"Claude does the work, runs the check, reads the result, and iterates until the check passes.","work":"anthropic-claude-code-best-practices"},{"label":"Theo Browne, What is a \"harness\"?","url":"https://t3.gg/blog/post/what-is-a-harness","kind":"practitioner","page":"LLMs use harnesses","quote":"A system that parses the message to identify that syntax and run the code … A method to give the response from the tool back to the LLM","work":"theo-browne-what-is-a-harness"},{"label":"Theo Browne, What is a \"harness\"?","url":"https://t3.gg/blog/post/what-is-a-harness","kind":"practitioner","page":"Isn't Claude Code a harness?","quote":"Claude Code has 2 parts: the interface and the harness","work":"theo-browne-what-is-a-harness"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/harness","kind":"practitioner","page":"Harness","quote":"Everything around the model that turns it into an agent: tools, system prompt, context-window management, permissions, hooks.","work":"matt-pocock-ai-coding-dictionary"}],"id":"agent-harness","look":{"parts":[{"slot":"terminal","shape":"box","size":[1.55,0.95,0.08],"pos":[0,0.55,0],"radius":0.04,"color":"#252b35","material":"matte"},{"slot":"screen","shape":"plane","size":[1.42,0.8],"pos":[0,0.58,0.05],"material":"screen","svg":"HEADLESS RUN$ agent -p \"implement\"Read Edit Bashcheck: passed"},{"slot":"tool-chip","shape":"plane","size":[0.44,0.22],"pos":[-0.42,0.05,0.1],"face":"camera","svg":"TOOLS"},{"slot":"permission-chip","shape":"plane","size":[0.52,0.22],"pos":[0.42,0.05,0.1],"face":"camera","svg":"PERMISSIONS"}]}},{"term":"Model choice","aka":["model routing","capability tier"],"kind":"knob","plain":"The policy that routes tasks to different model categories or tiers by difficulty and cost. Work that has become routine can run as plain code with no model.","does":"Matches model capability and spend to the task and its risk.","ask":"Which model can finish this task reliably, and what evidence justifies the cost?","cues":["A model name or tier in the run record","A routing rule that sends easy and hard tasks to different models","Cost and latency split by model choice"],"range":"Smaller, cost-efficient models for easy or common work; more capable models for hard or unusual work; no model for work already turned into code.","trouble":"A poor route sends distinct task categories to one model, wasting cost or reducing performance.","at":[-0.5,0.28,0.15],"links":[{"to":"token-cost","rel":"measured by"},{"to":"automated-review","rel":"see also"}],"sources":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Workflow: Routing","quote":"Routing easy/common questions to smaller, cost-efficient models like Claude Haiku 4.5 and hard/unusual questions to more capable models like Claude Sonnet 4.5 to optimize for best performance.","work":"anthropic-building-effective-agents"},{"label":"Kody, Documentation","url":"https://kody.codes/about","kind":"maker","page":"About Kody","quote":"Your connected agent does the thinking; Kody holds the durable result so the work can run again without another model in the loop.","work":"kody-docs"}],"id":"model-choice","look":{"parts":[{"slot":"dial","shape":"cylinder","radiusTop":0.38,"radiusBottom":0.38,"height":0.12,"pos":[0,0.4,0],"rot":[90,0,0],"color":"#3b6ea8","material":"metal"},{"slot":"needle","shape":"box","size":[0.04,0.3,0.04],"pos":[0.18,0.62,0.08],"rot":[0,0,35],"color":"#e0a526","material":"gloss"},{"slot":"label","shape":"plane","size":[1,0.42],"pos":[0,0.9,0.08],"svg":"MODEL ROUTEcost / skill / risk"}]}},{"term":"Effort","aka":["reasoning effort","thinking budget"],"kind":"knob","plain":"A dial for how much reasoning the model does before it answers.","does":"More effort spends more output tokens and time for a better shot at hard problems. Set it per task or subagent so each job gets the reasoning it needs.","ask":"How much reasoning should this task or subagent spend before it answers?","cues":["An effort level beside the model in a run record","Higher effort with more output tokens and longer latency","Different subagents carrying different effort settings"],"range":"Low for mechanical edits and lookups; Medium for everyday coding; High for tricky bugs and multi-step plans; Max for the hardest problems.","trouble":"Low effort on a hard problem produces a confident, shallow answer; max effort on a one-line rename spends time and cost without useful gain.","at":[0.64,0.21,0.17],"links":[{"to":"model-choice","rel":"see also"},{"to":"token-cost","rel":"raises"}],"sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/effort","kind":"practitioner","page":"Effort","quote":"A dial for how much reasoning the model does before it answers. More effort spends more output tokens for a better shot at hard problems.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/effort","kind":"practitioner","page":"Effort","quote":"Higher effort costs more and runs slower. The reasoning is emitted as tokens, billed as output tokens even when you never see them, and produced one token at a time — so turning effort up lengthens the wait before the answer arrives and adds to the bill.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/effort","kind":"practitioner","page":"Effort","quote":"Low | Mechanical edits, lookups, well-specified changes with one clear path.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/effort","kind":"practitioner","page":"Effort","quote":"Medium | Everyday coding — the usual default.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/effort","kind":"practitioner","page":"Effort","quote":"High | Tricky bugs, design decisions, multi-step plans.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/effort","kind":"practitioner","page":"Effort","quote":"Max | The hardest problems, where a wrong answer is expensive to unwind.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/effort","kind":"practitioner","page":"Effort","quote":"Set effort too low on a hard problem and you get a confident, shallow answer that skipped the reasoning the problem needed — it reads fine and is wrong in a way that costs you later. Set it to max for a one-line rename and you sit through a long think that produces nothing the lowest setting wouldn't have.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/skills/swarm/SKILL.md","kind":"practitioner","page":"Phase A: Frame","quote":"Pick each subagent's reasoning level and fast or standard speed for the job, per the Subagents section of the poteto-mode skill.","work":"lauren-tan-pstack"}],"id":"effort","look":{"parts":[{"slot":"panel","shape":"box","size":[1.3,0.62,0.1],"pos":[0,0.42,0],"radius":0.05,"color":"#26303f","material":"matte"},{"slot":"low","shape":"box","size":[0.2,0.14,0.08],"pos":[-0.42,0.27,0.08],"radius":0.02,"color":"#e0a526","material":"glow"},{"slot":"medium","shape":"box","size":[0.2,0.24,0.08],"pos":[-0.14,0.32,0.08],"radius":0.02,"color":"#e0a526","material":"glow"},{"slot":"high","shape":"box","size":[0.2,0.34,0.08],"pos":[0.14,0.37,0.08],"radius":0.02,"color":"#e0a526","material":"glow"},{"slot":"max","shape":"box","size":[0.2,0.44,0.08],"pos":[0.42,0.42,0.08],"radius":0.02,"color":"#4a5568","material":"matte"},{"slot":"scale","shape":"plane","size":[1.25,0.36],"pos":[0,0.95,0.06],"svg":"EFFORTlow · medium · high · max"}]}},{"term":"Context window","aka":["context budget","working context"],"kind":"concept","plain":"Everything the model sees on each model provider request. It is finite and is the only surface through which the model perceives.","does":"Carries a run's working state from turn to turn. A fresh run can start with an empty window, so sizing a task to one session keeps it in the sharpest part of the session.","ask":"What must this run see now, and what can it leave behind?","cues":["A context meter fills as messages, model responses, and tool results accumulate","A compaction or autocompact event replaces history with a summary","A new session opens with an empty context window"],"trouble":"Quality drops as the window fills. Attention degrades, and compaction or autocompact can lose detail; clearing discards the working state.","at":[0,0.16,0.12],"links":[{"to":"instructions","rel":"see also"},{"to":"tool-access","rel":"see also"}],"sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/context-window","kind":"practitioner","page":"Context window","quote":"Everything the model sees on each model provider request. Finite, model-specific, and the only surface through which the model perceives anything.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/context-window","kind":"practitioner","page":"Context window","quote":"Finite means it fills up. Every turn appends more — your messages, the model's responses, tool results — and a long session will eventually hit the limit, forcing compaction or clearing.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/clearing","kind":"practitioner","page":"Clearing","quote":"The next message begins with an empty session and an empty context window.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/smart-zone","kind":"practitioner","page":"Smart zone","quote":"Doing one task per session gives each task the sharpest part of the session.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/smart-zone","kind":"practitioner","page":"Smart zone","quote":"Early in a session the agent is sharp and focused. As the session grows it drifts into a dumb zone: sloppier, forgetful, more mistakes.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/attention-degradation","kind":"practitioner","page":"Attention degradation","quote":"As a session grows, each token's attention budget is spread across more competitors. The signal on any one meaningful relationship shrinks; noise from irrelevant context crowds in.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/compaction","kind":"practitioner","page":"Compaction","quote":"Whatever didn't make it into the summary is gone from the context.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/autocompact","kind":"practitioner","page":"Autocompact","quote":"Compaction is lossy, and autocompact is lossy at a moment you didn't choose.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/clearing","kind":"practitioner","page":"Clearing","quote":"What's gone is the agent's working state: the model is stateless, so the new session knows nothing the old one knew.","work":"matt-pocock-ai-coding-dictionary"}],"id":"context-window","feel":{"shape":0.22,"energy":0.45,"weight":0.15,"warmth":0.38,"grain":0.68,"motion":"pulse","why":"A finite, crowded working surface pulses as each turn adds another token."}},{"term":"Permission mode","aka":["approval mode","tool approval setting","auto-approve policy"],"kind":"knob","plain":"The setting that decides which tool calls run automatically and which stop for a permission request.","does":"An AFK run has nobody to answer a permission request, so pre-approve safe tools and let the sandbox contain what those tools can reach.","ask":"Which tool calls may run unattended, and which must wait for a person?","cues":["A mode selector such as read-only, default, auto-edit, or full-auto","An allowed-tools list in a headless command","A paused run showing a permission request"],"range":"Read-only / plan: reads auto, writes and shell blocked; Default: reads auto, writes ask; Auto-edit: edits auto, shell asks; Full-auto: reads, writes and shell auto.","trouble":"A run waits on a person who is away, or a loose mode lets an agent edit files and run commands you meant to review first.","at":[0.55,0.86,0.14],"links":[{"to":"security-boundary","rel":"see also"},{"to":"least-privilege","rel":"see also"}],"sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/permission-mode","kind":"practitioner","page":"Permission mode","quote":"The permission-gating slice of an agent mode — which tool calls trigger a permission request and which run automatically.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/permission-mode","kind":"practitioner","page":"Permission mode","quote":"Read-only / plan | Auto | Blocked | Research, planning, reviewing","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/permission-mode","kind":"practitioner","page":"Permission mode","quote":"Default | Auto | Ask | Day-to-day supervised work","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/permission-mode","kind":"practitioner","page":"Permission mode","quote":"Auto-edit | Auto | Edits auto, shell asks | Trusted repos, mechanical changes","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/permission-mode","kind":"practitioner","page":"Permission mode","quote":"\"Yolo\" / full-auto | Auto | Auto | Sandboxes, AFK runs","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/permission-request","kind":"practitioner","page":"Permission request","quote":"The model produces a tool call; instead of running it immediately, the harness pauses and asks.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/afk","kind":"practitioner","page":"AFK","quote":"Usually requires a permissive permission mode plus sandboxing to be safe.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/sandbox","kind":"practitioner","page":"Sandbox","quote":"The safety substrate that makes AFK practical.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/sandbox","kind":"practitioner","page":"Sandbox","quote":"Permissions ask before an action runs; a sandbox limits what the action can reach if it does run.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/permission-mode","kind":"practitioner","page":"Permission mode","quote":"Too loose, and the agent edits files and runs commands you'd have wanted to see first.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/permission-request","kind":"practitioner","page":"Permission request","quote":"The agent sits blocked until you answer, which is fine while you're watching and a problem when you're not.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Anthropic, Run Claude Code programmatically","url":"https://code.claude.com/docs/en/headless","kind":"maker","page":"Start faster with bare mode","quote":"This example runs a one-off summarize task in bare mode and pre-approves the Read tool so the call completes without a permission prompt:","work":"anthropic-claude-code-headless"}],"id":"permission-mode","look":{"parts":[{"slot":"plate","shape":"box","size":[1.42,0.82,0.1],"pos":[0,0.48,0],"radius":0.08,"color":"#263345","material":"matte"},{"slot":"track","shape":"box","size":[1,0.12,0.04],"pos":[0,0.48,0.08],"radius":0.06,"color":"#7a8798","material":"gloss"},{"slot":"switch","shape":"box","size":[0.24,0.3,0.1],"pos":[0.26,0.48,0.13],"radius":0.05,"color":"#4fc3a1","material":"gloss"},{"slot":"label","shape":"plane","size":[1.05,0.34],"pos":[0,0.92,0.08],"svg":"PERMISSION MODEask ← auto"},{"slot":"lock","shape":"torus","radius":0.09,"tube":0.025,"pos":[-0.42,0.2,0.12],"rot":[90,0,0],"color":"#e0a526","material":"metal"}]}},{"term":"Agent-ready codebase","aka":["agent experience","AX","agent legibility","harness engineering","the playground"],"kind":"part","plain":"The repository set up so an agent can find its way, make a change and prove the change works with no person watching. It holds a short instructions map, a glossary of the project's terms, one verify command, a tool that drives the running app, the tools and skills the agent may call, and checks that tighten after each repeated mistake.","does":"Lets the same agent and model do good work that it can prove, so less of its output needs a person to read it.","ask":"Can a fresh agent find the right code, run the one verify command, drive the app and show evidence, all without asking a person?","cues":["A short AGENTS.md map and a glossary near the repository root","One verify command that runs the same checks as CI","A checked-in CLI or script that starts and drives the app","Lint rules and checks that cite the review comment they replaced"],"at":[-1.75,1.65,-0.45],"children":["codebase-health","instructions","shared-vocabulary","verify-command","control-cli","tool-access","ratchet"],"links":[{"to":"merge-rate","rel":"raises"},{"to":"verification","rel":"see also"}],"sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/ax","kind":"practitioner","page":"AX","quote":"how well the environment is set up for an agent to do good work in a codebase.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/ax","kind":"practitioner","page":"AX","quote":"When the same agent performs well in one repo and badly in another … same model, same harness … the difference is usually AX.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/ax","kind":"practitioner","page":"AX","quote":"The instinct is to blame the model or rewrite the prompt; the fix is more often in the repo.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"Audit any repository and set it up so coding agents ship changes safely with little human review.","work":"kent-c-dodds-kcd-skills"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"The repo tells agents what to build and how to prove it.","work":"kent-c-dodds-kcd-skills"},{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Design the playground","quote":"Build good primitives. Agents are only as good as the building blocks you give them.","work":"kent-c-dodds-you-own-the-outcome"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/docs/contributing/harness-engineering.md","kind":"maker","page":"docs/contributing/harness-engineering.md","quote":"Humans steer outcomes; agents execute implementation details.","work":"kody-repository"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/06-verify-and-ship.md","kind":"practitioner","page":"Verify the result and open a PR","quote":"Make the agent able to do it, and you stop being the bottleneck.","work":"lauren-tan-pstack"}],"id":"agent-ready-codebase","look":{"parts":[{"slot":"board","shape":"box","size":[2.3,1.6,0.08],"pos":[0,0.9,0],"color":"#c9a46a","material":"matte","radius":0.04},{"slot":"board-face","shape":"plane","size":[2.2,1.5],"pos":[0,0.9,0.045],"svg":"AGENTS.md→ docs/→ glossary→ verifyglossaryverifyratchetmodulesCLIMCP · skills"},{"slot":"glossary","shape":"box","size":[0.28,0.4,0.06],"pos":[0,1.25,0.08],"color":"#6b5bd2","material":"gloss","radius":0.02},{"slot":"verify-button","shape":"cylinder","radiusTop":0.15,"radiusBottom":0.16,"height":0.08,"pos":[0.7,1.25,0.06],"rot":[90,0,0],"color":"#2e9b4f","material":"gloss"},{"slot":"verify-mark","shape":"plane","size":[0.16,0.16],"pos":[0.7,1.25,0.105],"svg":""},{"slot":"ratchet-head","shape":"torus","radius":0.075,"tube":0.028,"pos":[-0.8,0.73,0.06],"color":"#8a8f98","material":"metal"},{"slot":"ratchet-handle","shape":"box","size":[0.07,0.33,0.04],"pos":[-0.8,0.505,0.06],"color":"#8a8f98","material":"metal"},{"slot":"cli","shape":"box","size":[0.3,0.4,0.05],"pos":[0.27,0.5,0.065],"color":"#1d2a2a","material":"matte","radius":0.03},{"slot":"cli-screen","shape":"plane","size":[0.26,0.2],"pos":[0.27,0.57,0.095],"material":"screen","svg":"$ app start$ app login$ app screenshot✓ evidence saved"},{"slot":"plug","shape":"box","size":[0.18,0.28,0.1],"pos":[0.8,0.48,0.08],"color":"#3b6ea8","material":"gloss","radius":0.02}]}},{"term":"Codebase health","aka":["code health","deep modules","architecture hygiene"],"kind":"concept","plain":"How easy the code is to find and change. A healthy codebase has a predictable layout, much of its behaviour behind small interfaces, and names that say what things do. Agents write code fast, so a codebase can also decay fast, and its health needs regular work.","does":"Lets an agent find and change the right code without reading everything, which keeps its changes small enough to verify.","ask":"Can an agent find where this change belongs and make it behind one small interface?","cues":["Large modules with small public interfaces","Folder and file names that say what the code does","A regular architecture sweep that proposes deepening work"],"at":[-0.27,0.5,0.08],"links":[{"to":"merge-rate","rel":"raises"}],"sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/ax","kind":"practitioner","page":"AX","quote":"A codebase the agent can navigate without reading everything: predictable structure, a lot of behaviour behind small interfaces, names that say what things do","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, Skills For Real Engineers","url":"https://github.com/mattpocock/skills/blob/5beb35b040e3de97e54fa9e386d7e42d15c295c2/README.md","kind":"practitioner","page":"#4: We Built A Ball Of Mud","quote":"Because agents can radically speed up coding, they also accelerate software entropy. Codebases get more complex at an unprecedented rate.","work":"matt-pocock-skills"},{"label":"Matt Pocock, Skills For Real Engineers","url":"https://github.com/mattpocock/skills/blob/5beb35b040e3de97e54fa9e386d7e42d15c295c2/README.md","kind":"practitioner","page":"#4: We Built A Ball Of Mud","quote":"surveys a codebase for deepening opportunities and hands you the candidates. I recommend running it on your codebase once every few days.","work":"matt-pocock-skills"},{"label":"Steve McConnell, Code Complete, 2nd Edition","url":"https://ptgmedia.pearsoncmg.com/images/9780735619678/samplepages/9780735619678.pdf","kind":"book","isbn":"9780735619678","page":"Chapter 5, Design in Construction (publisher's sample chapter PDF)","quote":"Managing complexity is the most important technical topic in software development","work":"code-complete-steve-mcconnell"}],"id":"codebase-health","look":{"parts":[{"slot":"card","shape":"box","size":[1.4,1,0.05],"pos":[0,0.6,0],"color":"#f4f5f7","material":"matte","radius":0.04},{"slot":"map","shape":"plane","size":[1.32,0.92],"pos":[0,0.6,0.026],"svg":"MODULE MAPbilling/auth/tickets/"}]}},{"term":"Instructions","aka":["repository guidance","agent context","AGENTS.md"],"kind":"part","plain":"The durable guidance an agent receives about the repository, task boundaries, checks, style, and safety. It may live in AGENTS.md, CLAUDE.md, repository prompts, or equivalent configuration. Keep the root file a short map that points to deeper docs, and write down the \"no\" decisions so agents stop proposing them again.","does":"Supplies context the agent cannot infer reliably from the ticket or source tree alone.","ask":"Which instructions are always needed, which are local to a path, and which can the agent verify?","cues":["An AGENTS.md or CLAUDE.md file near the repository root","A run log showing loaded instruction files","A pull request that reports the checks and conventions it followed"],"at":[-0.7,1.25,0.08],"links":[{"to":"agent-harness","rel":"see also"},{"to":"improvement-loop","rel":"see also"}],"sources":[{"label":"AGENTS.md, AGENTS.md","url":"https://agents.md/","kind":"standard","page":"Why AGENTS.md?","quote":"Think of AGENTS.md as a README for agents: a dedicated, predictable place to provide the context and instructions to help AI coding agents work on your project.","work":"agentsmd-agents-md"},{"label":"GitHub, Adding repository custom instructions for GitHub Copilot","url":"https://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/add-custom-instructions/add-repository-instructions","kind":"maker","page":"Introduction","quote":"Repository custom instructions let you provide Copilot with repository-specific guidance and preferences on GitHub.","work":"github-copilot-custom-instructions"},{"label":"Anthropic, Best practices for Claude Code","url":"https://code.claude.com/docs/en/best-practices","kind":"maker","page":"Write an effective CLAUDE.md","quote":"CLAUDE.md is a special file that Claude reads at the start of every conversation.","work":"anthropic-claude-code-best-practices"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"Write down the \"no\": decisions agents would otherwise re-propose.","work":"kent-c-dodds-kcd-skills"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/docs/contributing/harness-engineering.md","kind":"maker","page":"docs/contributing/harness-engineering.md","quote":"Prefer small, enforceable rules over long, fragile instructions.","work":"kody-repository"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/ax","kind":"practitioner","page":"AX","quote":"AGENTS.md, skills, and tools kept lean, so most of the context window is available for the task","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/agents-md","kind":"practitioner","page":"AGENTS.md","quote":"A file in the environment that the harness loads into the context window at session start — the project's standing brief to the agent.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/progressive-disclosure","kind":"practitioner","page":"Progressive disclosure","quote":"Loading only the context an agent needs right now, with context pointers to the rest.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/context-pointer","kind":"practitioner","page":"Context pointer","quote":"A mention in one document that points to another, so the agent can pull it into the context window only when the task calls for it.","work":"matt-pocock-ai-coding-dictionary"}],"id":"instructions","look":{"parts":[{"slot":"paper","shape":"box","size":[1.35,1.02,0.05],"pos":[0,0.52,0],"radius":0.04,"color":"#f3efe4","material":"matte"},{"slot":"paper-face","shape":"plane","size":[1.24,0.91],"pos":[0,0.55,0.04],"svg":"AGENTS.mdsetup and checksstyle and boundariesevidence to report"},{"slot":"clip","shape":"torus","radius":0.14,"tube":0.025,"pos":[-0.48,0.99,0.08],"rot":[90,0,0],"color":"#e8743b","material":"metal"},{"slot":"tab","shape":"plane","size":[0.44,0.2],"pos":[0.48,0.1,0.1],"face":"camera","svg":"LOADED"}]}},{"term":"Shared vocabulary","aka":["glossary","ubiquitous language","domain language","primitives catalog"],"kind":"part","plain":"A short file of the project's terms, each with one meaning, that agents and people both use. It can also list the building blocks of the code and the words to avoid.","does":"Saves each agent from guessing the jargon, so tickets, prompts, code and reviews use the same short names.","ask":"Does each important concept have one written name, and does the code use it?","cues":["A glossary or primitives file in the repository","Entries with a term, a one-line meaning and words to avoid","Code, tickets and review comments that use the glossary's terms"],"at":[0,1.25,0.12],"links":[{"to":"instructions","rel":"see also"}],"sources":[{"label":"Matt Pocock, Skills For Real Engineers","url":"https://github.com/mattpocock/skills/blob/5beb35b040e3de97e54fa9e386d7e42d15c295c2/README.md","kind":"practitioner","page":"#2: The Agent Is Way Too Verbose","quote":"Agents are usually dropped into a project and asked to figure out the jargon as they go. So they use 20 words where 1 will do.","work":"matt-pocock-skills"},{"label":"Matt Pocock, Skills For Real Engineers","url":"https://github.com/mattpocock/skills/blob/5beb35b040e3de97e54fa9e386d7e42d15c295c2/README.md","kind":"practitioner","page":"#2: The Agent Is Way Too Verbose","quote":"The Fix for this is a shared language. It's a document that helps agents decode the jargon used in the project.","work":"matt-pocock-skills"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"Primitives catalog: building blocks and code roots, kept short.","work":"kent-c-dodds-kcd-skills"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/docs/contributing/architecture/primitives.yaml","kind":"maker","page":"docs/contributing/architecture/primitives.yaml","quote":"Stable vocabulary for the visual-recap skill","work":"kody-repository"}],"id":"shared-vocabulary","look":{"parts":[{"slot":"book","shape":"box","size":[0.62,0.9,0.14],"pos":[-0.35,0.47,0],"color":"#6b5bd2","material":"gloss","radius":0.03},{"slot":"book-title","shape":"plane","size":[0.5,0.2],"pos":[-0.35,0.68,0.071],"svg":"GLOSSARY"},{"slot":"entry-card","shape":"box","size":[0.86,0.6,0.02],"pos":[0.42,0.42,0.1],"color":"#fdfaf2","material":"matte","radius":0.03},{"slot":"entry","shape":"plane","size":[0.82,0.56],"pos":[0.42,0.42,0.111],"svg":"RunOne execution of one task.Ends when the agent stops.Avoid: session"}]}},{"term":"Verify command","aka":["validate script","single gate","local CI"],"kind":"part","plain":"One command that runs every check the repository requires, the same way on a laptop, on a runner and in CI. An agent runs it before it says a change is done.","does":"Gives every run one fast, trusted answer to whether the checks pass, with no guessing about which checks count.","ask":"Is there one command, and does CI run exactly the same checks?","cues":["A validate or verify script named in AGENTS.md","CI jobs that run the same command or the same list of checks","A pull request that reports the command's result"],"at":[0.7,1.25,0.12],"links":[{"to":"automated-checks","rel":"see also"}],"sources":[{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"One small AGENTS.md map and one verify command (local == CI).","work":"kent-c-dodds-kcd-skills"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/AGENTS.md","kind":"maker","page":"AGENTS.md","quote":"npm run validate is the single authoritative local gate.","work":"kody-repository"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/.github/workflows/validate.yml","kind":"maker","page":".github/workflows/validate.yml","quote":"Local npm run validate remains the authoritative single-command gate.","work":"kody-repository"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/references/verification.md","kind":"practitioner","page":"autopilot skill, references/verification.md","quote":"What helps agents ship quality, in order: unit and integration test suites, typecheck, agents running their own change, a control CLI to drive the app, end-to-end tests, and decision records.","work":"kent-c-dodds-kcd-skills"},{"label":"Matt Pocock, Skills For Real Engineers","url":"https://github.com/mattpocock/skills/blob/5beb35b040e3de97e54fa9e386d7e42d15c295c2/README.md","kind":"practitioner","page":"#3: The Code Doesn't Work","quote":"Without feedback on how the code it produces actually runs, the agent will be flying blind.","work":"matt-pocock-skills"}],"id":"verify-command","look":{"parts":[{"slot":"base","shape":"box","size":[0.9,0.18,0.7],"pos":[0,0.09,0.15],"color":"#2a2a2e","material":"matte","radius":0.03},{"slot":"button","shape":"cylinder","radiusTop":0.25,"radiusBottom":0.27,"height":0.12,"pos":[0,0.24,0.15],"color":"#2e9b4f","material":"gloss"},{"slot":"base-label","shape":"plane","size":[0.8,0.13],"pos":[0,0.09,0.502],"svg":"$ ./verify"},{"slot":"screen","shape":"box","size":[0.95,0.62,0.06],"pos":[0,0.78,-0.25],"color":"#1d2a2a","material":"matte","radius":0.03},{"slot":"screen-face","shape":"plane","size":[0.88,0.55],"pos":[0,0.78,-0.218],"material":"screen","svg":"$ ./verify✓ lint✓ types✓ tests✓ buildsame as CI"}]}},{"term":"Control CLI","aka":["app driver","verification skill","feature map"],"kind":"part","plain":"A checked-in command-line tool that starts the app, signs in and drives its features. An agent uses it to try its change the way a user would and to capture the evidence.","does":"Lets an agent prove a user-visible change itself, so no person has to click through it before merge.","ask":"Can an agent start the app, sign in and exercise this feature with one documented command?","cues":["A CLI or script in the repository that starts and drives the app","A feature map that lists each feature and the command that exercises it","Pull requests with captured output or screenshots from the driven app"],"at":[0.27,0.5,0.1],"links":[{"to":"self-verification","rel":"see also"}],"sources":[{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/.agents/skills/control-kody/SKILL.md","kind":"maker","page":".agents/skills/control-kody/SKILL.md","quote":"Drive and verify the Kody app with a Feature Map and one CLI.","work":"kody-repository"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/.agents/skills/control-kody/SKILL.md","kind":"maker","page":".agents/skills/control-kody/SKILL.md","quote":"Do not invent a throwaway curl script, scrape PR comments by hand, or rediscover account routes","work":"kody-repository"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/.agents/skills/control-kody/SKILL.md","kind":"maker","page":".agents/skills/control-kody/SKILL.md","quote":"CI green is not enough for a user-visible account change. Local verification is the default","work":"kody-repository"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/references/verification.md","kind":"practitioner","page":"autopilot skill, references/verification.md","quote":"An agent that starts the app, signs in, and asserts on real responses catches its own bugs before merge.","work":"kent-c-dodds-kcd-skills"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/skills/create-verification-skill/SKILL.md","kind":"practitioner","page":"Create a verification skill","quote":"Every serious project needs a scripted way to drive the real app and prove behavior: launch it, exercise a feature the way a user would, and capture evidence.","work":"lauren-tan-pstack"}],"id":"control-cli","look":{"parts":[{"slot":"terminal","shape":"box","size":[0.9,0.66,0.05],"pos":[-0.45,0.55,0.1],"color":"#1d2a2a","material":"matte","radius":0.03},{"slot":"terminal-screen","shape":"plane","size":[0.84,0.6],"pos":[-0.45,0.55,0.126],"material":"screen","svg":"$ app start$ app login --as test$ app open /billing$ app screenshotsaved shot-1.png"},{"slot":"app-window","shape":"box","size":[0.82,0.62,0.04],"pos":[0.5,0.66,-0.12],"color":"#f4f5f7","material":"matte","radius":0.03},{"slot":"app-screen","shape":"plane","size":[0.78,0.58],"pos":[0.5,0.66,-0.099],"svg":"BillingCard declinedRetry"}]}},{"term":"Tool access","aka":["MCP servers","skills","agent tools","MCP server","skill"],"kind":"part","plain":"The tools and skills an agent may call during a run. MCP servers let it query or act on outside systems, and skill folders hold repeatable procedures it loads only when a task needs them. Keep the set small, so most of the context window stays free for the task.","does":"Lets an agent reach systems and procedures it cannot rebuild from the source code alone.","ask":"Which tools does this task need, and which ones only fill the context window or widen what a run can reach?","cues":["An MCP server list in the agent's configuration","A skills folder with one SKILL.md file per skill","Tool calls in the run's trace"],"at":[0.8,0.5,0.14],"links":[{"to":"least-privilege","rel":"see also"},{"to":"agent-trace","rel":"measured by"}],"sources":[{"label":"Model Context Protocol, Specification 2025-06-18, Tools","url":"https://modelcontextprotocol.io/specification/2025-06-18/server/tools","kind":"standard","page":"Tools","quote":"The Model Context Protocol (MCP) allows servers to expose tools that can be invoked by language models.","work":"mcp-tools-specification"},{"label":"Model Context Protocol, Specification 2025-06-18, Tools","url":"https://modelcontextprotocol.io/specification/2025-06-18/server/tools","kind":"standard","page":"Tools","quote":"Tools enable models to interact with external systems, such as querying databases, calling APIs, or performing computations.","work":"mcp-tools-specification"},{"label":"Agent Skills, Specification","url":"https://agentskills.io/specification","kind":"standard","page":"Specification","quote":"A skill is a directory containing, at minimum, a SKILL.md file","work":"agent-skills-specification"},{"label":"Agent Skills, Specification","url":"https://agentskills.io/specification","kind":"standard","page":"Specification","quote":"Agents load skills progressively, pulling in more detail only as a task calls for it.","work":"agent-skills-specification"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/ax","kind":"practitioner","page":"AX","quote":"AGENTS.md, skills, and tools kept lean, so most of the context window is available for the task","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/tool","kind":"practitioner","page":"Tool","quote":"A function the harness exposes for the agent to call — Read, Write, Bash, Search. Tools are how an agent perceives and acts on the environment: it can't see the environment except through tool results, and can't change it except through tool calls.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/mcp","kind":"practitioner","page":"MCP","quote":"The agent never \"calls MCP\"; it calls a tool, and the harness happens to have gotten that tool from an MCP server.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/skill","kind":"practitioner","page":"Skill","quote":"A teachable capability bundled as a unit — instructions and resources for doing one task well, kept in the environment until a context pointer pulls it into the context window for the task at hand.","work":"matt-pocock-ai-coding-dictionary"}],"id":"tool-access","look":{"parts":[{"slot":"panel","shape":"box","size":[1.5,0.6,0.06],"pos":[0,0.6,0],"color":"#e9eaee","material":"matte","radius":0.04},{"slot":"sockets","shape":"plane","size":[1.44,0.54],"pos":[0,0.6,0.031],"svg":"tracker MCPbrowser MCPskills/"},{"slot":"plug","shape":"box","size":[0.2,0.26,0.14],"pos":[-0.45,0.69,0.1],"color":"#3b6ea8","material":"gloss","radius":0.02},{"slot":"cable","shape":"cylinder","radiusTop":0.025,"radiusBottom":0.025,"height":0.6,"pos":[-0.45,0.26,0.12],"color":"#2a2a2e","material":"matte"}]}},{"term":"Ratchet","aka":["baseline tightening","shrinking exception list"],"kind":"process","plain":"A stricter check is switched on while known failures remain in an exception list. New failures stop the run, and fixed exceptions are removed so the baseline only improves. The same move turns a review comment that keeps coming back into a lint rule, check or script.","does":"Converts a large legacy gap into a one-way improvement path without accepting new violations.","ask":"What is the current exception set, and what removes an exception from it?","cues":["A checked-in suppressions or baseline file","New violations fail while old entries remain listed","A periodic prune removes fixed entries"],"at":[-0.8,0.55,0.08],"pattern":"pattern-ratchet","links":[{"to":"automated-checks","rel":"controls"}],"sources":[{"label":"ESLint, Command Line Interface: Suppress Violations","url":"https://eslint.org/docs/latest/use/command-line-interface","kind":"maker","page":"--suppress-all","quote":"Suppresses existing violations, so that they are not being reported in subsequent runs.","work":"eslint-cli-suppressions"},{"label":"ESLint, Command Line Interface: Suppress Violations","url":"https://eslint.org/docs/latest/use/command-line-interface","kind":"maker","page":"--prune-suppressions","quote":"Prune unused suppressions from the suppressions file. This option is useful when you addressed one or more of the suppressed violations.","work":"eslint-cli-suppressions"},{"label":"Neal Ford, Rebecca Parsons and Patrick Kua, Architectural Fitness Functions","url":"https://evolutionaryarchitecture.com/ffkatas/","kind":"practitioner","page":"Architectural Fitness Functions","quote":"An architectural fitness function provides an objective integrity assessment of some architectural characteristic(s).","work":"evolutionaryarchitecture-fitness-functions"},{"label":"Neal Ford et al., Building Evolutionary Architectures, 2nd Edition","url":"https://www.thoughtworks.com/content/dam/thoughtworks/documents/books/bk_building_evolutionary_architectures_second_edition_free_chapter.pdf","kind":"book","isbn":"9781492097549","page":"Chapter 1, Evolving Software Architecture (free chapter PDF)","quote":"When an architect identifies an architectural characteristic they want to protect as things evolve, they define one or more fitness functions to protect that feature.","work":"building-evolutionary-architectures"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/docs/contributing/harness-engineering.md","kind":"maker","page":"docs/contributing/harness-engineering.md","quote":"Rule of thumb: if reviewers repeat the same comment twice, encode it.","work":"kody-repository"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/08-principles.md","kind":"practitioner","page":"Steer with principle names","quote":"turns advice you've repeated twice into a lint, check, or script","work":"lauren-tan-pstack"}],"id":"ratchet","look":{"parts":[{"slot":"rail","shape":"box","size":[1.55,0.12,0.2],"pos":[0,0.16,0],"color":"#3a3f47","material":"matte"},{"slot":"teeth","shape":"cone","radius":0.22,"height":0.32,"pos":[-0.42,0.38,0.08],"rot":[0,0,180],"color":"#c8432f","material":"gloss"},{"slot":"tooth-2","shape":"cone","radius":0.22,"height":0.32,"pos":[0,0.58,0.08],"rot":[0,0,180],"color":"#e0a526","material":"gloss"},{"slot":"tooth-3","shape":"cone","radius":0.22,"height":0.32,"pos":[0.42,0.78,0.08],"rot":[0,0,180],"color":"#2e9b4f","material":"glow"},{"slot":"baseline","shape":"plane","size":[1.22,0.3],"pos":[0,1.05,0.08],"svg":"BASELINE ONLY SHRINKS"}]}},{"term":"Task","aka":["work type","job definition","job","workflow"],"kind":"part","plain":"A repeatable work definition that names its trigger, action, and next handoff.","does":"Turns a broad request into a bounded run that can be selected and repeated by an orchestrator.","ask":"What triggers this task, what does it change, and what event should follow it?","cues":["A ticket or run page names the task type","A trigger rule maps an event to the task","A run result links to the next task or handoff"],"at":[-1.5,1.05,0.31],"children":["run","run-chain"],"links":[{"to":"coding-agent","rel":"see also"}],"variants":[{"label":"Task type","items":[{"id":"triage-explore","name":"Triage or explore","plain":"Trigger: an unclear ticket. Does: classifies whether its requirements are clear enough to assign. Next: a ready task or human clarification.","sources":[{"label":"GitHub, Best practices for using GitHub Copilot to work on tasks","url":"https://docs.github.com/en/copilot/tutorials/cloud-agent/get-the-best-results","kind":"maker","page":"Making sure your issues are well-scoped","quote":"An ideal task includes: A clear description of the problem to be solved or the work required.","work":"github-copilot-task-practices"},{"label":"GitHub, Best practices for using GitHub Copilot to work on tasks","url":"https://docs.github.com/en/copilot/tutorials/cloud-agent/get-the-best-results","kind":"maker","page":"Choosing the right type of tasks to give to Copilot","quote":"Tasks lacking clear definition: tasks with ambiguous requirements, open-ended tasks, tasks that require working through uncertainty to find a solution","work":"github-copilot-task-practices"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/07-overnight.md","kind":"practitioner","page":"Let loops start themselves","quote":"Software maintenance splits into stages that suit this well: triage a report, reproduce it, fix it, verify the fix.","work":"lauren-tan-pstack"},{"label":"Kent C. Dodds, Introducing Kody: Your Personal Software Factory","url":"https://kentcdodds.com/blog/introducing-kody-your-personal-software-factory","kind":"practitioner","page":"P.S.","quote":"Just ask your agent to submit feedback. My kody kicks off an agent to triage the feedback automatically.","work":"kent-c-dodds-introducing-kody"}]},{"id":"implement","name":"Implement","plain":"Trigger: a scoped task. Does: researches the repository, plans, and makes iterative changes on a branch before opening a pull request. Next: pull request review.","sources":[{"label":"GitHub, Best practices for using GitHub Copilot to work on tasks","url":"https://docs.github.com/en/copilot/tutorials/cloud-agent/get-the-best-results","kind":"maker","page":"Researching, planning, and iterating before opening a pull request","quote":"you can use Copilot cloud agent to research a repository, create an implementation plan, and make iterative code changes on a branch first.","work":"github-copilot-task-practices"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Popular use cases","quote":"Picks up ready tickets from Linear or Jira one at a time and drives each to an open pull request.","work":"factory-software-factory"}]},{"id":"automated-review-task","name":"Automated review","plain":"Trigger: a new or updated pull request. Does: checks it with a fresh model and posts a review finding. Next: rework or human review.","sources":[{"label":"Anthropic, Best practices for Claude Code","url":"https://code.claude.com/docs/en/best-practices","kind":"maker","page":"Give Claude a way to verify its work","quote":"By a second opinion: a verification subagent or a dynamic workflow that checks its own findings has a fresh model try to refute the result","work":"anthropic-claude-code-best-practices"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Popular use cases","quote":"Reviews new pull or merge requests and posts one review comment per head commit.","work":"factory-software-factory"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents","quote":"Agents can then pause for human feedback at checkpoints or when encountering blockers.","work":"anthropic-building-effective-agents"}]},{"id":"rework","name":"Rework","plain":"Trigger: a pull request needs further work. Does: makes the requested change and pushes it. Next: another review or merge.","sources":[{"label":"GitHub, Best practices for using GitHub Copilot to work on tasks","url":"https://docs.github.com/en/copilot/tutorials/cloud-agent/get-the-best-results","kind":"maker","page":"Using comments to iterate on a pull request","quote":"it’s common for the pull request to need further work before it can be merged.","work":"github-copilot-task-practices"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Popular use cases","quote":"Makes the change a teammate asks for in a pull request comment, pushes one commit, and replies.","work":"factory-software-factory"}]},{"id":"decompose","name":"Decompose","plain":"Trigger: a task can be split into subtasks. Does: decomposes it into a sequence whose steps pass outputs forward. Next: the following step.","sources":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Workflow: Prompt chaining","quote":"Prompt chaining decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one.","work":"anthropic-building-effective-agents"}]},{"id":"ratchet-task","name":"Ratchet","plain":"Trigger: a stricter check is enabled with known violations. Does: suppresses existing violations while surfacing new ones. Next: prune fixed suppressions.","sources":[{"label":"ESLint, Command Line Interface: Suppress Violations","url":"https://eslint.org/docs/latest/use/command-line-interface","kind":"maker","page":"--suppress-all","quote":"It allows you to enable one or more lint rules and be notified only when new violations show up.","work":"eslint-cli-suppressions"},{"label":"ESLint, Command Line Interface: Suppress Violations","url":"https://eslint.org/docs/latest/use/command-line-interface","kind":"maker","page":"--prune-suppressions","quote":"Prune unused suppressions from the suppressions file. This option is useful when you addressed one or more of the suppressed violations.","work":"eslint-cli-suppressions"}]},{"id":"dependency-update","name":"Dependency update","plain":"Trigger: a package schedule or update alert. Does: monitors package ecosystems and creates pull requests. Next: status checks and review.","sources":[{"label":"GitHub, Dependabot options reference","url":"https://docs.github.com/en/code-security/reference/supply-chain-security/dependabot-options-reference","kind":"maker","page":"Required keys","quote":"Use these options to customize how Dependabot monitors package ecosystems, schedules updates, and creates pull requests.","work":"github-dependabot-options"},{"label":"GitHub, Dependabot options reference","url":"https://docs.github.com/en/code-security/reference/supply-chain-security/dependabot-options-reference","kind":"maker","page":"Required keys","quote":"All dependencies defined in lock files with vulnerable dependencies are updated by security updates.","work":"github-dependabot-options"},{"label":"Renovate, Configuration Options","url":"https://docs.renovatebot.com/configuration-options/","kind":"maker","page":"Configuration Options","quote":"In that case Renovate first creates a branch and associated Pull Request, and then automerges the PR on a subsequent run once it detects the PR's status checks are \"green\".","work":"renovate-configuration-options"}]},{"id":"fix-factory","name":"Fix the factory","plain":"Trigger: a failed CI run. Does: classifies the failure and, when safe, drives it back to green or opens a fix pull request. Next: automated review.","sources":[{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Guided setups","quote":"watches your open pull requests and drives failing CI to green.","work":"factory-software-factory"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Popular use cases","quote":"investigates the errors that matter, and opens a fix PR when safe.","work":"factory-software-factory"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Popular use cases","quote":"Classifies failed GitHub Actions or GitLab pipelines as flaky, infrastructure, or regression, and names the responsible commit.","work":"factory-software-factory"},{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Watch the outcomes","quote":"Set things up so that failures get noticed and triaged (often by another agent) instead of silently rotting.","work":"kent-c-dodds-you-own-the-outcome"}]},{"id":"architecture-sweep","name":"Architecture sweep","plain":"Trigger: an architectural fitness check. Does: objectively assesses an architectural characteristic with tests or metrics. Next: an integrity assessment.","sources":[{"label":"Neal Ford, Rebecca Parsons and Patrick Kua, Architectural Fitness Functions","url":"https://evolutionaryarchitecture.com/ffkatas/","kind":"practitioner","page":"Architectural Fitness Functions","quote":"Fitness functions encompass existing engineering practices such as testing and metrics","work":"evolutionaryarchitecture-fitness-functions"},{"label":"Neal Ford, Rebecca Parsons and Patrick Kua, Architectural Fitness Functions","url":"https://evolutionaryarchitecture.com/ffkatas/","kind":"practitioner","page":"Architectural Fitness Functions","quote":"An architectural fitness function provides an objective integrity assessment of some architectural characteristic(s).","work":"evolutionaryarchitecture-fitness-functions"}]},{"id":"retro","name":"Retro","plain":"Trigger: a measured result. Does: measures performance and iterates on implementation. Next: another measured result.","sources":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Combining and customizing these patterns","quote":"the key to success, as with any LLM features, is measuring performance and iterating on implementations.","work":"anthropic-building-effective-agents"}]}]}],"sources":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"What are agents?","quote":"Workflows are systems where LLMs and tools are orchestrated through predefined code paths.","work":"anthropic-building-effective-agents"},{"label":"GitHub, Best practices for using GitHub Copilot to work on tasks","url":"https://docs.github.com/en/copilot/tutorials/cloud-agent/get-the-best-results","kind":"maker","page":"Making sure your issues are well-scoped","quote":"GitHub Copilot provides better results when assigned clear, well-scoped tasks.","work":"github-copilot-task-practices"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"What an automation is","quote":"Each automation starts a Droid session when its trigger fires: on a schedule, when a message lands in a Slack channel, on a GitHub event, or when another service sends a webhook.","work":"factory-software-factory"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/handoff","kind":"practitioner","page":"Handoff","quote":"Transferring agent context from one session to another, with no return path. Carry mechanism varies — artifact, compaction, others.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/handoff-artifact","kind":"practitioner","page":"Handoff artifact","quote":"Specs, tickets, and plan docs are all handoff artifacts.","work":"matt-pocock-ai-coding-dictionary"}],"id":"task","look":{"parts":[{"slot":"card-back","shape":"box","size":[1.45,1.18,0.06],"pos":[0,0.59,0],"radius":0.06,"color":"#b8ad8f","material":"matte"},{"slot":"card","shape":"plane","size":[1.36,1.1],"pos":[0,0.62,0.04],"svg":"TASKTriggerlabel: readyDoesimplementNextreviewHumanon PRbounded, repeatable work"},{"slot":"trigger","shape":"cone","radius":0.08,"height":0.22,"pos":[-0.55,0.12,0.08],"rot":[0,0,180],"color":"#e8743b","material":"glow"},{"slot":"next","shape":"cone","radius":0.08,"height":0.22,"pos":[0.55,0.12,0.08],"rot":[0,0,180],"color":"#2e9b4f","material":"glow"}]}},{"term":"Run","aka":["execution","agent run","session","AFK session"],"kind":"concept","plain":"One execution of one task. The agent uses tools in a loop, then the run ends and leaves anything needed later in backing services such as the tracker and the git host.","does":"Gives each task a clear start and end, and an outcome that the next step can act on.","ask":"What started this run, what changed, what evidence did it leave, and what event follows?","cues":["A run page with trigger, timestamps, status, logs, and artifacts","A link from one ticket or pull request to one execution","A finished status with a next event or human handoff"],"at":[-0.42,0.48,0.12],"links":[{"to":"runner","rel":"see also"},{"to":"run-state","rel":"measured by"},{"to":"agent-harness","rel":"made by"},{"to":"human-re-entry","rel":"see also"}],"pattern":"pattern-stateless-worker","sources":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents","quote":"They are typically just LLMs using tools based on environmental feedback in a loop.","work":"anthropic-building-effective-agents"},{"label":"Adam Wiggins, The Twelve-Factor App: Processes","url":"https://www.12factor.net/processes","kind":"standard","page":"VI. Processes","quote":"Twelve-factor processes are stateless and share-nothing. Any data that needs to persist must be stored in a stateful backing service, typically a database.","work":"twelve-factor-processes"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Events feed","quote":"Runs Each run's outcome and duration, with a link to its session.","work":"factory-software-factory"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"What an automation is","quote":"Each automation starts a Droid session when its trigger fires: on a schedule, when a message lands in a Slack channel, on a GitHub event, or when another service sends a webhook.","work":"factory-software-factory"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/session","kind":"practitioner","page":"Session","quote":"One bounded run of interaction with an agent. Starts empty, accumulates messages, tool results, and files read, and ends when cleared, closed, or compacted into a fresh session.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/afk","kind":"practitioner","page":"AFK","quote":"Away from keyboard. A working pattern where the user kicks off a session and leaves the agent to run unattended.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/stateless","kind":"practitioner","page":"Stateless","quote":"An agent is stateless across sessions by default: a new session starts empty, with no trace of prior ones.","work":"matt-pocock-ai-coding-dictionary"}],"id":"run","look":{"parts":[{"slot":"panel-back","shape":"box","size":[1.4,1,0.06],"pos":[0,0.5,0],"radius":0.05,"color":"#2a3a4c","material":"matte"},{"slot":"run-page","shape":"plane","size":[1.32,0.92],"pos":[0,0.52,0.04],"svg":"RUN #482SUCCESStrigger: label ready✓checkout✓agent: implement✓tests✓open pull request"},{"slot":"status-light","shape":"sphere","radius":0.07,"pos":[0.62,1.06,0.06],"color":"#2e9b4f","material":"glow"}]}},{"term":"Run chain","aka":["ticket to merge","the pipeline"],"kind":"loop","plain":"A trigger starts a run. The run's change to the tracker or git host triggers the next run, through checks, automated review and human review, until a pull request merges. Each run hands its evidence to the next, such as screenshots of the bug or proof that the fix works. A new ticket starts the cycle again. The simplest chain is a loop in which each run picks the next task from a plan and commits it.","does":"Joins short, stateless runs into one flow from ticket to merged code, with a gate or a human at each handoff.","ask":"Which change does each run make that starts the next one, and where can the chain stop?","trouble":"The chain stops silently between two runs, or one run's change starts two runs at once.","loop":{"type":"cycle","through":["trigger","run","automated-checks","automated-review","human-review","pull-request"]},"sources":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Workflow: Prompt chaining","quote":"Prompt chaining decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one.","work":"anthropic-building-effective-agents"},{"label":"GitHub, Events that trigger workflows","url":"https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows","kind":"maker","page":"About events that trigger workflows","quote":"Workflow triggers are events that cause a workflow to run.","work":"github-actions-events"},{"label":"Martin Fowler, Continuous Integration","url":"https://martinfowler.com/articles/continuousIntegration.html","kind":"practitioner","page":"Every Push to Mainline Should Trigger a Build","quote":"Every time the mainline receives a commit, the CI service checks out the head of the mainline into an integration environment and performs a full build.","work":"martin-fowler-continuous-integration"},{"label":"Martin Fowler, Continuous Delivery","url":"https://martinfowler.com/bliki/ContinuousDelivery.html","kind":"practitioner","page":"Continuous Delivery","quote":"You achieve continuous delivery by continuously integrating the software done by the development team, building executables, and running automated tests on those executables to detect problems.","work":"martin-fowler-continuous-delivery-article"},{"label":"Jez Humble and David Farley, Continuous Delivery","url":"https://www.informit.com/articles/article.aspx?p=1621865","kind":"book","isbn":"0321601912","page":"Anatomy of the Deployment Pipeline","quote":"The deployment pipeline is the key pattern that enables continuous delivery.","work":"martin-fowler-continuous-delivery"},{"label":"GitHub, About protected branches","url":"https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches","kind":"maker","page":"Require pull request reviews before merging","quote":"If you enable required reviews, collaborators can only push changes to a protected branch via a pull request that is approved by the required number of reviewers","work":"github-protected-branches"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/07-overnight.md","kind":"practitioner","page":"Let loops start themselves","quote":"Every stage hands over evidence. Repro attaches screenshots and video of the broken state, and the fix attaches before-and-after proof.","work":"lauren-tan-pstack"},{"label":"Matt Pocock, Getting Started With Ralph","url":"https://www.aihero.dev/getting-started-with-ralph","kind":"practitioner","page":"Getting Started With Ralph","quote":"Ralph is a technique for running AI coding agents in a loop. You run the same prompt repeatedly. The AI picks its own tasks from a PRD. It commits after each feature.","work":"matt-pocock-getting-started-with-ralph"}],"id":"run-chain"},{"term":"Verification","aka":["proof","evidence","prove it works","validation harness"],"kind":"part","plain":"The evidence that a change does what its spec asked. Scenarios run end to end against the real app or a twin of its outside services, and a run that did not write the code judges the result. When the evidence is strong, a person can check the outcome instead of reading the diff.","does":"Moves trust from reading code to checking evidence, so human review can leave the critical path for low-risk work.","ask":"What evidence shows this change works, which run produced it, and could the author have gamed it?","cues":["Scenario results and captured evidence attached to a pull request","A verifier run that is separate from the author run","A pass rate against scenarios the author never saw"],"at":[1.5,1.12,0.3],"children":["spec","scenario","holdout-scenario","digital-twin","self-verification","satisfaction","non-determinism"],"links":[{"to":"review-queue","rel":"lowers"},{"to":"merge-policy","rel":"see also"}],"sources":[{"label":"Simon Willison, How StrongDM’s AI team build serious software without even looking at the code","url":"https://simonwillison.net/2026/Feb/7/software-factory/","kind":"practitioner","quote":"I’m particularly invested in the question of what it takes to have agents prove that their code works without needing to review every line of code they produce.","work":"simon-willison-software-factory"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/06-verify-and-ship.md","kind":"practitioner","page":"Verify the result and open a PR","quote":"Verification is the slowest step in most agent work, because it's the step that usually waits on a human.","work":"lauren-tan-pstack"},{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/principles","kind":"maker","page":"The Principles","quote":"Your validation harness must be end-to-end, as close to the real environment as possible","work":"strongdm-software-factory"},{"label":"Delinea, The Delinea Software Factory: Building Software with AI","url":"https://delinea.com/blog/the-delinea-software-factory-building-software-with-ai","kind":"maker","page":"The Delinea Software Factory","quote":"It’s what happens when validation replaces code review.","work":"delinea-software-factory"},{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Shipping a feature without writing it","quote":"I didn't read that diff line by line. I read the outcome: does it do what we decided it should do, and did the gates I trust pass? That's only reasonable because of everything I'd set up ahead of time","work":"kent-c-dodds-you-own-the-outcome"}],"id":"verification","look":{"parts":[{"slot":"belt","shape":"box","size":[3.2,0.14,0.7],"pos":[0,0.22,0.2],"color":"#3a3f47","material":"matte"},{"slot":"parcel","shape":"box","size":[0.42,0.3,0.36],"pos":[-0.85,0.44,0.2],"color":"#c9a46a","material":"matte","radius":0.02},{"slot":"parcel-tag","shape":"plane","size":[0.3,0.18],"pos":[-0.85,0.45,0.385],"svg":"EVIDENCE✓ ran the app✓ screenshots"},{"slot":"scanner","shape":"box","size":[0.8,0.62,0.85],"pos":[0,0.6,0.2],"color":"#9cc4f0","material":"glass","opacity":0.28},{"slot":"scan-beam","shape":"box","size":[0.04,0.56,0.76],"pos":[0,0.6,0.2],"color":"#7fe08a","material":"glow","opacity":0.55},{"slot":"score-monitor","shape":"box","size":[0.66,0.42,0.06],"pos":[0,1.13,0.3],"color":"#2a2a2e","material":"matte","radius":0.03},{"slot":"score-screen","shape":"plane","size":[0.6,0.36],"pos":[0,1.13,0.335],"material":"screen","svg":"SCENARIOS47 / 50satisfaction 0.94"},{"slot":"spec-board","shape":"box","size":[0.44,0.58,0.03],"pos":[-1.3,0.47,-0.45],"rot":[-10,0,0],"color":"#8a6a3e","material":"matte"},{"slot":"spec-sheet","shape":"plane","size":[0.38,0.48],"pos":[-1.3,0.46,-0.43],"rot":[-10,0,0],"svg":"SPECAcceptanceuser can retryerror is loggedno double chargeOut of scopenew payment page"},{"slot":"safe","shape":"box","size":[0.5,0.5,0.45],"pos":[0.95,0.42,-0.45],"color":"#59626e","material":"metal","radius":0.03},{"slot":"safe-door","shape":"plane","size":[0.42,0.42],"pos":[0.95,0.42,-0.22],"svg":"HOLDOUT"},{"slot":"service","shape":"box","size":[0.28,0.34,0.28],"pos":[1.5,0.34,-0.45],"color":"#3b6ea8","material":"gloss","radius":0.02},{"slot":"twin","shape":"box","size":[0.28,0.34,0.28],"pos":[1.85,0.34,-0.45],"color":"#9cc4f0","material":"glass","opacity":0.35,"radius":0.02}]}},{"term":"Spec","aka":["specification","PRD","intent"],"kind":"part","plain":"A written description of the shared design concept: what a piece of work must do and how anyone will know it is done. It carries the intent across sessions, so the author run, the verifier and the person all work toward the same target.","does":"Gives verification a target that exists before the code does.","ask":"Does the spec say what must be true when the work is done, in terms a scenario can check?","cues":["A spec or PRD file linked from the ticket","Acceptance criteria written as statements a test can check","Out-of-scope notes and open questions"],"at":[-1.3,0.5,-0.4],"variants":[{"label":"Spec technique","items":[{"id":"grilling","name":"Grilling","plain":"The agent interviews the person one decision at a time and delays the handoff artifact until the design concept stabilises.","why":"Use it when the open questions are decisions a person can settle in conversation.","sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/grilling","kind":"practitioner","page":"Grilling","quote":"A technique for developing a design concept with an agent: the agent interviews the user Socratically, one decision at a time, proposing a recommended answer for each.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/grilling","kind":"practitioner","page":"Grilling","quote":"Slows the rush to a finished plan — no handoff artifact is written until the concept stabilises.","work":"matt-pocock-ai-coding-dictionary"}]},{"id":"prototyping","name":"Prototyping","plain":"The agent builds a quick, rough version when conversation is too low-fidelity, so the person can react to a real artifact.","why":"Use it when a question needs a real artifact to answer, such as how an interaction feels or whether an API shape reads well in calling code.","sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/prototyping","kind":"practitioner","page":"Prototyping","quote":"Having the agent build a quick, rough version of something, for when conversation is too low-fidelity and you need a real artifact to talk about.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/prototyping","kind":"practitioner","page":"Prototyping","quote":"Grilling resolves design decisions in conversation. Conversation is cheap, but it's low-fidelity: some questions can't be answered in words — how an interaction feels, whether an API shape is ergonomic in real calling code, whether the layout works at real data sizes.","work":"matt-pocock-ai-coding-dictionary"}]}]}],"sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/spec","kind":"practitioner","page":"Spec","quote":"A handoff artifact describing a multi-session piece of work","work":"matt-pocock-ai-coding-dictionary"},{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/","kind":"maker","page":"Software Factories And The Agentic Moment","quote":"non-interactive development where specs + scenarios drive agents that write code, run harnesses, and converge without human review.","work":"strongdm-software-factory"},{"label":"Delinea, The Delinea Software Factory: Building Software with AI","url":"https://delinea.com/blog/the-delinea-software-factory-building-software-with-ai","kind":"maker","page":"The Delinea Software Factory","quote":"Humans define intent: what the system should do, the scenarios it needs to handle, the constraints that matter.","work":"delinea-software-factory"},{"label":"Dan Shapiro, The Five Levels: from Spicy Autocomplete to the Dark Factory","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"practitioner","page":"Level 4","quote":"You write a spec. You argue with it about the spec. … Then you leave for 12 hours, and check to see if the tests pass.","work":"dan-shapiro-five-levels"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/design-concept","kind":"practitioner","page":"Design concept","quote":"The shared understanding of what's being built, held in common between user and agent but separate from any asset.","work":"matt-pocock-ai-coding-dictionary"}],"id":"spec","look":{"parts":[{"slot":"clipboard","shape":"box","size":[0.8,1.05,0.04],"pos":[0,0.55,0],"color":"#8a6a3e","material":"matte","radius":0.03},{"slot":"clip","shape":"box","size":[0.3,0.08,0.06],"pos":[0,1.05,0.03],"color":"#8a8f98","material":"metal","radius":0.01},{"slot":"sheet","shape":"plane","size":[0.72,0.9],"pos":[0,0.52,0.021],"svg":"SPECRetry failed paymentsAcceptance criteriauser can retry onceerror is loggedno double chargeOut of scopenew payment pageOpen: refund rule?"}]}},{"term":"Scenario","aka":["end-to-end user story","acceptance scenario","scenario test"],"kind":"part","plain":"A user story, written down, that a harness runs from start to finish against the built software. It checks what a user would see, and a model can judge results that are hard to assert exactly.","does":"Tests the whole change the way a user meets it, as close to the real environment as the factory can get.","ask":"Does each scenario run end to end against the real app or a faithful twin of its services?","cues":["A folder of scenario files written as user stories","Scenario runs listed beside the unit test results","A model's judgement recorded for each scenario run"],"at":[0,0.6,0.5],"links":[{"to":"spec","rel":"made from"}],"sources":[{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/","kind":"maker","page":"Software Factories And The Agentic Moment","quote":"We repurposed the word scenario to represent an end-to-end \"user story\", often stored outside the codebase (similar to a \"holdout\" set in model training), which could be intuitively understood and flexibly validated by an LLM.","work":"strongdm-software-factory"},{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/principles","kind":"maker","page":"The Principles","quote":"Your validation harness must be end-to-end, as close to the real environment as possible","work":"strongdm-software-factory"},{"label":"Simon Willison, How StrongDM’s AI team build serious software without even looking at the code","url":"https://simonwillison.net/2026/Feb/7/software-factory/","kind":"practitioner","quote":"Their scenario tests became scripts for agents to constantly execute against the new systems as they were being built.","work":"simon-willison-software-factory"}],"id":"scenario","look":{"parts":[{"slot":"stack","shape":"box","size":[0.92,0.62,0.12],"pos":[0.07,0.43,-0.1],"color":"#e9e4d4","material":"matte","radius":0.03},{"slot":"card","shape":"box","size":[0.92,0.62,0.02],"pos":[0,0.45,0.01],"color":"#fdfaf2","material":"matte","radius":0.03},{"slot":"card-face","shape":"plane","size":[0.88,0.58],"pos":[0,0.45,0.021],"svg":"SCENARIO 12A customer whose card failsretries and pays once.judge: satisfied"}]}},{"term":"Holdout scenario","aka":["hidden test","holdout set"],"kind":"part","plain":"A scenario kept where the coding agent cannot read it. Only the verifier runs it, so the author cannot shape its code to pass that test alone.","does":"Catches an agent that wrote code to pass the tests it could see, and not to do the job.","ask":"Which scenarios can the author run read, and which ones does only the verifier hold?","cues":["Scenarios stored outside the repository the agent works in","A verifier run that reports results the author never saw","The loop ends only when the hidden scenarios pass"],"at":[0.95,0.42,-0.2],"pattern":"pattern-holdout-set","sources":[{"label":"Simon Willison, How StrongDM’s AI team build serious software without even looking at the code","url":"https://simonwillison.net/2026/Feb/7/software-factory/","kind":"practitioner","quote":"That idea of treating scenarios as holdout sets … used to evaluate the software but not stored where the coding agents can see them … is fascinating.","work":"simon-willison-software-factory"},{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/principles","kind":"maker","page":"The Principles","quote":"The loop runs until the holdout scenarios pass (and stay passing)","work":"strongdm-software-factory"},{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/","kind":"maker","page":"Software Factories And The Agentic Moment","quote":"We repurposed the word scenario to represent an end-to-end \"user story\", often stored outside the codebase (similar to a \"holdout\" set in model training)","work":"strongdm-software-factory"}],"id":"holdout-scenario","look":{"parts":[{"slot":"safe","shape":"box","size":[0.8,0.8,0.7],"pos":[0,0.4,0],"color":"#59626e","material":"metal","radius":0.04},{"slot":"safe-door","shape":"plane","size":[0.68,0.68],"pos":[0,0.4,0.351],"svg":"HOLDOUTVERIFIER ONLY"},{"slot":"card","shape":"box","size":[0.42,0.16,0.01],"pos":[0,0.86,0],"color":"#fdfaf2","material":"matte"}]}},{"term":"Digital twin","aka":["behavioral clone","service twin"],"kind":"part","plain":"A local service that copies the behavior and edge cases of an outside dependency at its boundary. Scenarios can run against it quickly and safely instead of calling the live service.","does":"Makes integration verification fast, repeatable and safe at factory scale.","ask":"Which boundary must the twin reproduce, including its failures and observable behavior?","cues":["A local service with the same API boundary as a dependency","Fixtures for success, failure and edge cases","Scenario runs that use the twin instead of production"],"at":[1.85,0.34,-0.3],"sources":[{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/techniques/dtu","kind":"maker","page":"Digital Twin Universe","quote":"We built twins of Okta, Jira, Slack, Google Docs, Google Drive, and Google Sheets, replicating their APIs, edge cases, and observable behaviors.","work":"strongdm-software-factory"},{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/techniques/dtu","kind":"maker","page":"Digital Twin Universe","quote":"The key insight is to replicate behavior at the boundary.","work":"strongdm-software-factory"},{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/","kind":"maker","page":"Software Factories And The Agentic Moment","quote":"We can run thousands of scenarios per hour without hitting rate limits, triggering abuse detection, or accumulating API costs.","work":"strongdm-software-factory"}],"id":"digital-twin","look":{"parts":[{"slot":"service","shape":"box","size":[0.6,0.7,0.5],"pos":[-0.42,0.35,0],"color":"#3b6ea8","material":"gloss","radius":0.03},{"slot":"service-label","shape":"plane","size":[0.5,0.18],"pos":[-0.42,0.55,0.251],"svg":"Payments API"},{"slot":"twin","shape":"box","size":[0.6,0.7,0.5],"pos":[0.42,0.35,0],"color":"#9cc4f0","material":"glass","opacity":0.35,"radius":0.03},{"slot":"twin-label","shape":"plane","size":[0.5,0.18],"pos":[0.42,0.55,0.251],"svg":"Payments twin"},{"slot":"fixtures","shape":"plane","size":[0.5,0.3],"pos":[0.42,0.25,0],"svg":"200402timeout"}]}},{"term":"Self-verification","aka":["run your own change","author proof","evidence capture"],"kind":"process","plain":"The agent runs its change through the repository's checks and real user paths, then claims only what it actually verified. A separate verifier judges the result because an author naturally checks what it meant to build.","does":"Gives the next gate evidence from an exercised change instead of a summary from its author.","ask":"What did the agent run, what evidence did it capture, and who other than the author judged it?","cues":["A run that starts the app and exercises the changed path","Evidence links attached to the run or pull request","A fresh verifier that reads the result cold"],"at":[-0.85,0.45,0.4],"links":[{"to":"control-cli","rel":"made from"},{"to":"automated-review","rel":"see also"}],"sources":[{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"Agents run their own change before merge and claim only what they verified.","work":"kent-c-dodds-kcd-skills"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/references/verification.md","kind":"practitioner","page":"autopilot skill, references/verification.md","quote":"An agent that starts the app, signs in, and asserts on real responses catches its own bugs before merge.","work":"kent-c-dodds-kcd-skills"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/skills/poteto-mode/playbooks/shipping.md","kind":"practitioner","page":"Shipping","quote":"Only a verdict from an agent that did not write the code counts, because an author checks what it meant to build.","work":"lauren-tan-pstack"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/skills/poteto-mode/playbooks/shipping.md","kind":"practitioner","page":"Shipping","quote":"Green CI and an approving bot review are not verdicts, because neither runs the change as a user would.","work":"lauren-tan-pstack"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/hallucination","kind":"practitioner","page":"Hallucination","quote":"Hallucinated code is plausible by construction: it's what the API would look like if it existed, which is exactly what makes it slip past a skim-level review and fail only when run.","work":"matt-pocock-ai-coding-dictionary"}],"id":"self-verification","look":{"parts":[{"slot":"parcel","shape":"box","size":[0.8,0.55,0.6],"pos":[0,0.275,0],"color":"#c9a46a","material":"matte","radius":0.03},{"slot":"tag","shape":"plane","size":[0.56,0.34],"pos":[0,0.29,0.302],"svg":"EVIDENCE✓ started the app✓ ran the changed path✓ screenshots, logs"},{"slot":"camera","shape":"box","size":[0.34,0.22,0.2],"pos":[0.12,0.66,0],"color":"#2a2a2e","material":"gloss","radius":0.03},{"slot":"lens","shape":"cylinder","radiusTop":0.07,"radiusBottom":0.08,"height":0.08,"pos":[0.12,0.66,0.13],"rot":[90,0,0],"color":"#5a5f68","material":"metal"}]}},{"term":"Satisfaction","aka":["behavioral pass rate","user-fit measure","trajectory score"],"kind":"measure","plain":"The fraction of observed scenario trajectories that probably satisfy the user. It measures behavior across the whole run instead of counting only assertions.","does":"Shows whether the factory's changes satisfy users, even when a scenario needs judgement rather than an exact value.","ask":"Of all observed scenario trajectories, what fraction likely satisfies the user?","cues":["A scenario score with the evidence that led to it","A trend split by scenario and release","A threshold that blocks or permits the next run"],"at":[0,1.13,0.35],"sources":[{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/","kind":"maker","page":"Software Factories And The Agentic Moment","quote":"We use the term satisfaction to quantify this validation: of all the observed trajectories through all the scenarios, what fraction of them likely satisfy the user?","work":"strongdm-software-factory"}],"id":"satisfaction","look":{"parts":[{"slot":"dial","shape":"box","size":[0.9,0.7,0.06],"pos":[0,0.55,0],"color":"#2a2a2e","material":"matte","radius":0.04},{"slot":"dial-face","shape":"plane","size":[0.84,0.64],"pos":[0,0.55,0.031],"svg":"SATISFACTION0.94threshold 0.90"}]}},{"term":"Non-determinism","aka":["nondeterminism","run-to-run variance","different answer every run"],"kind":"concept","plain":"The same ticket on the same code can produce a different change on each run.","does":"Makes one passing run weak evidence, so the factory repeats scenarios and reports scores across several trials.","ask":"How many trials passed, and did each run reach the same user outcome?","range":"Use pass@k when one successful attempt is enough, and pass^k when every trial must succeed.","trouble":"A change passes one trial and fails or behaves differently on another, or a score is reported from one trial only.","at":[0.4,1.13,0.3],"sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/non-determinism","kind":"practitioner","page":"Non-determinism","quote":"The same input can produce different output. Run a model twice with identical context and you may get two different answers — sometimes a word, sometimes a completely different approach. Nothing in your code has to change for this to happen.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Anthropic, Demystifying evals for AI agents","url":"https://www.anthropic.com/research/demystifying-evals-for-ai-agents","kind":"maker","page":"The structure of an evaluation","quote":"Each attempt at a task is a trial. Because model outputs vary between runs, we run multiple trials to produce more consistent results.","work":"anthropic-demystifying-evals"},{"label":"Anthropic, Demystifying evals for AI agents","url":"https://www.anthropic.com/research/demystifying-evals-for-ai-agents","kind":"maker","page":"How to evaluate AI agents","quote":"pass@k measures the likelihood that an agent gets at least one correct solution in k attempts.","work":"anthropic-demystifying-evals"},{"label":"Anthropic, Demystifying evals for AI agents","url":"https://www.anthropic.com/research/demystifying-evals-for-ai-agents","kind":"maker","page":"How to evaluate AI agents","quote":"pass^k measures the probability that all k trials succeed.","work":"anthropic-demystifying-evals"},{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/","kind":"maker","page":"From Tests to Scenarios and Satisfaction","quote":"We use the term satisfaction to quantify this validation: of all the observed trajectories through all the scenarios, what fraction of them likely satisfy the user?","work":"strongdm-software-factory"}],"id":"non-determinism","feel":{"shape":0.68,"energy":0.72,"weight":0,"warmth":0.55,"grain":0.78,"motion":"jitter","why":"Jittering motion carries the idea that identical inputs can land on different trajectories and need repeated trials."}},{"term":"Brakes","aka":["quality gates","release gates","backpressure","feedback loop"],"kind":"part","plain":"The gates that slow or stop a run before its change moves forward. They include deterministic checks, a separate automated review and human review, plus the risk class and merge policy that decide which of them each change must pass.","does":"Keep accelerators from flooding the repository with changes that are hard to trust or costly to undo.","ask":"Which independent gate can stop this change, and what evidence does it require?","cues":["Red or green check dots on a pull request","An automated review with findings separate from the author run","A required human approval or a blocked merge button"],"at":[1.85,0.8,0.3],"children":["automated-checks","automated-review","review-rework-loop","human-review","risk-class","merge-policy","review-bottleneck"],"pattern":"pattern-accelerator-brake","sources":[{"label":"Anthropic, Best practices for Claude Code","url":"https://code.claude.com/docs/en/best-practices","kind":"maker","page":"Give Claude a way to verify its work","quote":"Give Claude a check it can run: tests, a build, a screenshot to compare.","work":"anthropic-claude-code-best-practices"},{"label":"Anthropic, Best practices for Claude Code","url":"https://code.claude.com/docs/en/best-practices","kind":"maker","page":"Give Claude a way to verify its work","quote":"By a second opinion: a verification subagent or a dynamic workflow that checks its own findings has a fresh model try to refute the result","work":"anthropic-claude-code-best-practices"},{"label":"GitHub, About protected branches","url":"https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches","kind":"maker","page":"Require pull request reviews before merging","quote":"can require that all pull requests receive a specific number of approving reviews before someone merges the pull request into a protected branch.","work":"github-protected-branches"},{"label":"Matt Pocock, Software Factories for Real Engineers","url":"https://www.aihero.dev/cohorts/software-factories-for-real-engineers-xdy1m","kind":"practitioner","page":"Software Factories for Real Engineers","quote":"Automated checks, automated & human review. Slows the agent down and increases quality. … Before you can trust your agent to go fast, you need to know that the quality of work is going to be high at any speed.","work":"matt-pocock-software-factories"},{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Shipping a feature without writing it","quote":"The agent wrote the implementation, tests, and PR. Automated gates and AI reviewers checked the work.","work":"kent-c-dodds-you-own-the-outcome"},{"label":"Matt Pocock, software-factory skill (pull request #1316)","url":"https://github.com/mattpocock/skills/blob/b59cab42bf1206e1363eafc37f71b21c3bcaa12f/skills/in-progress/software-factory/SKILL.md","kind":"practitioner","page":"Start slow","quote":"Every trigger is an accelerator. Accelerators without brakes make a slop cannon.","work":"matt-pocock-software-factory-skill"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/automated-check","kind":"practitioner","page":"Automated check","quote":"Avoid: \"feedback loop\" / \"backpressure\" … both lump checks together with review.","work":"matt-pocock-ai-coding-dictionary"}],"id":"brakes","look":{"parts":[{"slot":"gate","shape":"box","size":[1.7,0.24,0.72],"pos":[0,0.12,0],"color":"#3a3f47","material":"matte","radius":0.03},{"slot":"bar","shape":"box","size":[0.13,1.05,0.13],"pos":[0.78,0.75,0.1],"color":"#c8432f","material":"gloss","radius":0.03},{"slot":"light","shape":"sphere","radius":0.1,"pos":[0.78,1.37,0.1],"color":"#2e9b4f","material":"glow"},{"slot":"sign","shape":"plane","size":[1.28,0.62],"pos":[-0.1,0.76,0.09],"svg":"checksagent reviewhuman approval, high risk"},{"slot":"tag","shape":"plane","size":[0.42,0.18],"pos":[-0.55,0.12,0.362],"svg":"LOW"},{"slot":"switch","shape":"plane","size":[0.42,0.18],"pos":[0.45,0.12,0.362],"svg":"AUTO-MERGE"}]}},{"term":"Automated checks","aka":["deterministic gates","CI checks"],"kind":"part","plain":"Repeatable checks such as type checks, lint, tests, builds, security scans, and scenario tests. They return evidence that a run or pull request can read.","does":"Catches deterministic failures early and gives rework a concrete signal.","ask":"What must pass before merge, and can the check expose a failure without a human reading the diff?","cues":["Green or red check dots on a pull request","A check name, command, duration, and artifact on a run page","A required status check blocking the merge control"],"at":[-0.58,0.9,0.12],"pattern":"pattern-defense-in-depth","links":[{"to":"branch-protection","rel":"controls"},{"to":"ratchet","rel":"see also"}],"sources":[{"label":"Anthropic, Best practices for Claude Code","url":"https://code.claude.com/docs/en/best-practices","kind":"maker","page":"Give Claude a way to verify its work","quote":"Give Claude a check it can run: tests, a build, a screenshot to compare.","work":"anthropic-claude-code-best-practices"},{"label":"GitHub, About protected branches","url":"https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches","kind":"maker","page":"Require status checks before merging","quote":"After enabling required status checks, all required status checks must pass before collaborators can merge changes into the protected branch.","work":"github-protected-branches"},{"label":"Titus Winters et al., Software Engineering at Google","url":"https://abseil.io/resources/swe-book/html/ch23.html","kind":"book","isbn":"9781492082798","page":"Continuous Integration","quote":"Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.","work":"software-engineering-google"},{"label":"Martin Fowler, Continuous Integration","url":"https://martinfowler.com/articles/continuousIntegration.html","kind":"practitioner","page":"Every Push to Mainline Should Trigger a Build","quote":"We thus need to ensure that every commit is verified in a reference environment.","work":"martin-fowler-continuous-integration"},{"label":"Martin Fowler, Continuous Delivery","url":"https://martinfowler.com/bliki/ContinuousDelivery.html","kind":"practitioner","page":"Continuous Delivery","quote":"Anybody can get fast, automated feedback on the production readiness of their systems any time somebody makes a change to them","work":"martin-fowler-continuous-delivery-article"},{"label":"Neal Ford et al., Building Evolutionary Architectures, 2nd Edition","url":"https://www.thoughtworks.com/content/dam/thoughtworks/documents/books/bk_building_evolutionary_architectures_second_edition_free_chapter.pdf","kind":"book","isbn":"9781492097549","page":"Chapter 1, Evolving Software Architecture (free chapter PDF)","quote":"The fitness function metaphor encompasses a variety of mechanisms we employ to ensure architecture doesn’t change in undesirable ways, including metrics, tests, and other verification tools.","work":"building-evolutionary-architectures"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/skills/principle-prove-it-works/SKILL.md","kind":"practitioner","page":"Script the check when you can","quote":"The strongest proof is a deterministic script that re-runs the same comparison, not a one-time eyeball.","work":"lauren-tan-pstack"},{"label":"Matt Pocock, 11 Tips For AI Coding With Ralph Wiggum","url":"https://www.aihero.dev/tips-for-ai-coding-with-ralph-wiggum","kind":"practitioner","page":"5. Use Feedback Loops","quote":"The best setup blocks commits unless everything passes. Ralph can't declare victory if the tests are red.","work":"matt-pocock-ralph-tips"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/automated-check","kind":"practitioner","page":"Automated check","quote":"Determinism is what makes the loop trustworthy","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, software-factory skill (pull request #1316)","url":"https://github.com/mattpocock/skills/blob/b59cab42bf1206e1363eafc37f71b21c3bcaa12f/skills/in-progress/software-factory/SKILL.md","kind":"practitioner","page":"Start slow","quote":"deterministic gates (types, lint, tests) that every run must pass.","work":"matt-pocock-software-factory-skill"}],"id":"automated-checks","look":{"parts":[{"slot":"rack","shape":"box","size":[1.35,0.72,0.14],"pos":[0,0.4,0],"radius":0.05,"color":"#3a3f47","material":"matte"},{"slot":"check-1","shape":"sphere","radius":0.11,"pos":[-0.42,0.4,0.1],"color":"#2e9b4f","material":"glow"},{"slot":"check-2","shape":"sphere","radius":0.11,"pos":[0,0.4,0.1],"color":"#2e9b4f","material":"glow"},{"slot":"check-3","shape":"sphere","radius":0.11,"pos":[0.42,0.4,0.1],"color":"#e0a526","material":"glow"},{"slot":"label","shape":"plane","size":[1.15,0.3],"pos":[0,0.9,0.08],"svg":"TYPE · LINT · TEST"}]}},{"term":"Automated review","aka":["agent review","second opinion"],"kind":"process","plain":"A separate run checks a proposed change with fresh context and reports findings before human review.","does":"Adds a second, independent judgement, so the run that wrote the change is not the only one that checks it.","ask":"Is the reviewer separate enough to catch what the author missed, and does it report evidence for each finding?","cues":["A review check authored by a different bot or run","Findings linked to changed lines with severity and evidence","A second review status beside deterministic checks"],"at":[-0.58,0.74,0.12],"links":[{"to":"human-review","rel":"see also"},{"to":"review-rework-loop","rel":"see also"}],"sources":[{"label":"Anthropic, Best practices for Claude Code","url":"https://code.claude.com/docs/en/best-practices","kind":"maker","page":"Give Claude a way to verify its work","quote":"a verification subagent or a dynamic workflow that checks its own findings has a fresh model try to refute the result","work":"anthropic-claude-code-best-practices"},{"label":"Addy Osmani, Agentic Code Review","url":"https://addyosmani.com/blog/agentic-code-review/","kind":"practitioner","page":"The tools are good, but not always for the reason they advertise","quote":"A second model with different priors genuinely catches real bugs, and it catches a lot of them","work":"addy-osmani-agentic-code-review"},{"label":"Titus Winters et al., Software Engineering at Google","url":"https://abseil.io/resources/swe-book/html/ch19.html","kind":"book","isbn":"9781492082798","page":"Critique","quote":"Uploading a snapshot of the change triggers code analyzers","work":"software-engineering-google"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents in practice","quote":"However, whereas automated testing helps verify functionality, human review remains crucial for ensuring solutions align with broader system requirements.","work":"anthropic-building-effective-agents"},{"label":"Gene Kim and Steve Yegge, Vibe Coding: Building Production-Grade Software With GenAI, Chat, Agents, and Beyond","url":"https://itrevolution.com/articles/from-line-cookto-head-chef-orchestrating-ai-teams/","kind":"book","isbn":"9781966280026","page":"From Line Cook to Head Chef: Orchestrating AI Teams (book excerpt)","quote":"Generators and verifiers: Separate concerns by creating dedicated agents for implementation versus testing.","work":"gene-kim-steve-yegge-vibe-coding"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/07-overnight.md","kind":"practitioner","page":"When the night holds a queue, not a task","quote":"Each PR gets one owner agent that carries it from build through merge, and no owner merges on its own verdict. A swarm of fresh verifiers checks the owner's code-ready head.","work":"lauren-tan-pstack"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/06-verify-and-ship.md","kind":"practitioner","page":"Land the stack with Shipping","quote":"the agent that judges a change is never the one that wrote it.","work":"lauren-tan-pstack"},{"label":"Matt Pocock, software-factory skill (pull request #1316)","url":"https://github.com/mattpocock/skills/blob/b59cab42bf1206e1363eafc37f71b21c3bcaa12f/skills/in-progress/software-factory/SKILL.md","kind":"practitioner","page":"Start slow","quote":"a separate run that reads the diff cold against the codebase's standards and the ticket, when there is one.","work":"matt-pocock-software-factory-skill"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/automated-review","kind":"practitioner","page":"Automated review","quote":"An agent reviewing another agent's work, often with a different model or system prompt. Non-deterministic: it forms a judgement.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/automated-review","kind":"practitioner","page":"Automated review","quote":"A reviewer with a fresh context window has none of that attachment: it sees the diff the way a stranger would, which is what review depends on.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/sycophancy","kind":"practitioner","page":"Sycophancy","quote":"Biased framing — review skews positive when you signal you wrote it; negative when you signal someone else did. Same artifact, different verdict.","work":"matt-pocock-ai-coding-dictionary"}],"id":"automated-review","look":{"parts":[{"slot":"review-card","shape":"box","size":[1.38,0.9,0.06],"pos":[0,0.54,0],"radius":0.05,"color":"#f3efe4","material":"matte"},{"slot":"review-face","shape":"plane","size":[1.28,0.8],"pos":[0,0.57,0.04],"svg":"COLD REVIEWbug riskmissing testlooks sound"},{"slot":"lens","shape":"torus","radius":0.22,"tube":0.035,"pos":[-0.58,0.12,0.1],"rot":[90,0,0],"color":"#3b6ea8","material":"metal"},{"slot":"lens-handle","shape":"cylinder","radiusTop":0.04,"radiusBottom":0.04,"height":0.3,"pos":[-0.42,0,0.1],"rot":[0,0,-45],"color":"#3b6ea8","material":"metal"}]}},{"term":"Review and rework loop","aka":["evaluator loop","fix and recheck","ping-pong"],"kind":"loop","plain":"An automated review fails a pull request, a rework run fixes what the review named and pushes, and the push starts a new review. A try cap ends the loop and hands the pull request to a person.","does":"Fixes review findings without a human, and stops a disagreement between two runs from looping forever.","ask":"How many tries does the loop get before a human, and what does the human see when it stops?","trouble":"The same finding comes back on every try, or the loop runs all night on one pull request.","pattern":"pattern-accelerator-brake","loop":{"type":"balancing","through":["automated-review","run","automated-checks","try-cap"]},"links":[{"to":"try-cap","rel":"see also"}],"sources":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Workflow: Evaluator-optimizer","quote":"In the evaluator-optimizer workflow, one LLM call generates a response while another provides evaluation and feedback in a loop.","work":"anthropic-building-effective-agents"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents","quote":"Agents can then pause for human feedback at checkpoints or when encountering blockers.","work":"anthropic-building-effective-agents"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents","quote":"it’s also common to include stopping conditions (such as a maximum number of iterations) to maintain control.","work":"anthropic-building-effective-agents"}],"id":"review-rework-loop"},{"term":"Human review","aka":["approval","owner review","code review"],"kind":"process","plain":"A person reads the diff or the changed files before the change merges. Reading the agent's account of what it did does not count. Human review is the slowest gate, so a factory starts with it on every pull request and, as checks and verification earn trust, keeps it for high-risk changes, one-way doors and changes to the gates. For the rest, the person checks the outcome and the evidence.","does":"Supplies accountability and judgement for requirements and consequences that automated gates may miss, and teaches the factory what to check next.","ask":"Who owns the change, what did they actually read, and what can only a person decide here?","cues":["A named reviewer and approval state on a pull request","A review comment or resolved conversation","A merge control blocked until required approval exists"],"at":[-0.58,0.58,0.12],"links":[{"to":"pull-request","rel":"controls"},{"to":"autonomy-level","rel":"see also"}],"variants":[{"label":"Checkpoint","items":[{"id":"review-every-pr","name":"Every pull request","plain":"A person reads every diff before it merges. A factory starts here, and what the reviewer catches becomes new checks.","sources":[{"label":"Matt Pocock, software-factory skill (pull request #1316)","url":"https://github.com/mattpocock/skills/blob/b59cab42bf1206e1363eafc37f71b21c3bcaa12f/skills/in-progress/software-factory/SKILL.md","kind":"practitioner","page":"Start slow","quote":"Add one automated trigger at a time, throttled at first (one bug report in ten, a clock once a day), with human review on every PR.","work":"matt-pocock-software-factory-skill"},{"label":"GitHub, About protected branches","url":"https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches","kind":"maker","page":"Require pull request reviews before merging","quote":"If you enable required reviews, collaborators can only push changes to a protected branch via a pull request that is approved by the required number of reviewers","work":"github-protected-branches"}],"svg":{"approval-face":"EVERY PULL REQUESTdiff read before merge9 pull requests waitingAPPROVE"}},{"id":"review-plan","name":"Plan before work","plain":"A person reads the spec or plan before the run starts, while a change of direction costs minutes.","sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/human-review","kind":"practitioner","page":"Human review","quote":"Review is also cheaper earlier. Reading a plan before work starts, or a small diff mid-flight, takes minutes","work":"matt-pocock-ai-coding-dictionary"},{"label":"Dan Shapiro, The Five Levels: from Spicy Autocomplete to the Dark Factory","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"practitioner","page":"Level 4","quote":"You plan schedules. You review plans.","work":"dan-shapiro-five-levels"}],"svg":{"approval-face":"PLAN REVIEWspec and plan readbefore the run startsAPPROVE PLAN"}},{"id":"review-high-risk","name":"High-risk changes only","plain":"Low- and medium-risk changes merge on checks and evidence. A person reads the diff of high-risk changes, one-way doors and changes to the gates.","sources":[{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Design the playground","quote":"Well-structured PRs, tests, and AI reviewers let a lot of changes ship without you in the middle. Save your attention for the high-risk, one-way-door stuff.","work":"kent-c-dodds-you-own-the-outcome"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/human-review","kind":"practitioner","page":"Human review","quote":"human review is reserved for what only you can judge","work":"matt-pocock-ai-coding-dictionary"}],"svg":{"approval-face":"HIGH RISK ONLYauth change: diff readlow risk: merged on evidenceAPPROVE"}},{"id":"review-none","name":"None (dark factory)","plain":"No person reads the factory's code. Automated checks and automated review are the only gates, and people judge the software by scenarios and outcomes. This is credible only where agents cannot game that evidence, as with holdout scenarios run against twins of outside services.","sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/dark-factory","kind":"practitioner","page":"Dark factory","quote":"A codebase, or part of one, where a software factory writes the code and no human reads it.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/dark-factory","kind":"practitioner","page":"Dark factory","quote":"Automated checks and automated review are the only gates.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/","kind":"maker","page":"Software Factories And The Agentic Moment","quote":"Code must not be reviewed by humans","work":"strongdm-software-factory"},{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/","kind":"maker","page":"Software Factories And The Agentic Moment","quote":"Tests can be reward hacked - we needed validation that was less vulnerable to the model cheating","work":"strongdm-software-factory"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/vibe-coding","kind":"practitioner","page":"Vibe coding","quote":"With human review gone, whatever automated verification still runs — tests, types, automated review — is the only gate the code passes through.","work":"matt-pocock-ai-coding-dictionary"}],"svg":{"approval-face":"NO HUMAN REVIEWchecks and agent review onlypeople watch outcomesAUTO-MERGED"}}]}],"sources":[{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/human-review","kind":"practitioner","page":"Human review","quote":"Reading the diff or the changed files counts; reading the agent's description of what it did does not","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/human-review","kind":"practitioner","page":"Human review","quote":"Agents raise the volume of code produced, so review becomes the bottleneck.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/human-review","kind":"practitioner","page":"Human review","quote":"Where you place the review checkpoint is a human-in-the-loop decision","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, software-factory skill (pull request #1316)","url":"https://github.com/mattpocock/skills/blob/b59cab42bf1206e1363eafc37f71b21c3bcaa12f/skills/in-progress/software-factory/SKILL.md","kind":"practitioner","page":"Start slow","quote":"a human reads the PR before it merges; what it catches becomes new automated checks and sharper automated review.","work":"matt-pocock-software-factory-skill"},{"label":"Addy Osmani, Beyond Vibe Coding: From Coder to AI-Era Developer","url":"https://beyond.addy.ie/","kind":"book","isbn":"9798341634749","page":"Human oversight (author's book site)","quote":"Always maintain human oversight as the final reviewer and approver of all agent work","work":"addy-osmani-beyond-vibe-coding"},{"label":"GitHub, About protected branches","url":"https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches","kind":"maker","page":"Require pull request reviews before merging","quote":"If you enable required reviews, collaborators can only push changes to a protected branch via a pull request that is approved by the required number of reviewers","work":"github-protected-branches"},{"label":"Birgitta Böckeler, To vibe or not to vibe","url":"https://martinfowler.com/articles/exploring-gen-ai/to-vibe-or-not-vibe.html","kind":"practitioner","page":"A sliding scale of review effort","quote":"High probability + high impact + low detectability High level of review is advisable. Assume the AI might be wrong and cover for it.","work":"birgitta-bockeler-to-vibe"},{"label":"Titus Winters et al., Software Engineering at Google","url":"https://abseil.io/resources/swe-book/html/ch09.html","kind":"book","isbn":"9781492082798","page":"Code Review","quote":"Code review is a process in which code is reviewed by someone other than the author, often before the introduction of that code into a codebase.","work":"software-engineering-google"},{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Shipping a feature without writing it","quote":"I didn't read that diff line by line. I read the outcome: does it do what we decided it should do, and did the gates I trust pass?","work":"kent-c-dodds-you-own-the-outcome"}],"id":"human-review","look":{"parts":[{"slot":"approval-card","shape":"box","size":[1.35,0.85,0.06],"pos":[0,0.52,0],"radius":0.05,"color":"#e8743b","material":"matte"},{"slot":"approval-face","shape":"plane","size":[1.24,0.74],"pos":[0,0.55,0.04],"svg":"OWNER REVIEWrisk and intentevidence inspectedAPPROVE"},{"slot":"stamp","shape":"cylinder","radiusTop":0.13,"radiusBottom":0.13,"height":0.1,"pos":[0.56,0.12,0.12],"rot":[90,0,0],"color":"#2e9b4f","material":"gloss"}]}},{"term":"Risk class","aka":["risk tier","blast radius","door type"],"kind":"measure","plain":"A label for how much damage a change can do and how hard it is to undo. A change that only composes existing pieces is low risk. One that extends a piece's behavior or contract is medium. One that adds a piece, a data store, an outside service or a trust boundary, or that crosses a one-way door, is high. The class decides which gates the change must pass and whether an agent may merge it.","does":"Sends low-risk work through trusted automation and keeps human attention for high-risk and one-way-door changes.","ask":"Does this change compose, extend or add, and how costly is it to undo?","cues":["A low, medium or high label on a pull request","A pull request summary that says which pieces the change composes, extends or adds","High-risk pull requests waiting for a named human"],"at":[-0.55,0.12,0.38],"pattern":"pattern-two-way-door","links":[{"to":"merge-policy","rel":"controls"}],"variants":[{"label":"Risk tier","items":[{"id":"low-risk","name":"Low","plain":"The change composes existing pieces as designed. Green checks are enough to merge it.","why":"The change uses pieces that already exist and does not change how they behave.","sources":[{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/references/templates.md","kind":"practitioner","page":"autopilot skill, references/templates.md","quote":"composes = uses existing pieces as designed (low).","work":"kent-c-dodds-kcd-skills"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/.agents/skills/ship-pr/SKILL.md","kind":"maker","page":".agents/skills/ship-pr/SKILL.md","quote":"low: green CI. AI review is optional.","work":"kody-repository"}],"svg":{"tag":"LOW"}},{"id":"medium-risk","name":"Medium","plain":"The change extends an existing piece's behavior or contract. It needs green checks, an AI reviewer and the run's own proof that the change works.","why":"The change alters how an existing piece behaves, so its callers can break.","sources":[{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/references/templates.md","kind":"practitioner","page":"autopilot skill, references/templates.md","quote":"extends = changes an existing piece's behavior or contract (medium).","work":"kent-c-dodds-kcd-skills"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"P2 Risk tiers: low = CI green; medium = +AI reviewer +rung 4 proof; high = +human","work":"kent-c-dodds-kcd-skills"}],"svg":{"tag":"MEDIUM"}},{"id":"high-risk","name":"High","plain":"The change adds a new piece, data store, outside service or trust boundary, or it is a one-way door such as a public API name. A human approves it before merge.","why":"The change is costly to undo or widens what the system can reach.","sources":[{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/references/templates.md","kind":"practitioner","page":"autopilot skill, references/templates.md","quote":"adds = new piece, data store, external service, or trust boundary (high).","work":"kent-c-dodds-kcd-skills"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"P2 Risk tiers: low = CI green; medium = +AI reviewer +rung 4 proof; high = +human","work":"kent-c-dodds-kcd-skills"},{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Then, ask for options instead of an answer","quote":"Two-way doors you can walk through quickly and let the agent run. One-way doors (public API names, data shapes, pricing) are where you slow down and think.","work":"kent-c-dodds-you-own-the-outcome"}],"svg":{"tag":"HIGH"}}]}],"sources":[{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/docs/contributing/architecture/primitives.yaml","kind":"maker","page":"docs/contributing/architecture/primitives.yaml","quote":"wires existing primitives without changing them … changes a primitive's behavior, shape, or contract … introduces a new primitive","work":"kody-repository"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"P2 Risk tiers: low = CI green; medium = +AI reviewer +rung 4 proof; high = +human","work":"kent-c-dodds-kcd-skills"},{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Design the playground","quote":"Automate shipping where the risk is low or medium. Well-structured PRs, tests, and AI reviewers let a lot of changes ship without you in the middle. Save your attention for the high-risk, one-way-door stuff.","work":"kent-c-dodds-you-own-the-outcome"},{"label":"Addy Osmani, Agentic Code Review","url":"https://addyosmani.com/blog/agentic-code-review/","kind":"practitioner","page":"Should we just let AI review more of it?","quote":"How much human you keep is a dial, and you set it by blast radius, not by guilt.","work":"addy-osmani-agentic-code-review"},{"label":"Birgitta Böckeler, To vibe or not to vibe","url":"https://martinfowler.com/articles/exploring-gen-ai/to-vibe-or-not-vibe.html","kind":"practitioner","page":"A sliding scale of review effort","quote":"High probability + high impact + low detectability High level of review is advisable. Assume the AI might be wrong and cover for it.","work":"birgitta-bockeler-to-vibe"}],"id":"risk-class","look":{"parts":[{"slot":"pr-card","shape":"box","size":[1,1.2,0.04],"pos":[0,0.62,0],"color":"#fdf6f1","material":"matte","radius":0.04},{"slot":"pr-face","shape":"plane","size":[0.92,1.12],"pos":[0,0.62,0.021],"svg":"Pull requestRetry failed paymentsThis changecomposes existing piecesextends a contractadds a new piece"},{"slot":"tag","shape":"plane","size":[0.5,0.3],"pos":[0.22,0.2,0.06],"svg":"LOW"}]}},{"term":"Merge policy","aka":["merge authority","agent may merge","auto-merge policy","risk tolerance"],"kind":"knob","plain":"The risk classes an agent may merge on its own once the required checks and evidence pass. The rest wait for a person. A short \"always human\" list names areas such as auth, payments and migrations that always need a human approval, and a change to the gates themselves always goes to a human.","does":"Takes low-risk work out of the human review queue while keeping a person on the changes that need judgement.","ask":"Which risk classes may merge without a human today, which areas always need one, and what evidence would widen the policy?","cues":["A checked-in \"agent may merge\" setting beside an \"always human\" list","Auto-merge enabled on a pull request, waiting for required checks","High-risk or gate-changing pull requests parked for a person"],"range":"Start at none, with a person merging everything. Widen to low risk, then to low and medium, as the evidence holds.","trouble":"The policy widens on hope, or an agent edits the gates that decide what it may merge.","at":[0.45,0.12,0.38],"pattern":"pattern-two-way-door","links":[{"to":"human-review","rel":"controls"},{"to":"review-queue","rel":"lowers"}],"variants":[{"label":"Agent may merge","items":[{"id":"merge-none","name":"None","plain":"A person merges every pull request, whatever its risk class. A factory starts here.","sources":[{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"P2 Agent may merge: low risk (default) | low+medium | none","work":"kent-c-dodds-kcd-skills"},{"label":"Lauren Tan, Noodle","url":"https://raw.githubusercontent.com/poteto/noodle/82d2921c52370f23f29086de81ccfb600939c037/docs/concepts/modes.md","kind":"practitioner","page":"docs/concepts/modes.md","quote":"A new project starts in supervised. Critical infrastructure stays in manual.","work":"lauren-tan-noodle"}],"svg":{"switch":"AGENT MAY MERGENONELOWLOW+MED"}},{"id":"merge-low","name":"Low risk","plain":"Agents merge low-risk pull requests once the checks are green. Medium and high risk wait for a person.","sources":[{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/references/templates.md","kind":"practitioner","page":"autopilot skill, references/templates.md","quote":"Tiers the agent may merge (with --merge) without a human approval.","work":"kent-c-dodds-kcd-skills"},{"label":"GitHub, Automating Dependabot with GitHub Actions","url":"https://docs.github.com/en/code-security/tutorials/secure-your-dependencies/automate-dependabot-with-actions","kind":"maker","page":"Enabling automerge on a pull request","quote":"Here is an example that automerges all patch updates to my-dependency","work":"github-dependabot-actions"}],"svg":{"switch":"AGENT MAY MERGENONELOWLOW+MED"}},{"id":"merge-low-medium","name":"Low and medium","plain":"Agents merge low- and medium-risk pull requests once the checks, an AI review and the run's own proof pass. High risk waits for a person.","sources":[{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/references/templates.md","kind":"practitioner","page":"autopilot skill, references/templates.md","quote":"only low- and medium-risk PRs auto-merge, only with green CI, and within the Agent may merge answer.","work":"kent-c-dodds-kcd-skills"},{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Design the playground","quote":"Automate shipping where the risk is low or medium.","work":"kent-c-dodds-you-own-the-outcome"}],"svg":{"switch":"AGENT MAY MERGENONELOWLOW+MED"}}]}],"sources":[{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"only low/medium risk with green CI auto-merges.","work":"kent-c-dodds-kcd-skills"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"Name the changes that always need your approval (e.g. auth, payments, migrations, public API, store releases)","work":"kent-c-dodds-kcd-skills"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"the human always merges it (it changes the gates).","work":"kent-c-dodds-kcd-skills"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/references/templates.md","kind":"practitioner","page":"autopilot skill, references/templates.md","quote":"High risk, red or pending CI, and anything in Always human park for the human.","work":"kent-c-dodds-kcd-skills"},{"label":"GitHub, Automatically merging a pull request","url":"https://docs.github.com/en/pull-requests/how-tos/merge-and-close-pull-requests/automatically-merging-a-pull-request","kind":"maker","page":"Automatically merging a pull request","quote":"Auto-merge merges a pull request automatically after all required reviews and status checks pass","work":"github-auto-merge"},{"label":"Lauren Tan, Noodle","url":"https://raw.githubusercontent.com/poteto/noodle/82d2921c52370f23f29086de81ccfb600939c037/docs/concepts/modes.md","kind":"practitioner","page":"docs/concepts/modes.md","quote":"The mode is a trust dial, not a capability switch.","work":"lauren-tan-noodle"},{"label":"Mohannad Hammadeh et al. (AWS), Scaling agentic coding past the human bottleneck","url":"https://builder.aws.com/content/3Jzr7Zmo4OxkGxLZ83VofzQqT0W/scaling-agentic-coding-past-the-human-bottleneck","kind":"maker","page":"Scaling agentic coding past the human bottleneck","quote":"A low-risk ticket went the whole way on its own, code read, fix written, pull request raised and merged, ticket updated, with nobody touching a keyboard.","work":"aws-scaling-agentic-coding"}],"id":"merge-policy","look":{"parts":[{"slot":"panel","shape":"box","size":[1.5,0.85,0.06],"pos":[0,0.55,0],"color":"#2a2a2e","material":"matte","radius":0.04},{"slot":"switch","shape":"plane","size":[0.8,0.75],"pos":[-0.33,0.55,0.031],"svg":"AGENT MAY MERGENONELOWLOW+MED"},{"slot":"list","shape":"plane","size":[0.6,0.75],"pos":[0.42,0.55,0.031],"svg":"ALWAYS HUMANauthpaymentsmigrationsthe gates"}]}},{"term":"Review bottleneck","aka":["verification bottleneck","PRs nobody reads"],"kind":"loop","plain":"Opening the throttle makes more pull requests. They wait in the human review queue, so lead time grows. The team closes the throttle again, or the queue caps throughput by itself, and every agent added makes the queue longer. Speeding up the agents does not help, because the constraint is the reviewer. The way out is to let evidence carry low-risk work. An agent-ready codebase, verification and a merge policy by risk class take that work out of the queue.","does":"Shows why the factory's speed is set by how fast its output can be trusted, and where to work to raise it.","ask":"Is the review queue growing faster than merges, and which share of it could merge on evidence instead?","trouble":"Pull requests sit unreviewed for days, go stale and need rework, or reviewers start approving what they have not read.","pattern":"pattern-accelerator-brake","loop":{"type":"balancing","through":["throttle","pull-request","review-queue","lead-time"]},"sources":[{"label":"Addy Osmani, Agentic Code Review","url":"https://addyosmani.com/blog/agentic-code-review/","kind":"practitioner","page":"What the 2026 data actually shows","quote":"We poured machine-speed output into a system built for human-speed work. The bottleneck did not disappear; it moved to verification","work":"addy-osmani-agentic-code-review"},{"label":"Gene Kim and Steve Yegge, Vibe Coding: Building Production-Grade Software With GenAI, Chat, Agents, and Beyond","url":"https://itrevolution.com/articles/from-line-cookto-head-chef-orchestrating-ai-teams/","kind":"book","isbn":"9781966280026","page":"From Line Cook to Head Chef: Orchestrating AI Teams (book excerpt)","quote":"When coding is no longer the bottleneck, the rest of your organization becomes the bottleneck.","work":"gene-kim-steve-yegge-vibe-coding"},{"label":"Mohannad Hammadeh et al. (AWS), Scaling agentic coding past the human bottleneck","url":"https://builder.aws.com/content/3Jzr7Zmo4OxkGxLZ83VofzQqT0W/scaling-agentic-coding-past-the-human-bottleneck","kind":"maker","page":"Scaling agentic coding past the human bottleneck","quote":"Agentic coding today scales only as far as the person approving every step","work":"aws-scaling-agentic-coding"},{"label":"Mohannad Hammadeh et al. (AWS), Scaling agentic coding past the human bottleneck","url":"https://builder.aws.com/content/3Jzr7Zmo4OxkGxLZ83VofzQqT0W/scaling-agentic-coding-past-the-human-bottleneck","kind":"maker","page":"Scaling agentic coding past the human bottleneck","quote":"What caps agentic development is supervision bandwidth, not model quality.","work":"aws-scaling-agentic-coding"},{"label":"Gene Kim et al., The Phoenix Project","url":"https://reader.itrevolution.com/api/pub/phoenix-project-extra-excerpt/pdf/file","kind":"book","isbn":"9781950508945","page":"Chapter 7, p. 90 (5th Anniversary Edition excerpt, ISBN 9781942788294)","quote":"You don’t factor the availability of any of the other nineteen stations in your decision","work":"phoenix-project-gene-kim"},{"label":"Matthew Skelton and Manuel Pais, Team Topologies","url":"https://itrevolution.com/wp-content/uploads/2022/06/TTOP_excerpt.pdf","kind":"book","isbn":"9781942788812","page":"Chapter 1, p. 12 (publisher's excerpt)","quote":"Ultimately, the team becomes a delivery bottleneck, as their cognitive capacity has been largely exceeded, leading to delays, quality issues, and often, a decrease in team members’ motivation","work":"team-topologies-skelton-pais"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/human-review","kind":"practitioner","page":"Human review","quote":"Agents raise the volume of code produced, so review becomes the bottleneck.","work":"matt-pocock-ai-coding-dictionary"}],"id":"review-bottleneck"},{"term":"Observability platform","aka":["run telemetry","execution visibility","tracing backend","logs","monitoring","dashboard"],"kind":"part","plain":"The optional place where a factory records run status, logs, traces, token use and cost, and shows the factory's metrics. The orchestrator's run page may be enough for a small factory. Software reads the signals first, and a person is called only when they need to act.","does":"Makes failures, slow steps, tool calls, and spend visible across many runs.","ask":"What must a human learn from one failed run, and what must they compare across runs?","cues":["A run page with status, logs, artifacts, and timestamps","A trace tree linking model calls and tool calls","A dashboard for token use, cost, latency, or error rate"],"range":"Choose no extra platform, LLM or agent tracing, general APM, or an LLM gateway with usage accounting.","trouble":"A run fails without a useful log, spans cannot join to the run, sensitive prompts leak into telemetry, or spend cannot be assigned to a task.","at":[1.55,1.65,-0.5],"children":["run-log","agent-trace","token-cost","factory-metrics"],"variants":[{"label":"Platform type","items":[{"id":"none","name":"None","plain":"The orchestrator's own run records serve as the telemetry. A person can view each run and its logs, and re-run or cancel it, with no other platform.","why":"The run records answer the factory's operational questions without a separate telemetry platform.","sources":[{"label":"GitHub, REST API endpoints for workflow runs","url":"https://docs.github.com/en/rest/actions/workflow-runs","kind":"maker","page":"About workflow runs in GitHub Actions","quote":"You can use the REST API to view, re-run, cancel, and view logs for workflow runs in GitHub Actions.","work":"github-actions-workflow-runs"}]},{"id":"llm-agent-tracing","name":"LLM or agent tracing","plain":"A tracing product records LLM calls, tool calls, retrieval steps, and other observations.","why":"It shows the tool path inside an LLM run.","sources":[{"label":"Langfuse, Observability data model","url":"https://langfuse.com/docs/observability/data-model","kind":"maker","page":"Observations and Traces","quote":"Observations are the individual steps of your application: LLM calls, tool calls, retrieval steps, and so on.","work":"langfuse-data-model"}]},{"id":"general-apm","name":"General APM","plain":"A telemetry backend collects signals such as traces, metrics, and logs for the runner and its services.","why":"It covers signal types beyond model-specific events.","sources":[{"label":"OpenTelemetry, Signals","url":"https://opentelemetry.io/docs/concepts/signals/","kind":"standard","page":"Signals","quote":"Signals are system outputs that describe the underlying activity of the operating system and applications running on a platform.","work":"opentelemetry-signals"}]},{"id":"llm-gateway","name":"LLM gateway","plain":"A model gateway records usage and spend for requests.","why":"It attributes usage centrally on the model request path.","sources":[{"label":"LiteLLM, Spend Tracking","url":"https://docs.litellm.ai/docs/proxy/cost_tracking","kind":"maker","page":"Spend Tracking","quote":"Track spend for keys, users, and teams across 100+ LLMs.","work":"litellm-spend-tracking"}]}]}],"sources":[{"label":"Anthropic, Monitoring","url":"https://code.claude.com/docs/en/monitoring-usage","kind":"maker","page":"Monitoring","quote":"Track Claude Code usage, costs, and tool activity across your organization by exporting telemetry data through OpenTelemetry (OTel).","work":"claude-code-monitoring"},{"label":"OpenTelemetry, Signals","url":"https://opentelemetry.io/docs/concepts/signals/","kind":"standard","page":"Signals","quote":"The purpose of OpenTelemetry is to collect, process, and export signals.","work":"opentelemetry-signals"},{"label":"Arize Phoenix, Overview: Tracing","url":"https://arize.com/docs/phoenix/tracing/llm-traces","kind":"maker","page":"Overview: Tracing","quote":"LLM tracing records the paths taken by requests as they propagate through multiple steps or components of an LLM application.","work":"phoenix-llm-tracing"},{"label":"Betsy Beyer et al., Site Reliability Engineering","url":"https://sre.google/sre-book/introduction/","kind":"book","isbn":"9781491929124","page":"Chapter 1, Introduction","quote":"Monitoring is one of the primary means by which service owners keep track of a system’s health and availability.","work":"site-reliability-engineering"},{"label":"Betsy Beyer et al., Site Reliability Engineering","url":"https://sre.google/sre-book/introduction/","kind":"book","isbn":"9781491929124","page":"Chapter 1, Introduction","quote":"Monitoring should never require a human to interpret any part of the alerting domain. Instead, software should do the interpreting, and humans should be notified only when they need to take action.","work":"site-reliability-engineering"},{"label":"Matt Pocock, software-factory skill (pull request #1316)","url":"https://github.com/mattpocock/skills/blob/b59cab42bf1206e1363eafc37f71b21c3bcaa12f/skills/in-progress/software-factory/SKILL.md","kind":"practitioner","page":"Services","quote":"Where runners send logs, traces and status, so a human can see what happened inside each run.","work":"matt-pocock-software-factory-skill"}],"id":"observability","look":{"parts":[{"slot":"dashboard","shape":"box","size":[2.15,1.35,0.18],"pos":[0,0.68,0],"radius":0.08,"color":"#e0a526","material":"metal"},{"slot":"screen","shape":"plane","size":[1.7,0.9],"pos":[0,0.82,0.12],"svg":"RUN TELEMETRYRUN-184 · 2m 14s · $0.42"},{"slot":"log","shape":"box","size":[0.3,0.5,0.12],"pos":[-0.75,0.34,0.2],"radius":0.03,"color":"#f3efe4","material":"gloss"},{"slot":"trace","shape":"box","size":[0.3,0.5,0.12],"pos":[0,0.34,0.2],"radius":0.03,"color":"#3b6ea8","material":"gloss"},{"slot":"cost","shape":"box","size":[0.3,0.5,0.12],"pos":[0.75,0.34,0.2],"radius":0.03,"color":"#2e9b4f","material":"gloss"},{"slot":"metrics-gauge","shape":"box","size":[0.66,0.5,0.1],"pos":[0,1.6,0],"radius":0.04,"color":"#2a2a2e","material":"matte"},{"slot":"metrics-face","shape":"plane","size":[0.6,0.44],"pos":[0,1.6,0.051],"svg":"merged"}]}},{"term":"Run log","aka":["job log","execution transcript","build output"],"kind":"measure","plain":"The ordered record of a run's status changes, commands, tool results, warnings, and final outcome. It is the first account of what happened. An unattended agent run can also keep a decision log, with one row per step that names the decision, the reason, and the evidence.","does":"Lets a human locate the failing step and decide whether to retry, change the task, or stop the factory.","ask":"Can someone reconstruct the run from trigger to final result without opening the runner?","cues":["Timestamped steps with queued, running, and finished states","Command output, warnings, exit codes, and artifact links","A retention setting and a redaction marker"],"range":"Keep enough history for review and incident response. GitHub Actions retains workflow logs for 90 days by default, with repository limits from 1 to 90 or 400 days.","trouble":"The log ends before the failure, hides the command that changed state, contains a secret, or disappears when an ephemeral worker is deleted.","at":[-0.75,0.34,0.2],"links":[{"to":"agent-trace","rel":"see also"},{"to":"token-cost","rel":"see also"}],"sources":[{"label":"GitHub, Managing GitHub Actions settings for a repository","url":"https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository","kind":"maker","page":"Retention period for checks, workflow runs, artifacts, and logs","quote":"By default, checks, workflow runs, commit statuses, and the artifacts and log files generated by workflows are retained for 90 days before they are automatically deleted.","work":"github-actions-retention"},{"label":"GitHub, Self-hosted runners reference","url":"https://docs.github.com/en/actions/reference/runners/self-hosted-runners","kind":"maker","page":"Ephemeral runners for autoscaling","quote":"The runner application log files for ephemeral runners must be forwarded to an external log storage solution for troubleshooting and diagnostic purposes.","work":"github-self-hosted-runners"},{"label":"Betsy Beyer et al., Site Reliability Engineering","url":"https://sre.google/sre-book/introduction/","kind":"book","isbn":"9781491929124","page":"Chapter 1, Introduction","quote":"No one needs to look at this information, but it is recorded for diagnostic or forensic purposes. The expectation is that no one reads logs unless something else prompts them to do so.","work":"site-reliability-engineering"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/07-overnight.md","kind":"practitioner","page":"The morning audit","quote":"Each row records the time, phase, decision, reason, an evidence pointer, and the result","work":"lauren-tan-pstack"}],"id":"run-log","look":{"parts":[{"slot":"paper","shape":"box","size":[1.55,1.2,0.08],"pos":[0,0.62,0],"radius":0.04,"color":"#f3efe4","material":"matte"},{"slot":"lines","shape":"plane","size":[1.3,0.94],"pos":[0,0.68,0.06],"svg":"RUN LOGcheckout · 00:03tests · 01:42review · exit 1artifacts / logs / retention"}]}},{"term":"Agent trace","aka":["span tree","model trace","tool trace"],"kind":"measure","plain":"A trace links one run's model requests, tool calls, and hooks into a parent-child timeline. It keeps the agent's internal path attached to the run.","does":"Shows where the model waited, which tool acted, and which child operation caused a failure or cost.","ask":"Which model call or tool span changed the run's outcome?","cues":["A root interaction span with nested model and tool spans","Span duration, status, model, tool name, and run identifier","A trace link from the run page to the telemetry backend"],"range":"Record a root interaction span with child model, tool, and hook spans. Keep prompts, tool arguments, and results opt-in and redacted by default.","trouble":"Spans start new traces, tool calls are missing, retries are invisible, or sensitive content is recorded without a policy.","at":[0,0.34,0.2],"links":[{"to":"run-log","rel":"see also"},{"to":"token-cost","rel":"see also"}],"sources":[{"label":"Anthropic, Monitoring","url":"https://code.claude.com/docs/en/monitoring-usage","kind":"maker","page":"Span hierarchy","quote":"Each user prompt starts a `claude_code.interaction` root span. API calls, tool calls, and hook executions are recorded as its children.","work":"claude-code-monitoring"},{"label":"Langfuse, Observability data model","url":"https://langfuse.com/docs/observability/data-model","kind":"maker","page":"Observations and Traces","quote":"A `trace` represents a single request or operation, for example one chatbot interaction from the user's question to the final response.","work":"langfuse-data-model"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/tool-call","kind":"practitioner","page":"Tool call","quote":"The model's output naming a tool and its arguments — just structured text. The harness has to read it and execute.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/model-provider-request","kind":"practitioner","page":"Model provider request","quote":"One round-trip from the harness to the model provider. The harness sends the current context; the provider returns one response (a tool call or a final answer).","work":"matt-pocock-ai-coding-dictionary"}],"id":"agent-trace","look":{"parts":[{"slot":"panel","shape":"box","size":[1.7,1.25,0.08],"pos":[0,0.64,0],"radius":0.04,"color":"#d9e7f4","material":"matte"},{"slot":"tree","shape":"plane","size":[1.48,1.02],"pos":[0,0.7,0.06],"svg":"TRACE / RUN-184invoke_agentllm_requestexecute_toolhook"}]}},{"term":"Token cost","aka":["run spend","model cost","usage cost"],"kind":"measure","plain":"The money spent by one run on model input, model output, cached tokens, and related calls. It is attributed to a run, task, repository, or team.","does":"Shows whether a task's value justifies its model usage and reveals runaway retries or oversized context.","ask":"What did this run spend, and which model call or retry caused the bill?","cues":["Input, output, and cached token counts beside a model call","A currency total attached to a run or task","A spend header, budget meter, or cost report"],"range":"Report input and output tokens separately, then apply the provider's current rates. Aggregate by run, task, model, repository, and team.","trouble":"A run has tokens but no price, retries inflate spend, cost is assigned to the wrong task, or prompt content is logged to measure usage.","at":[0.75,0.34,0.2],"links":[{"to":"agent-trace","rel":"measured by"},{"to":"run-log","rel":"see also"}],"sources":[{"label":"Anthropic, Monitoring","url":"https://code.claude.com/docs/en/monitoring-usage","kind":"maker","page":"Span attributes","quote":"`input_tokens` | Input token count from the API usage block.","work":"claude-code-monitoring"},{"label":"OpenTelemetry, Semantic Conventions for GenAI agent and framework spans","url":"https://github.com/open-telemetry/semantic-conventions-genai/blob/9ffc7e00c639be7712e544e3e54331e885dccf84/docs/gen-ai/gen-ai-agent-spans.md","kind":"standard","page":"Invoke agent client span attributes","quote":"The number of tokens used in the GenAI input (prompt).","work":"opentelemetry-genai-agent-spans"},{"label":"OpenTelemetry, Semantic Conventions for GenAI agent and framework spans","url":"https://github.com/open-telemetry/semantic-conventions-genai/blob/9ffc7e00c639be7712e544e3e54331e885dccf84/docs/gen-ai/gen-ai-agent-spans.md","kind":"standard","page":"Invoke agent client span attributes","quote":"The number of tokens used in the GenAI response (completion).","work":"opentelemetry-genai-agent-spans"},{"label":"LiteLLM, Spend Tracking","url":"https://docs.litellm.ai/docs/proxy/cost_tracking","kind":"maker","page":"How to Track Spend with LiteLLM","quote":"Expect to see `x-litellm-response-cost` in the response headers with calculated cost","work":"litellm-spend-tracking"},{"label":"LiteLLM, Spend Tracking","url":"https://docs.litellm.ai/docs/proxy/cost_tracking","kind":"maker","page":"How to Track Spend with LiteLLM","quote":"LiteLLM automatically tracks spend for all known models.","work":"litellm-spend-tracking"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/token","kind":"practitioner","page":"Token","quote":"Tokens are the unit everything else is measured in. Cost is per token — providers bill input tokens and output tokens separately.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/input-tokens","kind":"practitioner","page":"Input tokens","quote":"Tokens the harness sends on each model provider request — the system prompt, the conversation history, tool results, everything the model reads before it writes. Billed at a lower rate than output tokens, because they are less expensive to process than output tokens.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/output-tokens","kind":"practitioner","page":"Output tokens","quote":"Tokens the model generates back. Billed at a higher rate than input tokens — commonly around five times the rate — since they cost more compute to produce.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/cache-tokens","kind":"practitioner","page":"Cache tokens","quote":"When consecutive requests share a prefix, the provider reuses the work via its prefix cache and bills the cached portion at a much lower rate.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/prefix-cache","kind":"practitioner","page":"Prefix cache","quote":"The provider-side store that lets consecutive model provider requests skip re-processing a shared prefix.","work":"matt-pocock-ai-coding-dictionary"}],"id":"token-cost","look":{"parts":[{"slot":"meter","shape":"cylinder","radiusTop":0.5,"radiusBottom":0.5,"height":0.12,"pos":[0,0.45,0],"rot":[90,0,0],"color":"#e0a526","material":"metal"},{"slot":"needle","shape":"box","size":[0.06,0.38,0.06],"pos":[0.13,0.45,0.08],"rot":[0,0,-35],"color":"#c8432f","material":"gloss"},{"slot":"readout","shape":"plane","size":[1.45,0.48],"pos":[0,0.98,0.08],"svg":"RUN COST$0.4218.4k tokens"}]}},{"term":"Factory metrics","aka":["delivery measures","factory dashboard"],"kind":"part","plain":"The measures that show whether automated work flows, waits, fails, and recovers in a useful way. They connect agent output to delivery and review capacity.","does":"Turns run and pull request history into evidence for throttle, staffing, gate, and task changes.","ask":"What moved faster, what failed, and where did a person have to wait?","cues":["A dashboard of merge, lead-time, failure, and queue readings","A time series split by task type or trigger","A review queue with age and volume"],"at":[0,1.62,0.08],"children":["merge-rate","lead-time","change-failure-rate","review-queue"],"sources":[{"label":"DORA, A history of DORA’s software delivery metrics","url":"https://dora.dev/insights/dora-metrics-history/","kind":"research","page":"From four keys to five (2024)","quote":"measuring these factors over time gives teams a high-level understanding of their performance, regardless of their technology stack.","work":"dora-metrics-history"},{"label":"Addy Osmani, Agentic Code Review","url":"https://addyosmani.com/blog/agentic-code-review/","kind":"practitioner","page":"What the 2026 data actually shows","quote":"Reviewers simply could not keep pace with the volume, so code began merging unread, and that became normal.","work":"addy-osmani-agentic-code-review"}],"id":"factory-metrics","look":{"parts":[{"slot":"board","shape":"box","size":[1.85,1.2,0.06],"pos":[0,0.6,0],"radius":0.05,"color":"#cfd2d7","material":"matte"},{"slot":"dashboard","shape":"plane","size":[1.72,1.05],"pos":[0,0.62,0.04],"svg":"FACTORY METRICSmergelead timefail ratequeue"},{"slot":"needle","shape":"cone","radius":0.06,"height":0.42,"pos":[-0.55,0.18,0.09],"rot":[0,0,35],"color":"#c8432f","material":"gloss"},{"slot":"dial","shape":"torus","radius":0.2,"tube":0.035,"pos":[-0.55,0.18,0.08],"rot":[90,0,0],"color":"#e0a526","material":"metal"}]}},{"term":"Merge rate","aka":["merge share","accepted PR rate"],"kind":"measure","plain":"The share of agent-assisted pull requests that merge during a stated period. Keep the denominator and time window fixed when comparing task types or throttles.","does":"Shows how much generated work survives checks and review to become repository history.","ask":"Of the agent pull requests opened in this window, how many merged, closed, or remain open?","cues":["A dashboard percentage labelled merged over opened agent PRs","Counts for merged, closed, and open pull requests","A trend split by task type or model"],"range":"83.8% accepted and merged in a 567-PR Claude Code sample; 54.9% merged without further modification.","trouble":"A high percentage hides many abandoned PRs, or merges rise while change failures and review wait rise too.","at":[-0.5,0.46,0.12],"sources":[{"label":"Miku Watanabe, On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub","url":"https://arxiv.org/abs/2509.14745","kind":"research","page":"Abstract","quote":"The results indicate that 83.8% of these agent-assisted PRs are eventually accepted and merged by project maintainers, with 54.9% of the merged PRs are integrated without further modification.","work":"watanabe-agentic-coding-pull-requests"}],"id":"merge-rate","look":{"parts":[{"slot":"dial","shape":"torus","radius":0.42,"tube":0.08,"pos":[0,0.55,0],"rot":[90,0,0],"color":"#2e9b4f","material":"metal"},{"slot":"needle","shape":"box","size":[0.05,0.38,0.05],"pos":[0.22,0.74,0.08],"rot":[0,0,40],"color":"#c8432f","material":"gloss"},{"slot":"readout","shape":"plane","size":[0.96,0.34],"pos":[0,1.1,0.08],"svg":"MERGED / OPENED74%"}]}},{"term":"Lead time","aka":["change lead time","delivery time"],"kind":"measure","plain":"The elapsed time from a code commit to a successful production deployment. For a ticket flow, pair it with the ticket-to-commit interval so the waiting point stays visible.","does":"Shows how long a change takes to travel from implementation to usable production software.","ask":"When did the measured change start, when did it run successfully in production, and where did it wait?","cues":["A duration from commit to production on a delivery dashboard","Percentiles or bands rather than one average","A breakdown of coding, checks, review, queue, and deployment time"],"at":[0,0.8,0.12],"sources":[{"label":"DORA, A history of DORA’s software delivery metrics","url":"https://dora.dev/insights/dora-metrics-history/","kind":"research","page":"From four keys to five (2024)","quote":"Change lead time: Time from code commit to successful production deployment.","work":"dora-metrics-history"},{"label":"Nicole Forsgren et al., Accelerate","url":"https://itrevolution.com/wp-content/uploads/2022/06/ACC_excerpt.pdf","kind":"book","isbn":"9781942788331","page":"Chapter 2: Measuring Performance, p. 15","quote":"We measured product delivery lead time as the time it takes to go from code committed to code successfully running in production","work":"accelerate-nicole-forsgren"}],"id":"lead-time","look":{"parts":[{"slot":"track","shape":"box","size":[1.8,0.1,0.16],"pos":[0,0.42,0],"color":"#3a3f47","material":"matte"},{"slot":"start","shape":"sphere","radius":0.11,"pos":[-0.78,0.42,0.1],"color":"#6b5bd2","material":"glow"},{"slot":"end","shape":"sphere","radius":0.11,"pos":[0.78,0.42,0.1],"color":"#2e9b4f","material":"glow"},{"slot":"clock","shape":"plane","size":[1.15,0.42],"pos":[0,0.85,0.08],"svg":"LEAD TIMEcommit → prod"}]}},{"term":"Change failure rate","aka":["failed change percentage","change fail rate"],"kind":"measure","plain":"The percentage of production deployments that cause a failure. Define the failure signal and remediation window before comparing periods. Most outages come from changes, so a factory that merges more also needs progressive rollouts and fast rollback after merge. That work sits outside the factory, which ends at merge.","does":"Shows whether faster delivery is producing changes that need rollback, hotfix, or other recovery work.","ask":"Which deployments caused a production failure, and what counts as a failure for this service?","cues":["A percentage of deployments marked failed","Links from a failed deployment to incident, rollback, or fix-forward work","A trend beside lead time and deployment frequency"],"at":[0.5,0.8,0.12],"sources":[{"label":"DORA, A history of DORA’s software delivery metrics","url":"https://dora.dev/insights/dora-metrics-history/","kind":"research","page":"From four keys to five (2024)","quote":"Change fail rate: Percentage of deployments causing failures in production.","work":"dora-metrics-history"},{"label":"Betsy Beyer et al., Site Reliability Engineering","url":"https://sre.google/sre-book/introduction/","kind":"book","isbn":"9781491929124","page":"Chapter 1, Introduction","quote":"SRE has found that roughly 70% of outages are due to changes in a live system. … Implementing progressive rollouts … Rolling back changes safely when problems arise","work":"site-reliability-engineering"}],"id":"change-failure-rate","look":{"parts":[{"slot":"dial","shape":"cylinder","radiusTop":0.45,"radiusBottom":0.45,"height":0.1,"pos":[0,0.5,0],"rot":[90,0,0],"color":"#f4f5f7","material":"matte"},{"slot":"red-zone","shape":"torus","radius":0.3,"tube":0.08,"pos":[0,0.5,0.08],"rot":[90,0,0],"color":"#c8432f","material":"gloss"},{"slot":"needle","shape":"box","size":[0.04,0.33,0.04],"pos":[-0.12,0.68,0.14],"rot":[0,0,-30],"color":"#3a3f47","material":"metal"},{"slot":"label","shape":"plane","size":[1.2,0.3],"pos":[0,1.04,0.08],"svg":"PROD FAILURES / DEPLOYS"}]}},{"term":"Review queue","aka":["PR backlog","reviewer load"],"kind":"measure","plain":"The count and age of pull requests waiting for an automated or human review. Read volume and wait together because either can hide the bottleneck.","does":"Shows whether generated work is arriving faster than the available review capacity.","ask":"How many changes await review, how old are they, and which gate is holding them?","cues":["A list of open pull requests sorted by review age","Queue depth and time-to-first-review charts","Labels for waiting on checks, bot review, or human review"],"at":[0,0.38,0.12],"trouble":"Queue age grows, reviewers skip evidence, or stale pull requests merge after attention has moved elsewhere.","sources":[{"label":"Addy Osmani, Agentic Code Review","url":"https://addyosmani.com/blog/agentic-code-review/","kind":"practitioner","page":"What the 2026 data actually shows","quote":"median review duration up 441.5%, with time-to-first-review and average review time both roughly doubling","work":"addy-osmani-agentic-code-review"},{"label":"Addy Osmani, Agentic Code Review","url":"https://addyosmani.com/blog/agentic-code-review/","kind":"practitioner","page":"What the 2026 data actually shows","quote":"This is already how I work on my own projects, including the open-source ones that now see more PRs in a day than I could carefully read in an evening.","work":"addy-osmani-agentic-code-review"}],"id":"review-queue","look":{"parts":[{"slot":"tray","shape":"box","size":[1.65,0.18,0.62],"pos":[0,0.18,0],"radius":0.05,"color":"#3a3f47","material":"matte"},{"slot":"pr-1","shape":"plane","size":[0.45,0.3],"pos":[-0.48,0.45,0.1],"face":"camera","svg":""},{"slot":"pr-2","shape":"plane","size":[0.45,0.3],"pos":[0,0.6,0.1],"face":"camera","svg":""},{"slot":"pr-3","shape":"plane","size":[0.45,0.3],"pos":[0.48,0.75,0.1],"face":"camera","svg":""},{"slot":"label","shape":"plane","size":[1.2,0.28],"pos":[0,1.08,0.08],"svg":"WAITING FOR REVIEW"}]}},{"term":"Factory design","aka":["operating model","automation policy"],"kind":"process","plain":"The deliberate work of choosing a problem, adding triggers and services in small steps, and deciding where people stay in the loop. It also covers the human role once agents carry most changes, and the evals that show whether a change to the factory helped.","does":"Turns an automation idea into a small system that earns more autonomy as evidence comes in.","ask":"What is the smallest useful loop, what can stop it, and what evidence earns the next level?","at":[2.55,0.21,0.85],"children":["start-slow","buy-vs-build","autonomy-level","human-re-entry","slop","human-role","factory-evals","trust-loop","improvement-loop"],"sources":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"When (and when not to) use agents","quote":"we recommend finding the simplest solution possible, and only increasing complexity when needed.","work":"anthropic-building-effective-agents"},{"label":"Gene Kim, Vibe Coding: Pairing vs. Delegation","url":"https://itrevolution.com/articles/vibe-coding-pairing-vs-delegation/","kind":"practitioner","page":"Vibe Coding: Pairing vs. Delegation","quote":"You delegate when you’re sure the task novelty is not terribly high","work":"gene-kim-vibe-coding-pairing"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Custom Automations","quote":"Create, configure, and monitor automations that run Droid on a schedule, from Slack messages, on GitHub events, or from inbound webhooks.","work":"factory-software-factory"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents","quote":"Agents can then pause for human feedback at checkpoints or when encountering blockers.","work":"anthropic-building-effective-agents"},{"label":"Matt Pocock, software-factory skill (pull request #1316)","url":"https://github.com/mattpocock/skills/blob/b59cab42bf1206e1363eafc37f71b21c3bcaa12f/skills/in-progress/software-factory/SKILL.md","kind":"practitioner","page":"Designing a factory","quote":"Buy the services and build the tasks: the tasks carry the user's design, the services are generic.","work":"matt-pocock-software-factory-skill"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/software-factory","kind":"practitioner","page":"Software factory","quote":"Deciding which of those decisions stay human is the main design question.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"Order: P1 first (checkable), then P2/P3 (trust), then loops.","work":"kent-c-dodds-kcd-skills"}],"id":"factory-design","look":{"parts":[{"slot":"blueprint","shape":"plane","size":[1.9,1.2],"pos":[0,0.62,0],"svg":"START SLOW / EARN TRUSTproblembrakesthrottlehumanevals"},{"slot":"dial","shape":"cylinder","radiusTop":0.18,"radiusBottom":0.18,"height":0.08,"pos":[-0.55,0.15,0.08],"rot":[90,0,0],"color":"#e0a526","material":"metal"},{"slot":"dial-mark","shape":"box","size":[0.03,0.22,0.03],"pos":[-0.55,0.3,0.12],"color":"#c8432f","material":"gloss"},{"slot":"flag","shape":"cone","radius":0.12,"height":0.3,"pos":[0.62,0.2,0.08],"rot":[0,0,180],"color":"#2e9b4f","material":"glow"}]}},{"term":"Start slow","aka":["canary rollout","throttled start"],"kind":"process","plain":"Start with one useful task and one automated trigger at a time, throttled to a small event sample or a once-daily clock. Require human review on every pull request and cap rework at a few tries. This is level 3, where a person reads every diff. Add triggers and open the throttle only after checks and review evidence earn trust; the trust loop then moves the factory toward level 4.","does":"Exposes weak instructions, missing checks, and unsafe permissions while the blast radius is small.","ask":"What is the smallest live loop that can prove value and reveal failure?","cues":["A canary label or sampled trigger rule","A low concurrency setting and human approval on every pull request","A change log showing one trigger or gate added at a time"],"at":[0.45,0.3,0.12],"sources":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"When (and when not to) use agents","quote":"we recommend finding the simplest solution possible, and only increasing complexity when needed.","work":"anthropic-building-effective-agents"},{"label":"GitHub, Best practices for using GitHub Copilot to work on tasks","url":"https://docs.github.com/en/copilot/tutorials/cloud-agent/get-the-best-results","kind":"maker","page":"Choosing the right type of tasks to give to Copilot","quote":"Initially, you might want to start by giving Copilot simpler tasks, to see how it works as a cloud agent.","work":"github-copilot-task-practices"},{"label":"Martin Fowler, Refactoring: Improving the Design of Existing Code","url":"https://martinfowler.com/books/refactoring.html","kind":"book","isbn":"9780134757599","page":"Refactoring","quote":"By doing them in small steps you reduce the risk of introducing errors.","work":"refactoring-martin-fowler"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents","quote":"We recommend extensive testing in sandboxed environments, along with the appropriate guardrails.","work":"anthropic-building-effective-agents"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents","quote":"Agents can then pause for human feedback at checkpoints or when encountering blockers.","work":"anthropic-building-effective-agents"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents","quote":"it’s also common to include stopping conditions (such as a maximum number of iterations) to maintain control.","work":"anthropic-building-effective-agents"},{"label":"GitHub, About protected branches","url":"https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches","kind":"maker","page":"Require pull request reviews before merging","quote":"can require that all pull requests receive a specific number of approving reviews before someone merges the pull request into a protected branch.","work":"github-protected-branches"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Filter which deliveries start a run","quote":"starts a run only for payloads that carry that field, which keeps a busy event stream from running the automation on every message.","work":"factory-software-factory"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Review deliveries","quote":"The automation already started 10 runs this hour. The limit resets at the top of the next hour.","work":"factory-software-factory"},{"label":"Factory, Software Factory documentation","url":"https://docs.factory.com/software-factory/automations","kind":"maker","page":"Schedules","quote":"A frequency without a time, such as daily, runs at 9:00 UTC.","work":"factory-software-factory"},{"label":"GitHub, Best practices for using GitHub Copilot to work on tasks","url":"https://docs.github.com/en/copilot/tutorials/cloud-agent/get-the-best-results","kind":"maker","page":"Adding custom instructions to your repository","quote":"If Copilot is able to build, test and validate its changes in its own development environment, it is more likely to produce good pull requests which can be merged quickly.","work":"github-copilot-task-practices"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/07-overnight.md","kind":"practitioner","page":"Earn the trust before the loop","quote":"Make the loop autonomous only after all four hold. Until then, run it while you watch.","work":"lauren-tan-pstack"},{"label":"Matt Pocock, 11 Tips For AI Coding With Ralph Wiggum","url":"https://www.aihero.dev/tips-for-ai-coding-with-ralph-wiggum","kind":"practitioner","page":"2. Start With HITL, Then Go AFK","quote":"You'll understand what it does, refine your prompt, and build confidence before going hands-off.","work":"matt-pocock-ralph-tips"},{"label":"Matt Pocock, software-factory skill (pull request #1316)","url":"https://github.com/mattpocock/skills/blob/b59cab42bf1206e1363eafc37f71b21c3bcaa12f/skills/in-progress/software-factory/SKILL.md","kind":"practitioner","page":"Start slow","quote":"Add one automated trigger at a time, throttled at first (one bug report in ten, a clock once a day), with human review on every PR.","work":"matt-pocock-software-factory-skill"},{"label":"Lauren Tan, Noodle","url":"https://raw.githubusercontent.com/poteto/noodle/82d2921c52370f23f29086de81ccfb600939c037/docs/reference/configuration.md","kind":"practitioner","page":"docs/reference/configuration.md","quote":"Start with max_concurrency = 1 or 2 while you’re learning the system, then scale up once you’ve seen the cost per session on your workload.","work":"lauren-tan-noodle"},{"label":"Dan Shapiro, The Five Levels: from Spicy Autocomplete to the Dark Factory","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"practitioner","page":"Level 3","quote":"You are the human in the loop. Your coding agent is always running multiple tabs. You spend your days reviewing code.","work":"dan-shapiro-five-levels"}],"id":"start-slow","look":{"parts":[{"slot":"ramp","shape":"box","size":[1.65,0.12,0.58],"pos":[0,0.18,0],"rot":[0,0,-18],"color":"#3b6ea8","material":"matte"},{"slot":"canary","shape":"sphere","radius":0.13,"pos":[-0.55,0.54,0.1],"color":"#e0a526","material":"glow"},{"slot":"brake","shape":"box","size":[0.12,0.7,0.12],"pos":[0.15,0.58,0.08],"color":"#c8432f","material":"gloss"},{"slot":"flag","shape":"plane","size":[0.86,0.3],"pos":[0.48,1.02,0.08],"svg":"CANARY 1 / 10"}]}},{"term":"Buy versus build","aka":["service choice","own the task"],"kind":"concept","plain":"Use third-party software for proven capabilities when customization matters less; build the task policies, checks, and feedback that differentiate the factory.","does":"Keeps local engineering effort on the work and controls that create differentiated value.","ask":"Which part is a commodity service, and which task or policy must the team understand and change itself?","cues":["Hosted tracker, git host and runner used with stock settings","Repository-owned task prompts and check definitions"],"at":[0.8,0.3,0.12],"sources":[{"label":"Thoughtworks, Build vs. buy: A strategic framework for evaluating third-party solutions","url":"https://www.thoughtworks.com/content/dam/thoughtworks/documents/e-book/tw_ebook_build_vs_buy_2022.pdf","kind":"practitioner","page":"Introduction","quote":"When you buy third-party software, you gain proven capabilities quickly, at the cost of customization and control.","work":"thoughtworks-build-vs-buy"},{"label":"Thoughtworks, Build vs. buy: A strategic framework for evaluating third-party solutions","url":"https://www.thoughtworks.com/content/dam/thoughtworks/documents/e-book/tw_ebook_build_vs_buy_2022.pdf","kind":"practitioner","page":"Defining your needs","quote":"Differentiator capabilities are how your company differentiates itself in the market.","work":"thoughtworks-build-vs-buy"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"How and when to use frameworks","quote":"We suggest that developers start by using LLM APIs directly: many patterns can be implemented in a few lines of code.","work":"anthropic-building-effective-agents"}],"id":"buy-vs-build","feel":{"shape":0.48,"energy":0.35,"weight":0.15,"warmth":0.45,"grain":0.28,"motion":"drift","why":"Service slots feel like stable purchased blocks while task policy remains a warm, changing piece of local craft."}},{"term":"Autonomy level","aka":["levels of AI coding","delegation level","trust dial"],"kind":"knob","plain":"How much of the work agents do with no person in the loop. Practitioners borrow the levels of driving automation, from a human who approves every character to a factory that turns specs into software with no human review. A software factory starts at level 3, with a person reading every diff, and aims at level 4. There, people write specs and check outcomes, while verification and the merge policy let low-risk work merge without a person reading the diff.","does":"Sets the human's job at each step (writer, reviewer, spec writer) and so sets how strong the brakes and the verification must be.","ask":"At this level, who starts the run, who reads the diff, and who can merge?","cues":["Whether a human or an event starts each agent session","Whether every pull request waits for a human approval","Work that continues while the team is away"],"range":"The scale runs from manual level 0 to level 5, where no human reviews the code. Levels 3 and 4 separate human diff review from spec-and-outcome oversight.","trouble":"The team claims a higher level than its brakes support, and defects reach production unreviewed.","at":[-0.05,0.68,0.12],"variants":[{"label":"Level","items":[{"id":"level-0","name":"Level 0: manual","plain":"The human writes the code. AI is at most a search engine or an occasional accepted suggestion.","sources":[{"label":"Dan Shapiro, The Five Levels: from Spicy Autocomplete to the Dark Factory","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"practitioner","page":"Level Zero","quote":"Whether it’s vi or Visual Studio, not a character hits the disk without your approval.","work":"dan-shapiro-five-levels"}],"look":{"parts":[{"slot":"needle","shape":"box","size":[0.05,0.3,0.05],"pos":[-0.145,0.499,0.09],"rot":[0,0,75],"color":"#e0a526","material":"gloss"}]},"svg":{"label":"OVERSIGHT012345human writes every line"}},{"id":"level-1","name":"Level 1: discrete tasks","plain":"The human hands small, separate jobs to the AI, such as a unit test or a docstring, and still writes the important code.","sources":[{"label":"Dan Shapiro, The Five Levels: from Spicy Autocomplete to the Dark Factory","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"practitioner","page":"Level 1","quote":"You’re writing the important stuff, but you offload specific, discrete tasks to your AI intern.","work":"dan-shapiro-five-levels"}],"look":{"parts":[{"slot":"needle","shape":"box","size":[0.05,0.3,0.05],"pos":[-0.106,0.566,0.09],"rot":[0,0,45],"color":"#e0a526","material":"gloss"}]},"svg":{"label":"OVERSIGHT012345AI does small tasks"}},{"id":"level-2","name":"Level 2: pairing","plain":"The human pairs with an AI coding tool like a colleague, in a flow state, and still drives every session.","sources":[{"label":"Dan Shapiro, The Five Levels: from Spicy Autocomplete to the Dark Factory","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"practitioner","page":"Level 2","quote":"You are pairing with the AI like a colleague. You get into a flow state; you’re more productive than you’ve ever been.","work":"dan-shapiro-five-levels"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/human-in-the-loop","kind":"practitioner","page":"Human-in-the-loop","quote":"A working pattern where one or more humans pair with the agent during a session — reviewing, redirecting, or collaborating in real time.","work":"matt-pocock-ai-coding-dictionary"}],"look":{"parts":[{"slot":"needle","shape":"box","size":[0.05,0.3,0.05],"pos":[-0.039,0.605,0.09],"rot":[0,0,15],"color":"#e0a526","material":"gloss"}]},"svg":{"label":"OVERSIGHT012345human pairs with AI"}},{"id":"level-3","name":"Level 3: human in the loop","plain":"Agents write the code in several sessions at once, and the human's day is reviewing their diffs. A software factory starts here, with human review on every pull request.","why":"Review of every diff makes it level 3, however many agents run.","sources":[{"label":"Dan Shapiro, The Five Levels: from Spicy Autocomplete to the Dark Factory","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"practitioner","page":"Level 3","quote":"You are the human in the loop. Your coding agent is always running multiple tabs. You spend your days reviewing code.","work":"dan-shapiro-five-levels"}],"look":{"parts":[{"slot":"needle","shape":"box","size":[0.05,0.3,0.05],"pos":[0.039,0.605,0.09],"rot":[0,0,-15],"color":"#e0a526","material":"gloss"}]},"svg":{"label":"OVERSIGHT012345human reads every diff"}},{"id":"level-4","name":"Level 4: spec and schedule","plain":"The human writes specs and plans, sets schedules, and leaves the agents to work for hours, then checks the results. A mature factory runs here. Evidence carries low-risk work to merge, and people read the diffs only of high-risk changes.","why":"The human checks outcomes such as passing tests and scenarios instead of reading each step.","sources":[{"label":"Dan Shapiro, The Five Levels: from Spicy Autocomplete to the Dark Factory","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"practitioner","page":"Level 4","quote":"You write a spec. You argue with it about the spec. You craft skills (for Claude Code, because most folks at level 4 seem to find their way to Claude Code). You plan schedules. You review plans. Then you leave for 12 hours, and check to see if the tests pass.","work":"dan-shapiro-five-levels"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/afk","kind":"practitioner","page":"AFK","quote":"Away from keyboard. A working pattern where the user kicks off a session and leaves the agent to run unattended.","work":"matt-pocock-ai-coding-dictionary"}],"look":{"parts":[{"slot":"needle","shape":"box","size":[0.05,0.3,0.05],"pos":[0.106,0.566,0.09],"rot":[0,0,-45],"color":"#e0a526","material":"gloss"}]},"svg":{"label":"OVERSIGHT012345human checks outcomes"}},{"id":"level-5","name":"Level 5: dark factory","plain":"Specs go in and software comes out, with no human writing or reviewing the code. It is credible only where agents cannot game the evidence, such as holdout scenarios run against twins of outside services. It may cover only part of a codebase, and only a handful of small teams work this way.","why":"\"Dark\" means no human is needed on the floor, as in a robot-staffed factory.","sources":[{"label":"Dan Shapiro, The Five Levels: from Spicy Autocomplete to the Dark Factory","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"practitioner","page":"Level 5","quote":"It’s a black box that turns specs into software.","work":"dan-shapiro-five-levels"},{"label":"Dan Shapiro, The Five Levels: from Spicy Autocomplete to the Dark Factory","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"practitioner","page":"Why Dark?","quote":"It’s dark, because it’s a place where humans are neither needed nor welcome.","work":"dan-shapiro-five-levels"},{"label":"Dan Shapiro, The Five Levels: from Spicy Autocomplete to the Dark Factory","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"practitioner","page":"Why Dark?","quote":"I know a handful of people who are doing this. They’re small teams, less than five people.","work":"dan-shapiro-five-levels"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/dark-factory","kind":"practitioner","page":"Dark factory","quote":"A codebase, or part of one, where a software factory writes the code and no human reads it.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Justin McCarthy (StrongDM), Software Factories And The Agentic Moment","url":"https://factory.strongdm.ai/","kind":"maker","page":"Software Factories And The Agentic Moment","quote":"Tests can be reward hacked - we needed validation that was less vulnerable to the model cheating","work":"strongdm-software-factory"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/vibe-coding","kind":"practitioner","page":"Vibe coding","quote":"With human review gone, whatever automated verification still runs — tests, types, automated review — is the only gate the code passes through.","work":"matt-pocock-ai-coding-dictionary"}],"look":{"parts":[{"slot":"needle","shape":"box","size":[0.05,0.3,0.05],"pos":[0.145,0.499,0.09],"rot":[0,0,-75],"color":"#e0a526","material":"gloss"}]},"svg":{"label":"OVERSIGHT012345no human reviews code"}}]}],"sources":[{"label":"Dan Shapiro, The Five Levels: from Spicy Autocomplete to the Dark Factory","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"practitioner","page":"Introduction","quote":"I’ve now seen dozens of companies struggling to put AI to work writing code, and each one has moved through five clear tiers of automation.","work":"dan-shapiro-five-levels"}],"id":"autonomy-level","look":{"parts":[{"slot":"dial","shape":"cylinder","radiusTop":0.45,"radiusBottom":0.45,"height":0.12,"pos":[0,0.46,0],"rot":[90,0,0],"color":"#3b6ea8","material":"metal"},{"slot":"needle","shape":"box","size":[0.05,0.4,0.05],"pos":[0.15,0.7,0.09],"rot":[0,0,36],"color":"#e0a526","material":"gloss"},{"slot":"label","shape":"plane","size":[1.35,0.52],"pos":[0,1.04,0.08],"svg":"OVERSIGHT0 human start3 human review5 dark factory"}]}},{"term":"Human re-entry","aka":["escalation point","handoff boundary"],"kind":"concept","plain":"The point where a person comes back after a run ends or a gate blocks progress. The person's change or decision becomes a new event that starts another run.","does":"Keeps automated work bounded while preserving a clear path for judgement, clarification, and recovery.","ask":"What exact evidence or failure brings a person back, and what event starts after their change?","cues":["A stopped run with a human-review or needs-clarification label","A comment, approval, or ticket edit that starts a new run","A handoff record naming the decision still needed"],"at":[-0.8,0.08,0.12],"pattern":"pattern-stop-the-line","links":[{"to":"try-cap","rel":"see also"},{"to":"trigger","rel":"see also"}],"sources":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Agents","quote":"Agents can then pause for human feedback at checkpoints or when encountering blockers.","work":"anthropic-building-effective-agents"},{"label":"Betsy Beyer et al., Site Reliability Engineering","url":"https://sre.google/sre-book/introduction/","kind":"book","isbn":"9781491929124","page":"Chapter 1, Introduction","quote":"Monitoring should never require a human to interpret any part of the alerting domain. Instead, software should do the interpreting, and humans should be notified only when they need to take action.","work":"site-reliability-engineering"},{"label":"GitHub, Best practices for using GitHub Copilot to work on tasks","url":"https://docs.github.com/en/copilot/tutorials/cloud-agent/get-the-best-results","kind":"maker","page":"Using comments to iterate on a pull request","quote":"After a user with write access mentions `@copilot` in a comment, Copilot will start to make any required changes, and will update the pull request when it's done.","work":"github-copilot-task-practices"}],"id":"human-re-entry","feel":{"shape":0.22,"energy":0.28,"weight":0.45,"warmth":0.72,"grain":0.38,"motion":"settle","why":"Human re-entry is a warm, heavy stop where a decision lands before another run can move."}},{"term":"Slop","aka":["low-value output","review debt"],"kind":"concept","plain":"Fast, plausible output that reaches review without enough evidence or attention to make it trustworthy. It raises triage work.","does":"Names the failure mode where generation outruns verification and human review capacity.","ask":"What evidence is missing, and what gate would make this output safe to merge?","cues":["Large pull requests with vague intent and little test evidence","Repeated review comments or abandoned runs","A growing queue beside a rising generation count"],"at":[-0.45,0.08,0.12],"links":[{"to":"review-queue","rel":"raises"},{"to":"throttle","rel":"lowers"}],"sources":[{"label":"Addy Osmani, Agentic Code Review","url":"https://addyosmani.com/blog/agentic-code-review/","kind":"practitioner","page":"What this means if you run a team","quote":"The steady stream of plausible but hollow contributions costs real triage time even when it is well-intentioned","work":"addy-osmani-agentic-code-review"},{"label":"Birgitta Böckeler, To vibe or not to vibe","url":"https://martinfowler.com/articles/exploring-gen-ai/to-vibe-or-not-vibe.html","kind":"practitioner","page":"Detectability: Will you notice when AI gets it wrong?","quote":"This dimension leans heavily on traditional engineering skills: test coverage, system knowledge, code review practices.","work":"birgitta-bockeler-to-vibe"}],"id":"slop","feel":{"shape":0.82,"energy":0.76,"weight":0.72,"warmth":0.52,"grain":0.9,"motion":"jitter","why":"Slop is sharp, restless, and heavy because it consumes attention while pretending to be finished work."}},{"term":"Human role","aka":["owner","product decider","factory engineer","head chef"],"kind":"concept","plain":"The work left for people once agents write and prove most changes. People decide what should exist and write the specs. They own the one-way doors and the gates, read the diffs of high-risk changes, watch outcomes and cost, and fix the factory when the same mistake repeats. A person attends to the factory when it stops or asks, the way a Toyota worker attends a machine only when it stops on an abnormality. In kitchen terms the person is the head chef, who designs the recipes while agents cook.","does":"Puts scarce human attention on the decisions that only a person can make.","ask":"Which decisions here can only a person make, and where is someone still spending attention on work the factory could prove?","at":[-0.575,0.76,0.06],"links":[{"to":"human-review","rel":"see also"},{"to":"human-re-entry","rel":"see also"},{"to":"pattern-kitchen-brigade","rel":"see also"}],"pattern":"pattern-stop-the-line","sources":[{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Decide what should exist","quote":"The valuable part is deciding whether the request should be built at all, and what it should actually be.","work":"kent-c-dodds-you-own-the-outcome"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"Agents ship most changes; the human owns product decisions.","work":"kent-c-dodds-kcd-skills"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"Product calls and one-way doors go to the human.","work":"kent-c-dodds-kcd-skills"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/docs/contributing/harness-engineering.md","kind":"maker","page":"docs/contributing/harness-engineering.md","quote":"Optimize for human attention as the scarce resource.","work":"kody-repository"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/software-factory","kind":"practitioner","page":"Software factory","quote":"Deciding which of those decisions stay human is the main design question.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Delinea, The Delinea Software Factory: Building Software with AI","url":"https://delinea.com/blog/the-delinea-software-factory-building-software-with-ai","kind":"maker","page":"The Delinea Software Factory","quote":"Humans define intent: what the system should do, the scenarios it needs to handle, the constraints that matter.","work":"delinea-software-factory"},{"label":"Taiichi Ohno, Toyota Production System: Beyond Large-Scale Production","url":"https://api.taylorfrancis.com/content/books/mono/download?identifierName=doi&identifierValue=10.4324/9780429273018&type=googlepdf","kind":"book","isbn":"9780915299140","page":"Chapter 1, Starting from Need (Taylor & Francis preview PDF)","quote":"Only when the machine stops because of an abnormal situation does it get human attention. As a result, one worker can attend several machines, making it possible to reduce the number of operators and increase production efficiency","work":"taiichi-ohno-toyota-production-system"},{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Watch the outcomes","quote":"Build self-healing automations. Things will break. Set things up so that failures get noticed and triaged (often by another agent) instead of silently rotting.","work":"kent-c-dodds-you-own-the-outcome"},{"label":"Gene Kim and Steve Yegge, Vibe Coding: Building Production-Grade Software With GenAI, Chat, Agents, and Beyond","url":"https://itrevolution.com/articles/from-line-cookto-head-chef-orchestrating-ai-teams/","kind":"book","isbn":"9781966280026","page":"From Line Cook to Head Chef: Orchestrating AI Teams (book excerpt)","quote":"Welcome back, head chefs. You’ve mastered working with your AI sous chef.","work":"gene-kim-steve-yegge-vibe-coding"},{"label":"Mary Poppendieck and Tom Poppendieck, Lean Software Development: An Agile Toolkit","url":"https://ptgmedia.pearsoncmg.com/images/0321150783/samplechapter/poppendieckch.pdf","kind":"book","isbn":"9780321150783","page":"Introduction, p. xxvi (publisher's sample chapter)","quote":"Development is like creating a recipe, while production is like making the dish.","work":"poppendieck-lean-software-development"}],"id":"human-role","feel":{"shape":0.28,"energy":0.3,"weight":0.4,"warmth":0.78,"grain":0.32,"motion":"pulse","why":"The human role is warm and steady, and it wakes when the line stops or asks, like a worker who attends a machine only when it signals."}},{"term":"Factory evals","aka":["eval suite","regression evals","replayed tickets"],"kind":"process","plain":"A fixed set of past tickets and scenarios that the factory reruns after any change to its models, prompts, tools or gates. Regression tasks should keep passing at close to 100%. Capability tasks show whether the change made the factory better at work it used to fail. Each eval names a failure mode and counts how often it happens.","does":"Shows whether a change to the factory helped or hurt before the change meets real tickets.","ask":"Which past tasks prove the factory still works, and what result blocks a change to the factory?","cues":["A suite of past tickets with known good outcomes","Pass rates per model, prompt or gate version","A factory change held back by a failing regression eval"],"at":[0.575,1.07,0.06],"links":[{"to":"improvement-loop","rel":"see also"},{"to":"model-choice","rel":"see also"}],"sources":[{"label":"Anthropic, Demystifying evals for AI agents","url":"https://www.anthropic.com/research/demystifying-evals-for-ai-agents","kind":"maker","page":"Demystifying evals for AI agents","quote":"Once evals exist, you get baselines and regression tests for free","work":"anthropic-demystifying-evals"},{"label":"Anthropic, Demystifying evals for AI agents","url":"https://www.anthropic.com/research/demystifying-evals-for-ai-agents","kind":"maker","page":"Demystifying evals for AI agents","quote":"Regression evals ask, ‘Does the agent still handle all the tasks it used to?’ and should have a nearly 100% pass rate.","work":"anthropic-demystifying-evals"},{"label":"Chip Huyen, AI Engineering: Building Applications with Foundation Models","url":"https://huyenchip.com/2025/01/07/agents","kind":"book","isbn":"9781098166267","page":"Agents, adapted from the Agents section of the book (author's blog)","quote":"To evaluate an agent, identify its failure modes and measure how often each of these failure modes happens","work":"ai-engineering-chip-huyen"}],"id":"factory-evals","look":{"parts":[{"slot":"board","shape":"box","size":[1.2,0.9,0.05],"pos":[0,0.55,0],"color":"#f4f5f7","material":"matte","radius":0.04},{"slot":"results","shape":"plane","size":[1.14,0.84],"pos":[0,0.55,0.026],"svg":"FACTORY EVALSv12v13regression99%94%capability41%52%v13 held back"},{"slot":"tickets","shape":"box","size":[0.4,0.16,0.28],"pos":[-0.75,0.08,0.2],"color":"#f7f4ec","material":"matte","radius":0.02}]}},{"term":"Trust loop","aka":["earned autonomy","opening the throttle"],"kind":"loop","plain":"What human review catches becomes new checks, sharper instructions and stronger verification. Fewer bad pull requests reach a person and more merge on evidence, so the team widens the merge policy to the next risk class and the human review queue shrinks. Growth levels off at the work that stays high-risk, where a person still reads the diff.","does":"Explains how a factory moves from human review on every pull request to level 4, where evidence carries low-risk work and people review only what needs them.","ask":"Which evidence earned the last widening of the merge policy, and what signal would narrow it again?","trouble":"The merge policy widens on hope instead of evidence, and defects reach production that a person would have caught.","loop":{"type":"s-curve","through":["human-review","automated-checks","merge-rate","merge-policy","review-queue"]},"sources":[{"label":"Donella Meadows, Leverage Points: Places to Intervene in a System","url":"https://donellameadows.org/archives/leverage-points-places-to-intervene-in-a-system/","kind":"practitioner","page":"8. The strength of negative feedback loops","quote":"Negative feedback loops are ubiquitous in systems. Nature evolves them and humans invent them as controls to keep important system states within safe bounds.","work":"donella-meadows-leverage-points"},{"label":"Addy Osmani, Agentic Code Review","url":"https://addyosmani.com/blog/agentic-code-review/","kind":"practitioner","page":"Should we just let AI review more of it?","quote":"How much human you keep is a dial, and you set it by blast radius, not by guilt.","work":"addy-osmani-agentic-code-review"},{"label":"GitHub, Best practices for using GitHub Copilot to work on tasks","url":"https://docs.github.com/en/copilot/tutorials/cloud-agent/get-the-best-results","kind":"maker","page":"Adding custom instructions to your repository","quote":"If Copilot is able to build, test and validate its changes in its own development environment, it is more likely to produce good pull requests which can be merged quickly.","work":"github-copilot-task-practices"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Combining and customizing these patterns","quote":"the key to success, as with any LLM features, is measuring performance and iterating on implementations.","work":"anthropic-building-effective-agents"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/README.md","kind":"practitioner","page":"pstack","quote":"when you can go deep on one agent and trust it to write good, verifiable code, you can truly parallelize with confidence.","work":"lauren-tan-pstack"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/07-overnight.md","kind":"practitioner","page":"Earn the trust before the loop","quote":"A loop you don't trust just produces unchecked work faster, and the mess compounds with every iteration.","work":"lauren-tan-pstack"},{"label":"Matt Pocock, software-factory skill (pull request #1316)","url":"https://github.com/mattpocock/skills/blob/b59cab42bf1206e1363eafc37f71b21c3bcaa12f/skills/in-progress/software-factory/SKILL.md","kind":"practitioner","page":"Start slow","quote":"a human reads the PR before it merges; what it catches becomes new automated checks and sharper automated review.","work":"matt-pocock-software-factory-skill"},{"label":"Matt Pocock, software-factory skill (pull request #1316)","url":"https://github.com/mattpocock/skills/blob/b59cab42bf1206e1363eafc37f71b21c3bcaa12f/skills/in-progress/software-factory/SKILL.md","kind":"practitioner","page":"Start slow","quote":"Open the throttle as the brakes earn trust, and cap every loop of runs at a few tries before a human.","work":"matt-pocock-software-factory-skill"},{"label":"Lauren Tan, Noodle","url":"https://raw.githubusercontent.com/poteto/noodle/82d2921c52370f23f29086de81ccfb600939c037/docs/concepts/modes.md","kind":"practitioner","page":"docs/concepts/modes.md","quote":"A mature project with good tests can run in auto. A new project starts in supervised. Critical infrastructure stays in manual. Move the dial as trust builds.","work":"lauren-tan-noodle"}],"id":"trust-loop"},{"term":"Improvement loop","aka":["retro loop","the factory learns","friction loop"],"kind":"loop","plain":"Finished runs leave logs and review findings. A retro, by a human or a retro run, finds a mistake that repeats and changes the factory's instructions or checks. A check holds better than a written rule, because nothing fails when an agent skips a rule, so each repeated review comment should become a check where it can. Later runs make that mistake less often, so more of their pull requests merge without a person.","does":"Turns each repeated mistake into a lasting fix in the factory, so quality grows with use.","ask":"Which repeated failure should change the factory next, and which measure will show the change worked?","trouble":"The same review comments appear week after week, and nobody edits the instructions or adds a check.","loop":{"type":"reinforcing","through":["run-log","instructions","ratchet","merge-rate"]},"sources":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Combining and customizing these patterns","quote":"the key to success, as with any LLM features, is measuring performance and iterating on implementations.","work":"anthropic-building-effective-agents"},{"label":"Donella Meadows, Leverage Points: Places to Intervene in a System","url":"https://donellameadows.org/archives/leverage-points-places-to-intervene-in-a-system/","kind":"practitioner","page":"7. The gain around driving positive feedback loops","quote":"Positive feedback loops are sources of growth, explosion, erosion, and collapse in systems.","work":"donella-meadows-leverage-points"},{"label":"Betsy Beyer et al., Site Reliability Engineering","url":"https://sre.google/sre-book/introduction/","kind":"book","isbn":"9781491929124","page":"Chapter 1, Introduction","quote":"This investigation should establish what happened in detail, find all root causes of the event, and assign actions to correct the problem or improve how it is addressed next time.","work":"site-reliability-engineering"},{"label":"Betsy Beyer et al., The Site Reliability Workbook","url":"https://sre.google/workbook/implementing-slos/","kind":"book","isbn":"9781492029458","page":"Chapter 2 - Implementing SLOs","quote":"Your first attempt at an SLI and SLO doesn’t have to be correct; the most important goal is to get something in place and measured, and to set up a feedback loop so you can improve.","work":"site-reliability-workbook"},{"label":"Gene Kim et al., The DevOps Handbook","url":"https://itrevolution.com/wp-content/uploads/2022/06/DOHB2_Excerpt.pdf","kind":"book","isbn":"9781950508402","page":"Part I - Introduction","quote":"The principles of Continual Learning and Experimentation, which foster a high-trust culture and a scientific approach to organizational improvement and risk-taking as part of our daily work.","work":"devops-handbook-gene-kim"},{"label":"GitHub, Best practices for using GitHub Copilot to work on tasks","url":"https://docs.github.com/en/copilot/tutorials/cloud-agent/get-the-best-results","kind":"maker","page":"Adding custom instructions to your repository","quote":"If Copilot is able to build, test and validate its changes in its own development environment, it is more likely to produce good pull requests which can be merged quickly.","work":"github-copilot-task-practices"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/09-make-it-yours.md","kind":"practitioner","page":"Fix the environment with /correct","quote":"When you correct agents for the same mistake again and again, the fix belongs in the repo, not in your next prompt.","work":"lauren-tan-pstack"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/09-make-it-yours.md","kind":"practitioner","page":"Fix the environment with /correct","quote":"Write it down as a doc or agent rule. Nothing fails when an agent skips a rule, so this comes last.","work":"lauren-tan-pstack"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/docs/contributing/harness-engineering.md","kind":"maker","page":"docs/contributing/harness-engineering.md","quote":"When a mistake repeats, encode it in the strongest cheap guardrail that actually works. Prefer a failing check over another paragraph of advice.","work":"kody-repository"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/docs/contributing/harness-engineering.md","kind":"maker","page":"docs/contributing/harness-engineering.md","quote":"Rule of thumb: if reviewers repeat the same comment twice, encode it.","work":"kody-repository"},{"label":"Kent C. Dodds, kcd-skills","url":"https://github.com/kentcdodds/kcd-skills/blob/ffa3faf9911210b89b1d9953e025d52247632070/skills/autopilot/SKILL.md","kind":"practitioner","page":"autopilot skill","quote":"Agents file friction after every PR with a priority; other agents fix it, highest priority first.","work":"kent-c-dodds-kcd-skills"}],"id":"improvement-loop"},{"term":"Accelerator with a brake","aka":["speed and gate"],"kind":"pattern","plain":"A fast path increases flow while an independent gate limits what can continue. The useful system tunes both together, and builds the brake before the accelerator.","ask":"What accelerates the work, and what stops unsafe output before it spreads?","sources":[{"label":"Donella Meadows, Leverage Points: Places to Intervene in a System","url":"https://donellameadows.org/archives/leverage-points-places-to-intervene-in-a-system/","kind":"practitioner","page":"8. The strength of negative feedback loops","quote":"Negative feedback loops are ubiquitous in systems. Nature evolves them and humans invent them as controls to keep important system states within safe bounds.","work":"donella-meadows-leverage-points"},{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","page":"Workflow: Evaluator-optimizer","quote":"In the evaluator-optimizer workflow, one LLM call generates a response while another provides evaluation and feedback in a loop.","work":"anthropic-building-effective-agents"},{"label":"Matt Pocock, Software Factories for Real Engineers","url":"https://www.aihero.dev/cohorts/software-factories-for-real-engineers-xdy1m","kind":"practitioner","page":"Software Factories for Real Engineers","quote":"So most folks start building accelerators … That's why starting with accelerators is backwards.","work":"matt-pocock-software-factories"}],"id":"pattern-accelerator-brake","feel":{"shape":0.42,"energy":0.72,"weight":0.18,"warmth":0.46,"grain":0.28,"motion":"pulse","why":"An accelerator with a brake feels fast and controlled because motion always meets a stopping force."}},{"term":"Defense in depth","aka":["layered gates","Swiss cheese barriers"],"kind":"pattern","plain":"Several independent layers reduce the chance that one failure reaches the final outcome. Each layer has gaps, so safety comes from their combination and from learning when gaps align.","ask":"Which independent layers stand between an unsafe change and merge, and how do they fail differently?","sources":[{"label":"James Reason, Human error: models and management","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC1117770/","kind":"research","page":"The Swiss cheese model of system accidents","quote":"Usually, this can happen only when the holes in many layers momentarily line up to permit a trajectory of accident opportunity","work":"james-reason-human-error"},{"label":"James Reason, Human error: models and management","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC1117770/","kind":"research","page":"System approach","quote":"The system approach concentrates on the conditions under which individuals work and tries to build defences to avert errors or mitigate their effects","work":"james-reason-human-error"},{"label":"Heather Adkins et al., Building Secure and Reliable Systems","url":"https://google.github.io/building-secure-and-reliable-systems/raw/ch08.html","kind":"book","isbn":"9781492083115","page":"Design for Resilience","quote":"Defense in depth protects systems by establishing multiple layers of defense perimeters.","work":"building-secure-reliable-systems"}],"id":"pattern-defense-in-depth","feel":{"shape":0.6,"energy":0.28,"weight":0.78,"warmth":0.3,"grain":0.72,"motion":"pulse","why":"Layered defenses feel dense and quiet because several imperfect barriers share the burden of stopping harm."}},{"term":"Holdout set","aka":["test set","held-out data","hidden test"],"kind":"pattern","plain":"Keep part of the evidence away from whatever is being tuned, and judge the result only on that hidden part. A score on examples the learner has already seen says little about how it will do on new ones.","ask":"What evidence has the learner never seen, and who holds it?","sources":[{"label":"scikit-learn 1.5, Cross-validation: evaluating estimator performance","url":"https://scikit-learn.org/1.5/modules/cross_validation.html","kind":"neighbour","page":"3.1. Cross-validation: evaluating estimator performance","quote":"Learning the parameters of a prediction function and testing it on the same data is a methodological mistake","work":"scikit-learn-cross-validation"},{"label":"scikit-learn 1.5, Cross-validation: evaluating estimator performance","url":"https://scikit-learn.org/1.5/modules/cross_validation.html","kind":"neighbour","page":"3.1. Cross-validation: evaluating estimator performance","quote":"it is common practice when performing a (supervised) machine learning experiment to hold out part of the available data as a test set","work":"scikit-learn-cross-validation"},{"label":"Simon Willison, How StrongDM’s AI team build serious software without even looking at the code","url":"https://simonwillison.net/2026/Feb/7/software-factory/","kind":"practitioner","quote":"That idea of treating scenarios as holdout sets … used to evaluate the software but not stored where the coding agents can see them … is fascinating.","work":"simon-willison-software-factory"}],"id":"pattern-holdout-set","feel":{"shape":0.66,"energy":0.2,"weight":0.62,"warmth":0.22,"grain":0.2,"motion":"settle","why":"A holdout set is sealed, cool and heavy: evidence locked away from whatever is being tuned."}},{"term":"Kitchen brigade","aka":["brigade de cuisine","brigade system","kitchen brigade model"],"kind":"pattern","plain":"A head chef designs the dishes and holds the standard, each cook works one station from ingredients prepared before service, and the chef checks plates at the pass and sends back any that fall short. Agent tools reuse the roles, with a scheduler that writes orders and agents that cook as line cooks, each in its own worktree.","ask":"Who designs the dishes, which station does each cook own, what is ready before service starts, and who checks each plate before it leaves?","trouble":"Service starts before mise en place is done, and cooks keep running back to the walk-in. The chef trusts no station and would rather cook every plate alone. Run as a strict line, the brigade keeps quality consistent but leaves little room to experiment.","sources":[{"label":"Conrad N. Hilton Library, Pages of the Past: Georges Auguste Escoffier","url":"https://library.culinary.edu/pagesofthepast/escoffier","kind":"neighbour","page":"French Brigade","quote":"The French Brigade system, inspired by Escoffier’s military experience in the Franco-Prussian War of 1870, became the organizational standard in professional kitchens.","work":"cia-library-escoffier"},{"label":"Conrad N. Hilton Library, Pages of the Past: Georges Auguste Escoffier","url":"https://library.culinary.edu/pagesofthepast/escoffier","kind":"neighbour","page":"French Brigade","quote":"The Sous Chef assists the Chef de Cuisine, and the Chefs de Partie manage specific stations based on a cooking method, such as grill or sauté.","work":"cia-library-escoffier"},{"label":"Conrad N. Hilton Library, Pages of the Past: Georges Auguste Escoffier","url":"https://library.culinary.edu/pagesofthepast/escoffier","kind":"neighbour","page":"French Brigade","quote":"What separated Auguste Escoffier from the other French Chefs of the 20th century was his standard for culinary excellence.","work":"cia-library-escoffier"},{"label":"Mary Poppendieck and Tom Poppendieck, Lean Software Development: An Agile Toolkit","url":"https://ptgmedia.pearsoncmg.com/images/0321150783/samplechapter/poppendieckch.pdf","kind":"book","isbn":"9780321150783","page":"Introduction, p. xxvi (publisher's sample chapter)","quote":"Recipes are designed by experienced chefs who have developed an instinct for what works and the capability to adapt available ingredients to suit the occasion.","work":"poppendieck-lean-software-development"},{"label":"Samuel Ashworth, Super Sad True Chef Story","url":"https://www.eater.com/2019/8/12/20799538/french-brigade-system-stage-chef-michelin-restaurants","kind":"neighbour","quote":"a system in which each cook is assigned a specific station within a hierarchy, with the head chef at the top","work":"samuel-ashworth-super-sad-true-chef-story"},{"label":"Samuel Ashworth, Super Sad True Chef Story","url":"https://www.eater.com/2019/8/12/20799538/french-brigade-system-stage-chef-michelin-restaurants","kind":"neighbour","quote":"mise en place was never quite done by the time service began and we would continually have to sprint back to the walk-in","work":"samuel-ashworth-super-sad-true-chef-story"},{"label":"Samuel Ashworth, Super Sad True Chef Story","url":"https://www.eater.com/2019/8/12/20799538/french-brigade-system-stage-chef-michelin-restaurants","kind":"neighbour","quote":"two rack of lambs hit the pass so underdone that she noticed from 8 feet away","work":"samuel-ashworth-super-sad-true-chef-story"},{"label":"Samuel Ashworth, Super Sad True Chef Story","url":"https://www.eater.com/2019/8/12/20799538/french-brigade-system-stage-chef-michelin-restaurants","kind":"neighbour","quote":"The solution to an undercooked fish wasn’t shouting or threatening; it was to cook it a little more.","work":"samuel-ashworth-super-sad-true-chef-story"},{"label":"Samuel Ashworth, Super Sad True Chef Story","url":"https://www.eater.com/2019/8/12/20799538/french-brigade-system-stage-chef-michelin-restaurants","kind":"neighbour","quote":"she was obliged to entrust any of her food to them at all. If she could have done it solo, she would have.","work":"samuel-ashworth-super-sad-true-chef-story"},{"label":"Samuel Ashworth, Super Sad True Chef Story","url":"https://www.eater.com/2019/8/12/20799538/french-brigade-system-stage-chef-michelin-restaurants","kind":"neighbour","quote":"This system made French cuisine a bastion of quality control and consistency, but it left little space for innovation or experimentation","work":"samuel-ashworth-super-sad-true-chef-story"},{"label":"Lauren Tan, Noodle","url":"https://raw.githubusercontent.com/poteto/noodle/82d2921c52370f23f29086de81ccfb600939c037/INSTALL.md","kind":"practitioner","page":"INSTALL.md","quote":"Think kitchen brigade: a head chef (the scheduler) reads the backlog, writes orders, and dispatches line cooks (agents) to do the work in isolated worktrees.","work":"lauren-tan-noodle"},{"label":"Lauren Tan, Noodle","url":"https://raw.githubusercontent.com/poteto/noodle/82d2921c52370f23f29086de81ccfb600939c037/docs/concepts/scheduling.md","kind":"practitioner","page":"docs/concepts/scheduling.md","quote":"Before each scheduling decision, Noodle builds a mise (a snapshot of everything the scheduler needs).","work":"lauren-tan-noodle"},{"label":"Gene Kim and Steve Yegge, Vibe Coding: Building Production-Grade Software With GenAI, Chat, Agents, and Beyond","url":"https://itrevolution.com/articles/from-line-cookto-head-chef-orchestrating-ai-teams/","kind":"book","isbn":"9781966280026","page":"From Line Cook to Head Chef: Orchestrating AI Teams (book excerpt)","quote":"Much like early-days chefs figuring out how to run a multi-station kitchen, we’re often improvising","work":"gene-kim-steve-yegge-vibe-coding"}],"id":"pattern-kitchen-brigade","feel":{"shape":0.45,"energy":0.72,"weight":0.3,"warmth":0.88,"grain":0.55,"motion":"orbit","why":"A brigade is a hot, busy ring of stations that turns around one chef and one pass."}},{"term":"Ratchet","aka":["shrinking exception set"],"kind":"pattern","plain":"A baseline check accepts known exceptions while rejecting new failures, then removes exceptions as the system improves. The boundary moves in one direction.","ask":"What exception can disappear next, and what prevents the baseline from loosening?","sources":[{"label":"Neal Ford, Rebecca Parsons and Patrick Kua, Architectural Fitness Functions","url":"https://evolutionaryarchitecture.com/ffkatas/","kind":"practitioner","page":"Architectural Fitness Functions","quote":"An architectural fitness function provides an objective integrity assessment of some architectural characteristic(s).","work":"evolutionaryarchitecture-fitness-functions"},{"label":"ESLint, Command Line Interface: Suppress Violations","url":"https://eslint.org/docs/latest/use/command-line-interface","kind":"maker","page":"--prune-suppressions","quote":"Prune unused suppressions from the suppressions file. This option is useful when you addressed one or more of the suppressed violations.","work":"eslint-cli-suppressions"},{"label":"ESLint, Command Line Interface: Suppress Violations","url":"https://eslint.org/docs/latest/use/command-line-interface","kind":"maker","page":"--suppress-all","quote":"It allows you to enable one or more lint rules and be notified only when new violations show up.","work":"eslint-cli-suppressions"}],"id":"pattern-ratchet","feel":{"shape":0.72,"energy":0.36,"weight":0.62,"warmth":0.38,"grain":0.58,"motion":"settle","why":"A ratchet has hard teeth and one-way motion, so each improvement holds while exceptions fall away."}},{"term":"Stateless worker with external memory","aka":["share-nothing worker"],"kind":"pattern","plain":"A fresh worker performs one bounded job, then stores any state needed later in durable services. A replacement worker can continue from those services. A coding-agent loop does this with a progress file and commits that the next run reads first.","ask":"What survives this worker, where is it stored, and can the next worker resume without local files?","sources":[{"label":"Adam Wiggins, The Twelve-Factor App: Processes","url":"https://www.12factor.net/processes","kind":"standard","page":"VI. Processes","quote":"Twelve-factor processes are stateless and share-nothing. Any data that needs to persist must be stored in a stateful backing service, typically a database.","work":"twelve-factor-processes"},{"label":"Matt Pocock, 11 Tips For AI Coding With Ralph Wiggum","url":"https://www.aihero.dev/tips-for-ai-coding-with-ralph-wiggum","kind":"practitioner","page":"4. Track Ralph's Progress","quote":"AI agents are like super-smart experts who forget everything between tasks. Each new context window starts fresh. Without a progress file, Ralph must explore the entire repo to understand the current state.","work":"matt-pocock-ralph-tips"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/stateless","kind":"practitioner","page":"Stateless","quote":"An agent is stateless across sessions by default: a new session starts empty, with no trace of prior ones.","work":"matt-pocock-ai-coding-dictionary"},{"label":"Matt Pocock, AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary/memory-system","kind":"practitioner","page":"Memory system","quote":"A system that attempts to make an agent stateful across sessions. Persists information into the environment during a session and reloads it into the context window at the start of future ones, so the agent carries continuity beyond the user clearing the session.","work":"matt-pocock-ai-coding-dictionary"}],"id":"pattern-stateless-worker","feel":{"shape":0.3,"energy":0.62,"weight":-0.08,"warmth":0.32,"grain":0.2,"motion":"drift","why":"A stateless worker is light and replaceable, with memory flowing through durable services outside the worker."}},{"term":"Stop the line","aka":["jidoka","andon"],"kind":"pattern","plain":"Detect an abnormal condition, stop propagation, show the signal, and call a person who can correct the cause. Work resumes after the abnormality is understood.","ask":"What signal stops the flow, who is called, and what evidence allows the line to resume?","sources":[{"label":"Toyota Motor Corporation, Toyota Production System","url":"https://global.toyota/en/company/vision-and-philosophy/production-system/","kind":"maker","page":"The Two Pillars of TPS","quote":"jidoka―which can be loosely translated as \"automation with a human touch\"―based on the concepts of stopping immediately when abnormalities are detected","work":"toyota-production-system"},{"label":"Toyota Motor Corporation, Toyota Production System","url":"https://global.toyota/en/company/vision-and-philosophy/production-system/","kind":"maker","page":"Jidoka","quote":"when an abnormality occurs, such as a machine or equipment abnormality, quality abnormality, or a work delay, the machine or equipment can detect the abnormality and stop automatically","work":"toyota-production-system"},{"label":"Lauren Tan, pstack","url":"https://raw.githubusercontent.com/cursor/plugins/9580c23696ac8d7abed0c166f2f9918800afb819/pstack/docs/guide/07-overnight.md","kind":"practitioner","page":"Let loops start themselves","quote":"Every stage can stop the line. Triage can decide the report is expected behavior, repro can fail to reproduce it, and the fixer can judge the change too risky.","work":"lauren-tan-pstack"},{"label":"Taiichi Ohno, Toyota Production System: Beyond Large-Scale Production","url":"https://api.taylorfrancis.com/content/books/mono/download?identifierName=doi&identifierValue=10.4324/9780429273018&type=googlepdf","kind":"book","isbn":"9780915299140","page":"Chapter 1, Starting from Need (Taylor & Francis preview PDF)","quote":"At Toyota, a machine automated with a human touch is one that is attached to an automatic stopping device","work":"taiichi-ohno-toyota-production-system"},{"label":"Taiichi Ohno, Toyota Production System: Beyond Large-Scale Production","url":"https://api.taylorfrancis.com/content/books/mono/download?identifierName=doi&identifierValue=10.4324/9780429273018&type=googlepdf","kind":"book","isbn":"9780915299140","page":"Chapter 1, Starting from Need (Taylor & Francis preview PDF)","quote":"the workers themselves should push the stop button to halt production if any abnormality appears.","work":"taiichi-ohno-toyota-production-system"}],"id":"pattern-stop-the-line","feel":{"shape":0.7,"energy":0.42,"weight":0.88,"warmth":0.68,"grain":0.5,"motion":"settle","why":"Stop the line is a heavy red pause with a warm human signal that prevents a defect from moving downstream."}},{"term":"Two-way door","aka":["reversible decision","one-way door","Type 1 and Type 2 decisions"],"kind":"pattern","plain":"Sort decisions by how hard they are to undo. Make the reversible ones quickly and close to the work. Slow down, and bring in more people, only for the few that are costly or impossible to reverse.","ask":"If this turns out to be wrong, how costly is it to undo?","sources":[{"label":"Jeff Bezos (Amazon), 2015 Letter to Shareholders","url":"https://www.sec.gov/Archives/edgar/data/1018724/000119312516530910/d168744dex991.htm","kind":"neighbour","page":"2015 Letter to Shareholders","quote":"Some decisions are consequential and irreversible or nearly irreversible … and these decisions must be made methodically, carefully, slowly, with great deliberation and consultation.","work":"amazon-2015-shareholder-letter"},{"label":"Jeff Bezos (Amazon), 2015 Letter to Shareholders","url":"https://www.sec.gov/Archives/edgar/data/1018724/000119312516530910/d168744dex991.htm","kind":"neighbour","page":"2015 Letter to Shareholders","quote":"But most decisions aren’t like that … they are changeable, reversible … they’re two-way doors.","work":"amazon-2015-shareholder-letter"},{"label":"Jeff Bezos (Amazon), 2015 Letter to Shareholders","url":"https://www.sec.gov/Archives/edgar/data/1018724/000119312516530910/d168744dex991.htm","kind":"neighbour","page":"2015 Letter to Shareholders","quote":"Type 2 decisions can and should be made quickly by high judgment individuals or small groups.","work":"amazon-2015-shareholder-letter"},{"label":"Kody, kody-hq/kody repository","url":"https://github.com/kody-hq/kody/blob/51c3a853c43e67600ab1bde6eb08a2965371b050/docs/principles/two-way-doors.md","kind":"maker","page":"docs/principles/two-way-doors.md","quote":"A two-way door is cheap to reverse. Ship it and change it if it is wrong. A one-way door is expensive to undo","work":"kody-repository"},{"label":"Kent C. Dodds, You Own the Outcome: Product Engineering in the Age of AI","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"practitioner","page":"Then, ask for options instead of an answer","quote":"Two-way doors you can walk through quickly and let the agent run. One-way doors (public API names, data shapes, pricing) are where you slow down and think.","work":"kent-c-dodds-you-own-the-outcome"}],"id":"pattern-two-way-door","feel":{"shape":0.42,"energy":0.56,"weight":-0.15,"warmth":0.55,"grain":0.26,"motion":"orbit","why":"A two-way door is light and swings both ways, so most decisions can be walked through and walked back."}}],"assembly":[{"title":"Pick the problem and the level","text":"Start from one problem the team has now, such as untriaged tickets or pull requests waiting for review. Choose how far agents may go without a human, and where the human comes back in. Plan to buy the services and build the tasks.","adds":["start-slow","buy-vs-build","autonomy-level","human-re-entry","slop"],"makes":"factory-design"},{"title":"Fill the ticket tracker slot","text":"Choose where tickets live and which labels or statuses route work to a task. Connect the sources that file tickets from outside, and decide what stops each one filing the same ticket twice.","adds":["ticket","routing-label","comment-thread","ticket-source"],"makes":"ticket-tracker"},{"title":"Fill the git host slot","text":"Give the factory its own bot identity, protect the main branch with required checks and reviews, and send signed webhooks to the orchestrator.","adds":["pull-request","bot-identity","branch-protection","webhook"],"makes":"git-host"},{"title":"Draw the security boundary","text":"Treat every ticket, comment and web page as untrusted input, and give each run only the permissions its task needs.","adds":["prompt-injection","least-privilege"],"makes":"security-boundary"},{"title":"Set up a disposable runner inside the boundary","text":"Give every run a fresh machine or container inside the security boundary, with a timeout, an optional cache, scoped secrets and a narrow network allowlist.","adds":["disposable-environment","run-timeout","cache","network-egress","secrets","security-boundary"],"makes":"runner"},{"title":"Install the brakes","text":"Make types, lint and tests a required check, and add an automated review run with a try cap on rework. Keep human review on every pull request for now.","adds":["automated-checks","automated-review","review-rework-loop","human-review"],"makes":"brakes"},{"title":"Configure the coding agent","text":"Pick the harness, the model and how much reasoning effort it spends. Pre-approve the tools an unattended run needs, because nobody is there to answer a permission request, and let the sandbox limit what they can reach. Size each task so it fits one fresh context window.","adds":["agent-harness","model-choice","effort","context-window","permission-mode"],"makes":"coding-agent"},{"title":"Write the first task","text":"Define the task's trigger, its work, the coding agent that runs it, the run it triggers next and where a human re-enters.","adds":["run","run-chain"],"makes":"task"},{"title":"Wire the orchestrator","text":"Map each trigger to a task, record every run's state, and claim work so two runs never do the same job. Set the concurrency limit, the throttle, the try cap and the budget cap low.","adds":["trigger","run-state","work-claim","concurrency-limit","throttle","try-cap","budget-cap"],"makes":"orchestrator"},{"title":"Measure the factory","text":"Track how many agent pull requests merge, how long work waits, how often merged changes fail, and how long the review queue is.","adds":["merge-rate","lead-time","change-failure-rate","review-queue"],"makes":"factory-metrics"},{"title":"Watch the runs","text":"Read each run's log on the run page. Add traces and token cost when the run page cannot explain a failure or a bill, and show the factory's metrics beside them.","adds":["run-log","agent-trace","token-cost","factory-metrics"],"makes":"observability"},{"title":"Make the codebase agent-ready","text":"Keep the code in deep modules with clear names. Write a short instructions map and a glossary, give agents one verify command and a CLI that drives the app, and connect the tools they need. Turn each review comment that keeps coming back into a check.","adds":["codebase-health","instructions","shared-vocabulary","verify-command","control-cli","tool-access","ratchet"],"makes":"agent-ready-codebase"},{"title":"Build verification","text":"Write a spec for each piece of work and turn it into scenarios. Keep some scenarios where the author cannot see them, and stand up twins of outside services. Each run proves its own change, then a separate verifier scores it against the scenarios over several trials, because the same ticket can give a different change on each run.","adds":["spec","scenario","holdout-scenario","digital-twin","self-verification","satisfaction","non-determinism"],"makes":"verification"},{"title":"Merge by risk","text":"Label each change by what it composes, extends or adds. Let agents merge low-risk work once checks and evidence pass, and keep a person on high-risk changes and on the gates. Widen the policy only as the evidence holds.","adds":["risk-class","merge-policy","review-bottleneck"],"makes":"brakes"},{"title":"Open the throttle and run in parallel","text":"Turn what review catches into checks, instructions and scenarios, and open the throttle as trust grows. Run several tasks at once, each in its own worktree or runner, and rerun the factory's evals after every change to it. People decide what to build and own the one-way doors. The services, agents, tasks and gates now run as one software factory.","adds":["ticket-tracker","git-host","orchestrator","runner","coding-agent","agent-ready-codebase","task","verification","brakes","observability","factory-design","parallel-runs","human-role","factory-evals","trust-loop","improvement-loop"],"makes":"software-factory"}],"survey":{"lists":[{"id":"continuous-delivery-reading-list","label":"Continuous Delivery, recommended reading","url":"https://continuousdelivery.com/","kind":"practitioner"},{"id":"dora-recommended-reading","label":"DORA, recommended reading and resources","url":"https://dora.dev/resources/","kind":"bibliography"},{"id":"octopus-devops-reading-list","label":"Octopus Deploy, DevOps reading list","url":"https://octopus.com/devops/reading-list/devops/","kind":"practitioner"},{"id":"it-revolution-devops-books","label":"IT Revolution, DevOps books catalogue","url":"https://itrevolution.com/devops-books/","kind":"publisher"},{"id":"awesome-devops-books","label":"jdrowne, Awesome DevOps books","url":"https://github.com/jdrowne/awesome-devops-books","kind":"community"},{"id":"devbooks-devops","label":"DevToolsD, DevOps book list","url":"https://raw.githubusercontent.com/devtoolsd/DevBooks/90462d87327dd1c055734c22e0c37fb4b6991b05/data/devops.md","kind":"community"},{"id":"ccsl-devops-readings","label":"CCSL, DevOps readings","url":"https://ccsl.ime.usp.br/devops/2019-11-06/devops-readings.html","kind":"course"},{"id":"leanpub-devops-reading-list","label":"Leanpub, DevOps reading list","url":"https://leanpub.com/read/devopstheopspective/leanpub-auto-a-devops-reading-list","kind":"practitioner"},{"id":"sre-google-books","label":"Google SRE, SRE books and online books","url":"https://sre.google/books/","kind":"publisher"},{"id":"sre-google-bibliography","label":"Google SRE, Site Reliability Engineering bibliography","url":"https://sre.google/sre-book/bibliography/","kind":"bibliography"},{"id":"sre-manifesto","label":"SRE Manifesto, SRE resources","url":"https://www.sre-manifesto.org/sre/resources/","kind":"practitioner"},{"id":"awesome-sre","label":"Dastergon, Awesome SRE","url":"https://github.com/dastergon/awesome-sre","kind":"community"},{"id":"microsoft-sre-books","label":"Microsoft Learn, SRE books","url":"https://learn.microsoft.com/en-us/azure/site-reliability-engineering/resources/books","kind":"publisher"},{"id":"team-topologies-reading-list","label":"Team Topologies, book and further reading","url":"https://teamtopologies.com/book","kind":"practitioner"},{"id":"devbooks-architecture","label":"DevToolsD, software architecture book list","url":"https://raw.githubusercontent.com/devtoolsd/DevBooks/90462d87327dd1c055734c22e0c37fb4b6991b05/data/architecture.md","kind":"community"},{"id":"atlassian-architecture-reading","label":"OpenDevEx, Architecture recommended reading list","url":"https://opendex.atlassian.net/wiki/spaces/AC/pages/679772564/Architecture+Recommended+Reading+List","kind":"bibliography"},{"id":"lean-institute-books","label":"Lean Enterprise Institute, recommended Lean books","url":"https://www.lean.org/the-lean-post/articles/ask-art-what-lean-books-should-i-start-with/","kind":"practitioner"},{"id":"umich-lean-guides","label":"University of Michigan, Lean and value-stream guides","url":"https://kresgeguides.bus.umich.edu/Lean-ValueStreams/Core-Guides","kind":"bibliography"},{"id":"toc-institute-recommended","label":"Theory of Constraints Institute, recommended books","url":"https://www.tocinstitute.org/recommended-books.html","kind":"publisher"},{"id":"cs-klub-swe-books","label":"CS Klub, essential software engineering books","url":"https://github.com/csklub/swe-essential-books","kind":"community"},{"id":"devbooks-software-engineering","label":"DevToolsD, software engineering book list","url":"https://raw.githubusercontent.com/devtoolsd/DevBooks/90462d87327dd1c055734c22e0c37fb4b6991b05/data/software-engineering.md","kind":"community"},{"id":"awesome-software-engineering","label":"Collided, Awesome software engineering","url":"https://github.com/Collided/awesome-software-engineering","kind":"community"},{"id":"stanford-cs146s","label":"Stanford CS146S, The Modern Software Developer syllabus","url":"https://themodernsoftware.dev/","kind":"course"},{"id":"stanford-cs329a","label":"Stanford CS329A, Self-Improving AI Agents syllabus","url":"https://cs329a.stanford.edu/","kind":"course"},{"id":"fullstack-llm-bootcamp","label":"Full Stack Deep Learning, LLM Bootcamp","url":"https://fullstackdeeplearning.com/llm-bootcamp/","kind":"course"},{"id":"huggingface-agents-course","label":"Hugging Face, Agents Course","url":"https://huggingface.co/learn/agents-course/en/unit1/introduction","kind":"course"},{"id":"fullstack-mlops-course","label":"Full Stack Deep Learning, MLOps course","url":"https://fullstackdeeplearning.com/course/2022/","kind":"course"},{"id":"devbooks-ai-engineering","label":"DevToolsD, AI engineering book list","url":"https://raw.githubusercontent.com/devtoolsd/DevBooks/90462d87327dd1c055734c22e0c37fb4b6991b05/data/ai-engineering.md","kind":"community"},{"id":"computingforgeeks-llm-books","label":"ComputingForGeeks, LLM engineering books","url":"https://computingforgeeks.com/best-llm-engineering-books/","kind":"practitioner"},{"id":"computingforgeeks-ai-coding-books","label":"ComputingForGeeks, best AI coding books","url":"https://computingforgeeks.com/best-ai-coding-books/","kind":"practitioner"},{"id":"stanford-cs329z","label":"Stanford CS329Z, Engineering AI Agents","url":"https://cs329z.stanford.edu/","kind":"course"},{"id":"duke-compsci390-readings","label":"Duke CompSci 390, Fall 2025 readings","url":"https://courses.cs.duke.edu/compsci390.2/fall25/announcements.php","kind":"course"},{"id":"cis4394-agentic-ai-resources","label":"CIS 4394 Agentic AI, resources and courses elsewhere","url":"https://xinyufu.org/cis4394-agentic-ai/courses.html","kind":"course"},{"id":"agentic-engineering-handbook","label":"keyuchen21, Agentic engineering handbook","url":"https://github.com/keyuchen21/agentic-engineering-handbook","kind":"community"},{"id":"agentic-engineering-field-study","label":"ianhxu, agentic engineering field study learning resources","url":"https://github.com/ianhxu/agentic-engineering-field-study/blob/2d062706483c5bca3faf79febb0f59cd1f639c25/08-learning-resources.md","kind":"community"},{"id":"owasp-genai-educational","label":"OWASP GenAI Security Project, educational resources","url":"https://github.com/OWASP/www-project-top-10-for-large-language-model-applications/wiki/Educational-Resources","kind":"bibliography"},{"id":"awesome-ai-security","label":"BrinhoSa, Awesome AI security","url":"https://github.com/BrinhoSa/awesome-ai-security","kind":"community"}],"canon":[{"label":"Anthropic, Building effective AI agents","url":"https://www.anthropic.com/research/building-effective-agents","kind":"maker","why":"The reading courses assign most for agent design: when a fixed workflow is enough, when to give a model autonomy, and how to keep agents simple, transparent and tested in a sandbox.","named":["stanford-cs329z","duke-compsci390-readings","cis4394-agentic-ai-resources","agentic-engineering-handbook","agentic-engineering-field-study"],"specialism":"AI coding agents","work":"anthropic-building-effective-agents"},{"label":"Betsy Beyer et al., Site Reliability Engineering","url":"https://sre.google/books/","kind":"book","isbn":"9781491929124","why":"The foundational operating model for reliability engineering, SLOs, toil reduction, monitoring, automation, and incident response.","named":["sre-google-books","sre-google-bibliography","sre-manifesto","microsoft-sre-books","awesome-sre"],"specialism":"site reliability and observability","work":"site-reliability-engineering"},{"label":"Gene Kim et al., The DevOps Handbook","url":"https://itrevolution.com/product/the-devops-handbook-second-edition/","kind":"book","isbn":"9781950508402","why":"A broad operating guide for flow, feedback, continuous delivery, reliability, security, and organizational improvement.","named":["continuous-delivery-reading-list","it-revolution-devops-books","octopus-devops-reading-list","awesome-devops-books"],"specialism":"continuous integration, continuous delivery and DevOps","work":"devops-handbook-gene-kim"},{"label":"Jez Humble and David Farley, Continuous Delivery","url":"https://martinfowler.com/books/continuousDelivery.html","kind":"book","isbn":"0321601912","why":"The practical foundation for turning integrated changes into repeatable, low-risk production releases.","named":["continuous-delivery-reading-list","dora-recommended-reading","it-revolution-devops-books","awesome-devops-books"],"specialism":"continuous integration and continuous delivery","work":"martin-fowler-continuous-delivery"},{"label":"Nicole Forsgren et al., Accelerate","url":"https://itrevolution.com/product/accelerate/","kind":"book","isbn":"9781942788331","why":"The research base for delivery throughput, stability, and the capabilities that improve both.","named":["dora-recommended-reading","it-revolution-devops-books","continuous-delivery-reading-list","octopus-devops-reading-list"],"specialism":"platform engineering and developer productivity metrics","work":"accelerate-nicole-forsgren"},{"label":"Betsy Beyer et al., The Site Reliability Workbook","url":"https://books.google.com/books?id=fElmDwAAQBAJ","kind":"book","isbn":"9781492029458","why":"Concrete SRE practices for SLOs, monitoring, incident response, and reducing operational overload.","named":["sre-google-books","microsoft-sre-books","awesome-sre"],"specialism":"site reliability and observability","work":"site-reliability-workbook"},{"label":"Chip Huyen, AI Engineering: Building Applications with Foundation Models","url":"https://books.google.com/books/about/AI_Engineering.html?id=abM1EQAAQBAJ","kind":"book","isbn":"9781098166267","why":"Practitioners get a systems guide to foundation-model applications, agents, evaluation, deployment, latency, and cost.","named":["devbooks-ai-engineering","computingforgeeks-llm-books","agentic-engineering-field-study"],"specialism":"AI and agent engineering","work":"ai-engineering-chip-huyen"},{"label":"Gene Kim et al., The Phoenix Project","url":"https://itrevolution.com/product/the-phoenix-project/","kind":"book","isbn":"9781950508945","why":"Practitioners get an accessible narrative of constraints, work types, flow, and operational pressure.","named":["it-revolution-devops-books","octopus-devops-reading-list","awesome-devops-books"],"specialism":"lean flow and DevOps operations","work":"phoenix-project-gene-kim"},{"label":"Heather Adkins et al., Building Secure and Reliable Systems","url":"https://sre.google/books/","kind":"book","isbn":"9781492083115","why":"A lifecycle view of security and reliability with design, coding, testing, incident response, and defense practices.","named":["sre-google-books","microsoft-sre-books","sre-manifesto"],"specialism":"secure and reliable systems","work":"building-secure-reliable-systems"},{"label":"Martin Fowler, Refactoring: Improving the Design of Existing Code","url":"https://www.informit.com/store/refactoring-improving-the-design-of-existing-code-9780134757599","kind":"book","isbn":"9780134757599","why":"The standard catalogue and rationale for small behavior-preserving changes that keep a codebase easy to evolve.","named":["cs-klub-swe-books","devbooks-software-engineering","awesome-software-engineering"],"specialism":"code review and software quality","work":"refactoring-martin-fowler"},{"label":"Matthew Skelton and Manuel Pais, Team Topologies","url":"https://teamtopologies.com/book","kind":"book","isbn":"9781942788812","why":"Practitioners get a model for team boundaries, platform teams, cognitive load, and fast flow.","named":["team-topologies-reading-list","it-revolution-devops-books","octopus-devops-reading-list"],"specialism":"platform engineering and team interaction","work":"team-topologies-skelton-pais"},{"label":"Steve McConnell, Code Complete, 2nd Edition","url":"https://www.microsoftpressstore.com/store/code-complete-9780735619678","kind":"book","isbn":"9780735619678","why":"Practitioners get a broad construction reference for complexity, defensive programming, testing, debugging, and quality gates.","named":["cs-klub-swe-books","devbooks-software-engineering","awesome-software-engineering"],"specialism":"code review and software quality","work":"code-complete-steve-mcconnell"},{"label":"Titus Winters et al., Software Engineering at Google","url":"https://abseil.io/resources/swe-book","kind":"book","isbn":"9781492082798","why":"A large-scale account of sustainable software engineering, code review, testing, ownership, and change over time.","named":["cs-klub-swe-books","devbooks-software-engineering","awesome-software-engineering"],"specialism":"code review and software quality","work":"software-engineering-google"},{"label":"Addy Osmani, Beyond Vibe Coding: From Coder to AI-Era Developer","url":"https://www.oreilly.com/library/view/beyond-vibe-coding/9798341634749/","kind":"book","isbn":"9798341634749","why":"Practitioners get a developer's path from prompting a chat assistant to handing whole tasks to autonomous coding agents and reviewing what comes back.","named":["computingforgeeks-ai-coding-books","agentic-engineering-field-study"],"specialism":"AI coding agents","work":"addy-osmani-beyond-vibe-coding"},{"label":"Anthropic, Best practices for Claude Code","url":"https://code.claude.com/docs/en/best-practices","kind":"maker","why":"The maker's guide to working with a coding agent: give it a way to verify its own work, explore and plan before coding, manage context, and run it headless or in parallel.","named":["stanford-cs329z","agentic-engineering-handbook"],"specialism":"AI coding agents","work":"anthropic-claude-code-best-practices"},{"label":"Gene Kim and Steve Yegge, Vibe Coding: Building Production-Grade Software With GenAI, Chat, Agents, and Beyond","url":"https://itrevolution.com/product/vibe-coding-book/","kind":"book","isbn":"9781966280026","why":"Practitioners get a guide to running many coding agents at once: generator and verifier agents, task graphs cut small enough for one agent, and the bottleneck moving from writing code to the rest of the organization.","named":["computingforgeeks-ai-coding-books","agentic-engineering-field-study"],"specialism":"AI coding agents","work":"gene-kim-steve-yegge-vibe-coding"},{"label":"Neal Ford et al., Building Evolutionary Architectures, 2nd Edition","url":"https://www.thoughtworks.com/content/dam/thoughtworks/documents/books/bk_building_evolutionary_architectures_second_edition_free_chapter.pdf","kind":"book","isbn":"9781492097549","why":"A practical model for guided incremental change and automated fitness functions that protect architecture qualities.","named":["devbooks-architecture","atlassian-architecture-reading"],"specialism":"evolutionary architecture and fitness functions","work":"building-evolutionary-architectures"},{"label":"OWASP GenAI Security Project, LLM Top 10 for Large Language Model Applications","url":"https://genai.owasp.org/llm-top-10/","kind":"standard","why":"A maintained threat catalogue for prompt injection, sensitive data, tool misuse, supply chain risk, and excessive agency.","named":["owasp-genai-educational","awesome-ai-security"],"specialism":"security of agents","work":"owasp-llm-top10"},{"label":"Taiichi Ohno, Toyota Production System: Beyond Large-Scale Production","url":"https://www.routledge.com/Toyota-Production-System-Beyond-Large-Scale-Production/Ohno/p/book/9780915299140","kind":"book","isbn":"9780915299140","why":"Practitioners get the foundational account of just-in-time flow, waste removal, and line stopping.","named":["lean-institute-books","umich-lean-guides"],"specialism":"lean flow and operations","work":"taiichi-ohno-toyota-production-system"}],"voices":[{"name":"Simon Willison","known":"Simon Willison created Datasette and publishes hands-on writing about data tools, language models, coding agents, and open source software.","links":[{"label":"Blog","url":"https://simonwillison.net/","kind":"blog"},{"label":"LLM and ChatGPT series","url":"https://simonwillison.net/series/using-llms/","kind":"newsletter"},{"label":"Podcast appearances","url":"https://simonwillison.net/tags/podcast/","kind":"podcast"},{"label":"GitHub","url":"https://github.com/simonw","kind":"social"}]},{"name":"Steve Yegge","known":"Steve Yegge helped build large-scale developer tools and writes about code search, coding agents, and software engineering practice.","links":[{"label":"Personal essays","url":"https://steve-yegge.medium.com/","kind":"blog"},{"label":"Yegge.ai","url":"https://yegge.ai/","kind":"site"},{"label":"Sourcegraph essays","url":"https://sourcegraph.com/blog/cody-is-cheating","kind":"blog"}]},{"name":"Gene Kim","known":"Gene Kim founded Tripwire, co-authored The Phoenix Project and The DevOps Handbook, and studies high-performing technology organizations.","links":[{"label":"IT Revolution author archive","url":"https://itrevolution.com/author/gene-kim/","kind":"blog"},{"label":"The Idealcast podcast","url":"https://itrevolution.com/the-idealcast-podcast/","kind":"podcast"},{"label":"Talks","url":"https://itrevolution.com/events/","kind":"talk"}]},{"name":"Addy Osmani","known":"Addy Osmani led work on Chrome, Lighthouse, and web performance and teaches software engineering and AI-assisted development.","links":[{"label":"Blog","url":"https://addyosmani.com/blog/","kind":"blog"},{"label":"Talks archive","url":"https://addyosmani.com/talks/","kind":"talk"},{"label":"YouTube talks","url":"https://www.youtube.com/@addyosmani","kind":"video"}]},{"name":"Birgitta Böckeler","known":"Birgitta Böckeler is a Thoughtworks technology leader who writes about coding agents, context engineering, and software architecture.","links":[{"label":"Martin Fowler articles","url":"https://martinfowler.com/articles/harness-engineering.html","kind":"blog"},{"label":"Exploring Generative AI series","url":"https://martinfowler.com/articles/exploring-gen-ai.html","kind":"blog"}]},{"name":"Charity Majors","known":"Charity Majors co-founded Honeycomb and teaches observability, software operations, engineering management, and production systems.","links":[{"label":"Personal blog and newsletter","url":"https://charity.wtf/","kind":"newsletter"},{"label":"Honeycomb author archive","url":"https://www.honeycomb.io/author/charity","kind":"blog"},{"label":"Honeycomb podcasts","url":"https://www.honeycomb.io/resources/podcasts","kind":"podcast"}]},{"name":"Kief Morris","known":"Kief Morris wrote Infrastructure as Code and teaches platform engineering, self-service infrastructure, and delivery systems.","links":[{"label":"Thoughtworks articles","url":"https://www.thoughtworks.com/insights/blog/infrastructure-as-code-where-are-we-today","kind":"blog"},{"label":"Infrastructure as Code site","url":"https://infrastructure-as-code.com/","kind":"site"},{"label":"Personal site","url":"https://kief.com/","kind":"site"}]},{"name":"Dan Shapiro","known":"Dan Shapiro writes about coding agents, software factories, and staged autonomy from assisted coding to unattended changes.","links":[{"label":"Blog","url":"https://www.danshapiro.com/blog/","kind":"blog"},{"label":"The Five Levels","url":"https://www.danshapiro.com/blog/2026/01/the-five-levels-from-spicy-autocomplete-to-the-software-factory/","kind":"blog"}]},{"name":"Hamel Husain","known":"Hamel Husain built and teaches evaluation systems for machine learning and LLM applications, including judges, traces, and production analysis.","links":[{"label":"Blog","url":"https://hamel.dev/blog/posts/evals-faq/","kind":"blog"},{"label":"Newsletter","url":"https://hamelhusain.substack.com/","kind":"newsletter"},{"label":"Interviews","url":"https://www.opensourceceo.com/p/hamel-husain-interview","kind":"podcast"}]},{"name":"Swyx (Shawn Wang)","known":"Swyx runs Latent Space and publishes interviews, analysis, and a newsletter about AI engineering, models, agents, and infrastructure.","links":[{"label":"Latent Space","url":"https://www.latent.space/","kind":"newsletter"},{"label":"Latent Space podcast","url":"https://www.latent.space/podcast","kind":"podcast"},{"label":"Personal site","url":"https://swyx.io/","kind":"blog"}]},{"name":"Nicole Forsgren","known":"Nicole Forsgren co-created the DORA research program and co-authored Accelerate, connecting delivery capabilities with organizational performance.","links":[{"label":"Personal site","url":"https://nicolefv.com/","kind":"site"},{"label":"DORA resources","url":"https://dora.dev/resources/","kind":"site"},{"label":"Podcast interview","url":"https://getdx.com/podcast/dora-metrics-space-framework/","kind":"podcast"}]},{"name":"Jez Humble","known":"Jez Humble co-authored Continuous Delivery and Accelerate and teaches delivery automation, DevOps, and software delivery research.","links":[{"label":"Continuous Delivery site","url":"https://continuousdelivery.com/","kind":"site"},{"label":"Personal blog","url":"https://jezhumble.net/","kind":"blog"},{"label":"IT Revolution video library","url":"https://videos.itrevolution.com/speakers/jez-humble/","kind":"video"}]},{"name":"Geoffrey Huntley","known":"Geoffrey Huntley develops and explains coding-agent loops and publishes practical material about autonomous software development.","links":[{"label":"Blog","url":"https://ghuntley.com/ralph/","kind":"blog"},{"label":"GitHub","url":"https://github.com/ghuntley","kind":"social"},{"label":"Podcast interview","url":"https://podcasts.apple.com/us/podcast/inventing-the-ralph-wiggum-loop-creator-geoffrey-huntley/id1537003676?i=1000744975721","kind":"podcast"}]},{"name":"Harper Reed","known":"Harper Reed led large civic technology programs and writes about practical coding-agent workflows, specifications, tests, and incremental delivery.","links":[{"label":"Blog","url":"https://harper.blog/","kind":"blog"},{"label":"Coding-agent workflow","url":"https://harper.blog/2025/02/16/my-llm-codegen-workflow-atm/","kind":"blog"}]},{"name":"Martin Fowler","known":"Martin Fowler documents software design, refactoring, architecture, and continuous delivery from his work at Thoughtworks.","links":[{"label":"Website","url":"https://martinfowler.com/","kind":"site"},{"label":"Continuous Delivery book page","url":"https://martinfowler.com/books/continuousDelivery.html","kind":"blog"},{"label":"Podcast archive","url":"https://martinfowler.com/tags/podcast.html","kind":"podcast"}]},{"name":"Lauren Tan (poteto)","known":"Lauren Tan has worked on large codebases at Meta, Netflix and Cursor, is on the React Compiler core team, and wrote pstack, a set of agent workflows built on trusting one verified agent before running many.","links":[{"label":"pstack","url":"https://github.com/cursor/plugins/tree/9580c23696ac8d7abed0c166f2f9918800afb819/pstack","kind":"site"},{"label":"Noodle","url":"https://github.com/poteto/noodle","kind":"site"},{"label":"Blog","url":"https://www.no.lol/","kind":"blog"},{"label":"GitHub","url":"https://github.com/poteto","kind":"social"},{"label":"X","url":"https://x.com/poteto","kind":"social"}]},{"name":"Matt Pocock","known":"Matt Pocock runs AI Hero, where he teaches engineers to run coding agents in loops, to move from watched runs to unattended ones, and to build a software factory.","links":[{"label":"AI Hero","url":"https://www.aihero.dev/","kind":"site"},{"label":"Getting Started With Ralph","url":"https://www.aihero.dev/getting-started-with-ralph","kind":"blog"},{"label":"11 Tips For AI Coding With Ralph Wiggum","url":"https://www.aihero.dev/tips-for-ai-coding-with-ralph-wiggum","kind":"blog"},{"label":"Software Factories for Real Engineers","url":"https://www.aihero.dev/cohorts/software-factories-for-real-engineers-xdy1m","kind":"site"},{"label":"AI Coding Dictionary","url":"https://www.aihero.dev/ai-coding-dictionary","kind":"site"},{"label":"Skills For Real Engineers","url":"https://github.com/mattpocock/skills","kind":"site"}]},{"name":"Kent C. Dodds","known":"Kent C. Dodds is a JavaScript engineer and teacher who builds Kody, a sandboxed cloud runtime where agents run code, use secrets they cannot see, and turn repeated work into packages that run without a model.","links":[{"label":"You Own the Outcome","url":"https://kentcdodds.com/blog/you-own-the-outcome","kind":"blog"},{"label":"Introducing Kody","url":"https://kentcdodds.com/blog/introducing-kody-your-personal-software-factory","kind":"blog"},{"label":"Kody","url":"https://kody.codes/","kind":"site"},{"label":"Kody source","url":"https://github.com/kody-hq/kody","kind":"site"},{"label":"kcd-skills","url":"https://github.com/kentcdodds/kcd-skills","kind":"site"}]},{"name":"Theo Browne","known":"Theo Browne builds developer tools, including create-t3-app and T3 Code, an open-source app that controls Claude Code, Codex and other coding agents, and explains them on YouTube.","links":[{"label":"What is a \"harness\"?","url":"https://t3.gg/blog/post/what-is-a-harness","kind":"blog"},{"label":"T3 Code","url":"https://t3.codes/","kind":"site"},{"label":"YouTube channel","url":"https://www.youtube.com/@t3dotgg","kind":"video"},{"label":"How I code with AI changed a lot","url":"https://www.youtube.com/watch?v=xJaMTo2YgO8","kind":"video"}]}]}}