- You don't need to jump to a big system on day one. Starting with one task you can measure is the safer path.
- The order that works: get answers right first, then let the system take action, and only then expand to more tasks.
- The one metric to hold on to the whole way is whether more customers are getting their business done from start to finish.
Where old websites and apps stop
Many companies start by putting a chatbot in the corner of the screen and then stay stuck there for years, because it can answer questions but has never finished a single task. So the question to ask is “what percentage of customers can now get their task done from start to finish?”, rather than “where else can we put AI?”
Many businesses start with a chat box and rush to connect actions because the demo answers well, while there is still no test set, source of truth or approval rules. The pilot then never feels safe enough to launch for real.
Many organizations have a chatbot demo that answers questions but can't go any further, because there is still no data owner, no API, no permissions, no measurement and no plan for when the system gets things wrong. Jumping straight from conversation to an agent that does the work makes risk grow faster than readiness, and the team wrongly concludes that AI can't be used for real.
For the development team · Technical detail
A good roadmap adds capability and accountability one level at a time. It starts with Search and Answer, then moves on to Assist, Recommend, Reversible Action and Orchestration. Each stage has its own outcome, guardrails and evidence, so the business invests according to the problem instead of building a big platform and waiting for future use cases.
| The old way | The new AI product approach |
|---|---|
| Pick a model, write a prompt and open it up for questions | Design the product loop (intent, context, tools, approval, action, feedback, evaluation and monitoring) before adding autonomy one level at a time |
For the development team · Technical detail
Every level of the roadmap has to share one foundation: identity, knowledge, APIs, policy, evaluation and monitoring. Keeping these layers separate from the model lets the product change technology and add actions without tearing up the whole journey.
New capabilities a business can put to work
The safe route goes one step at a time. First get the system to answer questions that have clear answers, and get them right. Once it is accurate, let it take actions that can be undone, such as booking appointments or issuing draft documents. Expand to work tied to money or contracts only after you have proven you can keep it under control.
The roadmap at a glance
The first level makes data searchable and citable. The next helps draft or summarize, with a person checking. After that, tools are connected so the system can take actions that can be reversed, and only then do you add coordination across several steps or several agents. Skipping a level isn't always wrong, but the foundation of the earlier levels still has to be there inside the system.
Capabilities to build up
Every phase should add more than prompts: knowledge ownership, identity, permissions, tool contracts, evaluation, approval, monitoring and feedback loops. On the user side, features may gradually shift from answers to a guided workspace, an action center and an exception queue as the level of autonomy rises.
Technology as reusable parts
The architecture should keep the web and mobile experience, backend and APIs, knowledge, agent runtime, policy, evaluation and observability separate, so you can swap the model or add use cases without tearing everything down. AI Autonomous Development can speed up prototypes, code and tests, but it has to work within clear architecture, review and security practice.
Why moving in phases pays off
Executives can see which question each piece of investment answers. The team learns from pilots without handing important processes to a system that hasn't proven itself, and the parts already built can be reused for the next use case. A roadmap also lets you decide to stop, for sound reasons, when the data, the value or the readiness isn't there.
- Level 1 Answer: answer from sources that can be cited
- Level 2 Assist: draft or recommend, and a person clicks to act
- Level 3 Act with Approval: prepare the action, then ask for approval
- Level 4 Bounded Autonomy: handle standard cases within set limits, with an audit trail
Hypothetical case
What it looks like in practice
A business starts its scheduling agent in recommendation mode, then lets it create draft appointments, and only switches on real booking confirmations once duplicates, time zones, cancellations and permissions pass the test set.
In a hypothetical case, a retailer starts with an assistant that answers questions about the returns policy and cites its sources. In phase two it summarizes order status, with staff checking. Phase three lets customers pick a new delivery slot, which can be reversed, and only in the phases after that does it coordinate stock, shipping and notifications.
Each phase has its own pass criteria, such as answer accuracy, completeness of handoffs, action success rate and duplicate operations. If an integration goes down, the system can fall back to giving information or passing the customer to a person, so the whole journey doesn't grind to a halt because the smartest part is unavailable.
Autonomy Ladder
Increase independence when evidence, reversibility and monitoring are ready, and never because the team wants to show off what it can do.
Every level needs stop/go criteria and an owner.
Scope, risks and how to measure results
For the development team · Technical metrics
A roadmap shouldn't be measured by the number of use cases or the highest level of autonomy reached. Measure outcome, adoption, error impact, recovery, cost per successful task and the support burden after launch. Every pilot needs stop/go criteria and an owner with the authority to change the process. Otherwise the system will be stuck between demo and production.
Don't tie the product to a single model without an abstraction layer, don't give the AI broad credentials, and don't scale before you have measured cost against outcome.
For the development team · Technical metrics
Metrics to track: Accepted Outcome, Tool Success, Approval Rate, Rollback, Incident, Latency and Total Cost
- Discover: follow the real work and collect examples of normal cases and exceptions
- Assist: let AI draft or recommend while people stay in control
- Act: switch on tools one at a time after the test set passes
- Scale: expand once monitoring, fallback, cost and an owner are in place
Stop expanding when a pilot has no owner, the data is unstable, cost per task goes over the ceiling, or old errors come back without anyone detecting them. Going back to fix the foundation is progress with AI, even if it feels like a step back.
BUSINESS & PRODUCT READINESS
Lay the product foundation before adding autonomy, so the pilot doesn't get stuck at the demo stage
For the development team · Technical detail
Build a capability inventory: what the model understands, where the data comes from, what the tools do, who holds the permissions, and whether results can be reversed. Then rank use cases by value, feasibility, risk and reversibility. Don't start at a high level on work where the data is still unclear.
For the development team · Technical detail
Build an evaluation set before switching on actions, covering the happy path, incomplete data, prompt injection, tool errors, duplicates and permission tests. Settle the owner, incident handling, cost budget and model change process. A product that uses AI has to be tested continuously, because models, data and user behavior all change.
For the development team · Technical metrics
Take a capability inventory of where the organization stands today on data, APIs, identity, ownership and measurement, then match each use case to the right level of autonomy. Work that can't be reversed or that involves high-level permissions may keep human approval permanently. There is no need to go all the way to fully autonomous.
For the development team · Technical detail
Build the evaluation set from the start, using normal cases, missing data, prompt injection, tool errors, repeated commands and exceptions, and estimate the running and operating costs. Knowing how you will detect a drop in quality matters as much as proving the prototype works on demo day.
What is the first outcome?
How much autonomy is the right amount?
Which actions can't be reversed?
How will we know when quality drops?
Start with the right problem, then choose the right amount of intelligence
DNA Maker helps business owners build an AI Opportunity/Autonomy Map from their real processes and data. We sort out what should be a website, an app, a workflow, knowledge search or an agent, and write the guardrails and value hypothesis before choosing any technology.
During product discovery we create the user journey, architecture options, a data and integration map, and a prototype to test with users and with the people who make the decision. The pilot is designed to answer business questions and has stop/go criteria, so it does more than demonstrate a model.
DNA Maker helps build an AI opportunity and autonomy map from real processes, then designs a product foundation you can keep building on, from data and integration, UX, architecture and guardrails through to evaluation. We don't force every problem into an agent, and we will propose a web app, a workflow or search when that is the simpler and more responsible answer.
Once a pilot is chosen, we can build the prototype, web and mobile apps, backend, AI agent, tools, approval, monitoring and support documentation, using AI Autonomous Development to speed up the work under engineers' review. If you have a list of ideas, bring your processes, sample data and concerns, and we can rank them down to a first project that is measurable and lays the foundation for the next phase.
The engineering team can build web, mobile and backend systems, AI agents, tools and APIs, human approval, evaluation, observability and admin operations all the way to production, using AI Autonomous Development to speed up the work under code review, testing and security practice.
If you have plenty of ideas but don't know where to start, bring your processes, sample data and concerns to a conversation. DNA Maker will help narrow the ideas down to a pilot that is clear, measurable and doesn't lock the organization into options that are still unproven.
SOFTWARE ENGINEERING GLOSSARY
Software engineering glossary
This last set of terms connects the idea of an agentic product to levels of autonomy, test sets, fallbacks and model separation. Use it as a shared language for planning a roadmap that can change providers or add use cases responsibly.
| Term | What it is | A simple example | What to ask the development team |
|---|---|---|---|
| Agentic Product | A product in which AI plans steps and uses tools within set limits. The AI plans or carries out several steps with tools, rules and tracking, which makes it much more than a chat screen that generates text. | An agent prepares an appointment and asks a person to confirm it | Who owns the outcome and the limits? |
| Autonomy | How independently a system is allowed to act. It should be set separately for each action, because one system may answer automatically yet still need a person's approval before it changes data or makes a transaction. | Handling standard cases without waiting at every step | What evidence has to pass before we increase independence? |
| Evaluation Set | A set of examples used to test quality again and again. It needs normal cases, missing data, exceptions and attacks, along with answers or criteria that experts accept. | Normal cases, risky cases and incomplete data | Does it cover the real exceptions yet? |
| Fallback | The backup route when AI or a tool fails. A fallback has to preserve the work and its context, for example by switching to a manual flow or handing it to a person, instead of only showing a message that the system is down. | Sending the job to a human queue without losing any data | Has the team rehearsed the fallback? |
| Model Abstraction | A layer that separates the product from the model provider. It keeps product logic apart from the provider, which makes it easier to change versions, control cost and test alternatives with less impact on the system. | Changing models without rebuilding the workflow | How are differences in quality and cost tested? |
Further reading from the original documents: https://openai.github.io/openai-agents-js/