Skip to main content
All posts
AI & Data18 min read

Workflows or Agents? When AI Autonomy Earns Its Cost

A decision framework for CTOs: four questions on determinism, risk and variability that set autonomy from level 0 to 4, with cost and reliability data.

Published Updated: 15 November 2026

On 25 June 2025, Gartner predicted that more than 40% of agentic AI projects will be cancelled by the end of 2027, because of escalating costs, unclear business value or inadequate risk controls. Seventeen months on, the agent projects we see struggling rarely fail because the models are weak. They fail because a fixed, auditable process was handed to a system that picks its own next step, and then nobody can explain the bill, the variance or the audit trail.

This article is for the people who sign off that choice: CTOs, AI platform leads, enterprise architects and the risk owners beside them. It gives you a four-question decision tree that sets an autonomy level from 0 to 4, the numbers behind the cost of autonomy, and a European governance view. We are not anti-agent. But autonomy is a cost you pay on every run, so it has to earn its place.

TL;DR / Key takeaways

  • Workflows and agents differ in who owns control flow. Cost, variance and auditability all follow from that one choice.
  • Autonomy is a dial, not a switch. Five levels: L0 deterministic code, L1 an LLM inside a fixed step, L2 a routed workflow, L3 a bounded agent, L4 an open agent.
  • Four questions set the level, in order: can you write the path down (determinism), is a wrong step costly (risk), can you enumerate the routes (variability), and is open-ended exploration worth the bill (autonomy).
  • The bill is real. Anthropic reports agents use about 4× the tokens of a chat and multi-agent systems about 15×. Reliability compounds against you: at 95% per step, a 20-step chain completes only 36% of the time.
  • The winning pattern is usually hybrid: a workflow backbone with a small, bounded agent island for the long tail, gated by evals, tracing, budgets and approvals.
Workflows or agents? Four questions, asked in order, set the autonomy level from L0 to L4
Workflows or agents? Four questions, asked in order, set the autonomy level from L0 to L4

Download: Editable draw.io file · Open in diagrams.net · Chart data (CSV)

Why this matters now

  1. The hype met the budget. Gartner's June 2025 forecast came with a warning about "agent washing": only about 130 of the thousands of agentic AI vendors are real, it estimated. A Gartner survey of 360 IT application leaders, published on 30 September 2025, found about three-quarters piloting or deploying some form of AI agent, but only 15% considering, piloting or deploying fully autonomous ones, and 74% seeing agents as a new attack vector. Most organisations already want bounded autonomy. What they lack is a way to decide how much.
  2. The vendors converged on the same advice. Anthropic's "Building effective agents" (December 2024) separates workflows, "orchestrated through predefined code paths", from agents that "dynamically direct their own processes and tool usage", and recommends "finding the simplest solution possible". OpenAI's 2025 guide to building agents warns that otherwise "a deterministic solution may suffice". Microsoft's Agent Framework documentation (updated October 2026) is bluntest: "If you can write a function to handle the task, do that instead of using an AI agent." When the companies selling the models tell you to use fewer agents, take the hint.
  3. Reliability data caught up with the demos. The τ-bench paper (Yao et al., 2024) introduced pass^k, the chance an agent solves the same task in all k attempts. GPT-4o solved about 61% of retail tasks on one try, but fewer than 25% consistently across eight. Demos show the first number; production runs on the second.

The autonomy ladder: five levels, one dial

"Agentic" now describes everything from a prompt chain to a self-directed swarm. We define five levels by one question: who decides the next step?

LevelNameWho decides the next stepTypical exampleHuman role
L0Deterministic automationYour code and rulesThree-way invoice match on ERP dataOwns the rules
L1LLM inside a fixed stepYour code; the model fills a stepExtract fields from a scanned claim, validate in codeReviews exceptions
L2Routed workflowThe model picks one of a few predefined branchesClassify a ticket into one of six queues with fixed handlersApproves writes
L3Bounded agentThe model, inside an allow-list, step cap and budgetInvestigate a failed payment with read-only tools, propose a fixApproves actions
L4Open agentThe model, including planning, delegation and tool choiceMulti-agent research across many sourcesMonitors and audits

L0 to L2 are workflows: control flow lives in your code, even with a model inside. L3 and L4 are agents: control flow lives in the model's reasoning at run time. That line decides how you test, how you explain a decision to an auditor and how you forecast the bill.

Anthropic's workflow patterns (prompt chaining, routing, parallelisation, orchestrator-workers, evaluator-optimiser) sit at L1 and L2. Feng, McDonald and Zhang's "Levels of Autonomy for AI Agents" (2025) grades the user's role, from operator to observer. We grade the architecture, because that is what you build, cost and govern.

The decision tree, question by question

Ask the four questions in order and stop at the first answer that fixes the level. The numbers match the figure.

① Determinism: can you write the path down in advance?

If a competent analyst can draw the process as a flowchart, you have a workflow; the only question is whether a step needs language judgement. If rules and structured data define the outcome, stay at L0: no model, which is cheaper, faster and easier to test. If one step must read a contract or classify free text, put the model inside that step and keep control flow in code: L1.

The most common mistake we see is confusing unstructured input with an unstructured process. An insurance claim arrives as a messy PDF, but the process behind it (check cover, check fraud indicators, calculate, approve or refer) is well defined. Messy input calls for a model in a step, not an agent in charge.

② Risk: what does a wrong step cost?

Ask this per action, not per use case. Reading a knowledge base is harmless. Issuing a refund, changing a firewall rule, sending a customer letter or deleting data is not.

When a wrong step is costly or irreversible (money moves, legal rights or personal data are affected, or customers see it), cap autonomy at L2. Let the model choose between paths you have reviewed, and put human approval in front of every write. A bounded agent can still do the read side, investigating and drafting, as long as the action passes an approval gate; our human-in-the-loop approval gates guide covers the patterns. Risk also means attack surface: an agent that reads untrusted content and holds write tools is the classic setup for indirect prompt injection.

③ Variability: can you enumerate the routes?

If the risk is contained, count the routes. If cases fall into a handful of patterns (say, fewer than ten) with known handlers, you still do not need an agent. A router (L2) classifies the case and hands it to a fixed sub-workflow. Each branch is testable on its own, and the router can be evaluated as a classifier with a confusion matrix, which auditors understand.

Model-directed control flow pays only when the routes form a long tail: every case differs a little, and the next useful step depends on what the last tool returned. That is the honest domain of agents: investigation, troubleshooting, research, coding and reconciliation across systems where the data surprises you.

④ Autonomy: is open-endedness worth the bill?

If you reached this question, you need an agent; the choice is how open. A bounded agent (L3) plans tool calls within an allow-list, with a step cap, a token budget, a timeout and approval on writes. That is the right enterprise default.

An open agent (L4) plans, delegates to sub-agents and picks its own tools. It earns its cost only when the task is open-ended and long-horizon and a better answer is worth many times the token spend. Anthropic's multi-agent research system is the reference case: it beat a single agent by 90.2% on Anthropic's internal research eval, but needs tasks whose value is "high enough to pay for the increased performance". Most enterprise processes are not open-ended research.

Either way, pass the gate in the figure first: evals on real cases, end-to-end tracing, token and step budgets, a kill switch, and approval on writes. Without all five you are not ready for L3; our agent evaluation harness covers the first.

The hybrid that usually wins

The answer for a whole process is rarely "workflow" or "agent". It is a workflow backbone with one or two bounded agent islands where the long tail lives.

Loading diagram...

The workflow owns the boundaries: what comes in, what gets written, who approves. The agent owns only the part you cannot specify, and its output is a proposal that re-enters the deterministic path, so the expensive, variable component stays small, measurable and replaceable. If the island runs for hours or waits on a human, give the backbone durable execution so a crash does not restart the investigation.

What the numbers say

Autonomy level vs cost, variance, governance and debugging effort and coping with novel inputs, scored 1 to 5 (our analysis)
Autonomy level vs cost, variance, governance and debugging effort and coping with novel inputs, scored 1 to 5 (our analysis)

The heat grid is our qualitative analysis, not measured data. Its point: every column rises together. Coping with novel inputs, the only column you actually want, is paid for in cost, variance, governance and debugging effort. The big step is from L2 to L3, where control flow moves from your code to the model and testing changes from "check each branch" to "sample many runs and measure the distribution".

End-to-end success over n sequential steps at 99%, 95% and 90% per-step success (p to the power of n)
End-to-end success over n sequential steps at 99%, 95% and 90% per-step success (p to the power of n)

This chart is plain arithmetic, assuming independent steps. If each autonomous step succeeds 95% of the time and nothing checks the result, a 20-step chain completes correctly 36% of the time (0.95^20). At 90% per step, ten steps already leave you at 35%. Real agents sometimes beat this by noticing errors and retrying; sometimes they do worse, because errors correlate and one misread fact poisons every later step. The lesson: put checkpoints into long chains, validate between steps in code, and keep the stretch the model controls short.

Token use relative to a chat: about 4× for a single agent, about 15× for a multi-agent system (Anthropic, June 2025)
Token use relative to a chat: about 4× for a single agent, about 15× for a multi-agent system (Anthropic, June 2025)

Anthropic's ratios make the cost concrete: about 4× the tokens of a chat for its agents, about 15× for its multi-agent system. On the BrowseComp evaluation, token usage alone explained 80% of performance variance: more autonomy buys better answers largely by spending more tokens. That is a good trade for a high-value research question and a bad one for routing invoices. Our agent cost governance guide shows how to budget it.

In a recent engagement…

During an architecture review this year, a European financial-services client showed us a customer-correspondence "agent". It read incoming letters, looked up the customer, drafted a reply and filed the case. The demo impressed; the pilot did not. The same kind of letter took different tool paths on different days, the token bill was hard to forecast, and when a reply was wrong nobody could say which step had failed. Within weeks, operations stopped trusting it.

We walked the four questions with the team. Not every letter's path could be written down (① no), and wrong replies to customers are costly, so ② capped the outbound action at L2 with approval. Question ③ was the revealing one: when we clustered a sample of past letters, most fell into a few recurring types with known procedures. Only a minority were genuinely novel.

The redesign was unglamorous. A model extracts fields and classifies the letter (L1 and L2). Recurring types go to fixed handlers that draft from compliance-approved templates. The novel remainder goes to a bounded, read-only agent that proposes a reply for a human to approve. The team feared "giving up on AI"; instead, the agent island became small enough to evaluate properly, and operations trusted it again.

The decision checklist

The screenshot-able part. Score each use case before anyone writes a prompt.

#QuestionIf yesIf noEvidence to ask for
1Can an analyst draw the path as a flowchart?L0 (rules suffice) or L1 (a step reads language)Go to 2The flowchart
2Is any action costly, irreversible, customer-visible or legally relevant?Cap at L2, approval on every write; L3 only read-onlyGo to 3Tool inventory with blast radius
3Do most cases fall into a few routes with known handlers?L2: router plus fixed sub-workflowsGo to 4Clustered sample of real cases
4Is the task open-ended and long-horizon, and is a better answer worth ~15× the tokens?L4 with budgetsL3 bounded agentValue per task vs. cost per run
5Gate for L3/L4: evals, tracing, budgets, kill switch, approval on writes?ProceedStay at L2Eval report, trace sample, budget config
6Can you explain one bad outcome to an auditor from the logs alone?ProceedAdd decision logs firstOne replayed failure

Two warnings. Levels apply per action, not per product: one assistant can be L3 for reading and L2 for writing. And re-score regularly, because new models change what a single classification step can handle.

Governance for European enterprises

The EU AI Act regulates uses and roles, not "agents": an agent and a workflow doing the same job face the same obligations. Autonomy changes how hard they are to demonstrate.

EU AI Act. The Article 4 AI literacy duty has applied since 2 February 2025. Article 50(1) has applied since 2 August 2026: systems that interact directly with people must tell them they are dealing with an AI, unless that is obvious, whether the system is an agent or a workflow. For Annex III high-risk uses such as credit scoring or recruitment, Regulation (EU) 2026/1744 moved the obligations, including human oversight under Article 14, to 2 December 2027. Effective oversight is far easier to show for an L2 workflow with an approval gate than for an L4 agent, which is why question ② caps such decisions at L2.

GDPR Article 22. Decisions based solely on automated processing with legal or similarly significant effects are restricted. In SCHUFA (C-634/21, 7 December 2023) the Court of Justice held that even an automated score can be such a decision when a third party draws strongly on it. An agent deciding a loan, claim or job application without meaningful human involvement is in Article 22 territory, whatever its architecture. Approval gates build in the human intervention the exceptions require.

DORA. For financial entities, Regulation (EU) 2022/2554 has applied since 17 January 2025. An agent supporting a critical or important function sits inside your ICT risk management framework, with change control, testing and incident classification. Version prompts, tools and models, and treat a model upgrade as a change.

A pragmatic adoption path

  1. Inventory and score (weeks 1–2). Score every AI use case in flight with the checklist and name an owner for each. Expect L4 ambitions on L1 problems.
  2. Push down first (weeks 3–6). Rebuild the frequent paths of over-scored use cases as L1/L2 steps, with the original agent in shadow mode for comparison.
  3. Build the gate (weeks 4–8). Stand up the five gate prerequisites as shared platform services. Our enterprise agent platform reference architecture shows where they live.
  4. Open one agent island (month 3). Pick the use case with the clearest long tail and lowest action risk. Run a read-only L3 agent on the residual cases and measure pass^k over repeated runs.
  5. Re-score quarterly. Promote a use case one level only when evals and traces show it is stable; demote it when cost or variance drifts. Record levels in your AI inventory so risk, audit and engineering argue from the same table.

The verdict

Agents are not a maturity stage every process should reach. They are a tool for problems whose path cannot be written down and whose routes cannot be enumerated. There they excel. Elsewhere, they spend tokens and audit effort to rediscover a flowchart you could have drawn.

Default to the lowest level that solves the problem, move up one level at a time with evidence, and give the model as little control flow as the problem allows. We expect most enterprise value in 2027 to come from L1 and L2 workflows with excellent model calls inside, plus a few well-bounded agents where the long tail really is long. Teams that work this way are unlikely to end up in Gartner's 40%.

Want a second opinion on where your use cases sit? Our AI architecture team runs this scoring as a short, fixed-scope portfolio review: each use case placed on the ladder, the gate items it still needs, and a list of agents that should become workflows.

FAQ

What is the difference between an agentic workflow and an AI agent?

Who owns the control flow. In a workflow, your code fixes the sequence of steps and the model works inside them. In an agent, the model decides at run time which tool to call next and when it is done. Anthropic draws the same line.

When should we use an AI agent instead of a workflow?

When you cannot write the path down, cannot enumerate the routes, and a wrong step is cheap or caught before it lands. Open, multi-agent setups also need tasks valuable enough to justify many times the token bill of a chat. If any condition fails, a workflow with model calls inside is usually cheaper and more reliable.

What are the levels of AI autonomy in this framework?

L0 is deterministic automation with no model; L1 puts an LLM inside a fixed step; L2 is a routed workflow where the model picks one of a few predefined branches. Those are workflows. L3 is a bounded agent with an allow-list, budgets and approval gates; L4 is an open agent that plans, delegates and chooses tools while humans monitor.

Are AI agents more expensive than workflows?

Usually, yes. Anthropic reported in June 2025 that its agents use about 4 times the tokens of chat interactions, and multi-agent systems about 15 times. Agents also cost more to test, trace, govern and debug, because the path changes from run to run.

Does the EU AI Act regulate AI agents differently from workflows?

No. The AI Act regulates use cases and roles, not architectures. Article 50(1) disclosure has applied since 2 August 2026; Annex III high-risk obligations, including human oversight under Article 14, apply from 2 December 2027 under Regulation (EU) 2026/1744. Higher autonomy just makes oversight harder to demonstrate.

Can we start with an agent and harden it into a workflow later?

Yes, and it is often the best way to discover the real process. Run a bounded agent in shadow mode, trace every run, and freeze its recurring paths into workflow steps. The agent keeps only the long tail, which is usually smaller than teams expect.

Sources

Checked on 15 November 2026. This article is not legal advice.

Tier A: primary and regulatory

Tier B: research and analysis

Tier C: vendor and directional

Topics

agentic workflow vs agentworkflows vs agentsAI agent decision frameworklevels of AI autonomywhen to use AI agentsLLM workflow orchestrationagentic AI cost and reliabilitybounded autonomy

Frequently Asked Questions

Who owns the control flow. In a workflow, your code fixes the sequence of steps and the model works inside them. In an agent, the model decides at run time which tool to call next and when it is done. Anthropic draws the same line.

Expert engagement

Need expert guidance?

Our team specializes in cloud architecture, security, AI platforms, and DevSecOps. Let's discuss how we can help your organization.

Get in touchNo commitment · No sales pressure

Related articles

All posts