AngoraTek FR
← Back to Blog
AI & Agents 6 min read

AI Agents Are Wildly Overapplied

Ninety-nine percent is the most dangerous autonomy score an AI agent can post. At 99 percent, the humans stop reading the audit trail, and the only cases that still reach them are the rare ones, where the stakes are highest. Fifty percent looks worse on a slide and is easier to operate: every case gets read, every correction teaches the pattern, and the system improves week over week.

The pitch usually arrives the other way around: autonomous agents that will "revolutionize your workflows." We build data and geospatial systems, so these pitches land weekly, and the appeal is not fake. Agents are powerful. They are also wildly overapplied, and the line is short: an agent earns its setup cost where decisions are frequent, low-stakes, and reversible. Everywhere else, a plain LLM call, or a plain human, is the better tool.

An agent decides. A chatbot answers.

An agent is an AI system that perceives its environment, decides on actions, executes them autonomously, and learns from feedback. A chatbot that answers questions is not an agent. A system that monitors your data pipeline, detects anomalies, investigates root causes, and queues remediation is.

The distinction matters because agency comes with a bill. Every decision an autonomous system makes is a decision that nobody made, at machine speed, in production. That complexity needs to earn its place, decision by decision.

Where agents earn their keep

Fifty feeds a day

Suppose your team ingests more than 50 data feeds daily, covering inventory, pricing, and shipments. Every feed needs validation, and the failure notices land in a Slack channel where an engineer triages them by hand. After a few weeks, the channel is the same ten failure types retitled by each vendor.

An LLM powered agent takes over that triage:

  • Reads failed validation logs
  • Classifies the failure: schema drift, data anomaly, or upstream bug
  • Acts on it: notifies the vendor, quarantines the data, or escalates to engineering
  • Logs the decision for review
The channel changes shape. What lands in it afterward are the cases that need a person, not the cases that resemble yesterday.

Why it works: the decisions are frequent, dozens a day, and the risk is low because quarantining data is reversible. Success is also checkable, which an agent needs. Validation either passes or fails.

The impossible-telemetry investigation

When a vehicle reports impossible telemetry, a sudden position jump or a speed spike, somebody has to figure out whether to trust the device. Check for GPS drift. Compare nearby vehicles. Look at historical patterns. Then decide: flag it or file it.

An agent runs that investigation and returns a recommendation. A human still makes the final call. The steps are well defined but tedious, which is exactly where an LLM pays for itself, and the standing human review catches the edge cases.

The playbook that spans systems

Pipeline failures always span systems. Logs live in Datadog, deploys in GitHub, jobs in Airflow, updates in Slack. An agent with a tool for each system can walk the response playbook on its own, and the audit trail of everything it touched comes free. The part that makes it safe is that the steps already exist in runbooks. The agent executes an approved script. It does not invent one.

The decisions that must stay human

Everything in this section fails the same test, from a different direction.

A decision you make once per quarter, like which cloud provider or database to standardize on, gives the agent no chance to learn from feedback. The setup cost never pays back. The workaround is to use an LLM as a research assistant, have it summarize the tradeoffs, then decide yourself.

Then high-stakes and infrequent together. Picture a pipeline migration that costs $200K and takes six months. Nothing about the error is recoverable at speed, and LLMs cannot reason about organizational factors like team skills and vendor relationships. These decisions need human judgment, even when it's slower.

Finally, problems without clear feedback. "Improve our data model design." What does better mean here, faster queries, or a schema a new hire can read? Without a measurable objective, the agent produces inconsistent results, because agents improve only where feedback exists. If you can't define success, you can't automate it.

Pick the pattern that limits the blast radius

ReAct: think, act, observe, stop

def agent_loop(task: str, tools: dict, max_iterations: int = 8):
    context = task
    for _ in range(max_iterations):
        thought = llm.generate(f"Context: {context}\nWhat should I do next?")
        action = parse_action(thought)

        if action == "DONE":
            return thought

        observation = tools[action.name](action.args)
        context += f"\nThought: {thought}\nAction: {action}\nObservation: {observation}"

    raise TimeoutError("Agent didn't converge")

Use this when the problem takes multiple steps with dependencies between them. The iteration cap and the timeout are not implementation details. They are the safety design, and the TimeoutError is the pattern working as intended.

A supervisor with specialist workers

A supervisor LLM decomposes the objective and hands subtasks to specialized workers: one fetches data, one analyzes, one composes the response. Use this when the steps need different expertise or different tools. The blast radius stays small because each worker sees only its own slice.

The human approves first

The agent proposes, the human approves, and the agent learns from every rejection. Use this when stakes are moderate and you are still building trust. It is also the pattern that gets skeptics on your side fastest, because reviewers watch the decision quality improve before autonomy ever expands. Every step toward more autonomy is this pattern with fewer meetings.

The 80 percent rule

Build agents when they can handle 80 percent of cases autonomously and the remaining 20 percent have a clear escalation path. Fifty percent is a fine starting point and a poor target. Below 80 percent you are operating a queue, not yet an agent; the agents we build start from the number the pilot produced. At 99 percent with catastrophic 1 percent failure modes, what you have built is an operational risk wearing a friendly name.

Boring components, chosen deliberately

ComponentRecommendation
LLMClaude Sonnet for reasoning-heavy tasks, Claude Haiku for latency-sensitive tool calls
OrchestrationLangChain for complex multi-tool workflows, plain Python loops for simpler patterns
ToolsREST APIs with JSON schemas. LLMs perform best with unambiguous interfaces
MonitoringLog every decision alongside its outcome
The model names above are families, not version pins: pin the exact version you ship and review it like any other dependency.

Start with the problem, not the technology. If the decision is frequent, low-stakes, and reversible, an agent delivers measurable returns. If it isn't, a single LLM call is the better choice, and you'll get there weeks sooner.

When in doubt, the right scope for an agent is smaller than the demo suggested.

AngoraTek builds agentic AI systems for data operations. Talk to us if you're figuring out where agents fit in your workflow.

Thanks for reading this far. If this sounds like the problem you're wrestling with, tell us about it. The team reads every message.

Start a conversation