AngoraTek
When AI Agents Make Sense (and When They Don't)
Every vendor claims their autonomous agents will revolutionize your workflows. After deploying AI agents for data quality monitoring, anomaly detection, and ETL orchestration, we have learned a simpler truth: agents are powerful, but they are widely overapplied. Here is a framework for deciding when they actually deliver.
What Is an Agent
An agent is an AI system that perceives its environment, decides on actions, executes autonomously, and learns from feedback. A chatbot that answers questions is not an agent. A system that monitors your data pipeline, detects anomalies, investigates root causes, and triggers remediation is.
The distinction matters. Agents add complexity. That complexity needs to earn its place.
Where Agents Deliver
High-Volume, Low-Stakes Decisions
A retail client receives over 50 data feeds daily — inventory, pricing, shipments. Each feed needs validation. Failures went to a Slack channel for manual triage.
An LLM-powered agent now handles this:
- Reads failed validation logs
- Classifies the failure: schema drift, data anomaly, or upstream bug
- Takes action: notifies the vendor, quarantines the data, or escalates to engineering
- Logs the decision for review
This works because decisions are frequent (dozens daily), low-risk (quarantining data is reversible), and have clear success criteria (validation passes or fails).
Multi-Step Investigation
When a vehicle reports impossible telemetry — a sudden position jump, a speed spike — someone must investigate. Check for GPS drift. Compare to nearby vehicles. Look at historical patterns. Decide: flag or accept.
An agent runs this investigation automatically and returns a recommendation. The human still makes the final call.
This works because the steps are well-defined but tedious, LLMs synthesize across multiple data sources effectively, and human oversight catches edge cases.
Cross-System Orchestration
Pipeline failures span systems: logs in Datadog, deploys in GitHub, jobs in Airflow, updates in Slack. An agent with tools for each system can execute the full response playbook automatically, producing an audit trail of every action.
This works because the steps are already defined in runbooks, tools have clear APIs, and the agent eliminates toil without introducing ambiguity.
Where Agents Fall Short
One-Shot Decisions
If you make a decision once per quarter — which cloud provider, which database — an agent will not pay back its setup cost. Use an LLM as a research assistant instead. Have it summarize tradeoffs, then decide yourself.
High-Stakes, Infrequent Decisions
A pipeline migration that costs $200K and takes six months. No opportunity for the agent to learn from feedback. The cost of error is catastrophic. LLMs cannot reason about organizational factors like team skills and vendor relationships. These decisions need human judgment, even if it is slower.
Problems Without Clear Feedback
"Improve our data model design." Faster queries? Simpler schema? Easier onboarding? Without a measurable objective function, the agent produces inconsistent results. Agents need feedback to improve. If you cannot define success, you cannot automate it.
Three Implementation Patterns
ReAct: Think, Act, Observe, Repeat
def agent_loop(task: str, tools: dict):
context = task
for _ in range(max_iterations):
thought = llm.generate(f"Context: {context}\nWhat should I do next?")
action = parse_action(thought)
if action == "DONE":
return thought
observation = tools[action.name](action.args)
context += f"\nThought: {thought}\nAction: {action}\nObservation: {observation}"
raise TimeoutError("Agent didn't converge")Use when the problem requires multiple steps with dependencies between them.
Supervisor and Workers
A supervisor LLM decomposes the objective and assigns subtasks to specialized workers — one to fetch data, one to analyze, one to compose. Use when different steps require different expertise or tools.
Human-in-the-Loop
The agent proposes, the human approves. The agent learns from the rejection. Use when stakes are moderate and you want to build trust before granting full autonomy.
The 80% Rule
Build agents when they can handle 80% of cases autonomously and the remaining 20% have a clear escalation path. At 50%, you have a complicated alerting system. At 99% with catastrophic 1% failure modes, you have an operational risk.
Technology Selection
| Component | Recommendation |
|---|---|
| LLM | Claude Sonnet for reasoning-heavy tasks; Claude Haiku for latency-sensitive tool calls |
| Orchestration | LangChain for complex multi-tool workflows; plain Python loops for simpler patterns |
| Tools | REST APIs with JSON schemas — LLMs perform best with unambiguous interfaces |
| Monitoring | Log every decision, action, and outcome |