Definition. An AI agent is a system that uses a language model to plan, call external tools, and take multiple sequential actions toward a goal.
The structural test: does it plan, or does it just respond?
The word "agent" has become a marketing label attached to almost any product with an LLM inside it. The useful, structural definition is narrower: a system is an agent if it can decide its own next step based on the result of its previous step, without a human re-prompting it each time. A support chatbot that answers a question is not an agent. A system that reads a support ticket, decides it needs account data, calls an API to fetch that data, evaluates whether the data resolves the issue, and either responds or escalates — that is an agent, because each step's output determined the next action.
The four components every agent needs
- Planner: decides what happens next — call a tool, ask a clarifying question, or conclude.
- Tool layer: lets the model act outside its own context window — APIs, databases, file systems, other services.
- State store: remembers what has already happened across steps, so the agent doesn't repeat work or lose context mid-task.
- Evaluator: decides when the task is actually complete, or when it has failed and should stop rather than loop.
A system missing any of these four is not incomplete by accident — it is a different kind of system. A tool layer without an evaluator will keep calling tools past the point the task is done. A planner without a state store will re-plan from scratch every step, which looks agentic in a demo and breaks on any task longer than one exchange.
Agent vs. chatbot vs. workflow automation
| Chatbot | Workflow automation | AI agent | |
|---|---|---|---|
| Decision logic | None — one input, one output | Fixed, predefined rules | Dynamic, based on intermediate results |
| Adapts mid-task | No | No | Yes |
| Tool/API access | Rare | Yes, but hardcoded sequence | Yes, model selects which tool and when |
| Failure handling | N/A | Manual, per-workflow | Built into the planning loop |
Where the definition gets abused
Two patterns account for most of the definitional inflation in the market. First, a single function call gets rebranded as "agentic" because it technically calls a tool — even though there is no loop, no re-planning, and no ability to recover from a bad result. Second, a fixed multi-step workflow (do A, then B, then C, always in that order) gets called an agent because an LLM is involved somewhere, even though the sequence itself is hardcoded and the model has no actual decision to make about what happens next.
Neither is wrong to build — both are often the right, simpler choice for a well-defined task. But conflating them with true agentic systems sets the wrong expectations for reliability, cost, and what happens when an input doesn't match the happy path.
Frequently asked questions
Is a chatbot with function calling an AI agent?
Not by itself. An agent needs a loop — observe the tool result, decide the next step, continue. A single tool call with no follow-up decision is function calling, not agency.
What is the minimum architecture for something to count as an agent?
A planner, a way to act via tools, and a way to know when the task is finished — plus persistent state for anything beyond one tool call.
Why do vendors call so many different products "AI agents"?
The term has commercial value. The structural test is whether the system plans and re-plans across multiple steps based on intermediate results.