⚡The short answer
A chatbot takes input and returns text. However sophisticated the model behind it, its output is words. It can tell you your refund policy.
An AI agent takes a goal and returns a changed world. It plans steps, calls tools, reads what comes back, decides what to do next, and acts. It does not tell you the refund policy — it checks the order, applies the policy, issues the refund, updates the CRM and closes the ticket.
The test is simple: if the system cannot change anything outside the conversation, it is a chatbot. The presence of an LLM is not the distinction — plenty of chatbots use frontier models, and plenty of useful agents run on small ones.
⚖️Side by side
| Dimension | Chatbot | AI agent |
|---|---|---|
| Output | Text | Actions, plus text |
| Can change your systems | No | Yes — that is the point |
| Decides its own next step | No — one turn, one reply | Yes — loops until done or escalates |
| Tools | None, or retrieval only | APIs, databases, email, CRM, payments |
| Failure mode | Unhelpful answer | Wrong action — which is why guardrails matter |
| Needs approval gates | Rarely | Always, for irreversible actions |
| Cost driver | Tokens per message | Tokens per completed task, including retries |
| Typical build time | Days to weeks | 4–6 weeks per workflow |
🎯Which one do you actually need?
The honest answer is that a lot of teams building agents needed a chatbot, and a lot of teams who built a chatbot are frustrated because they needed an agent. Deciding is cheap; discovering it after the build is not.
You need a chatbot when the answer is the deliverable
Documentation search, policy questions, product guidance, internal knowledge lookup. If a correct paragraph fully resolves the user's need, an agent adds cost, latency and risk for nothing.
You need an agent when the answer is not enough
If a satisfied user still has to go and do something — click the refund, update the record, file the ticket — then handing them text has only moved the work, not removed it.
Volume is the tie-breaker
An agent has real fixed costs: tool integration, guardrails, evaluation, monitoring. On a workflow that runs a handful of times a week, that engineering will not pay back. On one that runs thousands of times, it pays back quickly.
Reversibility decides how fast you can move
Automating reversible actions is comparatively safe — mistakes are recoverable. Automating irreversible ones (payments, deletions, external communications) requires approval gates and takes longer to earn trust.
A useful rule: if you cannot name the specific system the AI must write to, you are describing a chatbot. Building it as an agent will cost more and deliver less.
📚The thing in between: RAG assistants
There is a middle category that causes most of the confusion. A retrieval-augmented assistant reads your documents and answers from them with citations. It is grounded in your data and feels far more capable than a scripted bot — but it still only produces text, which makes it a chatbot by the definition above.
That is not a criticism. For a large share of internal knowledge use cases, a well-built RAG assistant is exactly right and considerably cheaper to run than an agent. The mistake is budgeting for an agent, building a RAG assistant, and then wondering why nothing downstream got automated.
🛡️Why agents need engineering that chatbots do not
When a chatbot is wrong, a user reads a bad paragraph. When an agent is wrong, money moves, records change, or an email goes out. The consequence gap is why production agents carry machinery that chatbots simply do not need.
Enforcement outside the model
Spend caps, permission scopes and blocks on destructive actions must be checked server-side on every call. An instruction in a prompt is not a control.
An explicit uncertainty path
Below a confidence threshold the agent stops and escalates with the context it gathered, rather than guessing. Chatbots can afford to always answer; agents cannot.
Evaluation from day one
A scored test set of real tasks, so you can tell whether a change improved the system or broke an edge case. Without it, teams become too afraid to change anything.
Full traces and replay
Every tool call and decision recorded, so a wrong outcome is a debuggable stack trace rather than a mystery you try to reproduce.