What an agent actually is

Agent is currently the most-used word in AI talk. It sounds as though it named a new kind of independently thinking machine. Most of the time it is something far more ordinary: a normal language model with a program built around it that gives the model tools and lets it try again.

An agent is a loop, not a model

An ordinary model call is one round: a question in, an answer out. An agent does the same thing several times over and does something between the rounds:

1. It sizes up the task and decides on the next step.
2. It picks a tool: a search, a database query, sending an email.
3. It looks at what the tool returned.
4. It revises the plan if the result was not what it expected.
5. It carries on until the task is done or it hits a limit.

The model produces text on every round. The software around it interprets that text as an action, performs it, and feeds the result back into the model. That is the whole trick.

Where an agent's capability comes from

In practice the quality of an agent is rarely decided by which model is under the bonnet. What decides it:

• Tools. An agent can only do what it has an interface to. Without access to the billing system it will not handle billing, however clever it is.
• Instructions. A clear description of the task, the limits and an acceptable result removes a large share of the errors.
• Context. The agent needs the right information at the right moment. Shoving everything into the prompt at once is expensive and makes it less accurate.
• Memory. What is worth keeping between rounds and what should be forgotten.
• Checks. Does anything or anyone assess whether a step went right?

Where agents go wrong

The typical failures are not exotic:

• It makes many model calls per task, so the cost multiplies without anyone noticing.
• One wrong intermediate step takes the whole chain off course, and the following steps are built on top of the error.
• It is given too broad a set of permissions and gets to do something irreversible.
• Nobody can see afterwards what it did and why.
• Success is established in a demo rather than by measurement.

What a company setting requires

An agent must not roam free. In production you typically need:

• narrow permissions: only the systems and rows the task requires
• approval points: a person confirms anything irreversible
• logging: which tools were called, with what parameters, and what came back
• a budget ceiling: a maximum number of rounds and a maximum cost per task
• evaluation: a set of real cases used to measure quality before and after changes.

In one sentence

An agent's intelligence does not live in the model but in what has been built around it. The same model can be excellent or useless depending on the tools, the instructions, the permissions and the checks.