An AI agent can inspect the results of a tool call and decide what to do next. That can be useful when the path to a goal depends on what the system finds. The same flexibility makes it harder to predict what the system will do and how much a run will cost. A useful starting question for a business is whether a task needs that flexibility at all.
Where an agent adds something
Consider a support inbox. If the goal is to route messages according to a known product code, a rule or simple classifier may be enough. A support investigation is different. The system might need to identify an order, check its delivery status and choose a different next step depending on whether the carrier reports a delay or a completed delivery. Anthropic’s engineering guide describes this distinction in terms of predefined workflows and systems that choose their own steps. Anthropic: Building effective agents
Start with a task a person can check
Give a pilot one narrow objective and one concrete output. For the support example, ask for a short case summary with links to the relevant records and a draft reply. Have a person check the order number, delivery status and proposed action. The reviewer should be able to tell whether the evidence supports the draft without rerunning the whole investigation.
Start with ordinary cases and a handful of exceptions. Try a missing order number, an order that belongs to another customer and a delivery record that conflicts with the message. A good result may be a question or a handoff when the evidence runs out. If every test ends in a confident answer, check whether the system is guessing.
Give tools limits that the application enforces
Allow only the lookups the pilot needs. Use the same customer access rules the existing application uses, so a request for an unrelated order cannot expose it. Keep email, refunds and account updates behind their normal approval steps. An instruction inside a customer message is part of the evidence the agent receives. It should have no power to widen access or authorize an action.
Put a time and tool-call budget on each run. If a lookup fails or the system repeatedly requests the same record, return a partial summary and let the reviewer take over. Keep a trace of the records consulted and the actions requested. Decide who can read that trace and how long it needs to be kept. NIST’s generative AI profile is a starting point for thinking through risks over the system’s lifecycle. NIST: Generative AI Profile
Measure the whole job
Compare the pilot with the current process on a similar set of cases. Track the time required to prepare and review a reply, how often the reviewer changes a material fact, and how many cases need a handoff. Include the cost of failed runs and the work of maintaining access rules. A draft that arrives in seconds can still take longer overall if a reviewer must verify several unsupported claims.
Write down in advance what would justify keeping the pilot. For example, a support team could require the workflow to reduce median handling time while preserving the accuracy of the order details and respecting access rules in every test. Choose thresholds based on the team’s baseline. If the gains come mainly from fetching records automatically, that part may be worth keeping even if the team decides a simpler workflow is easier to operate.
What a two-week support pilot could look like
Start by choosing one queue with enough similar cases to compare. A delivery-status queue is a better first candidate than a general complaints queue because the team can define which records count as evidence and which outcomes need escalation. During the first week, let the agent prepare summaries alongside the normal process. The person handling each case can mark missing evidence and incorrect assumptions without changing how the customer is served. In the second week, let staff use drafts that meet the team’s review criteria. Keep the original process available for cases with missing records or unusual requests.
Make the handoff useful. If the order search returns two possible matches, the summary should say which fields match, which fields differ and what information the reviewer needs next. If tracking data is stale, show its timestamp. This gives the reviewer a concrete starting point and makes it easier to distinguish a tool failure from a reasoning error. Record both kinds of failure separately. A better model may help with a reasoning error, while unreliable tracking data calls for a different fix.
Decide who owns the workflow after launch
Assign someone to review exceptions and update the instructions when the process changes. For example, if the company changes carriers, the agent’s tools may start returning different status labels. Test those labels against known cases before trusting the new summaries. Review the integration again when access rules, product names or refund policies change. Keep a way to disable the agent without taking down the support queue. A pilot earns a wider rollout when the team can explain its results, repeat the checks and recover from predictable failures.
