Build Autonomous AI Agents with Tool Use and Reasoning

Overview

A chatbot answers; an agent does. Where a simple chatbot generates a reply and stops, an agent keeps going — it looks at the task, decides what tool would help, calls it, reads the result, and decides again until the job is done. The jump from static prompts to agents is exactly that loop: observe → decide → act → repeat, with the model steering.

The loop is what makes the ReAct pattern work — reasoning steps alternate with tool calls so the agent can plan, act, and correct itself mid-flight. And for the cases where the loop still breaks, the LLM Fallback Pattern keeps the system standing.

When to use it

Reach for an agent when the task needs decisions between steps, not just a call. Questions that pull from several data sources, workflows that mix browsing with calculations and API calls, assistants that book or query or report — anything where the path isn’t known upfront and the model has to pick the next move. Research agents that search-summarize-synthesize on repeat are the canonical case; support bots that look up orders and know when to escalate are the boring real one. Skip it when a single LLM call or a RAG lookup can answer the question — an agent adds cost and failure modes for nothing.

Solution

ReAct Agent (Python / OpenAI)

from openai import OpenAI
import json

client = OpenAI()

def search_web(query: str) -> str:
    """Simulated web search tool."""
    return f"Search results for: {query}"

def calculate(expression: str) -> str:
    """Evaluates a mathematical expression."""
    try:
        return str(eval(expression))
    except Exception as e:
        return f"Error: {e}"

tools = [
    {
        "type": "function",
        "function": {
            "name": "search_web",
            "description": "Search the web for current information",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string", "description": "The search query"}
                },
                "required": ["query"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Evaluate a mathematical expression",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {"type": "string"}
                },
                "required": ["expression"]
            }
        }
    }
]

messages = [
    {"role": "system", "content": "You are a helpful assistant. Use tools when needed."},
    {"role": "user", "content": "What is the population of France divided by the population of Germany?"}
]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice="auto"
)

# Agent loop
max_iterations = 5
for i in range(max_iterations):
    message = response.choices[0].message

    if message.content:
        print(f"Assistant: {message.content}")
        break

    if message.tool_calls:
        messages.append(message)
        for tool_call in message.tool_calls:
            name = tool_call.function.name
            args = json.loads(tool_call.function.arguments)

            if name == "search_web":
                result = search_web(**args)
            elif name == "calculate":
                result = calculate(**args)
            else:
                result = "Unknown tool"

            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": result
            })

        response = client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            tools=tools,
            tool_choice="auto"
        )

LangChain Agent with Memory

from langchain.agents import AgentExecutor, create_react_agent
from langchain.memory import ConversationBufferMemory
from langchain.tools import Tool
from langchain_openai import ChatOpenAI
from langchain import hub

llm = ChatOpenAI(model="gpt-4o", temperature=0)

search_tool = Tool(
    name="web_search",
    func=lambda q: f"Results for {q}",
    description="Search the web for current information"
)

memory = ConversationBufferMemory(memory_key="chat_history")

prompt = hub.pull("hwchase17/react")
agent = create_react_agent(llm, [search_tool], prompt)
executor = AgentExecutor(agent=agent, tools=[search_tool], memory=memory, verbose=True)

executor.invoke({"input": "Find the capital of Japan and then calculate 15% of its population."})

Explanation

• ReAct (Reasoning + Acting): the agent alternates thinking and doing — a reasoning step says why it needs a tool, an action step calls it. That interleaving is what makes ReAct debuggable: you can read the agent’s rationale, not just its calls.
• Function calling: GPT-4/5, Claude, and Gemini all emit structured tool calls — a tool name plus JSON arguments the app parses and runs, with the result coming back as a tool message.
• The loop: it runs until the model produces plain text (done) or hits a step ceiling (guardrail). Without the ceiling, ambiguous goals loop forever.
• Memory: stateless agents forget everything between calls. Conversation buffers keep the session; vector stores keep long-term facts the tools surface.

The Agent Loop

Every agent — ReAct, planner-executor, or a homegrown loop — runs the same cycle: observe the state, decide the next action, call a tool, feed the result back, repeat until done or give up. The loop is where agents actually earn their complexity; get the loop right and the rest is plumbing.

def run_agent(task: str, max_steps: int = 10) -> str:
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": task},
    ]
    for step in range(max_steps):
        response = client.chat.completions.create(
            model="gpt-5",
            messages=messages,
            tools=tools,
            tool_choice="auto",
        )
        msg = response.choices[0].message
        messages.append(msg)

        # No tool calls -> the agent decided it's done
        if not msg.tool_calls:
            return msg.content

        # Execute each tool call and feed results back
        for call in msg.tool_calls:
            fn = TOOL_MAP.get(call.function.name)
            try:
                result = fn(**json.loads(call.function.arguments)) if fn else f"Unknown tool: {call.function.name}"
            except Exception as e:
                result = f"Tool error: {e}"  # feed the failure back — the model can retry
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": str(result)[:4000],  # cap tool output or context explodes
            })
    return "Max steps reached — task may be incomplete"

Three details separate a working loop from a demo: max_steps as a hard ceiling (agents loop forever on ambiguous tasks), tool errors fed back as results rather than thrown (the model can often recover), and tool output capped (a 100KB search result eats the context).

When the Loop Fails

The failure modes worth planning for: infinite loops (the agent can’t tell it’s done — always cap steps), tool hallucination (the model calls tools that don’t exist — validate against TOOL_MAP), context bloat (long tool outputs push early messages out — cap and summarize), and runaway cost (each iteration is a full call — a 10-step agent on premium models isn’t free).

Variants

Pattern	Reasoning	Tool use	Memory	Best for
ReAct	Explicit chain-of-thought	Structured functions	Buffer	General-purpose agents
Plan-and-execute	Pre-planned steps	Batch tool calls	Minimal	Predictable workflows
Reflection	Self-critique	Retry loops	Episodic	Code generation, writing
Multi-agent	Delegation	Agent-to-agent	Shared	Complex systems

What Works

The details that separate a working agent from a demo are boring and worth repeating. Tool descriptions are prompts — “search stuff” gets you a model guessing, “Search the web for current news” gets you a model aiming. Never trust LLM output as tool input: validate the JSON, sanitize the arguments, because a clever prompt is one injection away from eval("rm -rf"). Cap iterations at 5-10 — agents loop forever on goals they can’t reach, and the meter runs the whole time. Log the reasoning trace, always: when the agent makes a bad call at 2am, the trace is the only way to know whether the model chose wrong or the selector offered wrong. And when the output must be data, constrain it — response_format: json beats regex-parsing free text every time.

Common mistakes

The failures rhyme. Dangerous tools by default: an agent with shell access or write permissions is a loaded gun — least privilege, human approval for anything destructive, no exceptions. Latency blindness: each tool call is an API round-trip, so a 5-step agent on a 2-second model takes 10+ seconds — that’s a timeout problem, not a UX choice. Over-engineering: if a single RAG lookup answers the question, an agent adds cost and failure modes for nothing — the honest question is “does this need multi-step reasoning?” not “can I use an agent?”. And unhandled tool errors: a dead API returns an error string, and a helpful model will hallucinate an answer around it — catch the exception, return a structured failure, and let the agent retry or escalate rather than improvise.

Troubleshooting

Inconsistent outputs? Temperature to 0 for deterministic work, seed where the API supports it, and version the prompt so you can diff behavior across releases. Injection leaking context? Keep user input and system instructions on separate rails — the model can’t leak what it can’t reach. Costs running hot? Cache the embeddings, summarize long context, and push simple steps to a smaller model — the router doesn’t need the flagship. Retrieval returning garbage chunks? Tune size, overlap, and metadata filters; score retrieval separately from generation or you’ll chase the wrong metric. And when the eval scores disagree with what a human would call good — they will — trust the human: rubrics help, more judges help more, but human review is the ground truth the scores serve.

Scaling Considerations

When deploying agents at scale, consider these factors:

• Token consumption grows quadratically: each tool call adds tokens to the context window. After 10 iterations with verbose tool outputs, you may hit the model’s context limit.
• Concurrent agent sessions: sessions are independent state machines — nothing is shared, so any number of them can run side by side on separate instances. Horizontal scaling is the easy part; the session store is what you’ll fight if you persist state.
• Model latency compounds: an agent making 5 sequential calls at 2s per response is a 10-second-plus wait. For user-facing agents, stream intermediate results so people see progress; for background work, a job queue absorbs the latency entirely.
• Cost control at scale: a single agent session can consume 10K-50K tokens. At GPT-4 pricing, that’s $0. 30-$1. 50 per session. Set per-session cost limits and switch to cheaper models (GPT-4o-mini) for simple tool-routing decisions.

Cost Estimation

Component	Cost per session	Notes
LLM calls (5 iterations)	$0.15-$0.75	GPT-4o at $5/1M input, $15/1M output
Tool API calls	$0.00-$0.10	Depends on tools (search, database, etc.)
Embedding (if RAG tool)	$0.001	text-embedding-3-small
Total per session	$0.15-$0.85	10K-50K tokens per session

For 1000 sessions/day: $150-$850/day. Switch to GPT-4o-mini for routing decisions to cut costs by 80%.

When Not to Use Agents

The agent pattern earns its complexity only when the loop is actually needed. Skip it when:

• The task is one LLM call: a summary or a classification doesn’t need an agent — call the model directly. Wrapping it in a loop buys latency, cost, and new failure modes for nothing.
• Same input must give same output: agents are non-deterministic. If the business requires repeatability, use a fixed pipeline and drop the LLM router.
• The latency budget is tight: five tool calls at 2s each is a 10s+ loop. Under ~2 seconds, pre-compute or make a single call with no tools.
• The APIs change every week: tool definitions rot fast — if the contract isn’t stable, the agent breaks on every deploy.
• Volume is high and complexity low: a session costs 10-50x a single call. Simple high-volume tasks belong in a prompt chain.

Further Reading

• Anthropic — Building Effective Agents — when to use agents vs. workflows
• OpenAI — Function Calling Guide — the mechanics of tool use
• LangChain — Agents — building multi-turn tool agents
• ReAct: Synergizing Reasoning and Acting — the pattern’s foundational paper