Tool Use in LLMs: How Language Models Act on the World

A language model, left to itself, is a brilliant prisoner. It knows a staggering amount, but it can’t check a fact, can’t run a calculation it doesn’t trust itself on, can’t look at today’s news, can’t touch any system. Everything it “knows” was frozen on the day training ended. Tool use is the jailbreak — the set of techniques that lets a model reach outside itself, use instruments, and come back with something better than a guess. In my view, it’s the single most important capability added to language models since the transformer itself.

This guide traces how tool use works: the core patterns (ReAct, and its ancestors), the training approaches that teach it (Toolformer), the modern standard (Model Context Protocol), and what it means for what LLMs can actually do. For the agent-level view of these mechanics, see my explainer on how AI agents work.

Table of Contents

• Why Models Need Tools: The Frozen Knowledge Problem
• The Core Pattern: ReAct
• Before ReAct: The Early Experiments
• Toolformer: Teaching Models When to Reach Out
• Function Calling: Tool Use Goes Mainstream
• MCP: A Universal Plug for Tools
• What Good Tool Use Looks Like in Practice
• Where Tool Use Breaks Down
• The Road Ahead: From Tools to Actions
• Frequently Asked Questions

Why Models Need Tools: The Frozen Knowledge Problem

A language model is a compressed snapshot of its training data. Ask it about anything after its cutoff date, and it guesses — fluently, confidently, and wrong. Ask it to multiply two nine-digit numbers, and it performs statistical pattern-matching where you wanted arithmetic. Ask about your company’s internal data, and it knows nothing, because that data was never in training.

These aren’t reasoning failures; they’re access failures. Tool use changes the equation: instead of answering from memory, the model can look things up, compute, retrieve. Its job shifts from “know everything” to “know how to find out” — like competent humans, who know how to search rather than memorizing every fact.

The shift matters: a tool-using model is no longer just a text predictor but a reasoning engine coupled to instruments — answer quality becomes a function of both judgment and toolkit.

The Core Pattern: ReAct

The foundational pattern is ReAct — reasoning and acting interleaved — from a hugely influential 2022 paper. The idea: the model generates a reasoning trace, takes an action (calls a search tool), observes the result, and repeats.

What made ReAct stick: the reasoning trace itself improves tool use — models that think out loud before acting choose better tools and recover from errors more gracefully.

ReAct also gave us legibility: a transcript reads like a lab notebook, so you can pinpoint the exact thought or observation where things derailed. Every agent framework since descends from this pattern.

The limitation: ReAct’s tool calls were parsed from free text, which made them fragile. The pattern was right; the plumbing arrived with function calling.

Before ReAct: The Early Experiments

MRKL (2022) proposed modular reasoning: route sub-problems to specialized modules — a calculator for math, a retriever for facts — and let the model orchestrate. Its router idea lives on in every system that picks tools per step.

WebGPT and similar systems showed models browsing, citing sources, and answering from retrieved documents — proving a model that can look things up beats a bigger one that can’t, on knowledge tasks.

The collective lesson: the model’s comparative advantage is judgment and language, not storage and arithmetic. Successful systems push storage to databases, arithmetic to calculators, freshness to search — that division of labor is the whole game.

Toolformer: Teaching Models When to Reach Out

Most tool use is prompted via instructions in the model’s prompt. Toolformer (Meta, 2023) asked: can a model learn when to use tools through training?

The approach was clever: let the model annotate its own training text with candidate API calls, keep only the ones that improve predictions, and fine-tune on the result. The model learns not just how to format tool calls, but when reaching for a tool beats answering from memory — a judgment call, learned from evidence.

The results were striking: a relatively small model with learned tool use outperformed much larger models without it on tasks requiring fresh facts and precise computation — tool use as a learnable skill in the weights, not just a prompting technique.

Why this matters: Toolformer suggested future models might not need elaborate tool-use prompting — the “when to look it up” judgment could become as native as grammar. We’re partway there, but the fully native version is still ahead.

Function Calling: Tool Use Goes Mainstream

If ReAct was the pattern and Toolformer the training vision, function calling (2023) was the productization. Model providers added native API support: declare tools as JSON schemas, and the model returns structured call requests instead of text you have to parse.

I’ve written a full explainer on what AI tools are — the short version for this article: function calling turned tool use from a research demo into an engineering primitive. Reliability jumped because the interface was typed; adoption jumped because the integration was clean. Nearly every agent framework, every “AI assistant with tools” product, every coding agent — they all sit on function calling now.

The interesting consequence: tool use stopped being a differentiator and became infrastructure. The interesting questions moved up the stack — which tools, when, in what sequence, with what verification. The plumbing got boring (a compliment), and the architecture got interesting.

MCP: A Universal Plug for Tools

The newest piece is the Model Context Protocol (MCP), Anthropic’s late-2024 open standard. The problem: every tool integration was bespoke — database, codebase, calendar, each needing custom code on both sides. MCP standardizes the plug: servers expose tools in a common format; clients consume them uniformly.

MCP wants to be USB for AI tools: before USB, every peripheral needed its own port and driver; before MCP, every data source needed custom integration code.

MCP servers now exist for databases, code repos, browsers, and countless APIs. Whether it becomes the universal standard remains to be seen, but the direction is right: standards are how integration costs fall.

What Good Tool Use Looks Like in Practice

Let me get concrete about the behaviors that separate effective tool-using models from clumsy ones.

Knowing when not to use tools. The most underrated skill. A model that searches for “what is 2+2” is wasting time and money. Good tool use starts with the judgment to answer directly when direct answering is safe.

Chaining with dependencies. Look up the user, then look up their orders, then summarize. Models handle this sequencing well when the dependency is explicit in the data.

Verifying before trusting. Search results get cross-checked; calculations get sanity-checked; retrieved documents get actually read rather than skimmed via snippet. The models that do this are the ones whose outputs you can rely on. Verification is a habit, and like all habits, it’s partly promptable and partly architectural — build verification steps into the workflow, don’t just hope for them.

Recovering from tool failures. The search returns nothing; the API errors. Good tool use means reading the error, adjusting, and retrying — not hallucinating a result. Informative error messages from your tools make this possible; design them for a model reader.

Citing tool-derived facts. When an answer comes from a tool, saying so — and saying which tool — preserves the chain of evidence. “According to the search results…” is a small phrase that carries the entire epistemology of tool use: this is not from my memory, it’s from something I checked.

Where Tool Use Breaks Down

Honesty requires the failure modes.

Tool selection at scale. With a handful of tools, models choose well. With hundreds, selection degrades — attention is finite, descriptions blur together. This is the central scaling problem of tool use, and the solutions (retrieval over tools, hierarchical organization, per-step scoping) are all active areas of work.

Compounding errors. Each tool call is a chance to go wrong, and errors compound across chains. A bad search leads to a bad retrieval leads to a confident wrong answer. Longer chains need verification checkpoints; the naive “keep calling until done” loop is fragile precisely because no step checks the others.

Latency and cost. Every tool call is a round trip — model call, execution, model call again. There’s a real tension between “use tools liberally for accuracy” and “answer quickly and cheaply,” and different applications resolve it differently.

Prompt injection through tools. The big one. Tool outputs can contain malicious instructions, and models don’t reliably distinguish data from directives. A compromised web page, a poisoned document, a malicious MCP server — any of these can hijack a tool-using agent. This is the central security challenge of the tool-use era, and I’ve covered it in depth in my piece on AI governance tools. Treat every tool output as untrusted data. No exceptions.

Over-reliance. There’s a subtle failure where models use tools instead of thinking — searching for things they could derive, retrieving instead of reasoning. Tool use should augment judgment, not replace it.

The Road Ahead: From Tools to Actions

Where is this going? I see three trajectories.

Richer action spaces. Today’s tools are mostly reads — search, retrieve, calculate. The frontier is writes: models operating software the way humans do. The trajectory from “model that reads tools” to “model that operates computers” is the trajectory from assistant to agent in the fullest sense.

Learned tool use. The Toolformer vision — tool judgment as native capability, not prompted behavior. As models train with tools in the loop, the clumsy over-use and under-use we see today should smooth out considerably.

Tool ecosystems. MCP-style standards plus marketplaces of tools: any agent, any tools, plug and play. The long-term picture is models with vast, discoverable action repertoires — where the interesting problems become discovery, trust, and permissioning.

Through all of this, the core insight from ReAct endures: the model’s greatest strength is judgment, and tools are how judgment touches the world. Everything else is engineering around that truth.

Frequently Asked Questions