Function calling: letting the model use your tools

A plain LLM has a hard limit: it can only produce text from what it already knows. It can’t look up today’s order status, can’t query your database, can’t check live inventory, can’t do anything but talk. Function calling (OpenAI shipped it earlier this year and refined it since) removes that limit in a specific, controlled way: you describe your functions to the model, and when answering a question would require one, the model responds not with prose but with a request — “call get_order_status with order_id 1234.” Your code runs the function, hands the result back, and the model uses it to answer. A model that could only talk becomes one that can look things up and act. Let me unpack how it works, because under the buzzword it’s a simpler and more familiar mechanism than it sounds — and the control model is the part that matters most.

The problem it solves

The limit is worth stating plainly because it’s the whole motivation. An LLM’s knowledge is frozen at training time and confined to what was in its training data. It does not know your application’s data, it does not know anything that happened after training, and it cannot perform an action in the world. So “what’s the status of my order?” or “how many of these are in stock?” are unanswerable by the model alone — the answer lives in your systems, which the model has no access to. Function calling is the bridge: a structured way for the model to say “to answer this, I need you to run this function and tell me the result.”

How it actually works

The mechanism is a loop, and it’s less magical than the framing suggests. Step by step:

1. You describe your functions to the model — each one’s name, what it does, and the parameters it takes, as a structured schema. This is you telling the model what tools are on the table.
2. You send the user’s question along with those function descriptions.
3. The model decides. If it can answer directly, it returns text as usual. If answering needs one of your functions, it instead returns a structured request naming the function and the arguments it wants — extracted from the user’s question into the parameter shape you described.
4. Your code runs the function. This is ordinary code you wrote — a database query, an API call, whatever. The model doesn’t run anything; it only asks.
5. You send the result back to the model, which now has the data it was missing and produces a natural-language answer using it.

functions = [{
  name: "get_order_status",
  description: "Look up the current status of an order by its ID",
  parameters: {
    type: "object",
    properties: { order_id: { type: "integer" } },
    required: ["order_id"]
  }
}]

# The model, asked "where's order 1234?", responds with:
#   { name: "get_order_status", arguments: { order_id: 1234 } }
# You run YOUR code:
status = Order.find(args[:order_id]).status
# ...then hand `status` back to the model to phrase the answer.

That’s the whole shape. The model’s job is to (a) decide a function is needed and (b) extract the right arguments from the messy natural-language question into the clean structured form. Your job is to actually execute and return a result. Two collaborators with a clear division of labour.

It’s structured output with a loop around it

Here’s the demystifying observation, and it connects straight back to things we’ve already covered: function calling is structured output plus an execution loop. The model isn’t doing anything fundamentally new — it’s producing structured JSON (a function name and typed arguments) instead of prose, which is exactly the “make the model emit a specific shape” capability we leaned on for classification and extraction. The new part is purely the loop around it: you take that structured request, run real code, and feed the result back for another round. Seen this way, function calling isn’t an exotic capability bolted onto the model; it’s the structured-output capability you already understand, wired into a call-and-return cycle with your own code. That reframing makes it far less intimidating to build with.

You are in control of what runs

The single most important property — and the one that should shape how you think about the whole feature — is this: the model never executes anything. It only requests. When the model “calls” get_order_status, nothing runs on its side; it emits a request, and your code decides whether and how to honour it. That control point is where all your engineering judgement lives:

• You validate the arguments before acting on them. The model extracted order_id: 1234 from the user’s text — treat that exactly like any user-supplied input: validate it, check types, never trust it blindly. A model-extracted argument is untrusted input, full stop.
• You enforce authorization. Just because the model requested get_order_status for order 1234 doesn’t mean this user may see that order. Your code applies the same access checks it always would — the model’s request doesn’t bypass your authorization, because the model’s request doesn’t do anything until your code acts on it.
• You decide what’s on the menu. The model can only request functions you described to it. You choose what to expose, and you’d expose read-only lookups far more readily than anything destructive. A function that changes or deletes data, invoked off a model’s request, deserves real scrutiny — ideally a confirmation step, never a blind execution.

This is the part the hype skips and the part that actually matters for building responsibly. Function calling does not hand the model the keys to your systems. It gives the model a way to ask, and leaves every actual decision — validate, authorize, execute, or refuse — firmly in your code, where it belongs. Treat a model’s function request exactly as you’d treat an HTTP request from an untrusted client: as input to be checked, not a command to be obeyed.

What it’s good for, and what to watch

Function calling shines for the lookup case: answering questions from live data the model couldn’t otherwise reach — order status, account details, current inventory, anything that lives in your systems. It’s also the foundation of more ambitious “agent” patterns where a model chains several tool calls toward a goal, though that’s a much larger topic with its own failure modes. A few practical cautions:

• Each round-trip is a model call — slow and metered, exactly the cost model we’ve discussed throughout. A function-calling interaction may take several model calls (request, then answer), so the latency and cost add up. Budget for it.
• The model can pick the wrong function or wrong arguments. It’s a probabilistic extractor, not a guaranteed-correct one. Validate, and design so a wrong guess fails safely rather than doing damage.
• Keep the function set small and well-described. Too many vaguely-described functions and the model picks badly. Clear names and descriptions are, once again, the highest-leverage thing — it’s prompt engineering applied to your tool definitions.

Verdict

Function calling removes the LLM’s hard limit — that it can only talk — by giving it a structured way to ask your code to run a function and return a result, turning a model that’s frozen at training time into one that can look up live data and act through your systems. The mechanism is a simple loop: describe your functions, let the model emit a structured request when one is needed, run your code, feed the result back. Demystified, it’s nothing more than the structured-output capability you already use, wrapped in a call-and-return cycle — not an exotic new power. And the property that matters most is that you stay in control: the model never executes anything, it only requests, so every model-issued argument is untrusted input to validate, every request is subject to your authorization, and you decide what functions are even on the menu — exposing read-only lookups freely and guarding anything destructive behind real checks. Build with it for the lookup cases where it shines, budget for the extra round-trips, and treat the model’s requests exactly as you’d treat input from any untrusted client. Used that way, it’s a genuinely powerful bridge from a model that can only talk to one that can actually help — without ever handing over the keys.