What we learned shipping AI agents to production
Agents are fun in a demo and humbling in production. Here are the guardrails that made ours reliable enough to trust.
Agents are fun in a demo and humbling in production. Here are the guardrails that made ours reliable enough to trust.
Function calling, agents, and MCP all rest on the same thing: tools an LLM can invoke. But a tool built for a non-deterministic, fallible caller needs different design than one built for code. A deep-dive on what makes a tool reliable for an LLM — naming, scope, validation, error messages it can recover from, and the safety that can't live in the model.
When an AI agent does the implementation, the specification becomes the highest-leverage artifact in the project — a clear spec produces good code, a vague one produces a confident mess. A deep-dive on why spec-driven development has resurged in the agent era, what a good spec for an agent looks like, and how the workflow actually changes.
The most reliable AI systems in production aren't fully autonomous — they keep a human at the decision points that matter. But 'human in the loop' is easy to do badly: rubber-stamp approvals, alert fatigue, humans blamed for the model's mistakes. A deep-dive on designing the human-AI division of labour so both do what they're good at.
There are three ways to make an LLM do what your application needs — prompting, retrieval, and fine-tuning — and teams routinely reach for the wrong one. They solve different problems, and the costs differ enormously. A trend post on what each is actually for, and a decision order that saves you from over-engineering.
LLM calls are metered per token, so in an LLM-heavy app, cost is an engineering dimension you design for — not a bill you discover. A deep-dive on the patterns that keep it under control: model tiering, caching, token discipline, budgets, and the measurement that has to come before any of it.
The prototype was magic. Then we shipped it to real users, and everything the demo had hidden came due at once — cost, latency, abuse, the API going down, and the unique problem of supporting a feature whose output you can't fully predict. A war story about the gap between an LLM demo and an LLM product.
A year ago, AI in the editor mostly meant autocomplete. Now we have tools that edit across files, run commands, and work through a task while you watch. After a year of using them in real work, here's an honest review: what they're genuinely good at, where they fall down, and how they change the job.
Function calling lets an LLM ask your code to run a function and use the result — turning a model that can only talk into one that can look things up and act. A deep-dive on how it actually works, why it's just structured output with a loop around it, and why you stay firmly in control of what actually runs.
You can't assert exact equality on output that's different every time — so how do you test an LLM feature at all? A deep-dive on evals: building a representative test set, scoring properties instead of strings, catching regressions, and treating evaluation as the safety net that lets you change prompts with confidence.