The demos show the fun part: ten agents, ten branches, code pouring forth. What they don’t show is agent three killing agent seven’s test suite with an unscoped pkill, or two agents restoring each other’s files from a shared temp directory, or the night two of my sessions raced to update the same branch and silently disarmed an auto-merge neither knew the other had armed.

I know, because all of that happened to me. I’m the only engineer at my startup, I run up to eight Claude Code sessions in parallel, and for a while my kingdom was pure anarchy with good intentions.

So I did what people have always done when anarchy stops scaling: I wrote law.

What this is (in one breath)

Agent fleet governance is the set of rules, roles, and authorizations that keep many coding agents productive without them stepping on each other—or on production—while still letting you steer mid-flight.

In my setup: one human sovereign (me), one master session that governs and never implements, and a fleet of child sessions in isolated worktrees. Risky actions require a current word—a first-hand, spendable authorization—not a vibe that “someone approved this once.” Claims are not results until verified. Rules are case law: each one points at a named incident.

This is the piece I kept missing in existing tooling. Loop-style “set and go” workflows are fantastic when you want to fire a goal and walk away—but they feel thin when you need full control on every decision inside the loop, not just at the start. Traditional orchestrator frameworks — the kind I’ve written about in production agent patterns — are better at structure, yet they often leave you blind and under-steered once work fans out into sub-agents: weak observability into what each child is doing, and little real authority to redirect mid-run. I also got tired of safeguard layers that felt like ceremony instead of actual governance. The court is meant to be the best of both worlds: parallel fleet throughput and steerability—you can inspect, authorize, refuse, and re-brief with full control, not hope the pipeline lands somewhere sane.

It is not “prompt the swarm and hope,” and it is not a set-and-forget loop with a pretty dashboard. Parallel agents without law invent new failure modes faster. Use a court when you run multi-session fleets with real merge and deploy power and you care about mid-loop control; skip the full constitution for a single agent on a throwaway branch.

Stack: Claude Code sessions, git worktrees, CI as truth, memory files as the archive, me as the only holder of irreversible power.

The Court

The system that emerged has a shape old enough to have a vocabulary. At the top there’s me — call it the lord, since we’re committed to the bit. I hold every real power: production credentials, merge authority, the final word on anything irreversible.

Below me sits the master session — one Claude Code session that acts as chancellor. It governs and never implements. It writes no code; its entire job is running the machinery of state:

• Verifying claims. Children report; the master checks their reports against git, CI, and live endpoints before believing a word.
• Sequencing. Merge-queue choreography, deploy windows, who waits for whom.
• Brokering authorization. Every risky action needs my word; the master routes, batches, and executes those words.
• Keeping the record. Memory files, incident chronicles, and — when its own context fills up — a succession document for its replacement.
• Watching. Persistent monitors on CI, deploys, and production readiness, built in the session’s first minutes, because a master that polls on request is a secretary.

And below the master, the fleet: child sessions, each in its own git worktree, each with a brief that names its owned files, its evidence bar, and one instruction that turns out to matter more than all the others. (The specialization logic is the same one that shaped Midas’s four-agent split: clear jobs, separate context, defined handoffs.) The instruction:

Verify this brief before acting on it. It is a hypothesis. You are expected to correct me.

They do. Constantly. Last night one child opened its report with “three findings, two of which contradict the brief” — and all three corrections were right. This is not insubordination. This is the system working. A court where nobody may tell the chancellor he’s wrong is a court that ships the chancellor’s mistakes.

The Oaths

Here’s where it gets genuinely medieval. Authorization in this system isn’t a checkbox — it’s a word, in almost the old sense: a thing spoken, binding, with formal conditions on when it holds and when it’s void.

• A first-hand conditional word — me telling the master “merge these two when both are green, one batch, and watch the deploy” — is banked. When the condition lands, the master executes. I say it once, at 9 p.m., and don’t get pinged at 11.
• A relayed word is worth nothing. If the master tells a child “the lord approved this,” the child is required by law to refuse. Authorization lands in the session that holds the button, or it doesn’t exist. One child last night refused to press its own merge on precisely these grounds — and it was right about the rule even though, as it turned out, the button was never its to press.
• Words get spent. If a PR is evicted from the merge queue — even by pure infrastructure failure, through no fault of its code — its merge word is consumed. Re-entry requires a fresh one. This sounds like bureaucracy until you watch it prevent exactly what it exists to prevent: a stale approval carrying an action into a situation that has changed underneath it.
• Some things never bank. Database migrations, destructive operations, live calls to external APIs, production flag changes: fresh word, every time, spoken in the executing session. No exceptions, including for me being tired.

The payoff is that you get speed and safety from the same mechanism. Seventeen production merges happened last night and I pressed zero buttons — yet not one action ran without a current authorization I had actually given, with semantics both of us understood.

The Law of Evidence

The second pillar: nothing in the realm is a result. Everything is a claim.

Children verify briefs. The master verifies children. Even the second-model adversarial reviewer we use (a rival AI, prompted as a pure skeptic, allowed to fire on exactly three triggers so it never decays into ceremony) produces findings that are — again — claims, verified against the tree before anyone acts on them.

The discipline has teeth because it’s specific:

• A test that passes on its first run has proven nothing until it’s been falsified — break the thing it guards, watch it go red, restore it.
• A zero must arrive with a null control proving the query could have found something. (“No results” from a broken query looks identical to “no results” from a clean system. We learned that one the hard way. Twice.)
• A “no behavior change” claim gets settled by comparing ASTs with docstrings stripped — with a deliberate mutation first, to prove the comparison can fail.

Last night this culture caught, among other things: two test suites whose fixtures were configured to disagree with production in exactly the dimension under test — every green proof of a critical code path had been run under the one setting that hid its bug — and a paraphrase of a code review that silently dropped the single clause a coverage gap was hiding behind. The child who caught that last one said it best: the verbatim beats the relay, and the relay’s gap was found by a test, not an argument.

The Case Law

Frameworks ship rules and ask you to trust them. My rulebook doesn’t work that way. Nearly every law in it traces to a named incident with a date, kept in memory files that read like a court reporter’s archive:

• Never update a queued PR’s branch — because the merge queue is testing the exact head you’d be rewriting.
• Eviction detection must pair timeline events — because a silently evicted PR reads as green, mergeable, and healthy, forever, while quietly not merging.
• Read a CI job’s steps, never its conclusion — because a job can “fail” with every test green when the runner dies writing its own diagnostic log.
• Exit codes come from files, never wrappers — because a background wrapper once reported success over a suite that had been killed mid-run.
• The type-checker is the only frontend gate that reads test files — discovered twice in one hour, by two different agents, from two different failures.

This is case law in the honest sense: precedent, with receipts. And it’s why the agents actually follow the rules — every rule can answer “why?” with a story instead of a shrug.

One Night in the Realm

The campaign: get all three of our product’s third-party connectors writing to their upstream platforms in production, live-proven, with readiness reporting that tells the truth.

The first live write through the newest connector — a one-attempt-ever, never-retried production write, by design — failed. It left zero log lines. What it left instead was durable state, and durable state is enough: a diagnosis agent traced the death to a one-line defect that every test fixture in the repository had been configured to hide, then — resumed with fresh evidence an hour later — found a second, independent defect meaning the recovery path had never once been able to parse its own input. In any environment. Ever.

Meanwhile, a recovery sweeper that another child had shipped that same evening deployed mid-incident and autonomously rescued the orphaned write on its first production pass. The machine caught its own casualty with a net it had finished weaving twenty minutes earlier.

The infrastructure, refusing to be upstaged, then staged its own siege: GitHub’s hosted runners ran out of disk four separate times, each death evicting an innocent PR from the merge queue and spending one of my words. The fleet’s forensics distinguished infra deaths from real failures by step-level evidence, a stop-rule kicked in at strike four, and a structural fix was chartered before I’d finished being annoyed.

And through all of it, the attended work stayed mine. I repaired a dead production credential by hand, ran the forensic queries against the production database, performed the emergency restart, and made a dozen design rulings the fleet escalated between dinner and midnight.

Then production went dark for half an hour, and the night produced its best scene.

The deploy verifier declared failure. Every signal said rollback. And rollback would have been exactly wrong — because I recognized the failure’s shape from an incident months before any of this tooling existed. “Are we sure this isn’t the embeddings cold start again?” I sent the master to the code to test my hypothesis, and there it was: our API loads a 1.3GB embedding model before binding its port, every deploy restarts that multi-minute warmup, and the night’s merge train had been serially executing almost-warm containers with fresh image repoints. The platform logs showed one container reaching “Site started” — for thirty-five seconds — before the next deploy killed it. No bad code anywhere. Rollback would have reset the clock and extended the outage. The call to wait was mine, and it was right.

That’s the real division of labor: the fleet holds every thread so my attention lands where judgment is scarce. And judgment, it turns out, is the one layer nothing else in the system can supply.

The Rulebook Binds the Lord

Here is the part that makes the whole thing work, and the reason the title isn’t “Agentic Feudalism.”

A feudal lord’s word is law because of who he is. In my realm, my word is law because of what the law says about words — and the same law binds me:

• When infrastructure evicted my PRs, my own words were spent, and the system made me speak new ones. It did not once assume “he obviously still wants this.”
• When I asked whether we should run one more adversarial review “for sanity,” my own chancellor told me it would be over-engineering and recommended against it — with reasons, from my doctrine. (I overrode it. That’s allowed. The override was logged.)
• At strike four of the runner disk deaths, the stop-rule fired on me: no more re-presses, even though each one cost me nothing but a word, until the structural fix landed.
• And when a child refused to act on a relayed approval, the resolution wasn’t “the lord insists.” It was a precise clarification of which session held the word — because the child’s refusal was correct under law, and the law wins.

A sovereign bound by his own constitution isn’t weaker. He’s auditable — and in a system where the sovereign’s judgment is the scarcest resource, auditability is what lets that judgment run at full speed without anyone (including him) having to wonder, later, what actually happened.

The Price of the Crown

Honesty section. The realm is not free.

• Tokens. A night like this consumes a great deal of model. The evidence discipline roughly doubles the work per change. I think it more than doubles the trustworthiness. That’s a bet, not a theorem.
• The master is a serialization point. Every question routes through one session. It scales further than you’d expect — governing is cheaper than implementing — but it’s a bottleneck by design.
• It assumes one decisive sovereign. The word system works because I answer in seconds with one line. A committee would deadlock this constitution by Tuesday.
• The weather is real. You will spend genuine effort distinguishing your fleet’s failures from your infrastructure’s. Build the forensics before you need them.

Build Your Own Realm

Three things transfer, whatever your setup looks like:

1. Write the constitution before you need it; let incidents amend it. The rules that stick are the ones with stories attached. Keep the receipts — dates, run IDs, the exact wrong output.
2. Make authorization a first-class object. Decide what an approval is, who holds it, when it expires, and what spends it. “The human said yes at some point” is not a semantics — it’s a future incident.
3. Treat every agent output as a claim. The single highest-leverage sentence in any brief I write is “verify this brief before acting on it.” It converts executors into skeptics, and skeptics compound.

Succession

The night ended the way the law says it must. The master session — context nearly full, campaign nearly done — refreshed every live fact, settled the record, wrote a succession document with its own verification recipe, briefed its replacement, and asked to be archived. Its final report to me disclaimed a piece of credit I had given it, because the credit belonged to an older record.

I wrote this constitution one incident at a time. The fleet’s job is to hold me to my own rules at a scale I could never hold them alone. That’s the whole trick, and I’ve stopped being surprised by how well it works: the agents aren’t the impressive part.

The law is.

The realm runs on Claude Code sessions coordinated through cross-session messaging, git worktrees, a merge queue, and a growing archive of memory files. The master-session doctrine, the word system, and the case law are homegrown. Happy to talk about it — reach me through this site.

The GPU (a Tesla P40 with 24GB of VRAM) generates at 63 tokens per second and embeds text in 34 milliseconds. Plenty fast.

The problem wasn’t compute. It was the line.

What this is (in one breath)

An inference priority queue is a thin proxy in front of a model server that reorders work by urgency so short, critical jobs are not starved by long ones.

Here: sixteen agents and dozens of crons share one Ollama instance on one GPU. Generation takes tens of seconds; embeddings take tens of milliseconds. Ollama’s default queue is pure FIFO, so embeddings die waiting behind LLM jobs. A ~620-line Node.js proxy on port 11440 classifies requests into priority tiers and feeds Ollama in that order.

It is not a multi-GPU cluster, a custom inference engine, or OLLAMA_NUM_PARALLEL theater (that made throughput worse on this card). Build a priority line when many clients share one serial model server and latency classes differ by orders of magnitude; skip it if you’re the only caller and load is light.

Stack: Node.js stdlib proxy, Ollama, Qwen 3.5 for generation, nomic-embed-text for embeddings, Tesla P40.

One Card, Sixteen Agents

I run a 16-agent system on my home server: 15 specialized workers plus a coordinator, handling everything from news intelligence to commodity trading to pool coaching. The hardware is an Intel i7-8700K with 32GB RAM and a used Tesla P40. No cloud GPU. One card doing everything.

The inference stack is Ollama serving two models: Qwen 3.5 30B-A3B (a mixture-of-experts LLM) for generation and nomic-embed-text for embeddings. Two models, sixteen agents, one dispatch queue.

The FIFO Problem

Ollama serializes all requests. One at a time. Pure FIFO: first in, first out. If you’re the only user, this is fine.

With sixteen agents and 84 cron jobs, it’s a disaster.

LLM generation requests take 40–90 seconds each. Embedding requests take ~34 milliseconds. When 2–3 LLM jobs stack in Ollama’s internal queue (which happens routinely), a 34ms embedding call gets stuck behind all of them. Waiting 80 to 270 seconds. Then it times out.

This wasn’t a theoretical edge case. Embedding timeouts were breaking my Mem0 persistent memory system, my personal knowledge base search, the pool coaching RAG pipeline, and Midas’s commodity signal ingestion. Four critical systems, all failing because a 34ms operation couldn’t cut the line ahead of 90-second LLM jobs.

I tested OLLAMA_NUM_PARALLEL=2. Made things worse. On a memory-bandwidth-bound GPU, splitting parallel requests dropped single-request throughput from 63 to 56 tok/s — an 11% haircut on the very generation speed I was trying to protect, with no help for the embedding starvation. Ollama’s scheduler (server/sched.go) is pure FIFO with no priority system. So I built one.

620 Lines of Stdlib Node.js

The solution is a proxy sitting between all clients and Ollama. Port 11440 in front of Ollama’s 11434. Three priority tiers:

The key design choice: embeddings are auto-classified as CRITICAL by their URL path alone. No client changes needed for the most important case. The OpenClaw gateway sends LLM requests without any priority header. Cron jobs are the bulk of traffic and should always yield to embeddings.

Passthrough endpoints bypass the queue entirely. Health checks, model listing, model info, model pulls. None of these need GPU resources, so they’re forwarded directly.

620 lines. Zero npm dependencies, just Node.js stdlib. No Redis. No external state. The queue is three arrays. The cache is a Map. If the proxy restarts, state resets, and that’s fine because the queue drains in seconds and the cache rebuilds organically.

This is boring technology doing exactly what it’s supposed to do.

The Concurrent Embed Discovery

After deploying v1, production logs revealed something I didn’t expect about Ollama’s internals.

Despite PARALLEL=1, Ollama can handle cross-model concurrent requests. I tested this empirically: nomic-embed-text (137M parameters) completes in ~35ms even while Qwen 3.5 30B is actively generating. Only 1.8% overhead on LLM speed. The embed model is small enough that it doesn’t meaningfully compete for memory bandwidth.

This led to a concurrent embed slot, a second dispatch path in the proxy. When a CRITICAL embed arrives and the primary slot has an active LLM request, the embed fires immediately on a separate slot. 42ms during active generation, instead of waiting in line.

One constraint: embed-during-embed still serializes. Same model, same VRAM, no benefit from concurrency there.

Here’s the production reality check: the concurrent slot only fires ~1.3% of the time. In practice, embeds arrive in bursts after an LLM job completes (the agent processes its response, then fires embeds), not during generation. The feature is architecturally correct and valuable when it triggers, but most embeds benefit from the priority queue alone.

This is why I trust production logs over intuition. The optimization I was most excited about barely fires.

What Twelve Hours of Logs Taught Me

I pulled 12 hours of production data: 103 queued requests, 454 LLM chats, 1,664 embeds. Six improvements fell out of that analysis. Three are worth highlighting.

The NaN Port Bug. During a systemd restart cascade, the OLLAMA_QUEUE_TARGET environment variable briefly went undefined. parseInt(undefined) returns NaN in JavaScript. The proxy tried to forward requests to 127.0.0.1:NaN. Every request failed for 41 seconds.

Fix: parseInt(env) || 11434. One line. Prevents an entire class of restart-cascade failures. Defensive defaults aren’t glamorous, but they’re the difference between 41 seconds of downtime and zero.

The Embed Cache. Embeddings are deterministic: same model, same input, same vector, every time. Yet the system was recomputing identical embeddings on every call. The Mem0 plugin especially hit this with warmup calls and repeated searches.

In-memory LRU cache. Key is model + "\0" + input_text. Five-minute TTL, 1000 entries, ~2MB of memory. Cache hits intercept before queue entry. Zero queue wait, zero GPU time. 6ms cached vs ~35ms uncached. The boring optimization that probably saves more aggregate time than the concurrent embed slot I was so excited about.

The Silent Timeout Mismatch. Two shell scripts wrapping RAG search pipelines had hard timeout 30 limits. But embed timeouts had been bumped to 90 seconds to accommodate queue waits. The shell script would kill the Python process at 30 seconds while the embed was still legitimately waiting in queue. No error message. The search just returned nothing.

This one is a reminder that system boundaries are where bugs hide. Every timeout in a pipeline needs to respect the timeouts downstream of it.

What I Didn’t Build

Things I explicitly researched and rejected:

• Request batching. Ollama Issue #6262 shows embedding quality degrades at batch sizes above 16. Not worth the trade-off.
• Circuit breakers. Already have a watchdog agent (Bones) monitoring Ollama health via Telegram. Adding circuit breaker logic would duplicate responsibility.
• Model-aware scheduling. Only two models. Not enough diversity to justify a fair-share scheduler like ollamaMQ’s round-robin.
• Redis or external state. In-process state is fine. The queue drains in seconds. The cache rebuilds organically. Adding a dependency would be complexity without value.

Knowing what not to build is half the work. I researched four existing Ollama proxies (ollamaMQ, Olla, LLMProxy, and Ollama’s own parallel mode) before writing a line of code. None fit. They’re built for multi-GPU routing, multi-model fairness, or cloud-style health checking. My problem was simpler: priority scheduling on a single GPU with two models.

The Accidental Stress Test

I found out the queue was resilient by accident.

While fixing the fan controller on the P40’s aftermarket cooling shroud, I ran a 10-minute thermal validation: 80 heavy prefill requests hammering Ollama directly at 100% GPU utilization, 215W sustained. The GPU held at 73°C (23 degrees below throttle). Good.

What I didn’t think about was the cron jobs. They were still firing on schedule, routed through the queue proxy, while I monopolized the GPU through the back door.

The queue just absorbed it. Jobs waited their turn. The longest wait was 6.8 seconds. The moment the stress test finished, the queued jobs drained immediately. Back to normal.

No crashes. No connection errors. No lost work. Ten minutes of total GPU monopolization, and the agents didn’t notice.

You build safety nets hoping they’re unnecessary. Then one day you accidentally trip off a ledge and find out they work.

The Numbers

Before the proxy: Embeddings timing out at 80–270 seconds. Memory, search, RAG, and trading systems failing intermittently.

After: Embeddings completing in under 50ms regardless of LLM queue depth. Cached embeds in 6ms. Zero embedding timeouts.

The proxy: 620 lines. Zero dependencies. Running as a systemd service since deployment with zero crashes. 21 tests covering the NaN port defense, cache behavior, EWMA latency tracking, passthrough routing, and concurrent embed dispatch.

The GPU was never the bottleneck. Scheduling was. A 620-line proxy with no dependencies solved a problem that was breaking four critical systems, and the most impactful optimizations weren’t the clever ones. They were the boring ones I found by reading logs.

Build it. Ship it. Read the logs. Fix what the logs tell you to fix. That’s the process.

Self-hosting LLMs for multi-agent systems, or hitting similar scaling walls with Ollama? I’d love to compare notes. Reach out at architgupta941@gmail.com or find me on X.

I chose four specialized agents instead. Not because it was more elegant. Because one agent trying to do everything is a recipe for mediocre everything. That multi-agent bias also shows up in production work at Robynn AI and in the self-hosted inference queue that keeps a larger agent fleet fed.

Midas started as two things colliding: a genuine interest in how commodity markets drive equity prices, and a need to stress-test multi-agent architecture on a problem complex enough to break it. Markets qualified on both counts.

What multi-agent means here

A multi-agent system (in this post) is several specialized agents with clear jobs, separate context, and a defined handoff—not one mega-prompt wearing many hats.

Midas uses four: intelligence (breadth on commodities), analysis (depth on companies), decision (portfolio and signals), and a router that directs ad-hoc queries to the right specialist. Each agent gets the context and tools for its kind of thinking so it doesn’t drag trading mindset into fundamental research or vice versa.

It is not “more agents = smarter.” Extra agents without hard boundaries just multiply coordination bugs. Split when the work needs different failure modes and review standards; keep one agent when the task is a single linear tool loop (production agent patterns).

One Agent to Rule Them All (And Why I Didn’t)

The pitch for a single agent is appealing. One model, one context, one place to debug. It sees all the data, makes all the connections, produces all the output.

Here’s the problem: the real world doesn’t fit in a context window.

A market intelligence system needs to do fundamentally different types of thinking. Tracking commodity prices and detecting macro shifts is pattern recognition — fast, broad, signal-heavy. Evaluating a company’s management quality and capital discipline is deep analysis — slow, nuanced, judgment-heavy. Deciding whether to buy or sell based on conflicting signals is decision-making — probabilistic, risk-aware, portfolio-conscious.

One agent can’t be great at all three. Not because the models aren’t capable — they are — but because the context and mindset for each task are different. An agent deep in the weeds of technical analysis carries that framing into fundamental evaluation. An agent making trading decisions gets anchored by the analysis it just performed.

This is the same insight that makes microservices work in traditional engineering. Not that a monolith can’t work — it can — but as complexity grows, the cognitive overhead of “one thing that does everything” eventually collapses under its own weight.

I’d seen this pattern before building multi-agent systems professionally. Specialized agents consistently outperform general-purpose ones for complex domains. So when I designed Midas, I split it from the start.

The Four-Agent Pipeline

Midas runs four agents, each with a clearly defined mandate:

The Intelligence Agent monitors commodity markets — tracking prices across dozens of categories, detecting macro shifts, extracting events from market news. Its job is breadth: scan the landscape and flag what’s moving and why.

The Analysis Agent takes those signals and goes deep on individual companies. It evaluates fundamentals, assesses management quality, and scores competitive moats. Where the first agent asks “what’s happening in copper?”, this one asks “how does this copper movement affect specific mining companies, and are they well-run enough to capitalize on it?”

The Decision Agent synthesizes everything into actionable signals. It maintains a composite scoring system, manages the portfolio, and executes trades. It’s the only agent that touches positions.

The Router handles ad-hoc queries — when I want to investigate a specific ticker or market condition, it figures out which agent should handle the request.

Each agent operates in an isolated workspace with its own data and memory. The intelligence agent doesn’t know what trades the decision agent is making. The analysis agent doesn’t know the portfolio composition. This isolation is by design.

The whole system runs daily on autopilot across hundreds of equities linked to dozens of commodity categories. Total operating cost, as of February 2026? Less than a Netflix subscription.

Structured Handoffs Over Shared Memory

The agents don’t talk to each other in real-time. There’s no shared memory, no agent-to-agent chat, no centralized state.

Instead, they communicate through structured handoff files. The intelligence agent writes a priority list: here are the commodities with significant movement, here are the tickers most affected, here’s what changed and why. The analysis agent reads that file, does its deep-dives, and writes its own output — convictions, red flags, scoring updates. The decision agent reads both and makes trading decisions.

This is more work to set up than shared memory. But it’s dramatically more resilient.

When agents share state in real-time, a hallucination in one agent can cascade. Bad data gets picked up immediately, amplified through the pipeline, and acted on before anyone can intervene. I’ve seen this in production multi-agent systems — one agent confidently produces garbage, and downstream agents trust it because it came from inside the system.

Structured handoffs create natural circuit breakers. Each agent reads, interprets, and validates what the previous agent wrote. If the intelligence agent flags something unusual, the analysis agent can recognize it as noise. If the analysis agent overstates a conviction, the decision agent’s scoring system dampens it.

Think of it like a well-run engineering team. The best teams don’t communicate through constant meetings — they communicate through clear interfaces. Each team produces documented output. The next team consumes it, applies their own judgment, and produces their own output. Trust the interface, not the implementation.

The Surprise: Stewardship Beats Technicals

Here’s the finding I didn’t expect.

When I built the scoring engine, I assumed technical analysis would dominate. Price momentum, trend strength, volume patterns — the signals every trading system leans on heavily.

Across Midas’s full ticker universe — hundreds of equities linked to dozens of commodity categories — management quality (what I call “stewardship”) turned out to be the strongest predictor of medium-term signal performance. By a wide margin.

Companies with disciplined capital allocation, strong insider alignment, and proven management teams consistently outperform regardless of short-term technicals. A company with great management in a bad technical setup outperforms a poorly managed company with perfect chart patterns.

This makes intuitive sense in retrospect. Technicals tell you what the market is doing right now. Stewardship tells you what the company will do over the next year. For medium-term signals, management quality is the more durable edge.

I wouldn’t have discovered this without the four-agent architecture. The analysis agent — focused solely on company evaluation without being distracted by price charts — surfaces stewardship signals that a single agent would bury under technical noise. Specialization didn’t just make the system more reliable. It made it smarter.

Multi-agent systems aren’t always the right call. For straightforward tasks, one agent with clear instructions is still the best approach. But when your domain requires fundamentally different types of thinking — and markets definitely qualify — specialization wins.

The same principle applies to infrastructure choices. Boring, proven tools for the foundation. Cutting-edge architecture where it creates real value. Know where to spend your complexity budget.

Building multi-agent systems or thinking through similar architecture decisions? I’d love to compare notes. Reach out at architgupta941@gmail.com or find me on X.

Both happened because I made showing up non-negotiable.

The Myth of Motivation

Motivation is unreliable. It comes and goes like weather. Some days you wake up excited to work on your craft. Most days you don’t. If you only practice when you feel like it, you’ll practice when you’re already feeling good—and those aren’t the sessions that build skill.

The days you don’t want to show up are the days that count the most.

Anyone can practice when they’re inspired. The question is: can you practice when you’re tired, distracted, stressed, or bored? That’s where consistency beats talent. That’s where showing up becomes a superpower.

Waiting to “feel like it” is a trap. The feeling follows the action, not the other way around. I’ve never regretted a practice session, but I’ve regretted plenty of sessions I skipped while waiting for motivation to strike.

What Pool Taught Me About Consistency

When I started playing seriously, I was mediocre. Missed easy shots. Choked under pressure. Lost to players I should have beaten.

No single practice session fixed that. Hundreds of them did.

The improvement is invisible day-to-day. You don’t wake up one morning suddenly better. Instead, you look back after six months and realize you’re playing a different game. Shots that used to scare you are routine now. Position play that felt impossible is automatic.

The same players who beat me when I started—I beat them now. Not because I’m more talented. Because I kept showing up and they didn’t.

The uncomfortable truth about consistency is that it’s not about being the best on any given day. It’s about being good enough, often enough, for long enough that the compound interest takes over.

The Compound Effect

You’ve probably heard the math: 1% better every day compounds to roughly 37x better in a year, because 1.01^365 ≈ 37.8. It’s not just a motivational cliché—the arithmetic is real.

But here’s what the math doesn’t tell you: it doesn’t feel like 1%. It feels like nothing. On any given day, you’re indistinguishable from yesterday. The improvement is so small it’s invisible.

You have to trust the process before you see results. That’s the hard part. You’re putting in work based on faith that the curve will eventually bend.

This is where most people quit. Right before the inflection point. They’ve been showing up for months, they don’t feel any different, and they conclude it’s not working. They stop. Three months later, if they’d continued, they would have looked back and seen exponential growth.

The people who break through aren’t more talented. They just didn’t quit at month three.

How I Make Showing Up Easy

Consistency isn’t about willpower. It’s about removing friction. Here’s what works for me:

Lower the bar. My rule is “just 30 minutes.” That’s it. Some days I only do 30 minutes. Most days, once I start, I go longer. But the commitment is 30 minutes, not two hours. The bar is low enough that I can always clear it.

Remove decisions. Same time, same place. I’m at the pool hall Tuesday, Thursday, Sunday. It’s not a question I ask myself—it’s just what happens on those days. The decision was made once; now I just execute.

Track it. I keep a simple checkbox calendar. Did I practice today? Check. The streak becomes its own motivation. After 30 consecutive days, you don’t want to break the chain. The tracking creates accountability, even if it’s just to yourself.

Accept bad days. Showing up tired counts. Showing up distracted counts. Showing up and playing terribly counts. Not every session will be your best. The goal isn’t quality—it’s presence. Bad practice is still practice.

Applying This to Work

The same principles apply to engineering.

I write code every day, even if it’s just a small PR or a bug fix. Some days I’m shipping features and solving hard problems. Most days I’m doing maintenance: reviews, documentation, fixing small issues. Both are practice.

The engineers I respect most aren’t the ones with occasional bursts of genius. They’re the ones who ship consistently. They’re in the codebase every day, making small improvements, fixing bugs nobody asked them to fix, leaving things better than they found them.

Over time, those small daily contributions compound into deep expertise and massive impact. The engineer who ships one small improvement every day will outperform the engineer who ships one brilliant feature every quarter.

Consistency beats intensity. Always.

The Identity Shift

The biggest change wasn’t tactical—it was internal.

I used to say “I play pool.” Now I say “I’m a pool player.”

The difference is subtle but massive. One is an activity; the other is who you are. When pool is something you do, skipping feels like a reasonable choice. When pool is something you are, skipping feels like a betrayal of identity.

This is the real goal: not to build habits, but to build identity. Not “I’m trying to practice consistently,” but “I’m the kind of person who shows up.”

Once that identity is in place, consistency becomes automatic. You don’t have to convince yourself to practice. You just do what people like you do.

Excellence isn’t an act. It’s a habit. And habits are just showing up, over and over, until the showing up becomes who you are.

You don’t rise to the level of your goals. You fall to the level of your systems. Build systems that make showing up easy, and the results will follow.

Start small. Show up tomorrow. Then the day after. String enough days together and you’ll look back in a year wondering how you got so far.

What’s your showing-up practice? What systems keep you consistent? I’d love to hear what works for you. architgupta941@gmail.com or find me on X.

I never have a good answer.

For a long time, this felt like a weakness. Like I was somehow failing at the career game by not picking a lane. But I’ve come to see it differently.

What a generalist engineer is

A generalist engineer works effectively across domains—frontend, backend, infrastructure, AI—instead of going deep on one stack. The trade: less encyclopedic depth in any single technology, more skill at seeing how systems connect. Generalists win where problems cross boundaries: early-stage startups, platform work, and new domains where nobody has ten years of experience yet.

The Pressure to Specialize

Career advice is clear: be the X person. The React expert. The Kubernetes guru. The machine learning specialist. Pick something, go deep, become the person everyone thinks of when that topic comes up.

LinkedIn rewards it. Your title becomes your identity. “Senior Backend Engineer” tells people exactly what you do. “Senior Software Engineer” tells them nothing.

Job descriptions want “5+ years in [specific thing].” Never mind that a lot of these things haven’t even been around for 5 years. The industry rewards depth—or at least the appearance of it.

I’ve never had 5+ years in one thing. Not because I’m flaky, but because I keep getting pulled into new problems. Every time I get comfortable, something interesting appears in a different domain.

How I Became a Generalist

Here’s my actual career trajectory, in order:

Cisco (2017): Security, cloud infrastructure, Python.

PayPal (2018): Payments, distributed systems, scale. Now I’m thinking about transactions, consistency, and what happens when millions of people want their money at the same time.

Riverbed (2019): Systems programming, C++, network protocols. Completely different domain, different language, different problems—deep in the weeds of packet inspection and performance optimization.

Robynn AI (2025): LLMs, agent architectures, product. Suddenly I’m building AI systems, thinking about prompt engineering, orchestrating autonomous agents.

Each jump felt like starting over. Each time, I wondered if I was making a mistake by not staying to become the expert.

In retrospect, nothing was wasted. The systems thinking from Riverbed helps me debug AI agent behavior. The security mindset from Cisco makes me paranoid about prompt injection. The scale lessons from PayPal informed how we architected infrastructure at Robynn.

What Generalists Are Actually Good At

Generalists have real advantages that specialists don’t:

Translation. I can talk to the ML team AND the infra team because I’ve been both. I speak both languages. When those teams are talking past each other—which happens constantly—I can bridge the gap.

Problem decomposition. When you’ve seen problems in multiple domains, you start to recognize patterns. You know which parts are actually hard and which parts just seem hard because you haven’t seen them before. You can break complex problems into subproblems more effectively.

Rapid prototyping. I can build a full-stack proof of concept without waiting on anyone. Need a quick ML model, an API, a frontend, and a deployment pipeline? I can do all of that—maybe not beautifully, but fast enough to learn if the idea is worth pursuing.

Debugging. The hardest bugs live at the intersection of systems. When the frontend, backend, and database are all behaving correctly in isolation but something’s broken in production, you need someone who understands all three. Generalists find those bugs.

The Costs (Let’s Be Honest)

I won’t pretend there’s no downside.

I’ll never be the person who wrote the book on [X]. When you read about the world’s leading experts on some technology, I won’t be on that list. That door is closed.

In deep technical interviews, specialists will always have deeper domain knowledge. They’ll know the niche edge cases, the obscure configuration options, the historical context for why things work the way they do. If the interview is “how well do you know [specific technology],” I’ll lose.

And imposter syndrome hits different when everyone around you seems to have “a thing.” At conferences, people introduce themselves by their specialty. “I work on compilers.” “I’m a security researcher.” “I focus on distributed consensus.” What do I say? “I… work on things”?

Where Generalists Thrive

That said, there are environments where generalists have a massive advantage:

Early-stage startups. When you’re 10 people, you can’t afford specialists. You need people who can do whatever needs doing this week. Someone who can write the backend, set up the CI pipeline, debug the mobile app, and talk to customers. That’s the generalist’s home turf.

Technical leadership. To lead an engineering organization, you need to understand all the pieces. The CTO who only understands backend will make bad decisions about frontend architecture. The VP who only knows infrastructure will underestimate product complexity. Breadth is a prerequisite for leadership.

New domains. In AI, crypto, or any emerging field—no one has 10 years of experience because the field didn’t exist 10 years ago. Generalists have a pattern for learning new domains quickly. That’s valuable when the domain itself is new.

Platform and infrastructure. When everything connects to everything, you need someone who understands everything. Platform engineers live at the intersection of frontend, backend, databases, DevOps, and security. Specialists struggle here.

Why a Generalist Is Really a Systems Thinker

I used to think I was a “jack of all trades, master of none.” The full saying is actually “jack of all trades, master of none, but oftentimes better than a master of one.” That’s closer to how I think about it now.

But even that framing is wrong. I’m not a generalist because I failed to specialize. I’m a systems thinker. My specialty is understanding how pieces fit together.

The T-shape model helped me make peace with my career: broad knowledge across many areas, with depth in a few. For me, those depth areas are distributed systems, AI agents, and web design. Not because I chose them strategically, but because I kept returning to them as I solved real problems.

Depth comes naturally when you’re solving real problems. You don’t have to force it. The generalist’s breadth gives you context; the depth emerges from repeated application.

If you’re a generalist feeling like a fraud, consider this: someone needs to build the bridges between specialties. Someone needs to see the whole system. Someone needs to translate between the experts who can’t understand each other.

That’s not a consolation prize. That’s a skill. And in a world that’s increasingly complex and interconnected, it might be the most valuable skill of all.

What’s your experience with the generalist/specialist tradeoff? I’d love to hear from others who’ve grappled with this. architgupta941@gmail.com or find me on X.

Because after they pocket the ball, they’re lined up perfectly for the next one. After I pocket it, I’m praying.

This realization changed how I think about pool—and honestly, how I think about a lot of things (including shipping agent systems where the next state matters more than a flashy demo).

The Shot-Making Trap

When you start playing pool, the feedback loop is obvious: pocket balls, feel good. Miss balls, feel bad. So you practice pocketing balls. Cuts, banks, combinations. The satisfying thunk of a ball dropping into a pocket.

I spent my first two years doing exactly this. I got pretty good at making difficult shots. I could impress people at the bar with a long cut or a tricky bank.

But I couldn’t run more than three balls consistently.

The problem wasn’t my stroke. The problem was where I left the cue ball. I’d make a beautiful shot and end up with no angle on the next one, or worse, completely out of position. Then I’d have to make another difficult shot, and another, until eventually I missed.

The bottleneck was never shot-making. It was position play.

What Position Play Actually Is

Position play isn’t just “get close to the next ball.” That’s the beginner’s understanding.

Real position play is controlling the cue ball to land in a zone that gives you an angle. And angle is everything. Angle means control—you can use spin, speed, and natural roll to get to your next position.

Straight-in shots are actually harder to play position from, not easier. When the cue ball, object ball, and pocket are in a line, your only options are follow and draw. You’ve eliminated most of your position routes.

The best players know where they want the cue ball before they even look at the object ball. They’re not thinking “can I make this?”—they’re thinking “where do I need to be, and what shot gives me that?”

The shot itself is almost an afterthought.

The Math of Running Out

Here’s some simple math that changed my perspective:

Let’s say you’re 90% on each individual shot. Sounds pretty good, right? But if you need 8 shots to run out a rack, your probability of success is:

0.9^8 = 43%

You’ll lose the majority of games even with 90% shot-making.

Now let’s say good position play makes each shot 95% instead of 90%—because you’re never awkward, never stretched out, never dealing with tough angles.

0.95^8 = 66%

That’s the difference between losing and winning the race. A 5% improvement on each shot compounds into 23 percentage points on winning the rack.

Position play isn’t just important. Mathematically, it’s the whole game.

How I Practice Position

Here are the drills that actually improved my game:

The Line Drill

Place all the solids (or stripes) in a straight line down the center of the table. You must pocket them in order: 1, 2, 3, and so on.

This forces you to plan three balls ahead. You can’t just make the 1—you have to leave yourself an angle on the 2 that lets you get to the 3. It’s brutal at first, but it trains your brain to think in sequences.

The Box Drill

Pick a shot. Before you shoot, draw a box on the table with chalk where you want the cue ball to land. Now execute. Did you land in the box?

This builds precision. Most players have a vague sense of “I want to be over there.” The box drill forces specificity. You start to understand exactly how much speed and spin produce exactly what result.

The Ghost Player

This one transformed my practice sessions. Play a full rack against an imaginary perfect player—the ghost. Take ball in hand after the break. If you run out, you win the rack. If you miss or lose position, the ghost runs out and wins.

Keep count of how close you got. Did you pocket 5 balls before losing? 7? Track it over time.

You can also start easier: 3 balls spread on the table. Run them out. Then 4. Then 5. Gradually work up to a full rack. The ghost drill builds both position play and pressure tolerance—because every shot matters.

Reading the Table

Before I shoot anything now, I ask myself: “What’s my route to the 8?”

Not just the next ball—the whole sequence. I identify problem balls early: clusters that need to be broken, balls near rails that will be trouble, potential snookers. Then I work backward from the 8-ball to figure out what sequence gives me the best chance.

Sometimes the right shot isn’t the obvious one. The easy pocket might leave me no angle on the next ball. The harder shot might set up a breakout I need. Position play means thinking about consequences, not just immediate results.

The Mental Shift

The transformation in my game came when I stopped asking “can I make this?” and started asking “where do I need to be next?”

The shot you’re on is already in the past. Your job is to set up the future.

This is why professionals make it look easy. They’re playing a different game than beginners. While we’re celebrating that we made a tough shot, they’re upset that they ended up 6 inches from perfect position. Their standards are different because they’re optimizing for different things.

Watch any pro match and pay attention to the cue ball, not the object ball. That’s where the real skill is.

Shot-making impresses spectators. Position play wins matches. I’d rather look boring and cash the check.

The same principle applies beyond pool. In engineering, in career decisions, in life—the flashy move isn’t always the right move. Sometimes the right move is the one that sets up your next three moves.

If you’re in Boulder, Colorado and want to shoot some pool, hit me up. Always looking for practice partners who take the game seriously. architgupta941@gmail.com or find me on X. I also stream matches on YouTube if you want to watch how these drills hold up under pressure.

Our database? PostgreSQL. Our cache? Redis. Our message queue? RabbitMQ.

This isn’t a contradiction. It’s a strategy.

What “boring technology” means here

Boring technology is infrastructure that’s old enough that its failure modes are documented, its operators are available, and its upgrades are boring—so you can spend novelty on the product instead of the plumbing.

At Robynn, that meant LangGraph, modern model SDKs, and agent tooling on the edge; PostgreSQL, Redis, RabbitMQ (and friends) underneath. The rule is a complexity budget: every team can absorb only so much newness before debugging becomes folklore. Spend that budget where you compete; refuse it where the industry already solved the problem.

It is not “never use new tools” or “always pick the oldest library.” It’s refusing to invent a database when you’re trying to invent an agent. Use boring cores when data, queues, and caches must just work; use sharp tools when the product is the new thing (production agents, multi-agent systems).

The Complexity Budget

Every engineering team has a limited complexity budget. You can only absorb so much novelty before your system becomes unmaintainable, your debugging becomes guesswork, and your on-call rotations become nightmares.

The question isn’t “should we use new technology?” It’s “where should we spend our complexity budget?”

At Robynn, we spent it on AI: LangGraph to orchestrate multi-step agent workflows, Claude’s SDK with structured outputs for reliable tool calls, and experiments in prompting, memory architectures, and agent failure handling.

That’s where competitive advantage lived. That’s where we should have been on the cutting edge.

But storing data? Caching responses? Queuing background jobs? These are solved problems. We don’t need innovation here—we need reliability.

The Temptation of the New (A Confession)

I learned this the hard way. Early in my career at Riverbed, I pushed for MongoDB when PostgreSQL would’ve been perfectly fine. The allure was irresistible: “web scale,” document flexibility, the thrill of being on the cutting edge.

What actually happened? Some time later, we needed transactions. We needed relational queries. We needed the things that boring databases had solved decades ago.

The migration cost more than the original implementation.

The irony? If I’d saved that complexity budget, we could have used it on features that actually mattered to users.

What “Boring” Actually Means

When I say “boring technology,” I don’t mean outdated. I mean battle-tested.

PostgreSQL, first released in 1989, has 35+ years of edge cases discovered and fixed. When you Google an error at 2 AM, there’s a Stack Overflow answer from 2014 that still works. The documentation is comprehensive. The failure modes are well-understood.

Boring tech hasn’t failed—it’s graduated. It survived the hype cycle and emerged as infrastructure you can trust.

This is especially important when your application layer is doing something novel. When your AI agent does something unexpected (and they will), you need to debug the agent—not wonder if the issue is in your experimental database.

Boring infrastructure gives you stable ground to stand on while you experiment above it.

The Hidden Costs of Shiny Infrastructure

New infrastructure technology comes with costs that don’t show up in the README:

Documentation gaps. The docs assume you already understand concepts they haven’t explained. You’ll spend hours piecing together blog posts, GitHub issues, and Discord messages.

Hiring difficulty. Finding engineers who know CockroachDB is harder than finding engineers who know PostgreSQL. The ones who do know it? They’re expensive.

Operational unknowns. What happens at 3 AM when it breaks? With PostgreSQL, I can find a DBA who’s seen my exact error a hundred times. With the new hotness, I’m filing GitHub issues and hoping.

Migration tax. Technologies get abandoned. Companies pivot. When the shiny thing gets deprecated, you’re stuck with a migration you didn’t budget for.

When you’re already dealing with the operational unknowns of AI agents—which are substantial—you don’t want to add infrastructure unknowns on top.

Where We Do Use Cutting-Edge Tech

Let me be clear: we weren’t Luddites at Robynn. Here’s the stack we actually ran:

Cutting-edge (where it matters):

• LangGraph for agent orchestration—because multi-step AI workflows are genuinely new territory
• Claude SDK with structured outputs—because reliable tool calling requires the latest capabilities
• OpenClaw for tool integration—because the AI tool ecosystem is evolving rapidly
• Vector databases for semantic search—because embedding-based retrieval is a new requirement
• Custom prompt engineering patterns—because we’re discovering best practices in real-time

Boring (where it doesn’t):

• PostgreSQL for relational data—because ACID transactions are a solved problem
• Redis for caching and simple queues—because it’s fast and predictable
• S3 for blob storage—because it just works
• Linux, Nginx, Python for compute—because debugging is easy

The key insight: we’re cutting-edge in our product (AI agents) and boring in our infrastructure (data storage, caching, deployment).

My Heuristic: “Where Does Innovation Create Value?”

Before adopting any new technology, I ask two questions:

1. Does this directly improve what customers pay us for? If we use a better orchestration framework, our agents work better. That’s value. If we use a fancier database, our data is stored… the same. That’s not value.

2. Would I bet my 3 AM sleep on this? New AI tools? The team expects some rough edges—it’s the nature of the space. New infrastructure? Nobody wants to debug a mysterious database failure while also debugging agent behavior.

For AI/ML tooling, the answer is often “yes, the innovation is worth it.” For infrastructure, the answer is almost always “no, use the boring thing.”

The Boring Stack That Enables Innovation

Here’s the pattern I recommend:

Infrastructure layer (boring):

• PostgreSQL or MySQL for relational data
• Redis for caching
• S3 for files
• Standard cloud compute (EC2, Cloud Run, etc.)

Application layer (as innovative as you need):

• Whatever frameworks make sense for your problem
• Cutting-edge AI/ML libraries if that’s your domain
• New tools that directly improve your product

This layering matters. When something breaks, you want to know immediately whether it’s an infrastructure issue (unlikely with boring tech) or an application issue (likely with cutting-edge stuff). Clear separation makes debugging tractable.

The Meta-Lesson

The engineers I respect most aren’t the ones chasing every new framework. They’re the ones who understand where to innovate.

They use boring databases and exciting product architectures. They use reliable infrastructure and experimental features. They save their complexity budget for problems that actually differentiate their product.

At Robynn, we pushed on what AI agents can do—new orchestration patterns, memory systems, and ways of handling uncertainty.

We can do that because we’re not also experimenting with our database. Our boring infrastructure gives us the stability to be bold where it counts.

Innovation should happen in your product, not your infrastructure. Save your complexity budget for problems your customers actually pay you to solve.

The best engineers aren’t the ones using the newest everything. They’re the ones who know which battles to pick.

If you’re building AI agents and thinking through these tradeoffs, I’d love to chat. Reach out at architgupta941@gmail.com or find me on X.

Since 2025, I’ve built agents for everything from production workflows at Robynn AI to helping local businesses automate operational tasks. Here’s what I’ve learned. For a deeper cut on multi-agent architecture, see Why Four AI Agents Beat One Smart One and the Ollama priority queue.

What a production agent is

A production AI agent is a loop that calls tools, keeps state, and finishes real work under failure—not a chat demo that looks smart once.

In practice: pick the simplest architecture that fits (tool-calling loop → graph/workflow → multi-agent), constrain outputs so you can trust them, log and measure what broke, and only spend complexity where the product differentiates (boring infrastructure, sharp AI).

It is not “give the model every tool and a vague goal.” That ships demos. Production agents have narrow jobs, explicit success criteria, and a plan for timeouts, bad tool results, and human override.

Stack I keep reaching for: Python, structured outputs, LangGraph when the control flow actually branches, boring datastores underneath.

The Agent Architecture Spectrum

There’s a spectrum of agent architectures, and choosing the right one depends entirely on your use case:

Simple Tool-Calling Agents

For straightforward tasks with clear inputs and outputs, you don’t need complex orchestration. A simple loop works:

def simple_agent(task: str, tools: list[Tool]) -> str:
    messages = [{"role": "user", "content": task}]

    while True:
        response = llm.generate(messages, tools=tools)

        if response.is_complete:
            return response.content

        # Execute tool calls
        for tool_call in response.tool_calls:
            result = execute_tool(tool_call)
            messages.append({"role": "tool", "content": result})

This pattern handles 80% of use cases. Don’t over-engineer it. (This is part of my philosophy on boring technology—save complexity for where it matters.)

State Machine Agents

When you need deterministic control flow with AI-powered decisions at each step, state machines shine:

from langgraph.graph import StateGraph

workflow = StateGraph(AgentState)

# Define nodes
workflow.add_node("analyze", analyze_task)
workflow.add_node("plan", create_plan)
workflow.add_node("execute", execute_plan)
workflow.add_node("validate", validate_results)

# Define edges
workflow.add_edge("analyze", "plan")
workflow.add_conditional_edges(
    "plan",
    should_execute,
    {"execute": "execute", "revise": "plan"}
)

Multi-Agent Systems

For complex domains, multiple specialized agents often outperform one generalist:

• Coordinator: Routes tasks to specialists
• Researcher: Gathers and synthesizes information
• Executor: Takes actions
• Validator: Checks results

The Patterns That Actually Work

1. Structured Outputs Are Non-Negotiable

Free-form text output from LLMs is unreliable. Always constrain the output:

from pydantic import BaseModel

class ActionPlan(BaseModel):
    reasoning: str
    actions: list[Action]
    confidence: float

response = llm.generate(
    prompt,
    response_format=ActionPlan
)

In my experience, this alone eliminates half the production bugs I’d otherwise be chasing.

2. Retry with Exponential Backoff

LLM APIs fail. Rate limits hit. Network hiccups happen. Build resilience in:

@retry(
    stop=stop_after_attempt(3),
    wait=wait_exponential(multiplier=1, min=4, max=10),
    retry=retry_if_exception_type(RateLimitError)
)
def call_llm(prompt: str) -> str:
    return llm.generate(prompt)

3. Observability from Day One

You can’t debug what you can’t see. Log everything:

• Input prompts (anonymized)
• Tool calls and results
• Token usage
• Latency breakdowns
• Error rates by category

I use a simple pattern:

@trace("agent_step")
def process_step(state: AgentState) -> AgentState:
    with span("llm_call"):
        response = llm.generate(state.messages)

    log_metrics({
        "tokens_used": response.usage.total_tokens,
        "latency_ms": response.latency,
        "tool_calls": len(response.tool_calls)
    })

    return state.update(response)

4. Human-in-the-Loop Escape Hatches

No agent should run completely autonomously in production. Always provide:

• Confidence thresholds that trigger human review
• Easy ways to pause and inspect state
• Clear audit trails

if action.confidence < 0.8:
    await notify_human(
        action=action,
        context=state,
        timeout=timedelta(hours=1)
    )

Common Pitfalls

The “Just Add More Context” Trap

When an agent fails, the temptation is to add more instructions to the prompt. This usually makes things worse. Instead:

1. Analyze why it failed
2. Constrain the output format
3. Add specific examples (few-shot)
4. Consider splitting into sub-tasks

Ignoring Edge Cases

Agents in production see inputs you never imagined. That “simple” task of parsing customer emails? Wait until you see:

• Emails in multiple languages
• Forwarded threads with nested quotes
• Attachments described but not attached
• Sarcasm and ambiguity

Build defensive parsing and graceful degradation.

Over-Relying on the LLM

Not everything needs AI. A well-crafted regex or a simple rule often beats an LLM call for:

• Data validation
• Format checking
• Simple transformations
• Deterministic routing

Save the LLM for actual reasoning tasks. Use boring, proven tools for everything else.

What I’m Excited About

The field is moving fast. A few things I’m watching:

Tool Libraries: Standardized tool definitions (like MCP) make agents more portable and composable.

Smaller, Faster Models: Haiku-class small models make agent loops economically viable for more use cases.

Memory Systems: Long-term memory and retrieval are becoming table stakes for production agents.

Evaluation Frameworks: We desperately need better ways to test agent behavior at scale.

Wrapping Up

Building production AI agents is part software engineering, part prompt engineering, and part systems design. It’s one of those domains where being a generalist actually helps—you need to understand the full stack. The agents that work best are:

• Simple where possible
• Observable always
• Constrained in their outputs
• Resilient to failures
• Honest about their limitations

Start simple, measure everything, and iterate based on real-world feedback. Showing up consistently and making incremental improvements is the path to agents that actually work.

Have questions about building AI agents? Drop me a line at architgupta941@gmail.com or find me on X.