Back to Thoughts

Your Agent Architecture Has an Expiration Date

Jan 30, 20268 min readView raw

The architecture that makes your AI agent reliable today can become the thing that holds it back six months from now.

That is not because the original design was wrong. It is because model capability keeps moving. Tool calling improves. Reasoning gets cheaper. Context windows grow. Structured output gets less fragile. Every improvement changes which pieces of scaffolding are still useful.

This is the uncomfortable sequel to "the hard part of agents is design": good design in AI has a shelf life.

Link to this sectionThe Bitter Lesson applies to agent architecture

In 2019, Rich Sutton published "The Bitter Lesson." The argument was simple: across AI history, general methods that use computation tend to beat hand-crafted structure.

Chess is the familiar example. Researchers encoded human chess knowledge for decades: openings, positional rules, endgame patterns. Then systems that leaned harder into search and self-play changed what mattered.

The same pattern shows up in speech recognition, computer vision, and natural language processing. Clever human structure helps for a while. Then scale, learning, and compute push past it.

For agent builders, the bitter part is practical. The workaround you add today because a model cannot reliably do something may become unnecessary when the next model can. If that workaround is buried deep in your architecture, progress turns into migration work.

Link to this sectionModel improvements bulldoze hidden assumptions

PostHog spent a year building production agents. One of their biggest lessons was that model improvements change more than you expect.

Reasoning models went from experimental to central. Tool calling became more reliable. Models became better at following complex instructions over long runs.

Those changes did not just improve outputs. They changed architecture.

The same thing happened in research-agent systems. Architectures that were reasonable when models were weaker became bottlenecks when models could plan, call tools, and recover better on their own.

The lesson is not "never add structure." Today's models still need structure.

The lesson is to name the assumption behind every piece of structure. If a graph node exists because a model cannot choose the next step reliably, write that down. If a parser exists because structured output fails too often, write that down. If a retry loop exists because a provider drifts, write that down.

When models improve, those notes tell you what to remove.

Link to this sectionAgents can beat workflows when models can self-correct

A year ago, graph-style workflows made sense for many agent tasks. A simple tool-calling loop often got confused. Frameworks added reliability by forcing the model through predefined steps.

Now that trade-off is less obvious.

For some tasks, a single model loop with simple tools can work better than a complex graph. The model keeps context, sees its own prior actions, and can self-correct without state being split across nodes.

That does not make workflows obsolete. Workflows are still the right choice when the path is known, compliance matters, or every step must be deterministic.

But it does mean a workflow should justify itself. If the graph exists only because older models needed hand-holding, it may be time to retest the assumption.

Link to this sectionSubagents create context loss before they create value

It is tempting to decompose agent systems into roles.

The research agent gathers sources. The analyst agent finds insights. The writer agent drafts. The critic agent reviews. The orchestrator agent coordinates everyone.

Sometimes that works, especially when tasks are independent and can run in parallel.

Sequential work is different. If each agent needs the previous agent's full context, splitting the task can make the system worse. Every handoff compresses context. Every role boundary creates a place for nuance to disappear.

The result is a system that looks modular but behaves forgetfully.

For sequential tasks, start with one loop. Add subagents only when the boundary is real: independent work, different permissions, different tools, or a clear need for parallelism.

Link to this sectionA to-do tool is boring and useful

If one loop is doing multi-step work, it needs a way to stay oriented.

One useful pattern is a simple to-do tool. The agent writes down the next steps, updates them as work progresses, and uses the list as lightweight working memory.

This does not need to be sophisticated. The value is that the model externalizes intent. It can inspect the plan, notice drift, and adjust without relying entirely on hidden context.

For long-running tasks, this is often more useful than another layer of orchestration.

Link to this sectionContext engineering is the main cost lever

Most agent prototypes stuff full tool responses back into the next prompt.

That works for short tasks. It breaks down when the agent researches, compares, retries, or runs for many iterations.

The context grows with every step:

  • iteration 1: query, prompt, and tool response
  • iteration 2: all of iteration 1 plus another response
  • iteration 3: all previous responses plus another response
  • iteration 10: a prompt full of stale detail the model may not need

The cost is not only tokens. The model also loses focus.

A better pattern is context distillation. Keep the insight, not always the raw output. Store source references separately. Bring raw data back only when the next step actually needs it, such as citation, verification, or detailed extraction.

The useful question is: does the next iteration need the evidence, or does it need the conclusion from the evidence?

Answer that aggressively. A clean context window is an architecture feature.

Link to this sectionStructured output gets better, but it still needs ownership

Structured output is necessary whenever downstream code consumes model output.

JSON, typed schemas, and Pydantic models make integration possible. They also create a reliability problem: the model might produce malformed output, incomplete objects, or values that satisfy the syntax but violate the business rule.

As models improve, some of this gets better. Strict schema following becomes more reliable. Long nested objects fail less often. Provider tooling improves.

That improvement can make old defensive code look silly: fallback parsers, repair prompts, extra wrappers, and retry logic that used to be necessary but now mostly adds latency.

Do not remove those safeguards blindly. Measure the failure rate first.

But do revisit them. In AI systems, defensive structure has a half-life.

Link to this sectionFrameworks add convenience and lock-in

Frameworks are useful when they save you from undifferentiated work.

They are dangerous when they force your architecture to match their mental model after the underlying models have changed.

Agent frameworks often wrap provider APIs, normalize tool calling, provide orchestration primitives, and manage state. That helps early. Later, it can make new provider features harder to use or old assumptions harder to remove.

The practical approach is not "never use frameworks." It is:

  • keep provider boundaries explicit
  • avoid burying business logic inside framework-specific nodes
  • own your prompts, tools, schemas, and traces
  • document why each orchestration layer exists
  • make removal possible

AI may eventually settle around stable framework patterns. Until then, prefer primitives you can replace.

Link to this sectionEvals matter, but traces explain reality

Evals are necessary. They are not sufficient.

For agentic systems, the hard cases come from real user behavior: vague goals, partial context, unexpected tool outputs, contradictory sources, and tasks that are easy to describe but hard to judge.

That is why trace review matters. Production traces show what the agent actually saw, what it decided, which tools it called, where it recovered, and where it drifted.

Evals should come from that evidence. A trace reveals a failure pattern. You turn the pattern into a repeatable eval. Then you test whether a change improves the behavior without breaking something else.

If you optimize only for a score, you can miss the product. If you review only anecdotes, you can fool yourself. You need both.

Link to this sectionShow users the work

Agent UX has a trust problem.

When a system takes several steps, calls tools, retries, and synthesizes an answer, hiding the process makes the result harder to believe. Users do not need every internal token, but they do need enough visibility to understand what happened.

Show tool calls. Show progress. Show important assumptions. Show when the system is waiting for approval. Show when it failed and what it tried.

This is not decoration. It is the interface for trust.

Link to this sectionFive rules for agent systems that will change

Link to this section1. Name the model assumptions

Write down which parts of the architecture exist because current models are weak at something. Those are the first places to revisit when capabilities improve.

Link to this section2. Prefer one loop until decomposition earns its keep

Use subagents and graphs when they create real boundaries: parallelism, permissions, deterministic sequencing, or isolated responsibilities. Do not add them because the diagram feels professional.

Link to this section3. Treat context as a product surface

Distill aggressively. Deduplicate sources. Carry forward what the next step needs, not everything the previous step saw.

Link to this section4. Keep frameworks replaceable

Use framework conveniences where they help, but avoid handing over the shape of your product. Own the contracts that matter.

Link to this section5. Let traces drive evals

Review real runs. Turn repeated failures into eval cases. Use metrics to validate fixes, not to avoid looking at behavior.

Link to this sectionBuild like the scaffolding will expire

You still have to build for the models you have today.

Current models need guidance. They need schemas, tools, retries, context shaping, human approval, and product-specific constraints.

The mistake is pretending those constraints are timeless.

Build the scaffolding. Label it. Measure it. Remove it when it stops helping.

The best agent architecture is not the one that survives unchanged. It is the one that can evolve without requiring a rewrite every time the models get better.

Keep reading

All thoughts
Gopal Khadka