I Thought Context Was the Product. I Was Wrong.

When I started building Context System Design, I thought the primary engineering challenge was creating richer context for AI systems.

I don’t think that’s true anymore.

After spending about a week experimenting with multiple coding agents and language models, I came away with a different understanding of the problem.

The real bottleneck isn’t simply generating context.

It’s understanding what different models actually do with it when given the chance.

That distinction turned out to matter much more than I expected.

It Started with a Framework

The original premise was straightforward: If AI systems make better decisions with better context, then we need a way to design that context intentionally rather than treating prompt engineering as the complete solution.

That led me to build Context System Design v0.1 — a framework for thinking about what information an AI system should receive, how that information should be organized, and how it should evolve as the system and its environment change.

But I didn’t want to simply design context for an AI system myself. I wanted to see whether the system could help determine what context it needed.

So I tried a different approach.

I gave coding agents the same framework and asked them to generate the context they believed they would need to execute a long-running software project.

In other words: I asked the AI to help build the system that would guide the AI.

That immediately introduced a chicken-and-egg problem.

How do you generate good context without already possessing good context?

The framework was my attempt to break that loop. It became the initial context from which the agents could determine what additional context they needed.

Same Framework. Different Results.

I ran the experiment across multiple models. Each received the same high-level framework and instructions.

The results were not the same.

The more capable models didn’t just produce passive documentation — they built scaffolding.

• They created task records.
• They maintained handoff files.
• They updated codebase conventions.
• They created relationships between documents and maintained project state.

As the implementation progressed, they continued updating the artifacts that helped them understand what they were doing. They behaved as though maintaining the system’s working context was part of the work itself.

The less capable models also generated context, but they filled far fewer gaps. They were passive about maintaining documentation, less likely to infer missing architectural pieces, and required explicit direction to remain aligned with the intended state of the project.

The framework was identical. The operational capabilities of the models were not.

That was the first major surprise.

The Shift in the Experiment

Originally, I thought I was testing a relatively simple proposition: Does richer context produce better AI performance?

The answer is fairly intuitive: more relevant information should generally help, particularly as the problem becomes more complex. But the experiment exposed another variable that I hadn’t given enough weight to:

Can the model actually use the context effectively?

Providing a model with exhaustive documentation doesn’t guarantee that it will consult it. It doesn’t guarantee that it will recognize which parts are relevant, connect information across documents, or maintain that context as the environment changes.

You can lead a horse to water, but you can’t make it drink. Context can exist without becoming useful context.

Context Has Two Halves

I now think about context architecture as having two distinct halves.

1. The System Design (Defined by the Architect)

This is the part the system architect defines, including:

• Goals and architectural constraints
• Project conventions and state tracking
• Relevant documentation and relationships between artifacts
• Definitions of success

The purpose is to create an environment in which the model has access to the information it needs.

2. The Execution Engine (Performed by the Model)

This is the part the model has to perform, including:

• Discovering relevant information and connecting disparate pieces
• Recognizing missing information and inferring what is needed
• Maintaining context as the environment changes
• Deciding which information matters at a given moment

The first half can be designed. The second half depends heavily on the capabilities of the model — and that difference turned out to be much larger than I expected.

Context to Context

There is an interesting paradox here.

I originally thought of context as something that humans create for AI. But the experiment suggested something far more dynamic: A capable agent can create context for itself.

It can recognize that a project needs a task record, that a convention needs to be documented, that a handoff needs to preserve state, or that a previous decision needs to remain accessible. It can create those artifacts and then use them later.

The agent isn’t simply consuming context. It can participate in the creation and maintenance of the context system itself.

This is where model capability becomes critical:

• Stronger models were better at filling gaps, recognizing missing pieces, maintaining state over longer horizons, and creating necessary scaffolding.
• Weaker models could still use the scaffolding once it existed, but were far less likely to build it themselves.

The Strongest Models Don’t Just Use the Environment

One of the most interesting observations was how differently models behaved once the project became large enough to require real coordination.

The strongest models weren’t simply executing instructions. They appeared to treat the surrounding project structure as part of the problem. As the codebase evolved, they updated task tracking, kept documentation synchronized, maintained conventions, and left behind artifacts that made future work easier.

They weren’t just completing tasks — they were maintaining the system that allowed them to keep completing tasks.

That represents a fundamentally different paradigm from simple code generation:

• Basic Paradigm: Read context → write code
• Agentic Paradigm: Read context → identify gaps → create context → use context → update context → continue

That is a much more interesting system.

But More Context Isn’t Automatically Better

This is where I had to revise another assumption. It is tempting to conclude that the solution is simply to give the model more context.

That isn’t quite right. Context has a cost.

• Too little context: The model lacks crucial information.
• Too much context: Relevant information becomes harder to identify, more expensive to process, or simply ignored.

So the core question isn’t: How much context can we give the model?

It’s closer to: How much relevant context can the model effectively use, and how reliably can it maintain that context over time?

That is a much harder question, and it is a moving target. As models improve, their ability to reason across larger environments changes. The amount of context that is useful today may not be the amount that is useful tomorrow.

The Model Became Part of the Context System

This was probably the most important thing I learned.

I originally treated the model as the thing consuming the context system. Now I realize the model is actually part of the context system.

[ System Architecture ] ---> Determines what information exists & how it's organized
          +
[ Execution Engine ]     ---> Determines how information is discovered, connected & updated
          =
[ Useful Context ]

Context design cannot be evaluated independently of the model using it. The same context architecture can produce wildly different outcomes depending on the execution engine. That makes context engineering less like designing a static information repository and more like designing an environment for a probabilistic system.

What I Thought I Was Building

When I started this project, I thought I was building a better way to provide context to AI. Then I thought I was building a knowledge system. Then I thought I was testing whether richer context produced better answers.

All of those things were partially true, but none captured what I ultimately found most interesting.

The deeper question became:

How do we design an environment in which an AI system can discover, create, and maintain the context it needs to operate effectively over time?

That changes the role of the framework. Context System Design isn’t a recipe for stuffing more information into a prompt. It’s a way of thinking about the environment surrounding an AI system:

• What information exists and where does it come from?
• How trustworthy is it, and how does it change?
• What relationships does it have to other information?
• How does the system know when it needs something?
• And, perhaps most importantly: Can the model actually use it?

What I Learned

I started this experiment believing that the primary challenge was designing better context. I still believe that matters, but context isn’t the whole problem.

There are really two capabilities involved:

1. Environment Design: Designing an environment that contains the right information, structure, boundaries, and state (Human/Architect role).
2. Execution Engine: Relying on an engine capable of using that environment effectively (Model role).

This leads to a more nuanced version of my original hypothesis:

Better context can enable better AI performance, but only to the extent that the model can effectively discover, interpret, and maintain that context.

As models become more capable, they will increasingly build and maintain their own operating context. The future of context design may not be about humans manually deciding what information an AI should receive — it may be about designing systems that allow AI to continuously determine what context it needs.

We started with: How do we give AI better context?

I’m now more interested in: How do we design systems that enable different models to dynamically maintain and utilize context over long horizons?

I don’t have a complete answer yet. But after watching these systems work, I think that’s the more interesting problem.

And perhaps that’s the real lesson of building AI products: Sometimes the most valuable thing you can build isn’t the product you started with.

It’s a better question.