Why My Agent's Context Window Fills Up Faster Than I Expect

This post's canonical version is I Am The Context Window on danstolts.com.
Direct answer

The practitioner version: why an agent session runs out of working memory long before you expect it to, what I actually watch to see it coming, and the habits that keep a long session useful instead of degrading into guesswork.

What actually eats my context window faster than I expect?

The line I underestimate every time, even after doing this for a while: it's never the conversation itself, it's the file reads. A single "let me check that file" during a debugging session can pull in hundreds of lines I only needed three of. Multiply that by a dozen files across a long session and the working memory is gone long before the actual conversation feels long. The fix I use now is deliberate: read the smallest slice of a file that answers the current question -- a specific line range or a grep result -- before reaching for the whole file, and only widen the read if the narrow one doesn't answer it.

How do I tell my session is getting close to full before it degrades on me?

I don't wait for an explicit warning, because by the time one fires I've already lost some quality. What I actually watch for: the agent starts re-asking something I told it three messages ago, or it re-derives a fact it already looked up instead of referencing the earlier answer. That's the practical tell that older material is being deprioritized even if it hasn't been formally dropped yet. The moment I see that pattern once, I checkpoint state to a file and either continue in a fresh session or explicitly re-state the constraint that just got lost.

Before continuing: write the current state (what's done, what's pending,
the one constraint that must survive) to a checkpoint file. Then continue.

I run that instruction the moment I notice re-derivation, not after the session has visibly gone downhill.

What's the actual habit that keeps a long session usable?

Load the index first, not the library. When I start a task that touches a big codebase or a big knowledge base, my first read is a routing file or a summary -- never the full set of documents "just in case I need them." I learned this the hard way on a session where I had the agent read every rule file in a repository up front "to be thorough," and by the time real work started there was no room left for the actual task files. The fix wasn't "use a bigger model" -- it was reordering what got read first. A one-line routing index that says "read file X only if you're doing Y" saves more working memory than any amount of raw window size.

What do I do when a task genuinely needs more working memory than fits?

I stop trying to cram it into one session. The real fix is architectural, not a bigger prompt: split the task into subtasks, each with its own fresh context, and have each one report back a short result instead of carrying its full working set forward. Concretely, if I'm auditing twenty files for the same pattern, I don't read all twenty into one session -- I dispatch the audit in batches, each batch starting clean, and only the findings (not the raw file contents) come back to the session that's tracking the overall task. That's the difference between a session that stays sharp for hours and one that's unusable by file twelve.

What do I do differently after a session has been compacted?

I treat everything before the compaction event as a lossy paraphrase, not a verified fact. The specific number, the exact file path, the precise wording of a constraint -- those are exactly the details a summary is most likely to have smoothed over. So after a compaction, before I rely on anything from before it, I re-verify: re-grep the file path, re-read the number, re-state the constraint back to the agent and get it confirmed. It costs a few extra tool calls. It's cheaper than shipping a change built on a detail the summary quietly got wrong.


*Read the underlying mechanism -- why context is a fixed-size working memory that nothing protects by default -- in the companion piece on danstolts.com: [What Is a Context Window and Why Does It Limit What an AI Can Remember?](https://danstolts.com/writing/i-am-the-context-window/)*

Dan Stolts

Founder and Chief AI Officer of Just In Time AI, with an IT career that began in 1988 -- nearly four decades of experience. Over a decade of that career went into building ITProGuru into a 250,000-uniques/mo IT-community resource, and he now applies the same teach-first approach to AI systems that run real businesses. Full profile →