You have a long-running agent that accumulates conversation and tool output until it overruns its context window mid-task. Walk me through the techniques you'd use to keep it coherent over a long task without just truncating history.

technical-conceptual · Senior level · data-ml

What the interviewer is really asking

Probes practical context-management techniques for long-horizon agents — compaction, external memory, and tool-result pruning — versus naive truncation that drops critical decisions.

What to say

• Use compaction: when the conversation nears the limit, summarize it and reinitialize a fresh window with that summary, preserving key decisions and state rather than dropping the oldest messages blindly.
• Push durable state to external memory: have the agent write structured notes (decisions, open tasks, dependencies) to storage outside the window and retrieve them when needed — and keep lightweight references (IDs, file paths) it can re-load just-in-time instead of holding everything inline.
• Prune tool-result pollution: once a tool result deep in the history has been used, you usually don't need its raw payload again — clear or collapse it so the window stays high-signal.

What to avoid

• Naive truncation of the oldest turns — that silently drops the architectural decisions and constraints the agent still needs, so it contradicts itself later.
• Keeping every raw tool result in the transcript forever, letting redundant payloads pollute the window and inflate cost.
• Treating it purely as 'buy a bigger window' rather than actively managing what's in the one you have.

Example answers

Strong: I avoid blind truncation — it drops the decisions the agent still depends on. First, compaction: as we near the limit, summarize the conversation and reinitialize a fresh window seeded with that summary so coherence and key state survive. Second, external memory: the agent writes structured notes — decisions, open tasks, dependencies — to storage outside the window and keeps references like IDs or paths to re-load just-in-time, instead of carrying everything inline. Third, prune tool-result pollution: once a tool's raw output deep in history has been consumed, I collapse it, since the agent rarely needs the raw payload again. Net effect is the smallest high-signal window that still remembers what matters.

Weak: When it gets close to the limit I'd drop the oldest messages to make room for new ones — usually the most recent context is the most relevant anyway. If that loses too much, I'd move it to a model with a bigger context window so we can keep the whole history in the prompt and not have to manage it.