The context window is a resource you engineer, not a buffer you fill

A context window rarely fails at its advertised limit. It fails earlier and more quietly: the model rereads files it already opened, forgets a constraint from the brief, and starts producing strangely empty edits while the provider still accepts another hundred thousand tokens.

That happened in an efferent whole-repo port. The coder’s input grew from 73k to 110k tokens across gate retries; the model’s window was 256k, but the work had already degraded. The useful threshold was not “request rejected.” It was “the trajectory stopped being healthy.” Efferent now folds at 80k.

The number is the least interesting part. The implementation has to answer four harder questions: where a fold is safe, which recent work stays verbatim, how a checkpoint lines up with durable rows, and which instructions must be restored from source rather than entrusted to a summary.

The live cost is already in the turn#

You do not need a second tokenizer running beside the agent. The last model response reports the input tokens it actually consumed; that is the live cost of the next-turn buffer, including the system prompt, tools, summary, and active history. Smith records that value from each settled assistant event:

const contextRef = yield* Ref.make(0)

const onEvent = (event: LoopEvent) =>
  event.type === "assistant_message"
    ? Ref.set(contextRef, event.usage.inputTokens) 
    : Effect.void

This avoids an attractive lie: summing every turn’s usage measures cumulative spend, not current context. The fold decision wants the latest input, because that is the buffer the model is trying to reason over now.

The TUI reads the same field for its ctx 80k/256k gauge. Measurement and policy share one source; a dashboard estimate cannot drift away from the threshold that actually folds the run.

There are two natural fold points#

Smith uses the same 80k policy at two boundaries.

The first is between forge attempts. A rejected attempt has just produced a dense, deterministic gate brief. Before the next attempt, the old trail can become a handoff summary and the fresh findings can become the first new instruction. Rejection is a natural chapter break.

const grown = yield* Ref.get(contextRef)
const willFold = Option.isSome(input.feedback) && grown > CHECKPOINT_THRESHOLD_TOKENS

yield* willFold
  ? foldConversation({ conversationId: cid, attempt: input.attempt, contextTokens: grown })
  : Effect.void

The second is inside one long attempt. Waiting for a gate rejection is too late when a single exploratory pass can consume the healthy range. The engine therefore asks a compact callback after every completed model/tool turn while the loop still intends to continue:

const plan =
  phase === "continue" && options.compact !== undefined
    ? yield* options.compact(nextMessages, usage)
    : Option.none<CompactionPlan>()

One policy, two seams. The outer seam uses the workflow’s chapter boundary; the inner seam prevents one chapter from becoming pathological before it ends.

A fold may only cut on an assistant boundary#

Tool transcripts are paired protocols. An assistant asks for tools; the following tool message answers those call ids. Cut between them and the next provider rejects the history before the model sees it.

safeKeepFrom finds the largest cut that still keeps the configured number of recent assistant turns, and the loop validates the returned plan again:

const applied = Option.filter(
  plan,
  (p) =>
    p.keepFrom > 0 &&
    p.keepFrom < nextMessages.length &&
    nextMessages[p.keepFrom]?.role === "assistant", 
)

Smith keeps two recent assistant turns verbatim. The summary carries the older narrative; the tail preserves the exact newest tool calls, results, and provider metadata. The overlap is deliberate. Summaries are good at task state and bad at protocol detail.

The ordering around human steering is equally deliberate. Pending input is checked after compaction, so a message typed while the agent is busy can never be included in the material immediately folded away. The newest instruction is guaranteed one real model read.

The position mirror is the difficult bit#

Rewriting an in-memory buffer is easy. Making the next process resume from the same fold is not.

The conversation store is append-only. A synthetic handoff summary has no message row, while every real buffer entry does. runAgent keeps a mirror beside the buffer: Some(position) for persisted messages, None for a synthetic handoff. When a fold chooses keepFrom, the mirror tells the store the exact last durable row covered by the summary:

const covered = mirror.slice(0, cut.value).flatMap(Option.toArray)
const coveredPosition = covered[covered.length - 1]
const firstKept = mirror.slice(cut.value).flatMap(Option.toArray)[0]

if (
  coveredPosition === undefined ||
  (firstKept !== undefined && coveredPosition >= firstKept)
) return Option.none()

yield* store.checkpointAt(conversationId, summary, coveredPosition) 

That defensive inequality looks redundant until persistence and memory drift by one entry. The code refuses to checkpoint rather than risk covering a row that the active tail still needs. A missed fold costs tokens; a wrong fold manufactures amnesia. The failure preference is obvious once stated and easy to get backwards in code.

A summary is not an authority#

The handoff prompt asks for four things: task, workspace state, verification, and dead ends. It does not ask the summarizer to preserve the complete standing quality contract. That contract may be long, repetitive, and easy to paraphrase into something weaker.

So after an attempt-boundary fold, Smith deterministically reattaches the original context blocks in authority order:

1. the human’s AGENTS.md, CLAUDE.md, or .efferent/rules.md;
2. the quality bar rendered from the armed gate config;
3. lessons distilled from prior gate history;
4. curated workspace memory, explicitly marked “verify before relying.”

export const renderPostFoldRetryBrief = (
  feedback: string,
  extras: BriefExtras,
): string => [renderRetryBrief(feedback), ...extrasBlocks(extras)].join("\n\n")

This is one of the sharpest design choices in the implementation. Teaching the summarizer to retain doctrine would make enforcement probabilistic at exactly the moment the model is least reliable. The summary remembers work. Source files and gate config restate authority.

The same idea keeps the system prompt lean. Skills enter it as name plus description; the full markdown body arrives only when the model calls load_skill. Permanent context holds indexes and rules, not every procedure the workspace might contain.

Folding is best-effort, authority is not#

Compaction can fail: the utility model can rate-limit, the digest can be empty, the position mirror can fail its consistency check. Every one of those cases leaves the trail unfolded and lets the run continue. Context pressure is serious, but destroying history in order to relieve it is worse.

By contrast, the gate feedback and quality bar are not best-effort. They are regenerated from deterministic sources on the new side of the fold. This asymmetry is the point:

• the lossy optimization may decline to run;
• the instructions that govern the next attempt may not quietly disappear.

There is a cost. A cold post-fold turn rebuilds the provider cache. The digest itself spends a fast-model call. Two kept turns duplicate facts already present in the summary. And 80k is an empirical threshold from the models Efferent runs today, not a law of cognition.

Still, the implementation gives the threshold somewhere honest to live. Context management is not “summarize when full.” It is a transaction across an in-memory protocol, an append-only store, a provider cache, and a hierarchy of instructions. The summary is the visible artifact. The safe cut, position mirror, and deterministic reattachment are the actual feature.