Name spans for humans, query them by attribute

An agent can finish successfully and still have had a terrible run.

Twenty-five turns, five failed tools, three malformed-response corrections, and a last-minute answer may all collapse into the same green “completed” state as a clean three-turn implementation. If the trace records only duration and status, the trajectory disappears exactly when you need to compare the two.

Efferent’s current tracing design stays deliberately small — run, engine loop, turns, provider calls — and makes those spans carry enough evidence to distinguish a healthy success from an expensive rescue.

The tree is smaller than the program#

One persisted user turn creates this shape:

agent.run
└─ engine.run
   ├─ engine.turn
   │  └─ providers.generate
   ├─ engine.turn
   │  └─ providers.generate
   └─ …

There is no span for every helper function and currently no child span for every tool handler. The loop already emits structured tool events and counters; turning every read_file into a span would deepen the tree without answering a new operational question. The one other child is engine.compact, wrapped around within-run compaction on the turns where it fires.

The names are short because the waterfall already supplies hierarchy. Dynamic facts belong in attributes, not in strings a query has to parse.

Identity belongs at the outer run#

runAgent owns the durable conversation id and the user’s prompt, so it opens the outer span:

Effect.withSpan("agent.run", {
  attributes: {
    "agent.kind": "run",
    "agent.conversation_id": String(conversationId), 
    "agent.prompt": traceContent ? prompt.slice(0, 500) : "[redacted]",
    "agent.prompt_chars": prompt.length,
  },
})

The name is for a human scanning the waterfall. agent.kind and agent.conversation_id are the machine contract: low-cardinality discriminator plus join key. A dashboard can group all runs without matching agent.run, and an incident can jump from a conversation row to its trace without relying on timestamps.

Prompt content is redacted by default. Length stays visible because “this run began with a 40k-character prompt” is operationally useful even when the text is too sensitive to export. Opt-in capture clips at 500 characters; tracing should help debug the agent, not become a second conversation database.

The run span carries trajectory vitals#

At the end of runLoop, the state already knows how the trajectory ended and what happened along the way. Efferent annotates the still-open engine.run span once:

yield* Effect.annotateCurrentSpan({
  "engine.outcome": outcome,
  "engine.reason": reason,
  "engine.turns": final.turnIndex,
  "engine.usage.total_tokens": final.usage.totalTokens,
  "engine.tool_calls": final.toolCalls,
  "engine.tool_calls.failed": final.toolFailures,
  "engine.corrections": final.corrections,
  error: outcome !== "ok",
})

This is more useful than one span per counter increment. A trace list can immediately filter for completed runs with corrections, unusually high tool-failure ratios, or step-cap exits. The waterfall remains readable; the terminal span acts as the run’s compact receipt.

outcome and reason are separate on purpose. partial is the broad machine state; step-cap and degenerate-loop explain how it became partial. Conflating those would make two different fixes — increase a limit versus investigate repeated no-progress calls — look identical.

Turn spans own model-call latency#

Each loop iteration wraps the streaming or settled model path in engine.turn, tagged by index:

const outcome = yield* modelCall.pipe(
  Effect.withSpan("engine.turn", {
    attributes: { "engine.turn": state.turnIndex },
  }),
)

The provider router opens providers.generate underneath. That split lets the trace answer two different questions:

• Was the turn slow because the provider was slow?
• Or did work around the call — stream folding, corrective handling, persistence — dominate?

Most turns will show nearly identical durations. The cases where they do not are why both spans exist.

The provider span records the resolved reality#

Model settings and credentials are resolved per call, so the configured model at session start is not sufficient evidence. providers.generate carries the exact provider:modelId used by that call plus finish reason, input/output usage, response size, tool names, and optionally clipped reasoning.

For the settled path:

Effect.annotateCurrentSpan({
  "llm.finish_reason": String(response.finishReason),
  "llm.usage.input_tokens": response.usage.inputTokens ?? 0,
  "llm.usage.output_tokens": response.usage.outputTokens ?? 0,
  "llm.response_chars": response.text.length,
  "llm.tool_calls": toolNames.join(","),
  "llm.reasoning": tracedContent(response.reasoningText ?? ""),
  "llm.fallback": isFallback,
})

Free-text reasoning is [redacted] unless EFFERENT_TRACE_CONTENT=1, and even then clips at 500 characters. Token counts and tool names are useful by default; model content requires an explicit privacy decision.

Streaming and settled calls must tell the same story#

Streaming produces no final response object to inspect. Telemetry has to accumulate while parts flow: text characters, reasoning fragments, and tool names in a Ref. When the finish part arrives, it writes the same attributes and counters as the settled path.

if (part.type !== "finish") {
  return Ref.update(stats, (current) => accumulateStats(current, part))
}

return Ref.get(stats).pipe(
  Effect.flatMap((current) =>
    Effect.annotateCurrentSpan({
      "llm.finish_reason": String(part.reason),
      "llm.usage.input_tokens": part.usage?.inputTokens ?? 0,
      "llm.usage.output_tokens": part.usage?.outputTokens ?? 0,
      "llm.response_chars": current.textChars,
      "llm.tool_calls": current.toolNames.join(","),
    }),
  ),
)

The parity is a correctness property. Switching streaming: true should alter presentation latency, not the numbers in Grafana. Efferent’s router tests compare the two paths because duplicated instrumentation drifts unless equality is treated as a contract.

Duration has to follow stream terminal state too. onDone increments one successful request and records elapsed time; tapErrorCause does the same for error. Retries underneath count as one routed call because the metric measures what the caller experienced, not how many transport attempts the adapter made.

Trace data is not the durable audit trail#

Telemetry may be absent, sampled, expired, or pointed at the wrong endpoint. The conversation database must still answer which model authored each message.

So the router stamps the resolved model into the finish part’s metadata before the message is persisted. There is a sharp JavaScript trap here: GenerateTextResponse exposes fields such as usage and finishReason through prototype getters. Spreading it into { ...response } strips those getters and once caused the loop to read every tool turn as finished.

The router rebuilds the real class instead:

export const stampResponse = (response, label) =>
  new LanguageModel.GenerateTextResponse(
    response.content.map((part) => stampModel(part, label)), 
  )

This is a useful boundary: spans make a run queryable; stamped messages make it auditable after telemetry is gone. Do not make an exporter the only keeper of a fact the product itself needs.

Metrics are the aggregate view#

The provider seam records request count, input/output tokens, and duration tagged by resolved model and outcome. The engine records runs by outcome/reason, tool calls by name/success, and corrections by kind.

Tags stay bounded. Model ids come from configuration; outcomes and correction kinds are closed sets. Conversation ids belong on spans, not metrics — putting one on a counter would create a time series per conversation and turn a useful join key into a cardinality bomb.

That span/metric split is the operational version of the title:

• names make one trace pleasant to read;
• span attributes make individual runs queryable;
• low-cardinality metric tags make populations aggregatable.

What the trace admits it cannot tell you#

There are no per-tool spans today, so a slow shell command is visible through loop events and logs rather than its own waterfall child. Content capture is too clipped to reconstruct a conversation. Silent OTLP export failure means a typoed endpoint looks like no traffic. And the attribute vocabulary is an API: renaming engine.corrections breaks dashboards just as surely as renaming a JSON field breaks clients.

Those limits keep the design honest. The trace is not a transcript and not a second application database. It is a causal performance record with a compact trajectory receipt.

The result is enough to answer the expensive questions quickly: which model actually served the turn, where time accumulated, whether the run was partial, whether a “success” needed repeated correction, and how much context it consumed. A readable name gets you into the trace. Stable attributes tell you whether the run was any good.