Skip to main content
The overmind package instruments your LLM stack over OpenTelemetry and exports spans to Overmind. One init() call captures every supported provider call automatically; run() brackets one agent run; decorators trace the code around the model calls; a few one-line calls declare what the run was asked to do and what it delivered, which is what trace scoring judges. The package is MIT.
The base package is the CLI and the OpenAI-compatible inference client. Tracing is the one extra: on a bare install, accessing overmind.init or any tracing name raises ImportError with the install line to run. There are no other extras.

init()

Call once at process startup, before any LLM call. Idempotent and thread-safe — calling again refreshes identity, enables more providers, and updates the orphan-export policy without rebuilding the exporter. Without an API key it logs once, returns False, and every decorator and helper below becomes a no-op; set OVERMIND_STRICT_MODE=true to make a missing key raise.
Passing either capability_id or capability suppresses the env fallback for both — a declared half never picks up the other from the environment. Environment variables: OVERMIND_API_KEY, OVERMIND_API_URL, OVERMIND_SERVICE_NAME, OVERMIND_ENVIRONMENT, OVERMIND_CAPABILITY_ID, OVERMIND_CAPABILITY_NAME, OVERMIND_PROJECT_ID, OVERMIND_STRICT_MODE; OVERMIND_SPAN_FLUSH_INTERVAL_MS (default 2000) and OVERMIND_SPAN_MAX_EXPORT_BATCH_SIZE (default 256) tune the batch exporter; OVERMIND_GIT_SHA overrides commit detection (then GIT_SHA, GIT_COMMIT, GITHUB_SHA, RENDER_GIT_COMMIT, VERCEL_GIT_COMMIT_SHA, HEROKU_SLUG_COMMIT, CI_COMMIT_SHA, then .git/HEAD); SERVICE_VERSION sets service.version; a TRACEPARENT env var is attached as the remote parent. In a repository where overmind sync has run, the SDK reads the key from .overmind/credentials.toml and the API URL and project id from overmind.toml. Deployments provide credentials through their runtime secret configuration. After init(), calls made with the OpenAI, Anthropic, Google Gemini, Agno, and LangChain client libraries produce llm_call spans automatically, capturing messages and tool calls, model and request parameters, token usage, latency, and errors. Spans export via a batching OTLP/HTTP exporter to POST {base_url}/api/v1/traces with the key in X-Api-Key. The resource carries vcs.ref.head.revision, so a unit binds to the behaviour contract analysed at that commit. If the app already owns an OpenTelemetry TracerProvider, skip init(), add an OTLP exporter for the endpoint to that provider, and call overmind.tracing.enable_tracing(providers): it attaches the SDK’s stamping and usage processors to the existing provider and enables the instrumentors, so decorators and run() work unchanged.

run() — the run boundary

Every agent execution needs exactly one run boundary. run() is the one scope that covers it: capability identity, the entry-point span (overmind.unit_kind = "run"), the intent, the conversation id, tags, error status, and a flush on exit. Use it as a context manager or as a decorator on the entry point:
As a decorator, every parameter except name also accepts a callable receiving the wrapped call’s arguments, resolved per invocation (a failing callable resolves to None). The run span carries the function’s code.namespace / code.function.name, so one decoration also satisfies the entry-point anchor a scanned behaviour expects. An exception marks the run span failed and re-raises; the exporter is flushed either way. The return value is not delivered automatically: call deliver() (or the handle’s run.deliver()) inside the unit that produced it. A function span (the @observe and start_span default) that would start its own trace outside a run boundary is dropped as an orphan fragment, and the SDK warns once. Boundary spans, @tool / @workflow roots, auto-instrumented roots, and spans continuing a remote TRACEPARENT still export. If a trace is missing, add the bracket — do not reach for export_orphan_spans.

deliver() — the terminal deliverable

Captures the run’s result on its own child span: the payload is serialised into outputs and the span is stamped overmind.delivery = true. Trace scoring judges this span as the unit’s terminal. grounded_by names the evidence spans the deliverable rests on (span-id hex strings or span handles), written as overmind.grounded_by; when omitted, the environment-provenance spans (tools, retrieval) collected in the current trace are used, so call it inside the run.

Tracing your own code

Auto-instrumentation only sees model calls. Wrapping the code around them — tools, retrieval steps, the phases of a run — turns flat spans into a tree that mirrors your agent’s structure, and gives trace scoring the anchors it binds to.

Span types

The value lands in overmind.span.type; ingest stores it verbatim.

Decorators

@observe is the general decorator; @entry_point, @workflow, @tool, and @retrieval are typed shortcuts. All work on sync and async functions, record duration and status, and re-raise exceptions after recording them. An @entry_point interrupted by KeyboardInterrupt or cancellation force-flushes before re-raising, so the run still lands.
Captured arguments land in the inputs attribute and the return value in outputs, both as JSON. Payloads are scrubbed: keys containing password, secret, token, credential, authorization, api_key, apikey, or sensitive are redacted, data URLs and base64-looking strings of 512+ characters become placeholders, byte strings over 256 bytes become placeholders, dataclasses and Pydantic models are dumped, and text is kept in full. For code that handles credentials, use capture="none" — and prefer masking values before they reach traced functions. normalize_messages is exported for callers that build chat evidence themselves. The span name may be a callable receiving the call’s arguments — for a polymorphic dispatcher, each invocation then emits its own tool span and tool.name follows the resolved action:
Every decorated span carries the function’s code.namespace / code.function.name; tool spans add tool.name and tool.arg_keys.
Decorate every function the scan anchors on. An undecorated anchor emits no code.namespace / code.function.name, so the behaviour it belongs to never binds and its step judges skip it.

Instrumenting a scanned repository

After overmind sync, each capability’s behaviours carry the anchors trace scoring binds to. The get_instrumentation_plan MCP tool turns them into tickets: call it with no capability for project-wide work, or with a capability and optional behaviour for a scoped change. Each ticket names the target file and qualname, import line, required scope, required decorators, capability id, behaviour key, version, and grain. Apply the placements verbatim so the resulting spans bind to the pushed contracts; the /overmind ensure-tracing command does exactly this. A missing registry means capabilities have not been synced — run /overmind setup then overmind sync first. Two identity forms tie code to a capability declared in overmind.toml:
  • overmind.init(capability_id="<slug>") (or run(capability_id=...)) — ingest binds by the capability’s UUID; a slug resolves server-side.
  • @overmind.capability("<slug>", id="...") on an entry function — at runtime the name is a label only; pass id= or set capability_id in init() to bind.
Verification is a bounded, read-only MCP smoke: run with a unique conversation_id, flush, find the trace with query_traces(session=..., all_spans=true), read overmind://traces/{trace_id}, and hand its spans to verify_instrumentation, which checks capability attribution, nesting, inputs and outputs, usage, and timing without writing traces or scores.

task() — carve a run into units

task(key, unit="turn") makes a phase an independently scored unit bound to a declared behaviour; key is the behaviour’s key (its slug — the mode id from the scan). Re-entering the same key re-uses the still-open turn span, so a re-entrant phase (a tool loop, debate rounds) lands in one unit; the span ends when the run-boundary span ends. Without unit, task() only stamps overmind.behaviour.key on the spans inside. unit accepts "turn" only — run boundaries come from run() / entry_point.
Rules that matter: call deliver() inside the unit that produced the deliverable; internal fan-out, retries, and loop bodies must not declare unit.

capability() — multi-capability agents

capability(name=None, *, id=None) declares that all work inside belongs to one capability: every span created inside carries overmind.capability.id / .name, and the outer identity is restored on exit. It works as a context manager (with / async with) or a decorator. id — the capability’s UUID — is the binding; a name-only scope labels the spans and leaves them unbound. Entering a different capability mid-trace (by id, or by name on the slug grain) is a handoff — the first span of the new scope is stamped overmind.unit_kind = "turn", so the platform scores it as a new unit against that capability’s eval set. Only declared identities are stamped; nothing is created.

start_span() — blocks and loops

A context manager for regions that aren’t whole functions. It opens a child span under whatever span is active and captures nothing automatically — attach metadata via attributes or set_tag(). Before init() it yields a non-recording span.

LangChain / LangGraph

providers=["langchain"] (or "auto") mounts the OpenInference LangChain instrumentor that ships with overmind[tracing], which covers LangGraph: every chain, model, and tool invocation gets a span with model, token, and cost evidence. For the scoring semantics no instrumentor can know, overmind.integrations.langgraph.bind maps graph nodes to behaviour units — call it on the StateGraph after add_node(), before compile():
Each node runs inside task(key, unit="turn"); nodes backed by your own function also get an @observe(capture="none") span carrying that function’s code identity.

Runtime expectations

These declare evidence the platform evaluates server-side; each is a span event (overmind.eval.*, with a JSON payload) on the current span and a no-op when nothing is recording.
How the platform uses them is on Trace scoring.

Context helpers

All operate on the current span or context — call them inside a traced function or request handler.

PromptString — explicit prompt structure

PromptString (re-exported from the opentelemetry-overmind package that ships with the tracing extra) is a str subclass that keeps the template and its arguments. When a provider instrumentor sees one in a request, it stamps overmind.prompt.template and overmind.prompt.kwargs on the llm_call span, and the eval envelope hands the template and rendered prompt to the judges.
Use one PromptString per LLM call — the instrumentor raises ValueError when it finds more.

Full example

One trace: the handle_support_query run → lookup_order (tool, environment evidence) → the OpenAI llm_call → the deliver span, attributed to the support-triage capability, grouped into the user’s session, with the question as its intent. Trace scoring binds it to the capability’s behaviour and judges the delivered answer against the eval set.