Skip to main content
Overmind improves production AI agents with their own data. It ingests OpenTelemetry traces, scores them as they arrive, turns them into curated datasets, and feeds three kinds of work: eval experiments, optimisation runs that open pull requests, and training runs that produce models benchmarked against the one you ship today.
Everything speaks standard protocols: OTLP/HTTP for ingest, REST JSON for the API, OpenAI-compatible serving for models. Authenticate with a project-scoped ovr_ key as X-Api-Key: <key> or Authorization: Bearer <key>. There is no proprietary agent framework — the SDKs auto-instrument OpenAI, Anthropic, and Gemini over OpenTelemetry, and any OTel exporter can POST to the same endpoint. Overmind gives you:
  • Live scoring — evaluators run on root spans as they land
  • Datasets from traffic — convert traces into eval and training data
  • Eval runs — variants measured against a baseline with typed scores
  • Optimiser — local execution, scored candidates, a PR with the winning diff
  • Training — managed GPU runs, loss curves, benchmark vs your production model
  • Inference — OpenAI-compatible API for your trained models and frontier models

Guides

Quickstart

API key, agent setup prompt, first trace in the console.

Observability

OTLP ingest, span model, attribute mapping, explorer.

Python SDK

init(), auto-instrumentation, decorators, context.

TypeScript SDK

OvermindClient, initTracing(), manual spans.

Agent & Capabilities

Repo scans, telemetry attribution, eval specs.

Datasets

Schema, intents, Workshop, trace-to-dataset.

Eval

Evaluator kinds, eval sets, run reports.

Optimisers

Loop, local executioner, resulting PR.

Models & API

Training

Runs, validation, loss curves, baseline benchmark.

Inference

Serving lifecycle and /api/v1/chat/completions.

REST API

Auth, endpoint map, conventions, Swagger.

Model Library

Base models, context windows, training and inference pricing.