Detailed guide

1. how to use overmind to your llm stack.
2. prebuilt integrations for

• langchain

3. deep dive into overmind layers

how to use overmind to your workflow.

1. set it up, overmind comes with drop in replacement for popular sdks, openai and anthropic

import os

~~from openai import OpenAI~~
from overmind.clients import OpenAI


os.environ["OVERMIND_API_KEY"] = "your_overmind_api_key"
os.environ["OPENAI_API_KEY"] = "your_openai_api_key"

openai_client = OpenAI(
  ...all_openai_parameters,
  overmind_api_key="not optional" # this will throw an error
  overmind_base_url="optional",
) # this automatically adds tracing to your llm code, all inputs, outputs are now visible in overmind dashboard

# you can add feedback or run evals from overmind dashboard

# then all existing openai methods will work

2. start using overmind to add more security to your LLM workflow

response=openai_client.chat.completions.create(
    messages=[],
    tools=[],
    ...all_openai_parameters,
    # Overmind built-in policies
    input_policies=['anonymize_pii'],
    output_policies=['reject_irrelevant_answer'],
)
print(response.choices[0].message.content) # as usual

print(response.summary()) # this will give you summary from overmind

please note: streaming not fully suppported yet.

3. advance usage of policies

from overmind.policies import AnonymizePii, RejectLLMJudgeWithCriteria

anonymize_pii = AnonymizePii(
  pii_types=["DEMOGRAPHIC_DATA", "FINANCIAL_ID"],
)

reject_irrelevant_answer = RejectLLMJudgeWithCriteria(
    criteria=[
        "Must not contain financial advice",
        "Must be succinct and easy to understand",
    ]
)
response = openai_client.chat.completions.create(
    messages=[],
    tools=[],
    # ...all_openai_parameters,
    # Overmind built-in policies
    input_policies=[anonymize_pii],
    output_policies=[reject_irrelevant_answer],
)

print(response.choices[0].message.content) # as usual

print(response.summary()) # this will give you summary from overmind

working with non-textual data

input_document = """
Dear Jon Smith,

We're glad to inform you that your visa application for passport number 19318291 has been successfull...
"""

result = openai_client.embeddings.create(
    model="text-embedding-3-small",
    input=input_document,
    encoding_format="float",
    # ...all_openai_parameters,
    input_policies=['anonymize_pii'],
)

print(response.summary()) # this will give you summary from overmind