BYO-LLM — Use Your Own SDK¶

If you already use the Anthropic, OpenAI, Gemini, or any other SDK for your LLM calls, you can use mnesis purely as a memory and compaction layer — without routing any calls through litellm.

Use session.record() instead of session.send(). It persists a completed user/assistant turn to the store, accumulates token usage, and triggers compaction automatically — but makes no LLM call itself.

How it works¶

Your code                           mnesis
──────────────────────────────      ──────────────────────────────
1. Build messages from context  ←── session.context_for_next_turn()
2. Call your LLM SDK
3. Record the result            ──► session.record(user, assistant)
                                     └─ persists both messages
                                     └─ updates token counters
                                     └─ triggers compaction if needed

Examples¶

Anthropic SDKOpenAI SDK

import anthropic
from mnesis import MnesisSession, TokenUsage

client = anthropic.Anthropic()
session = await MnesisSession.create(model="anthropic/claude-opus-4-6")

user_text = "Explain quantum entanglement."

# Compaction-aware context: summaries replace compacted turns
messages = await session.context_for_next_turn()
messages.append({"role": "user", "content": user_text})
# After compaction the first item is the summary, with role "assistant".
# Anthropic's Messages API docs only show conversations that start with a
# user turn and do not say a leading assistant turn is accepted, so lead
# with a user turn to be safe.
if messages[0]["role"] != "user":
    messages.insert(0, {"role": "user", "content": "[Conversation so far]"})

# Call your SDK
response = client.messages.create(
    model="claude-opus-4-6",
    max_tokens=1024,
    messages=messages,
)

# Record the turn — mnesis handles the rest
await session.record(
    user_message=user_text,
    assistant_response=response.content[0].text,
    tokens=TokenUsage(
        input=response.usage.input_tokens,
        output=response.usage.output_tokens,
    ),
)

from openai import AsyncOpenAI
from mnesis import MnesisSession, TokenUsage

client = AsyncOpenAI()
session = await MnesisSession.create(model="openai/gpt-4o")

user_text = "What is the capital of France?"

messages = await session.context_for_next_turn()
messages.append({"role": "user", "content": user_text})

response = await client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
)

await session.record(
    user_message=user_text,
    assistant_response=response.choices[0].message.content,
    tokens=TokenUsage(
        input=response.usage.prompt_tokens,
        output=response.usage.completion_tokens,
    ),
)

Do not build prompts from session.messages()

messages() returns the complete append-only history, including compaction summaries and turns the summaries already replaced. Sending it to your LLM resends content that was intentionally removed from the context window. Use context_for_next_turn() for the prompt; use messages() only to inspect or audit stored history. The pending user message is not stored until record(), so append it yourself as shown above.

Token usage is optional¶

If you omit tokens, mnesis estimates them from the text length using its built-in TokenEstimator. The estimate is a heuristic — pass real usage from your SDK response for accurate compaction budgeting:

# Without token usage — mnesis estimates
await session.record(
    user_message="Hello",
    assistant_response="Hi there!",
)

# With token usage — accurate compaction budgeting
await session.record(
    user_message="Hello",
    assistant_response="Hi there!",
    tokens=TokenUsage(input=5, output=3),
)

Leading summary role

context_for_next_turn() returns compaction summaries as assistant messages, so after a compaction the list can start with an assistant entry. The Anthropic Messages API docs only show user-first conversations and state no rule either way for a leading assistant turn, so the Anthropic example inserts a short user turn first. Mnesis does not reorder the list for you.

Compaction still works automatically¶

After session.close() (or while it is closing), record() and context_for_next_turn() raise SessionClosedError (a MnesisStoreError). close() waits for a record() already in flight to finish persisting.

record() checks for overflow after every turn, exactly as send() does. If the size of the current context window (system prompt + summaries + raw messages that context_for_next_turn() would return, un-truncated) reaches the soft threshold, compaction is triggered in the background — no extra configuration required. The tokens you pass are lifetime accounting (session.token_usage); they do not drive compaction, so a session whose context was already compacted does not compact again every turn.

RecordResult¶

record() returns a RecordResult:

result = await session.record(...)

result.user_message_id      # ID of the persisted user message
result.assistant_message_id # ID of the persisted assistant message
result.tokens               # TokenUsage (provided or estimated)
result.compaction_triggered # True if compaction was scheduled
result.doom_loop_detected   # True if the same tool call was recorded `doom_loop_threshold` times in a row

record() tracks ToolPart calls (tool name plus input) for doom-loop detection. When doom_loop_threshold consecutive identical tool calls are seen (within one turn or across turns), doom_loop_detected is True and a DOOM_LOOP_DETECTED event is published. A turn with no tool call breaks the run, as does a managed send() turn. The detection window lives in memory only: MnesisSession.load() does not restore it, so detection starts fresh after a reload.

Runnable example¶

See examples/06_byo_llm.py for a complete, self-contained example that runs without any API key.

Streaming is for managed mode only¶

session.stream() is a wrapper around session.send() — it routes the LLM call through litellm and is only available in managed mode.

If you are using BYO-LLM (i.e. you call your own LLM SDK and use record() to persist the turn), there is no streaming path through mnesis. Streaming is handled entirely by your SDK. Persist the completed turn with record() as usual once your SDK has finished streaming.