Context Overview

This page explains how SimpleLLMFunc compiles context — the information the LLM sees at each reasoning step. The final provider-facing message list is built from invocation configuration, a base transcript, and internal runtime patches.

The Core Data Structures

ContextState

The runtime representation of a conversation’s current state:

@dataclass
class ContextState:
    messages: NormalizedMessageList           # The conversation transcript
    data_from_selfref: Optional[DataFromSelfRef]  # Durable self-reference state
    pending_mutations: List[ContextMutation]  # Changes waiting to be applied

messages is the base transcript for the current runtime state — assistant messages, tool results, user messages
data_from_selfref carries durable state from the SelfReference backend (experiences, summaries)
pending_mutations accumulates internal runtime patches during a ReAct iteration, applied to the transcript at the next compile boundary

CompileSource

The input boundary — everything needed to produce an LLM request:

@dataclass(frozen=True)
class CompileSource:
    data_from_agent_config: DataFromAgentConfig  # Static: system prompt, tool specs
    data_from_selfref: Optional[DataFromSelfRef] # Durable: experiences, summaries
    input_messages: NormalizedMessageList         # Dynamic: conversation messages

DataFromAgentConfig

Static configuration from the decorator:

@dataclass(frozen=True)
class DataFromAgentConfig:
    base_system_prompt: str                    # From docstring (possibly templated)
    template_params: Optional[Dict[str, Any]]  # Runtime template values
    tool_prompt_specs: List[Dict[str, Any]]    # Tool best-practice injections
    include_must_principles: bool              # Whether to add structured-call rules

DataFromSelfRef

Durable state from the SelfReference backend:

@dataclass(frozen=True)
class DataFromSelfRef:
    base_system_prompt: str                    # System prompt with selfref markers
    experiences: List[Dict[str, str]]          # Durable remembered experiences
    summary: Optional[Dict[str, Any]]          # Compaction summary metadata
    summary_message: Optional[Dict[str, Any]]  # The summary as a message
    working_messages: NormalizedMessageList     # Post-compaction working transcript

Context Inputs

SimpleLLMFunc does not build LLM context from mutations alone. Each LLM request is compiled from three inputs:

Invocation configuration — the InvocationSpec / prompt contract built from the decorated function, docstring, template parameters, tool guidance, return mode, and SelfRef snapshot.
Base transcript — the message list created from history / chat_history, the current user input, and the previous finalized messages.
Runtime patches — ContextMutation objects produced internally by LLM calls, tools, SelfRef primitives, compaction, and abort/cancel handling.

provider messages = render(invocation config, patch(base transcript, runtime patches))

Data Flow: From Your Code to the LLM

┌─ Your Code ──────────────────────────────────────────────┐
│                                                           │
│  @llm_chat(llm_interface=llm, toolkit=[...])             │
│  async def agent(message: str, history: list):            │
│      """System prompt here."""                            │
│      pass                                                 │
│                                                           │
│  async for output in agent("hello", history):             │
│      ...                                                  │
│                                                           │
└───────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─ Decorator Layer ────────────────────────────────────────┐
│                                                           │
│  Build InvocationSpec:                                    │
│    - Parse docstring → base_system_prompt                 │
│    - Collect tool specs → tool_prompt_specs               │
│    - Resolve template_params                              │
│    - Build initial transcript from history + user message │
│                                                           │
└───────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─ ReAct Loop ─────────────────────────────────────────────┐
│                                                           │
│  while has_more_work:                                     │
│                                                           │
│    1. Collect hook mutations (selfref, etc.)              │
│    2. compile_context(state, pending_mutations)           │
│         → apply_mutations → CompiledContext               │
│    3. compile_invocation_turn(spec, messages, [], selfref)│
│         → reduce + convert → CompiledTurnContext          │
│    4. execute_single_llm_phase(compiled.llm_messages)     │
│         → yields events + produces mutations              │
│    5. If tool calls: schedule_tool_batch(...)             │
│         → executes tools → produces mutations             │
│    6. Merge all mutations → loop back to step 1           │
│                                                           │
└───────────────────────────────────────────────────────────┘
                          │
                          ▼
┌─ Interface Layer ────────────────────────────────────────┐
│                                                           │
│  OpenAICompatible.chat_stream(messages=[...])             │
│  → HTTP request to provider                               │
│  → Stream response back                                   │
│                                                           │
└───────────────────────────────────────────────────────────┘

The Key Invariants

Runtime side effects do not edit the live transcript directly — LLM calls, tools, SelfRef primitives, and abort handling produce ContextMutation patches instead of mutating ContextState.messages in place.
Mutations are transcript patches, not the whole context — docstrings, template params, tool guidance, SelfRef snapshots, and initial history come from invocation configuration and the base transcript.
Mutations accumulate, then apply atomically — During a ReAct iteration, runtime patches collect in pending_mutations. They’re applied together at the compile boundary.
Compile produces a snapshot — The compiled output is a clone. The original state is never mutated in place.
SelfRef state propagates — DataFromSelfRef is carried forward through compilations. If selfref markers are detected in the transcript after mutation application, the snapshot is refreshed.
System prompt resolution has priority order:
- SelfRef snapshot (if present) → renders base prompt + experiences
- PromptContract.system_prompt (if set explicitly)
- Latest system message in transcript
- PromptContract.base_instruction (docstring fallback)

Next Steps

Runtime Patches

The internal mutation types — when runtime effects produce transcript patches.

Compile Pipeline

How invocation config, transcript, and patches become the final LLM request.

Documentation Index

​Context Overview

​The Core Data Structures

​ContextState

​CompileSource

​DataFromAgentConfig

​DataFromSelfRef

​Context Inputs

​Data Flow: From Your Code to the LLM

​The Key Invariants

​Next Steps