Documentation Index
Fetch the complete documentation index at: https://simplellmfunc.cn/llms.txt
Use this file to discover all available pages before exploring further.
Interfaces API Reference
LLM_Interface (ABC)
from SimpleLLMFunc.interface import LLM_Interface
Constructor
LLM_Interface(
api_key_pool: APIKeyPool,
model_name: str,
base_url: Optional[str] = None,
context_window: int = 200_000,
)
Abstract Methods
| Method | Signature | Description |
|---|
chat | async def chat(*, trace_id, stream=False, messages, timeout=None, **kwargs) | Non-streaming call |
chat_stream | async def chat_stream(*, trace_id, stream=True, messages, timeout=None, **kwargs) -> AsyncGenerator | Streaming call |
Properties
| Property | Type | Description |
|---|
model_name | str | Model identifier |
base_url | str | None | API endpoint |
context_window | int | Context window size in tokens |
api_key_pool | APIKeyPool | Key rotation pool |
OpenAICompatible
from SimpleLLMFunc import OpenAICompatible
Constructor
OpenAICompatible(
api_key_pool: APIKeyPool,
model_name: str,
base_url: str,
max_retries: int = 5,
retry_delay: float = 1.0,
rate_limit_capacity: int = 10,
rate_limit_refill_rate: float = 1.0,
context_window: int = 200_000,
)
Class Methods
| Method | Returns | Description |
|---|
load_from_json_file(path) | Dict[str, Dict[str, OpenAICompatible]] | Load all models from provider.json |
Instance Methods
| Method | Returns | Description |
|---|
get_rate_limit_status() | Dict[str, Any] | Current rate limiter state |
reset_rate_limit() | None | Reset the token bucket |
Example
from SimpleLLMFunc import OpenAICompatible, APIKeyPool
# From file
models = OpenAICompatible.load_from_json_file("provider.json")
llm = models["openrouter"]["openai/gpt-4o"]
# Direct
llm = OpenAICompatible(
api_key_pool=APIKeyPool(api_keys=["sk-..."], provider_id="openai"),
model_name="gpt-4o",
base_url="https://api.openai.com/v1",
rate_limit_capacity=20,
rate_limit_refill_rate=5.0,
)
OpenAIResponsesCompatible
from SimpleLLMFunc import OpenAIResponsesCompatible
OpenAICompatible.
Differences from OpenAICompatible
| Aspect | OpenAICompatible | OpenAIResponsesCompatible |
|---|
| System prompt | messages[0].role="system" | instructions field |
| Streaming format | Chat Completion chunks | Responses stream events |
| Reasoning support | N/A | reasoning={...} kwargs |
| Wire protocol | Chat Completions API | Responses API |
Example
from SimpleLLMFunc import OpenAIResponsesCompatible, APIKeyPool
llm = OpenAIResponsesCompatible(
api_key_pool=APIKeyPool(api_keys=["sk-..."], provider_id="openai"),
model_name="gpt-4o",
base_url="https://api.openai.com/v1",
)
@llm_chat(llm_interface=llm, reasoning_effort="high")
async def reasoning_agent(message: str, history: list | None = None):
"""Reason carefully about the question."""
pass
APIKeyPool
from SimpleLLMFunc import APIKeyPool
Constructor
APIKeyPool(
api_keys: List[str],
provider_id: str,
)
Example
pool = APIKeyPool(
api_keys=["sk-key-1", "sk-key-2", "sk-key-3"],
provider_id="openrouter-gpt4o",
)
# Keys are rotated on each call automatically
TokenBucket
from SimpleLLMFunc import TokenBucket
Constructor
TokenBucket(
capacity: int,
refill_rate: float,
)
| Parameter | Description |
|---|
capacity | Maximum tokens in the bucket |
refill_rate | Tokens added per second |
OpenAICompatible. Exposed for custom implementations.