Skip to main content

​
Quickstart

​
Install

pip install SimpleLLMFunc
Or with Poetry: 
poetry add SimpleLLMFunc

​
Configure a Model

Create provider.json in your project root: 
{
  "openrouter": [
    {
      "model_name": "openai/gpt-4o",
      "api_keys": ["sk-your-key"],
      "base_url": "https://openrouter.ai/api/v1",
      "max_retries": 3,
      "retry_delay": 1.0,
      "rate_limit_capacity": 20,
      "rate_limit_refill_rate": 3.0
    }
  ]
}
This works with any OpenAI-compatible endpoint — OpenRouter, Together, local vLLM, etc.

​
Your First LLM Function

import asyncio
from pydantic import BaseModel, Field
from SimpleLLMFunc import OpenAICompatible, llm_function


class Summary(BaseModel):
    headline: str = Field(description="One-sentence summary")
    key_points: list[str] = Field(description="3-5 bullet points")
    tone: str = Field(description="formal, casual, or technical")


models = OpenAICompatible.load_from_json_file("provider.json")
llm = models["openrouter"]["openai/gpt-4o"]


@llm_function(llm_interface=llm)
async def summarize(text: str) -> Summary:
    """
    Summarize the input text concisely.
    Focus on the most actionable information.
    """
    pass


async def main():
    result = await summarize("SimpleLLMFunc treats every LLM call as a typed Python function...")
    print(result.headline)
    print(result.key_points)


asyncio.run(main())
That’s it. The decorator:
  1. Builds a system prompt from the docstring + parameter types + return type schema
  2. Sends the function arguments as the user message
  3. Parses the LLM response into your Pydantic model
No message lists, no JSON mode flags, no output parsers.

​
Direct Construction (No provider.json)

For quick scripts, construct the model directly: 
from SimpleLLMFunc import APIKeyPool, OpenAICompatible

llm = OpenAICompatible(
    api_key_pool=APIKeyPool(
        api_keys=["sk-your-key"],
        provider_id="openai",
    ),
    model_name="gpt-4o",
    base_url="https://api.openai.com/v1",
)

​
What’s Next

Build a Tool-Using Agent

Add tools and see the ReAct loop in action.

Build a Multi-Turn Chat

History management, streaming, and stateful conversations.