Skip to content
GraphStaff Starter

Swapping models

Model selection is data, not code. The server reads models.toml once at startup; nothing else in the system names a model.

Swap the answer model

Edit one line

models.toml
[models]
synthesis = "anthropic/claude-sonnet-4.5"

The value is an OpenRouter model slug by default, so hundreds of models are one edit away.

Set the key

Terminal window
export OPENROUTER_API_KEY=sk-or-...

A different variable name can be configured with endpoint.api_key_env.

Restart the server

No feature code changes anywhere.

Use any OpenAI-compatible endpoint

Point the endpoint at any server that speaks the OpenAI chat-completions API:

models.toml
[models]
synthesis = "llama3.1"


[endpoint]
api_base = "http://localhost:11434/v1"
api_key_env = "OLLAMA_API_KEY"