← All docs

Model servers

Sovereign AI Workspace expects an OpenAI-compatible HTTP API for chat and embeddings. Configure per tenant in /ops or via environment defaults.

Ollama (local)

ollama pull llama3.2
ollama serve

MODEL_BASE_URL=http://127.0.0.1:11434/v1
MODEL_API_KEY=
MODEL_ID=llama3.2
EMBEDDING_MODEL=nomic-embed-text

Ollama exposes OpenAI-compatible endpoints at /v1/chat/completions and /v1/embeddings. Use providerKind: openai_compatible with MODEL_BASE_URL=http://127.0.0.1:11434/v1.

Ollama (native /api/chat)

For tenants configured with providerKind: ollama, the app calls Ollama's native streaming API (not /v1):

MODEL_BASE_URL=http://127.0.0.1:11434
MODEL_ID=llama3.2

Set the tenant model profile providerKind to ollama in /ops. Embeddings still use the OpenAI-compatible /v1/embeddings path via the embedding adapter — run a small OpenAI-compatible proxy or use nomic-embed-text through Ollama's /v1 endpoint for RAG.

LiteLLM (multi-provider proxy)

# litellm config.yaml
model_list:
  - model_name: default
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY
  - model_name: text-embedding-3-small
    litellm_params:
      model: openai/text-embedding-3-small
      api_key: os.environ/OPENAI_API_KEY

litellm --config config.yaml --port 8000

MODEL_BASE_URL=http://127.0.0.1:8000/v1
MODEL_ID=default
EMBEDDING_MODEL=text-embedding-3-small

Cloud (OpenAI)

MODEL_BASE_URL=https://api.openai.com/v1
MODEL_API_KEY=sk-...
MODEL_ID=gpt-5.5
OPENAI_REASONING_EFFORT=medium
EMBEDDING_MODEL=text-embedding-3-small

When MODEL_BASE_URL points at api.openai.com, the app automatically uses OpenAI's Responses API (/v1/responses) with the configured reasoning effort. This is recommended for GPT-5.x and other reasoning models.

All other OpenAI-compatible hosts (Ollama, LiteLLM, vLLM, Azure gateways) use Chat Completions (/v1/chat/completions).

Azure OpenAI

MODEL_BASE_URL=https://YOUR_RESOURCE.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT
MODEL_API_KEY=your-azure-key
MODEL_ID=your-deployment-name
EMBEDDING_MODEL=text-embedding-3-small

Verify connectivity

npm run smoke

Or manually:

curl http://127.0.0.1:8000/v1/models