Why this Configuration?
- True Data Sovereignty: 100% GDPR-compliant on-premises deployment. You own your configuration and data without vendor lock-in.
- Hardware Agnostic: Optimized and verified across low-power ARM64 SBCs (Raspberry Pi 5) and x86_64 hypervisors (Proxmox VE LXC & VM).
- Engine Freedom: Tested and supported under standard Docker Engine and unprivileged rootless Podman.
- Platform Verification: Tested on Proxmox LXC: docker (main-stable, 2026-09-14), podman (1.19, 2026-09-14); Proxmox VM: docker (main-stable, 2026-09-14), podman (16.15-1.pgdg13+2, 2026-09-14).
Quick Start (Standalone Docker Compose)
The snippet below is immediately ready to run in any standard Docker or Podman environment:
services:
litellm:
container_name: njorddeploy-litellm
image: "ghcr.io/berriai/litellm:main-stable"
ports:
- "4000:4000"
volumes:
- "data_root/litellm/config.yaml:/app/config.yaml:ro"
command:
- "--config=/app/config.yaml"
environment:
- "DATABASE_URL=postgresql://llmproxy:njorddeploy_db_password_123@njorddeploy-litellm-db:5432/litellm"
- "STORE_MODEL_IN_DB=True"
- "OPENAI_API_KEY=openai_api_key"
- "ANTHROPIC_API_KEY=anthropic_api_key"
- "COHERE_API_KEY=cohere_api_key"
- "AZURE_API_KEY=azure_api_key"
- "AZURE_API_BASE=https://YOUR_RESOURCE_NAME.openai.azure.com/"
- "AZURE_API_VERSION=2023-07-01-preview"
- "GEMINI_API_KEY=gemini_api_key"
- "HOSTYOURAI_API_KEY=hostyourai_api_key"
- "HOSTYOURAI_API_BASE=https://hostyourai.com/api/v1"
- "AWS_ACCESS_KEY_ID=aws_access_key_id"
- "AWS_SECRET_ACCESS_KEY=ChangeMeSecurePassword123!"
- "AWS_REGION_NAME=us-east-1"
networks:
- njorddeploy_net
depends_on:
- njorddeploy-litellm-db
healthcheck:
test: ["CMD-SHELL", "python3 -c \"import urllib.request; urllib.request.urlopen('http://localhost:4000/health/liveliness')\""]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
labels:
- "traefik.enable=true"
- "traefik.http.routers.litellm.rule=Host(`litellm.yourdomain.com`)"
- "traefik.http.routers.litellm.entrypoints=websecure"
- "traefik.http.routers.litellm.tls.certresolver=default"
- "traefik.http.services.litellm.loadbalancer.server.port=4000"
user: "0:0"
njorddeploy-litellm-db:
container_name: njorddeploy-litellm-db
image: postgres:16
restart: always
environment:
- "POSTGRES_DB=litellm"
- "POSTGRES_USER=llmproxy"
- "POSTGRES_PASSWORD=njorddeploy_db_password_123"
volumes:
- "./data/litellm/postgres_data:/var/lib/postgresql/data"
networks:
- njorddeploy_net
healthcheck:
test: ["CMD-SHELL", "pg_isready -d litellm -U llmproxy"]
interval: 1s
timeout: 5s
retries: 10
user: "0:0"
networks:
njorddeploy_net:
Start the service directly via the command line: docker compose up -d
Configuration & Environment Variables
Key configuration parameters and defaults derived from the NjordDeploy component template:
| Variable | Default Value | Description |
|---|---|---|
LITELLM_WEB_PORT |
4000 |
The external port for accessing the LiteLLM AI Gateway web interface. |
LITELLM_DB_NAME |
litellm |
The name of the PostgreSQL database for LiteLLM. |
LITELLM_DB_USER |
llmproxy |
The username for accessing the PostgreSQL database. |
LITELLM_DB_PASSWORD |
njorddeploy_db_password_123 |
The password for the PostgreSQL database user. |
LITELLM_STORE_MODEL_IN_DB |
True |
Set to 'True' to allow adding models to the proxy via the UI, 'False' otherwise. |
OPENAI_API_KEY |
*None* |
Your OpenAI API Key. Required for using OpenAI models. This key will be available as an environment variable for LiteLLM. |
ANTHROPIC_API_KEY |
*None* |
Your Anthropic API Key. Required for using Anthropic models. This key will be available as an environment variable for LiteLLM. |
COHERE_API_KEY |
*None* |
Your Cohere API Key. Required for using Cohere models. This key will be available as an environment variable for LiteLLM. |
AZURE_API_KEY |
*None* |
Your Azure OpenAI API Key. Required for using Azure OpenAI models. This key will be available as an environment variable for LiteLLM. |
AZURE_API_BASE |
https://YOUR_RESOURCE_NAME.openai.azure.com/ |
Your Azure OpenAI API Base URL (e.g., https://YOUR_RESOURCE_NAME.openai.azure.com/). Required for Azure OpenAI models. |
AZURE_API_VERSION |
2023-07-01-preview |
Your Azure OpenAI API Version (e.g., 2023-07-01-preview). Required for Azure OpenAI models. |
GEMINI_API_KEY |
*None* |
Your Google Gemini API Key. Required for using Google Gemini models. This key will be available as an environment variable for LiteLLM. |
HOSTYOURAI_API_KEY |
*None* |
Your HostYourAI API Key for accessing models on HostYourAI. |
HOSTYOURAI_API_BASE |
https://hostyourai.com/api/v1 |
The API Base URL for your HostYourAI instance or gateway endpoint. |
AWS_ACCESS_KEY_ID |
*None* |
Your AWS Access Key ID. Required for using AWS Bedrock/Sagemaker models. This key will be available as an environment variable for LiteLLM. |
AWS_SECRET_ACCESS_KEY |
*None* |
Your AWS Secret Access Key. Required for using AWS Bedrock/Sagemaker models. This key will be available as an environment variable for LiteLLM. |
AWS_REGION_NAME |
us-east-1 |
Your AWS Region Name (e.g., us-east-1). Required for using AWS Bedrock/Sagemaker models. This key will be available as an environment variable for LiteLLM. |
TRAEFIK_HOST |
litellm.yourdomain.com |
The hostname for Traefik to route traffic to LiteLLM (e.g., litellm.yourdomain.com). |
Ecosystem & Enterprise Integration
- Reverse Proxy Ingress Ready: Pre-configured for Caddy, Traefik, or Nginx Proxy Manager with automatic Let's Encrypt TLS certificates.
- Zero-Trust Mesh VPN: Seamless integration with WireGuard or Tailscale/Headscale mesh networks for secure remote administration.
- Transactional State Backups: Ready for point-in-time database dumps and container-safe persistent volume freezing.
Effortless Management with NjordDeploy
Deploy, monitor, and update this service with a single click on your own hardware via the NjordDeploy Configurator.