Open WebUI & Ollama AI Studio

Integrated sovereign AI chatbot and inference platform combining Open WebUI with the local Ollama LLM engine and LiteLLM unified gateway.

3 Verified Services

100% Proxmox Tested (Docker & Podman)

Isolated Subnet (Zero Port Conflicts)

Local Data Storage

Key Architectural Capabilities

Local AI inference with Ollama (Llama, Mistral, Qwen, DeepSeek)
Open WebUI interface with document support and multi-modal vision
LiteLLM OpenAI-compatible API gateway and proxy
Supports GPU passthrough or CPU-only execution
Zero API token costs and local data confidentiality

Included Services in Stack

All containers in this suite are pre-configured with bridge networking, environment variables, and persistent volume paths.

Ollama LLM Engine

ollama

Ollama is an open-source, lightweight, and extensible framework for running Large Language Models (LLMs) locally, such as Llama 3, Qwen, Mistral, and DeepSeek.

Open WebUI

open-webui

Open WebUI is an extensible, feature-rich, and user-friendly web interface for AI models and LLMs, supporting Ollama and OpenAI-compatible APIs with chat, voice, and RAG capabilities.

LiteLLM AI Gateway

litellm

LiteLLM is an AI gateway and LLM proxy that unifies access to 100+ Large Language Models (OpenAI, Gemini, Anthropic, Ollama, Azure, Bedrock, HostYourAI) behind a single OpenAI-compatible API format.

Deploy with NjordDeploy

NjordDeploy handles the deployment of this multi-container suite over SSH. Compose templates are rendered and started on Docker or Rootless Podman without host agents.

Download NjordDeploy Suite View Proxmox Test Report Inspect Compose Templates