7 Commits
9 changed files with 325 additions and 39 deletions

No files matched your search

+45 -6
View File
@@ -1,11 +1,50 @@
.git/
# Python
__pycache__/
*.pyc
*.pyo
*.pyd
.venv/
venv/
*.egg-info/
.pytest_cache/
# Tests
backend/tests/
# Environment & secrets
.env
*.env.local
# Databases
*.db
*.sqlite3
# Logs
logs/
*.log
# IDE & tools
.idea/
.vscode/
.claude/
# Git
.git/
.gitignore
# Frontend
frontend/node_modules/
frontend/dist/
backend/__pycache__/
backend/**/__pycache__/
backend/*.pyc
backend/test.db
backend/tests/
# Docker
docker-compose.yml
.dockerignore
# Docs
*.md
# Dev & build scripts
run_dev.py
build_push.sh
start.sh
test_api.sh
+217
View File
@@ -0,0 +1,217 @@
# mediaeng/llmproxy
A lightweight reverse proxy for [Ollama](https://ollama.com) that manages API keys with configurable token and request quotas. Incoming requests in OpenAI-compatible format are authenticated, checked against the quota, and forwarded to the configured Ollama server.
Ollama does not need to run on the same host — `OLLAMA_URL` can point to any reachable server: the Docker host itself, another machine on the network, or a remote server.
## Features
- OpenAI-compatible endpoint (`/v1/chat/completions`, `/v1/models`)
- API key management with daily and monthly token/request limits
- Web-based admin interface (port 8001)
- Streaming support (Server-Sent Events)
- Tool use / function calling passthrough
- Rotating usage logs
- SQLite (default) or PostgreSQL
## Ports
| Port | Service |
|------|---------|
| `8000` | Proxy endpoint (OpenAI API) |
| `8001` | Admin API + web interface |
Port 8001 must be exposed because the container serves the admin interface directly on this port. To restrict access to the local machine, bind it to `127.0.0.1` — this makes the port reachable only from the host, not from the network:
```
ports:
- "127.0.0.1:8001:8001"
```
## Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `ADMIN_PASSWORD` | – | **Required.** Password for the admin interface |
| `OLLAMA_URL` | `http://localhost:11434` | URL of the Ollama server (without `/v1` suffix) |
| `DEFAULT_MODEL` | `llama3` | Model used when the client does not specify one |
| `DATABASE_URL` | `sqlite:///./test.db` | Database connection string (SQLite or PostgreSQL) |
| `PROXY_HOST` | `0.0.0.0` | Proxy bind address |
| `PROXY_PORT` | `8000` | Proxy port |
| `ADMIN_PORT` | `8001` | Admin API port |
| `APP_TZ` | `Europe/Berlin` | Timezone for daily/monthly quota resets |
| `LOG_FILE` | `logs/usage.log` | Path of the rotating usage log file |
## Docker Compose – External Ollama, SQLite
Use this when Ollama runs outside of Docker — on the Docker host or any other reachable server. Adjust `OLLAMA_URL` accordingly.
```yaml
services:
llmproxy:
image: mediaeng/llmproxy:latest
restart: unless-stopped
ports:
- "8000:8000"
- "127.0.0.1:8001:8001"
environment:
ADMIN_PASSWORD: changeme
OLLAMA_URL: http://host.docker.internal:11434 # or http://<ip>:11434
DEFAULT_MODEL: llama3
APP_TZ: Europe/Berlin
volumes:
- llmproxy-data:/app/backend
# On Linux, add extra_hosts since host.docker.internal is not
# available automatically:
# extra_hosts:
# - "host.docker.internal:host-gateway"
volumes:
llmproxy-data:
```
## Docker Compose – External Ollama, PostgreSQL
```yaml
services:
llmproxy:
image: mediaeng/llmproxy:latest
restart: unless-stopped
ports:
- "8000:8000"
- "127.0.0.1:8001:8001"
environment:
ADMIN_PASSWORD: changeme
OLLAMA_URL: http://host.docker.internal:11434 # or http://<ip>:11434
DEFAULT_MODEL: llama3
APP_TZ: Europe/Berlin
DATABASE_URL: postgresql://llmproxy:secret@db:5432/llmproxy
volumes:
- llmproxy-data:/app/backend
depends_on:
db:
condition: service_healthy
# extra_hosts:
# - "host.docker.internal:host-gateway"
db:
image: postgres:16-alpine
restart: unless-stopped
environment:
POSTGRES_DB: llmproxy
POSTGRES_USER: llmproxy
POSTGRES_PASSWORD: secret
volumes:
- pg-data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U llmproxy"]
interval: 5s
timeout: 5s
retries: 5
volumes:
pg-data:
```
## Docker Compose – Ollama as Container, SQLite
Ollama and llmproxy run together in Docker, data persisted in a volume.
```yaml
services:
llmproxy:
image: mediaeng/llmproxy:latest
restart: unless-stopped
ports:
- "8000:8000"
- "127.0.0.1:8001:8001"
environment:
ADMIN_PASSWORD: changeme
OLLAMA_URL: http://ollama:11434
DEFAULT_MODEL: llama3
APP_TZ: Europe/Berlin
volumes:
- llmproxy-data:/app/backend
depends_on:
- ollama
ollama:
image: ollama/ollama:latest
restart: unless-stopped
volumes:
- ollama-data:/root/.ollama
volumes:
llmproxy-data:
ollama-data:
```
## Docker Compose – Ollama as Container, PostgreSQL
For production environments with an external database.
```yaml
services:
llmproxy:
image: mediaeng/llmproxy:latest
restart: unless-stopped
ports:
- "8000:8000"
- "127.0.0.1:8001:8001"
environment:
ADMIN_PASSWORD: changeme
OLLAMA_URL: http://ollama:11434
DEFAULT_MODEL: llama3
APP_TZ: Europe/Berlin
DATABASE_URL: postgresql://llmproxy:secret@db:5432/llmproxy
depends_on:
db:
condition: service_healthy
ollama:
condition: service_started
db:
image: postgres:16-alpine
restart: unless-stopped
environment:
POSTGRES_DB: llmproxy
POSTGRES_USER: llmproxy
POSTGRES_PASSWORD: secret
volumes:
- pg-data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U llmproxy"]
interval: 5s
timeout: 5s
retries: 5
ollama:
image: ollama/ollama:latest
restart: unless-stopped
volumes:
- ollama-data:/root/.ollama
volumes:
pg-data:
ollama-data:
```
## Quick Start
```bash
docker run -d \
-p 8000:8000 \
-e ADMIN_PASSWORD=changeme \
-e OLLAMA_URL=http://host.docker.internal:11434 \
-v llmproxy-data:/app/backend \
mediaeng/llmproxy:latest
```
## Client Configuration
Configure the proxy as an OpenAI-compatible endpoint:
```
Base URL: http://<host>:8000/v1
API Key: <API key created in the admin interface>
```
+12 -1
View File
@@ -19,7 +19,14 @@ Ollama muss dabei nicht auf demselben Host laufen — `OLLAMA_URL` kann auf jede
| Port | Dienst |
|------|--------|
| `8000` | Proxy-Endpunkt (OpenAI-API) |
| `8001` | Admin-API + Web-Oberfläche (nicht exponieren) |
| `8001` | Admin-API + Web-Oberfläche |
Port 8001 muss exposed werden, da der Container die Admin-Oberfläche selbst auf diesem Port ausliefert. Um den Zugriff auf den lokalen Rechner zu beschränken, die Portbindung auf `127.0.0.1` setzen — so ist der Port nur vom Host erreichbar, nicht aus dem Netzwerk:
```
ports:
- "127.0.0.1:8001:8001"
```
## Umgebungsvariablen
@@ -46,6 +53,7 @@ services:
restart: unless-stopped
ports:
- "8000:8000"
- "127.0.0.1:8001:8001"
environment:
ADMIN_PASSWORD: changeme
OLLAMA_URL: http://host.docker.internal:11434 # oder http://<ip>:11434
@@ -71,6 +79,7 @@ services:
restart: unless-stopped
ports:
- "8000:8000"
- "127.0.0.1:8001:8001"
environment:
ADMIN_PASSWORD: changeme
OLLAMA_URL: http://host.docker.internal:11434 # oder http://<ip>:11434
@@ -115,6 +124,7 @@ services:
restart: unless-stopped
ports:
- "8000:8000"
- "127.0.0.1:8001:8001"
environment:
ADMIN_PASSWORD: changeme
OLLAMA_URL: http://ollama:11434
@@ -147,6 +157,7 @@ services:
restart: unless-stopped
ports:
- "8000:8000"
- "127.0.0.1:8001:8001"
environment:
ADMIN_PASSWORD: changeme
OLLAMA_URL: http://ollama:11434
+2
View File
@@ -47,6 +47,8 @@ async def read_api_keys(
item.tokens_used_month = usage.tokens_used_month or 0
item.requests_today = usage.requests_today or 0
item.requests_month = usage.requests_month or 0
item.daily_reset_at = usage.daily_reset_at
item.monthly_reset_at = usage.monthly_reset_at
result.append(item)
return result
+2
View File
@@ -58,6 +58,8 @@ class APIKeyWithUsage(APIKey):
tokens_used_month: int = 0
requests_today: int = 0
requests_month: int = 0
daily_reset_at: Optional[datetime] = None
monthly_reset_at: Optional[datetime] = None
class Config:
from_attributes = True
+13 -3
View File
@@ -5,13 +5,23 @@ cd "$(dirname "$0")"
IMAGE=mediaeng/llmproxy
PLATFORM=linux/arm64
VERSION=$(git describe --tags --always)
if [ -z "$VERSION" ]; then
CURRENT=$(git describe --tags --always)
if [ -z "$CURRENT" ]; then
echo "Fehler: git describe liefert kein Ergebnis"
exit 1
fi
echo "Version : $VERSION"
echo "Aktueller Tag: $CURRENT"
read -rp "Neuer Tag [${CURRENT}]: " INPUT
VERSION="${INPUT:-$CURRENT}"
if [ "$VERSION" != "$CURRENT" ]; then
git tag "$VERSION"
git push origin "$VERSION"
echo "Tag '$VERSION' gesetzt und gepusht."
fi
echo ""
echo "Image : $IMAGE"
echo "Platform: $PLATFORM"
echo "Tags : $IMAGE:$VERSION $IMAGE:latest"
+9 -22
View File
@@ -1,24 +1,11 @@
backend:
build:
context: ./backend
dockerfile: Dockerfile
services:
llmproxy:
image: mediaeng/llmproxy:latest
restart: unless-stopped
env_file: .env
ports:
- "8000:8000"
environment:
- DATABASE_URL=postgresql://ollama:password@db:5432/ollama_proxy
- OLLAMA_URL=http://ollama:11434
- SECRET_KEY=your-secret-key-change-me
depends_on:
- db
db:
image: postgres:16
environment:
- POSTGRES_USER=ollama
- POSTGRES_PASSWORD=password
- POSTGRES_DB=ollama_proxy
- "${PROXY_PORT:-8000}:${PROXY_PORT:-8000}"
- "127.0.0.1:8001:8001"
volumes:
- postgres_data:/var/lib/postgresql/data
volumes:
postgres_data:
- ./backend/test.db:/app/backend/test.db
- ./backend/logs:/app/backend/logs
+18 -7
View File
@@ -11,17 +11,28 @@ function authHeaders(token) {
const fmtK = (n) => { const k = n / 1000; return k % 1 === 0 ? `${k}k` : `${k.toFixed(1)}k`; };
function QuotaBar({ used, limit, isToken = false }) {
if (limit == null) return <span className="quota-unlimited">∞</span>;
function QuotaBar({ used, limit, isToken = false, since = null }) {
const fmt = isToken ? fmtK : (n) => n.toLocaleString('de-DE');
const sinceLabel = since
? new Date(since).toLocaleDateString('de-DE', { day: '2-digit', month: '2-digit' })
: null;
if (limit == null) return (
<div className="quota-cell">
<span className="quota-unlimited">∞</span>
{sinceLabel && <span className="quota-since">seit {sinceLabel}</span>}
</div>
);
const pct = Math.min(100, (used / limit) * 100);
const color = pct >= 90 ? '#e74c3c' : pct >= 70 ? '#e67e22' : '#27ae60';
const fmt = isToken ? fmtK : (n) => n.toLocaleString('de-DE');
return (
<div className="quota-cell">
<span className="quota-label">{fmt(used)} / {fmt(limit)}</span>
<div className="progress-bar">
<div className="progress-fill" style={{ width: `${pct}%`, backgroundColor: color }} />
</div>
{sinceLabel && <span className="quota-since">seit {sinceLabel}</span>}
</div>
);
}
@@ -400,10 +411,10 @@ function App() {
<td>{key.name}</td>
<td>{displayKey(key.key_prefix)}</td>
<td>{key.expires_at ? new Date(key.expires_at).toLocaleDateString('de-DE', { timeZone: 'Europe/Berlin' }) : '∞'}</td>
<td><QuotaBar used={key.tokens_used_today} limit={key.daily_tokens} isToken /></td>
<td><QuotaBar used={key.tokens_used_month} limit={key.monthly_tokens} isToken /></td>
<td><QuotaBar used={key.requests_today} limit={key.daily_requests} /></td>
<td><QuotaBar used={key.requests_month} limit={key.monthly_requests} /></td>
<td><QuotaBar used={key.tokens_used_today} limit={key.daily_tokens} isToken since={key.daily_reset_at} /></td>
<td><QuotaBar used={key.tokens_used_month} limit={key.monthly_tokens} isToken since={key.monthly_reset_at} /></td>
<td><QuotaBar used={key.requests_today} limit={key.daily_requests} since={key.daily_reset_at} /></td>
<td><QuotaBar used={key.requests_month} limit={key.monthly_requests} since={key.monthly_reset_at} /></td>
<td className="action-cell">
<button className="btn-icon btn-icon-edit" data-tooltip="Bearbeiten" onClick={() => handleEdit(key)}>✏</button>
{key.is_active ? (
+7
View File
@@ -246,6 +246,13 @@ tr:hover {
font-size: 14px;
}
.quota-since {
display: block;
font-size: 10px;
color: #aaa;
margin-top: 2px;
}
.progress-bar {
height: 4px;
background: #e2e8f0;