7 Commits
Author SHA1 Message Date
hofmannol 3974010156 Use network_mode: host so container can reach Ollama on 127.0.0.1 2026-05-07 13:58:26 +02:00
hofmannol 8d3f9a7661 Fix OpenAI array content, add error logging, Ollama reachability warning
- Normalize OpenAI array-format content to string to fix connection reset
- Add error.log with rotating handler for proxy and stream errors
- Add global unhandled exception handler returning JSON 500
- Write OLLAMA_URL/DEFAULT_MODEL env vars to DB on startup (reset on restart)
- Add extra_hosts to docker-compose.yml for host.docker.internal on Linux
- Show warning in admin UI when Ollama URL is unreachable
- Return reachable: true/false from /api/ollama-models endpoint
2026-05-07 11:43:17 +02:00
Oliver Hofmann 280b3b0762 Add open: true to Vite dev server config 2026-04-29 17:13:14 +02:00
Oliver Hofmann 89661dafcc Update README: new endpoints, Docker Hub, streaming, PyCharm config, port 8001 security 2026-04-29 10:19:31 +02:00
Oliver Hofmann 3ccc7f325d Explain port 8001 security model in docs 2026-04-29 10:17:59 +02:00
Oliver Hofmann 25f19b6ada Show reset date below quota progress bars in admin UI 2026-04-29 09:55:25 +02:00
Oliver Hofmann 5b97ed0ef7 Expose port 8001 on 127.0.0.1 only, explain why in docs 2026-04-29 09:52:13 +02:00
10 changed files with 246 additions and 112 deletions

No files matched your search

+16 -1
View File
@@ -19,7 +19,18 @@ Ollama does not need to run on the same host — `OLLAMA_URL` can point to any r
| Port | Service | | Port | Service |
|------|---------| |------|---------|
| `8000` | Proxy endpoint (OpenAI API) | | `8000` | Proxy endpoint (OpenAI API) |
| `8001` | Admin API + web interface (do not expose) | | `8001` | Admin API + web interface |
Port 8001 must be exposed because the container serves the admin interface directly on this port. All API endpoints require the `ADMIN_PASSWORD` — without a valid token, only the public frontend files (HTML/JS/CSS of the login page) are accessible. The password is therefore the primary protection.
Additional hardening: binding to `127.0.0.1` restricts access to the local host and prevents direct network access:
```
ports:
- "127.0.0.1:8001:8001" # local access only
# or:
- "8001:8001" # network-wide, protected by ADMIN_PASSWORD only
```
## Environment Variables ## Environment Variables
@@ -46,6 +57,7 @@ services:
restart: unless-stopped restart: unless-stopped
ports: ports:
- "8000:8000" - "8000:8000"
- "127.0.0.1:8001:8001"
environment: environment:
ADMIN_PASSWORD: changeme ADMIN_PASSWORD: changeme
OLLAMA_URL: http://host.docker.internal:11434 # or http://<ip>:11434 OLLAMA_URL: http://host.docker.internal:11434 # or http://<ip>:11434
@@ -71,6 +83,7 @@ services:
restart: unless-stopped restart: unless-stopped
ports: ports:
- "8000:8000" - "8000:8000"
- "127.0.0.1:8001:8001"
environment: environment:
ADMIN_PASSWORD: changeme ADMIN_PASSWORD: changeme
OLLAMA_URL: http://host.docker.internal:11434 # or http://<ip>:11434 OLLAMA_URL: http://host.docker.internal:11434 # or http://<ip>:11434
@@ -115,6 +128,7 @@ services:
restart: unless-stopped restart: unless-stopped
ports: ports:
- "8000:8000" - "8000:8000"
- "127.0.0.1:8001:8001"
environment: environment:
ADMIN_PASSWORD: changeme ADMIN_PASSWORD: changeme
OLLAMA_URL: http://ollama:11434 OLLAMA_URL: http://ollama:11434
@@ -147,6 +161,7 @@ services:
restart: unless-stopped restart: unless-stopped
ports: ports:
- "8000:8000" - "8000:8000"
- "127.0.0.1:8001:8001"
environment: environment:
ADMIN_PASSWORD: changeme ADMIN_PASSWORD: changeme
OLLAMA_URL: http://ollama:11434 OLLAMA_URL: http://ollama:11434
+16 -1
View File
@@ -19,7 +19,18 @@ Ollama muss dabei nicht auf demselben Host laufen — `OLLAMA_URL` kann auf jede
| Port | Dienst | | Port | Dienst |
|------|--------| |------|--------|
| `8000` | Proxy-Endpunkt (OpenAI-API) | | `8000` | Proxy-Endpunkt (OpenAI-API) |
| `8001` | Admin-API + Web-Oberfläche (nicht exponieren) | | `8001` | Admin-API + Web-Oberfläche |
Port 8001 muss exposed werden, da der Container die Admin-Oberfläche selbst auf diesem Port ausliefert. Alle API-Endpunkte erfordern das `ADMIN_PASSWORD` — ein Zugriff ohne gültiges Token liefert nur die öffentlichen Frontend-Dateien (HTML/JS/CSS der Login-Seite). Das Passwort ist damit die primäre Schutzmaßnahme.
Zusätzliche Härtung: Portbindung auf `127.0.0.1` beschränkt den Zugriff auf den lokalen Host und verhindert direkten Netzwerkzugriff:
```
ports:
- "127.0.0.1:8001:8001" # nur lokal erreichbar
# oder:
- "8001:8001" # netzwerkweit, Schutz nur durch ADMIN_PASSWORD
```
## Umgebungsvariablen ## Umgebungsvariablen
@@ -46,6 +57,7 @@ services:
restart: unless-stopped restart: unless-stopped
ports: ports:
- "8000:8000" - "8000:8000"
- "127.0.0.1:8001:8001"
environment: environment:
ADMIN_PASSWORD: changeme ADMIN_PASSWORD: changeme
OLLAMA_URL: http://host.docker.internal:11434 # oder http://<ip>:11434 OLLAMA_URL: http://host.docker.internal:11434 # oder http://<ip>:11434
@@ -71,6 +83,7 @@ services:
restart: unless-stopped restart: unless-stopped
ports: ports:
- "8000:8000" - "8000:8000"
- "127.0.0.1:8001:8001"
environment: environment:
ADMIN_PASSWORD: changeme ADMIN_PASSWORD: changeme
OLLAMA_URL: http://host.docker.internal:11434 # oder http://<ip>:11434 OLLAMA_URL: http://host.docker.internal:11434 # oder http://<ip>:11434
@@ -115,6 +128,7 @@ services:
restart: unless-stopped restart: unless-stopped
ports: ports:
- "8000:8000" - "8000:8000"
- "127.0.0.1:8001:8001"
environment: environment:
ADMIN_PASSWORD: changeme ADMIN_PASSWORD: changeme
OLLAMA_URL: http://ollama:11434 OLLAMA_URL: http://ollama:11434
@@ -147,6 +161,7 @@ services:
restart: unless-stopped restart: unless-stopped
ports: ports:
- "8000:8000" - "8000:8000"
- "127.0.0.1:8001:8001"
environment: environment:
ADMIN_PASSWORD: changeme ADMIN_PASSWORD: changeme
OLLAMA_URL: http://ollama:11434 OLLAMA_URL: http://ollama:11434
+59 -72
View File
@@ -8,20 +8,22 @@ Ein Reverse-Proxy für Ollama mit API-Key-Authentifizierung, Quota-Management un
- Optionales Ablaufdatum pro API-Key - Optionales Ablaufdatum pro API-Key
- Quota-Management mit getrennten Tages- und Monatslimits (Tokens & Requests) - Quota-Management mit getrennten Tages- und Monatslimits (Tokens & Requests)
- Token-Zählung via tiktoken, Reset-Grenzen in der Zeitzone Europe/Berlin - Token-Zählung via tiktoken, Reset-Grenzen in der Zeitzone Europe/Berlin
- Web-Admin-Oberfläche (API-Keys verwalten, Ollama-Einstellungen, Proxy-Info) - Web-Admin-Oberfläche (API-Keys verwalten, Ollama-Einstellungen, Verbrauchsanzeige)
- OpenAI-kompatibler `/v1/chat/completions`-Endpunkt - OpenAI-kompatibler `/v1/chat/completions`-Endpunkt mit Streaming und Tool-Use
- Rotierende Nutzungs-Logs
- SQLite (Standard) oder PostgreSQL
- Docker-Image auf DockerHub: `mediaeng/llmproxy`
## Sicherheit ## Sicherheit
- Admin-Oberfläche passwortgeschützt (`ADMIN_PASSWORD`) - Admin-Oberfläche passwortgeschützt (`ADMIN_PASSWORD`) — alle API-Endpunkte erfordern den Token
- Admin-API bindet lokal auf `127.0.0.1` (nicht von außen erreichbar)
- API-Keys als SHA-256-Hash in der DB — Plaintext nur einmalig bei Erstellung - API-Keys als SHA-256-Hash in der DB — Plaintext nur einmalig bei Erstellung
- Quota-Check atomar mit `SELECT FOR UPDATE` (kein TOCTOU-Race) - Quota-Check atomar mit `SELECT FOR UPDATE` (kein TOCTOU-Race)
- CORS-Origins konfigurierbar via `ALLOWED_ORIGINS` - Port 8001 kann optional auf `127.0.0.1` gebunden werden (zusätzliche Härtung)
## Konfiguration ## Konfiguration
`.env`-Datei im Projektverzeichnis anlegen (Vorlage: `.env.example`): `.env`-Datei im Projektverzeichnis anlegen:
```env ```env
ADMIN_PASSWORD=change-me ADMIN_PASSWORD=change-me
@@ -32,6 +34,7 @@ DATABASE_URL=sqlite:///./test.db
OLLAMA_URL=http://localhost:11434 OLLAMA_URL=http://localhost:11434
DEFAULT_MODEL=llama3 DEFAULT_MODEL=llama3
APP_TZ=Europe/Berlin APP_TZ=Europe/Berlin
LOG_FILE=logs/usage.log
``` ```
| Variable | Standard | Beschreibung | | Variable | Standard | Beschreibung |
@@ -40,25 +43,15 @@ APP_TZ=Europe/Berlin
| `PROXY_HOST` | `0.0.0.0` | Bind-Adresse des Proxys | | `PROXY_HOST` | `0.0.0.0` | Bind-Adresse des Proxys |
| `PROXY_PORT` | `8000` | Port des Proxys | | `PROXY_PORT` | `8000` | Port des Proxys |
| `ADMIN_PORT` | `8001` | Port der Admin-API | | `ADMIN_PORT` | `8001` | Port der Admin-API |
| `DATABASE_URL` | `sqlite:///./test.db` | DB-Verbindungsstring | | `DATABASE_URL` | `sqlite:///./test.db` | DB-Verbindungsstring (SQLite oder PostgreSQL) |
| `OLLAMA_URL` | `http://localhost:11434` | Adresse der Ollama-Instanz (auch in der UI änderbar) | | `OLLAMA_URL` | `http://localhost:11434` | Adresse der Ollama-Instanz (auch in der UI änderbar) |
| `DEFAULT_MODEL` | `llama3` | Standard-Modell für `/v1/chat/completions` (auch in der UI änderbar) | | `DEFAULT_MODEL` | `llama3` | Standard-Modell für `/v1/chat/completions` (auch in der UI änderbar) |
| `APP_TZ` | `Europe/Berlin` | Zeitzone für tägliche/monatliche Quota-Resets | | `APP_TZ` | `Europe/Berlin` | Zeitzone für tägliche/monatliche Quota-Resets |
| `ALLOWED_ORIGINS` | `http://localhost:5173` | Kommagetrennte CORS-Origins | | `LOG_FILE` | `logs/usage.log` | Pfad der rotierenden Nutzungs-Logdatei |
| `ALLOWED_ORIGINS` | `http://localhost:5173` | CORS-Origins (nur für Entwicklung relevant) |
## Entwicklung (lokal) ## Entwicklung (lokal)
```bash
cp .env.example .env
# ADMIN_PASSWORD in .env setzen
./start.sh
```
Das Script prüft alle Ports auf Belegung, aktiviert automatisch eine vorhandene `.venv`, initialisiert die Datenbank und startet Proxy, Admin-API und Vite-Dev-Server.
Admin-Oberfläche: `http://localhost:5173`
### Voraussetzungen ### Voraussetzungen
- Python 3.12+ mit virtualenv - Python 3.12+ mit virtualenv
@@ -68,63 +61,50 @@ Admin-Oberfläche: `http://localhost:5173`
python -m venv .venv python -m venv .venv
source .venv/bin/activate source .venv/bin/activate
pip install -r backend/requirements-dev.txt pip install -r backend/requirements-dev.txt
cd frontend && npm install cd frontend && npm install
``` ```
### Starten
**Per Script:**
```bash
cp .env.example .env # ADMIN_PASSWORD setzen
./start.sh
```
**Per PyCharm:** Run-Config „Dev" starten (startet Proxy, Admin-API und Vite-Dev-Server gemeinsam).
Das Script prüft alle Ports auf Belegung, initialisiert die Datenbank und startet alle drei Dienste.
Admin-Oberfläche: `http://localhost:5173`
## Produktion (Docker) ## Produktion (Docker)
### Image bauen ### Docker Compose (empfohlen)
```bash ```bash
docker build -t llm-quota . docker compose up -d
``` ```
### Container starten Zieht das Image von DockerHub, lädt Variablen aus `.env` und verwendet die lokale SQLite-Datenbank. Weitere Compose-Varianten (PostgreSQL, Ollama als Container) siehe `DOCKERHUB.md`.
### Image selbst bauen und pushen
```bash ```bash
docker run -d \ ./build_push.sh
-p 8000:8000 \
-p 127.0.0.1:8001:8001 \
-e ADMIN_PASSWORD=geheim \
-e OLLAMA_URL=http://host.docker.internal:11434 \
-e DATABASE_URL=sqlite:///./data/quota.db \
-v $(pwd)/data:/app/backend/data \
--name llm-quota \
llm-quota
``` ```
| Port | Bindung | Dienst | Das Script zeigt den aktuellen Git-Tag, bietet an einen neuen zu setzen, baut das Image für `linux/arm64` und pusht zu `mediaeng/llmproxy`.
|------|---------|--------|
| `8000` | `0.0.0.0` — öffentlich | Proxy (für LLM-Clients) |
| `8001` | `127.0.0.1` — nur lokal am Server | Admin-API + Admin-Oberfläche |
Docker unterscheidet beim Port-Mapping zwischen `0.0.0.0` (alle Interfaces, öffentlich erreichbar) und `127.0.0.1` (nur der Server selbst kann zugreifen). Mit `-p 127.0.0.1:8001:8001` ist Port 8001 am Server verfügbar, aber von außen nicht direkt ansprechbar. ### Port 8001 (Admin)
### Admin-Oberfläche per SSH-Tunnel erreichbar machen Port 8001 muss exposed werden, da der Container die Admin-Oberfläche auf diesem Port ausliefert. Alle API-Endpunkte erfordern das `ADMIN_PASSWORD` — der Token ist der primäre Schutz. Optionale zusätzliche Härtung: Bindung auf `127.0.0.1`:
Der SSH-Tunnel leitet einen lokalen Port auf den Server weiter und nutzt dabei, dass Port 8001 dort auf `127.0.0.1` erreichbar ist: ```yaml
ports:
``` - "127.0.0.1:8001:8001" # nur lokal
Admin-Laptop:8001 ──SSH──► Server:127.0.0.1:8001 ──► Container:8001 # oder:
``` - "8001:8001" # netzwerkweit, Schutz durch ADMIN_PASSWORD
```bash
ssh -L 8001:localhost:8001 user@server
```
Danach ist die Admin-Oberfläche auf dem Laptop unter `http://localhost:8001` erreichbar — ohne dass Port 8001 öffentlich exponiert wird.
### Mit PostgreSQL
```bash
docker run -d \
-p 8000:8000 \
-p 127.0.0.1:8001:8001 \
-e ADMIN_PASSWORD=geheim \
-e DATABASE_URL=postgresql://user:pass@db-host:5432/llm_quota \
-e OLLAMA_URL=http://ollama:11434 \
llm-quota
``` ```
## Proxy-Endpunkte (Port 8000) ## Proxy-Endpunkte (Port 8000)
@@ -132,7 +112,7 @@ docker run -d \
Alle Endpunkte erfordern einen gültigen API-Key im `Authorization`-Header. Alle Endpunkte erfordern einen gültigen API-Key im `Authorization`-Header.
```bash ```bash
curl -X POST http://localhost:8000/api/chat \ curl -X POST http://localhost:8000/v1/chat/completions \
-H "Authorization: Bearer sk-xxxxxx" \ -H "Authorization: Bearer sk-xxxxxx" \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{"model":"llama3","messages":[{"role":"user","content":"Hallo"}]}' -d '{"model":"llama3","messages":[{"role":"user","content":"Hallo"}]}'
@@ -140,12 +120,12 @@ curl -X POST http://localhost:8000/api/chat \
| Endpunkt | Methode | Beschreibung | | Endpunkt | Methode | Beschreibung |
|----------|---------|--------------| |----------|---------|--------------|
| `/api/generate` | POST | Ollama generate | | `/v1/chat/completions` | POST | Chat (OpenAI-Format, Streaming + Tool-Use) |
| `/api/chat` | POST | Ollama chat | | `/v1/models` | GET | Modelle (OpenAI-Format) |
| `/api/generate` | POST | Ollama generate (nativ) |
| `/api/chat` | POST | Ollama chat (nativ) |
| `/api/tags` | GET | Verfügbare Modelle | | `/api/tags` | GET | Verfügbare Modelle |
| `/api/versions` | GET | Ollama-Version | | `/api/versions` | GET | Ollama-Version |
| `/v1/models` | GET | Modelle (OpenAI-Format) |
| `/v1/chat/completions` | POST | Chat (OpenAI-Format) |
## Admin-API (Port 8001) ## Admin-API (Port 8001)
@@ -153,10 +133,12 @@ Alle Endpunkte erfordern `Authorization: Bearer <ADMIN_PASSWORD>`.
| Endpunkt | Methode | Beschreibung | | Endpunkt | Methode | Beschreibung |
|----------|---------|--------------| |----------|---------|--------------|
| `/api/api-keys` | GET | Alle API-Keys auflisten | | `/api/api-keys` | GET | Alle API-Keys mit Verbrauchsdaten |
| `/api/api-keys` | POST | Neuen API-Key erstellen | | `/api/api-keys` | POST | Neuen API-Key erstellen |
| `/api/api-keys/{id}/quota` | PATCH | Limits eines Keys aktualisieren |
| `/api/api-keys/{id}/activate` | PUT | API-Key aktivieren |
| `/api/api-keys/{id}/deactivate` | PUT | API-Key deaktivieren | | `/api/api-keys/{id}/deactivate` | PUT | API-Key deaktivieren |
| `/api/api-keys/{id}/quota` | PATCH | Quota eines Keys aktualisieren | | `/api/api-keys/{id}` | DELETE | API-Key löschen |
| `/api/settings` | GET/PUT | Ollama-URL und Standard-Modell | | `/api/settings` | GET/PUT | Ollama-URL und Standard-Modell |
| `/api/ollama-models` | GET | Verfügbare Modelle von Ollama | | `/api/ollama-models` | GET | Verfügbare Modelle von Ollama |
| `/api/proxy-info` | GET | Lokaler Proxy-Endpunkt | | `/api/proxy-info` | GET | Lokaler Proxy-Endpunkt |
@@ -180,22 +162,27 @@ llm_quota/
│ ├── schemas.py # Pydantic-Schemas │ ├── schemas.py # Pydantic-Schemas
│ ├── crud.py # DB-Operationen, Token-Zählung, Quota-Logik │ ├── crud.py # DB-Operationen, Token-Zählung, Quota-Logik
│ ├── init_db.py # Tabellen anlegen & Settings seeden │ ├── init_db.py # Tabellen anlegen & Settings seeden
│ ├── setup_admin.py # Standard-API-Key erstellen
│ ├── requirements.txt # Produktiv-Dependencies │ ├── requirements.txt # Produktiv-Dependencies
│ ├── requirements-dev.txt # Test-Dependencies │ ├── requirements-dev.txt # Test-Dependencies
│ └── tests/ │ └── tests/
│ ├── conftest.py # Fixtures │ ├── conftest.py
│ ├── test_auth.py # Authentifizierungs-Tests │ ├── test_auth.py
│ └── test_quota.py # Quota-, Token- und Ablauf-Tests │ └── test_quota.py
├── frontend/ ├── frontend/
│ └── src/ │ └── src/
│ ├── main.jsx # React-Admin-UI │ ├── main.jsx # React-Admin-UI
│ └── styles.css │ └── styles.css
├── .idea/runConfigurations/
│ └── Dev.xml # PyCharm Run-Config
├── Dockerfile ├── Dockerfile
├── docker-compose.yml # Produktiv-Start mit DockerHub-Image
├── docker-entrypoint.sh ├── docker-entrypoint.sh
├── .dockerignore ├── .dockerignore
├── .env.example
├── start.sh # Entwicklungs-Startscript ├── start.sh # Entwicklungs-Startscript
├── run_dev.py # Entwicklungs-Runner für PyCharm
├── build_push.sh # Docker-Build & Push zu DockerHub
├── DOCKERHUB.md # DockerHub-Beschreibung (deutsch)
├── DOCKERHUB.en.md # DockerHub-Beschreibung (englisch)
└── .gitignore └── .gitignore
``` ```
+4 -2
View File
@@ -47,6 +47,8 @@ async def read_api_keys(
item.tokens_used_month = usage.tokens_used_month or 0 item.tokens_used_month = usage.tokens_used_month or 0
item.requests_today = usage.requests_today or 0 item.requests_today = usage.requests_today or 0
item.requests_month = usage.requests_month or 0 item.requests_month = usage.requests_month or 0
item.daily_reset_at = usage.daily_reset_at
item.monthly_reset_at = usage.monthly_reset_at
result.append(item) result.append(item)
return result return result
@@ -160,9 +162,9 @@ async def get_ollama_models(
async with httpx.AsyncClient(timeout=5.0) as client: async with httpx.AsyncClient(timeout=5.0) as client:
response = await client.get(f"{ollama_url}/api/tags") response = await client.get(f"{ollama_url}/api/tags")
models = [m["name"] for m in response.json().get("models", [])] models = [m["name"] for m in response.json().get("models", [])]
return {"models": models, "reachable": True}
except Exception: except Exception:
models = [] return {"models": [], "reachable": False}
return {"models": models}
# Statisches Frontend ausliefern (nur im Produktivbetrieb, wenn dist/ existiert) # Statisches Frontend ausliefern (nur im Produktivbetrieb, wenn dist/ existiert)
_dist = Path(__file__).parent.parent / "frontend" / "dist" _dist = Path(__file__).parent.parent / "frontend" / "dist"
+77 -18
View File
@@ -10,20 +10,40 @@ from database import get_db
import crud import crud
import httpx import httpx
_log_dir = Path(os.getenv("LOG_FILE", "logs/usage.log")).parent
_log_dir.mkdir(parents=True, exist_ok=True)
_fmt = logging.Formatter("%(asctime)s | %(message)s", datefmt="%Y-%m-%d %H:%M:%S")
# Rotating usage log (8 KB per file, 3 backups) # Rotating usage log (8 KB per file, 3 backups)
_log_path = Path(os.getenv("LOG_FILE", "logs/usage.log")) _usage_handler = RotatingFileHandler(str(_log_dir / "usage.log"), maxBytes=8192, backupCount=3, encoding="utf-8")
_log_path.parent.mkdir(parents=True, exist_ok=True) _usage_handler.setFormatter(_fmt)
_handler = RotatingFileHandler(str(_log_path), maxBytes=8192, backupCount=3, encoding="utf-8")
_handler.setFormatter(logging.Formatter("%(asctime)s | %(message)s", datefmt="%Y-%m-%d %H:%M:%S"))
usage_log = logging.getLogger("proxy.usage") usage_log = logging.getLogger("proxy.usage")
usage_log.setLevel(logging.INFO) usage_log.setLevel(logging.INFO)
usage_log.addHandler(_handler) usage_log.addHandler(_usage_handler)
usage_log.propagate = False usage_log.propagate = False
# Rotating error log (64 KB per file, 5 backups)
_error_handler = RotatingFileHandler(str(_log_dir / "error.log"), maxBytes=65536, backupCount=5, encoding="utf-8")
_error_handler.setFormatter(_fmt)
error_log = logging.getLogger("proxy.error")
error_log.setLevel(logging.ERROR)
error_log.addHandler(_error_handler)
error_log.propagate = False
def _content_to_str(content) -> str:
"""Normalize OpenAI content: string or array of content parts → plain string."""
if isinstance(content, list):
return " ".join(
part.get("text", "") if isinstance(part, dict) else str(part)
for part in content
)
return content or ""
def _last_user_msg(messages: list, max_len: int = 120) -> str: def _last_user_msg(messages: list, max_len: int = 120) -> str:
for msg in reversed(messages): for msg in reversed(messages):
if msg.get("role") == "user": if msg.get("role") == "user":
text = (msg.get("content") or "").replace("\n", " ").strip() text = _content_to_str(msg.get("content")).replace("\n", " ").strip()
return text[:max_len] + ("…" if len(text) > max_len else "") return text[:max_len] + ("…" if len(text) > max_len else "")
return "" return ""
@@ -43,6 +63,25 @@ async def require_api_key(request: Request, db: Session = Depends(get_db)):
app = FastAPI(title="Ollama Proxy", dependencies=[Depends(require_api_key)]) app = FastAPI(title="Ollama Proxy", dependencies=[Depends(require_api_key)])
@app.on_event("startup")
def apply_env_settings():
"""Write env-configured values into DB so they take effect until next restart."""
db = next(get_db())
try:
if url := os.getenv("OLLAMA_URL"):
crud.set_setting(db, "ollama_url", url)
if model := os.getenv("DEFAULT_MODEL"):
crud.set_setting(db, "default_model", model)
db.commit()
finally:
db.close()
@app.exception_handler(Exception)
async def unhandled_exception_handler(request: Request, exc: Exception):
error_log.error("Unhandled exception | %s %s | %s: %s",
request.method, request.url.path, type(exc).__name__, exc, exc_info=exc)
return JSONResponse(status_code=500, content={"error": {"message": "Internal server error", "type": "server_error"}})
async def proxy_request(url: str, method: str = "GET", json_data: dict = None): async def proxy_request(url: str, method: str = "GET", json_data: dict = None):
async with httpx.AsyncClient(timeout=300.0) as client: async with httpx.AsyncClient(timeout=300.0) as client:
response = await client.request(method=method, url=url, json=json_data) response = await client.request(method=method, url=url, json=json_data)
@@ -60,23 +99,33 @@ async def generate(request: Request, db: Session = Depends(get_db)):
prompt_preview = (body.get("prompt", "").replace("\n", " ").strip())[:120] prompt_preview = (body.get("prompt", "").replace("\n", " ").strip())[:120]
usage_log.info('%s | /api/generate | %s | ~%d tokens | "%s"', usage_log.info('%s | /api/generate | %s | ~%d tokens | "%s"',
request.state.api_key_name, body.get("model", "?"), prompt_tokens, prompt_preview) request.state.api_key_name, body.get("model", "?"), prompt_tokens, prompt_preview)
response = await proxy_request(f"{ollama_url}/api/generate", method="POST", json_data=body) try:
return JSONResponse(content=response.json(), status_code=response.status_code) response = await proxy_request(f"{ollama_url}/api/generate", method="POST", json_data=body)
return JSONResponse(content=response.json(), status_code=response.status_code)
except Exception as exc:
error_log.error("Proxy error | %s | /api/generate | %s | %s: %s",
request.state.api_key_name, body.get("model", "?"), type(exc).__name__, exc, exc_info=exc)
raise
@app.post("/api/chat") @app.post("/api/chat")
async def chat(request: Request, db: Session = Depends(get_db)): async def chat(request: Request, db: Session = Depends(get_db)):
ollama_url = crud.get_setting(db, "ollama_url", os.getenv("OLLAMA_URL", "http://localhost:11434")) ollama_url = crud.get_setting(db, "ollama_url", os.getenv("OLLAMA_URL", "http://localhost:11434"))
body = await request.json() body = await request.json()
messages = body.get("messages", []) messages = body.get("messages", [])
prompt_tokens = sum(crud.count_tokens(msg.get("content") or "") for msg in messages) prompt_tokens = sum(crud.count_tokens(_content_to_str(msg.get("content"))) for msg in messages)
if not crud.check_and_increment_quota(db, request.state.api_key_id, tokens=prompt_tokens, requests=1): if not crud.check_and_increment_quota(db, request.state.api_key_id, tokens=prompt_tokens, requests=1):
raise HTTPException(status_code=429, detail="Quota exceeded") raise HTTPException(status_code=429, detail="Quota exceeded")
usage_log.info('%s | /api/chat | %s | ~%d tokens | "%s"', usage_log.info('%s | /api/chat | %s | ~%d tokens | "%s"',
request.state.api_key_name, body.get("model", "?"), prompt_tokens, _last_user_msg(messages)) request.state.api_key_name, body.get("model", "?"), prompt_tokens, _last_user_msg(messages))
response = await proxy_request(f"{ollama_url}/api/chat", method="POST", json_data=body) try:
return JSONResponse(content=response.json(), status_code=response.status_code) response = await proxy_request(f"{ollama_url}/api/chat", method="POST", json_data=body)
return JSONResponse(content=response.json(), status_code=response.status_code)
except Exception as exc:
error_log.error("Proxy error | %s | /api/chat | %s | %s: %s",
request.state.api_key_name, body.get("model", "?"), type(exc).__name__, exc, exc_info=exc)
raise
@app.get("/api/tags") @app.get("/api/tags")
async def list_models(db: Session = Depends(get_db)): async def list_models(db: Session = Depends(get_db)):
@@ -103,7 +152,7 @@ async def openai_chat_completions(request: Request, db: Session = Depends(get_db
body = await request.json() body = await request.json()
messages = body.get("messages", []) messages = body.get("messages", [])
prompt_tokens = sum(crud.count_tokens(msg.get("content") or "") for msg in messages) prompt_tokens = sum(crud.count_tokens(_content_to_str(msg.get("content"))) for msg in messages)
if not crud.check_and_increment_quota(db, request.state.api_key_id, tokens=prompt_tokens, requests=1): if not crud.check_and_increment_quota(db, request.state.api_key_id, tokens=prompt_tokens, requests=1):
raise HTTPException(status_code=429, detail="Quota exceeded") raise HTTPException(status_code=429, detail="Quota exceeded")
@@ -119,15 +168,25 @@ async def openai_chat_completions(request: Request, db: Session = Depends(get_db
if body.get("stream"): if body.get("stream"):
async def generate(): async def generate():
async with httpx.AsyncClient(timeout=300.0) as client: try:
async with client.stream("POST", target, json=body) as resp: async with httpx.AsyncClient(timeout=300.0) as client:
async for chunk in resp.aiter_bytes(): async with client.stream("POST", target, json=body) as resp:
yield chunk async for chunk in resp.aiter_bytes():
yield chunk
except Exception as exc:
error_log.error("Stream error | %s | /v1/chat/completions | %s | %s: %s",
request.state.api_key_name, model_name, type(exc).__name__, exc, exc_info=exc)
raise
return StreamingResponse( return StreamingResponse(
generate(), generate(),
media_type="text/event-stream", media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"}, headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
) )
response = await proxy_request(target, method="POST", json_data=body) try:
return JSONResponse(content=response.json(), status_code=response.status_code) response = await proxy_request(target, method="POST", json_data=body)
return JSONResponse(content=response.json(), status_code=response.status_code)
except Exception as exc:
error_log.error("Proxy error | %s | /v1/chat/completions | %s | %s: %s",
request.state.api_key_name, model_name, type(exc).__name__, exc, exc_info=exc)
raise
+2
View File
@@ -58,6 +58,8 @@ class APIKeyWithUsage(APIKey):
tokens_used_month: int = 0 tokens_used_month: int = 0
requests_today: int = 0 requests_today: int = 0
requests_month: int = 0 requests_month: int = 0
daily_reset_at: Optional[datetime] = None
monthly_reset_at: Optional[datetime] = None
class Config: class Config:
from_attributes = True from_attributes = True
+4 -2
View File
@@ -2,9 +2,11 @@ services:
llmproxy: llmproxy:
image: mediaeng/llmproxy:latest image: mediaeng/llmproxy:latest
restart: unless-stopped restart: unless-stopped
network_mode: host
env_file: .env env_file: .env
ports: extra_hosts:
- "${PROXY_PORT:-8000}:${PROXY_PORT:-8000}" - "host.docker.internal:host-gateway"
volumes: volumes:
- ./backend/test.db:/app/backend/test.db - ./backend/test.db:/app/backend/test.db
- ./backend/logs:/app/backend/logs - ./backend/logs:/app/backend/logs
+35 -16
View File
@@ -11,17 +11,28 @@ function authHeaders(token) {
const fmtK = (n) => { const k = n / 1000; return k % 1 === 0 ? `${k}k` : `${k.toFixed(1)}k`; }; const fmtK = (n) => { const k = n / 1000; return k % 1 === 0 ? `${k}k` : `${k.toFixed(1)}k`; };
function QuotaBar({ used, limit, isToken = false }) { function QuotaBar({ used, limit, isToken = false, since = null }) {
if (limit == null) return <span className="quota-unlimited">∞</span>; const fmt = isToken ? fmtK : (n) => n.toLocaleString('de-DE');
const sinceLabel = since
? new Date(since).toLocaleDateString('de-DE', { day: '2-digit', month: '2-digit' })
: null;
if (limit == null) return (
<div className="quota-cell">
<span className="quota-unlimited">∞</span>
{sinceLabel && <span className="quota-since">seit {sinceLabel}</span>}
</div>
);
const pct = Math.min(100, (used / limit) * 100); const pct = Math.min(100, (used / limit) * 100);
const color = pct >= 90 ? '#e74c3c' : pct >= 70 ? '#e67e22' : '#27ae60'; const color = pct >= 90 ? '#e74c3c' : pct >= 70 ? '#e67e22' : '#27ae60';
const fmt = isToken ? fmtK : (n) => n.toLocaleString('de-DE');
return ( return (
<div className="quota-cell"> <div className="quota-cell">
<span className="quota-label">{fmt(used)} / {fmt(limit)}</span> <span className="quota-label">{fmt(used)} / {fmt(limit)}</span>
<div className="progress-bar"> <div className="progress-bar">
<div className="progress-fill" style={{ width: `${pct}%`, backgroundColor: color }} /> <div className="progress-fill" style={{ width: `${pct}%`, backgroundColor: color }} />
</div> </div>
{sinceLabel && <span className="quota-since">seit {sinceLabel}</span>}
</div> </div>
); );
} }
@@ -69,6 +80,7 @@ function SettingsSection({ password }) {
const [settings, setSettings] = useState(null); const [settings, setSettings] = useState(null);
const [availableModels, setAvailableModels] = useState([]); const [availableModels, setAvailableModels] = useState([]);
const [modelsLoading, setModelsLoading] = useState(false); const [modelsLoading, setModelsLoading] = useState(false);
const [ollamaReachable, setOllamaReachable] = useState(true);
const [proxyEndpoint, setProxyEndpoint] = useState(null); const [proxyEndpoint, setProxyEndpoint] = useState(null);
const [saved, setSaved] = useState(false); const [saved, setSaved] = useState(false);
const [error, setError] = useState(null); const [error, setError] = useState(null);
@@ -80,12 +92,14 @@ function SettingsSection({ password }) {
headers: authHeaders(password), headers: authHeaders(password),
params: url ? { url } : {}, params: url ? { url } : {},
}); });
const models = res.data.models; const { models, reachable } = res.data;
setOllamaReachable(reachable);
setAvailableModels(models); setAvailableModels(models);
if (models.length > 0 && !models.includes(currentModel)) { if (models.length > 0 && !models.includes(currentModel)) {
setSettings(s => ({ ...s, default_model: models[0] })); setSettings(s => ({ ...s, default_model: models[0] }));
} }
} catch { } catch {
setOllamaReachable(false);
setAvailableModels([]); setAvailableModels([]);
} finally { } finally {
setModelsLoading(false); setModelsLoading(false);
@@ -133,14 +147,19 @@ function SettingsSection({ password }) {
</div> </div>
<div className="settings-row"> <div className="settings-row">
<label>Ollama-Endpunkt</label> <label>Ollama-Endpunkt</label>
<input <div className="settings-input-wrap">
type="url" <input
value={settings.ollama_url} type="url"
onChange={(e) => setSettings({ ...settings, ollama_url: e.target.value })} value={settings.ollama_url}
onBlur={(e) => fetchModels(e.target.value, settings.default_model)} onChange={(e) => setSettings({ ...settings, ollama_url: e.target.value })}
placeholder="http://localhost:11434" onBlur={(e) => fetchModels(e.target.value, settings.default_model)}
required placeholder="http://localhost:11434"
/> required
/>
{!ollamaReachable && !modelsLoading && (
<div className="warning">⚠ Ollama nicht erreichbar unter {settings.ollama_url}</div>
)}
</div>
</div> </div>
<div className="settings-row"> <div className="settings-row">
<label>Standard-Modell</label> <label>Standard-Modell</label>
@@ -400,10 +419,10 @@ function App() {
<td>{key.name}</td> <td>{key.name}</td>
<td>{displayKey(key.key_prefix)}</td> <td>{displayKey(key.key_prefix)}</td>
<td>{key.expires_at ? new Date(key.expires_at).toLocaleDateString('de-DE', { timeZone: 'Europe/Berlin' }) : '∞'}</td> <td>{key.expires_at ? new Date(key.expires_at).toLocaleDateString('de-DE', { timeZone: 'Europe/Berlin' }) : '∞'}</td>
<td><QuotaBar used={key.tokens_used_today} limit={key.daily_tokens} isToken /></td> <td><QuotaBar used={key.tokens_used_today} limit={key.daily_tokens} isToken since={key.daily_reset_at} /></td>
<td><QuotaBar used={key.tokens_used_month} limit={key.monthly_tokens} isToken /></td> <td><QuotaBar used={key.tokens_used_month} limit={key.monthly_tokens} isToken since={key.monthly_reset_at} /></td>
<td><QuotaBar used={key.requests_today} limit={key.daily_requests} /></td> <td><QuotaBar used={key.requests_today} limit={key.daily_requests} since={key.daily_reset_at} /></td>
<td><QuotaBar used={key.requests_month} limit={key.monthly_requests} /></td> <td><QuotaBar used={key.requests_month} limit={key.monthly_requests} since={key.monthly_reset_at} /></td>
<td className="action-cell"> <td className="action-cell">
<button className="btn-icon btn-icon-edit" data-tooltip="Bearbeiten" onClick={() => handleEdit(key)}>✏</button> <button className="btn-icon btn-icon-edit" data-tooltip="Bearbeiten" onClick={() => handleEdit(key)}>✏</button>
{key.is_active ? ( {key.is_active ? (
+32
View File
@@ -194,6 +194,31 @@ tr:hover {
font-size: 14px; font-size: 14px;
} }
.settings-input-wrap {
flex: 1;
display: flex;
flex-direction: column;
gap: 4px;
}
.settings-input-wrap input {
width: 100%;
padding: 8px 10px;
border: 1px solid #ccc;
border-radius: 4px;
font-size: 14px;
box-sizing: border-box;
}
.warning {
color: #b8520a;
background: #fff3e0;
border: 1px solid #e67e22;
border-radius: 4px;
padding: 6px 10px;
font-size: 13px;
}
.settings-form button { .settings-form button {
align-self: flex-start; align-self: flex-start;
padding: 8px 20px; padding: 8px 20px;
@@ -246,6 +271,13 @@ tr:hover {
font-size: 14px; font-size: 14px;
} }
.quota-since {
display: block;
font-size: 10px;
color: #aaa;
margin-top: 2px;
}
.progress-bar { .progress-bar {
height: 4px; height: 4px;
background: #e2e8f0; background: #e2e8f0;
+1
View File
@@ -5,6 +5,7 @@ export default defineConfig({
plugins: [react()], plugins: [react()],
clearScreen: false, clearScreen: false,
server: { server: {
open: true,
proxy: { proxy: {
'/api/api-keys': 'http://localhost:8001', '/api/api-keys': 'http://localhost:8001',
'/api/settings': 'http://localhost:8001', '/api/settings': 'http://localhost:8001',