chore: add mise toolchain for local Quellenatlas development
Pin Python 3.12, Node 22 and uv; add tasks for setup, dev, ingest and tests.
This commit is contained in:
1 parent
872f836a47
commit
57fce463cd
5 files changed
+245
-73
No files matched your search
@@ -18,3 +18,5 @@ data/postgres/
|
||||
*.log
|
||||
htmlcov/
|
||||
.coverage
|
||||
mise.local.toml
|
||||
.mise.local.toml
|
||||
+94
@@ -0,0 +1,94 @@
|
||||
# HANDOFF — Quellenatlas
|
||||
|
||||
Stand: 2026-08-21. Neue Session hier ansetzen, Architektur nicht neu aufrollen.
|
||||
|
||||
## Repo
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Pfad | `/Volumes/SSD2TB/work/cdu` |
|
||||
| Remote | `http://10.27.27.139:3000/admin/cdu-vorfaelle.git` |
|
||||
| Branch | `main` = `origin/main` |
|
||||
| Commit | `872f836` — `feat: add Quellenatlas knowledge base` (103 Dateien) |
|
||||
| Produktname | Quellenatlas (nicht „cdu“ — Ordnername ist Zufall, keine Parteikategorie) |
|
||||
|
||||
`.env` ist gitignored. Vorlage: `.env.example`.
|
||||
|
||||
## Was das System ist
|
||||
|
||||
Private, selbst gehostete Knowledge Base für **quellenbasierte** Dokumentation politischer/gesellschaftlicher Auffälligkeiten. Kein öffentliches Portal, kein Meinungsprodukt.
|
||||
|
||||
Schwerpunkte von Anfang an: Ursula von der Leyen (Sorgfalts-Cluster, keine Schuldzuschreibung), Ukraine-Vorgänge, DACH + EU.
|
||||
|
||||
## Festgelegte Entscheidungen (nicht wieder öffnen)
|
||||
|
||||
- Zugang: privat, Login Pflicht (`admin` / `changeme` lokal)
|
||||
- Stack: FastAPI + SQLAlchemy 2 + Alembic + PostgreSQL 16 + Next.js
|
||||
- Ingest v1: nur RSS/API → `SourceEntry` + `ReviewItem`. **Nie Cases.** Kein HTML-Scraping.
|
||||
- Graph: SQL-abgeleiteter `/api/v1/graph`, kein Neo4j
|
||||
- UI: Deutsch
|
||||
- Ethik im Code (`backend/app/core/ethics.py`): Claim ≥ 1 Quelle; Case ≥ 1 Quelle; `confirmed` nur mit Amts-/Gerichts-/Parlamentsquelle **oder** `confirmation_basis`; `care_level=high` braucht `accessed_at`; Summary-Lint gegen Werturteile
|
||||
- Admin nur über dieselben Services (kein SQLAdmin, damit Ethik nicht umgangen wird)
|
||||
- Postgres-Port lokal **5433** (5432 ist von einer anderen lokalen Postgres belegt). In Docker intern weiter 5432.
|
||||
|
||||
## Laufender Stand
|
||||
|
||||
v1 ist implementiert und gepusht:
|
||||
|
||||
- Docs: `docs/architecture.md`, `data-model.md`, `ethics.md`, `adding-sources.md`
|
||||
- Schema + Migration `backend/alembic/versions/a2bddb90c586_initial_schema.py`
|
||||
- Katalog: 30 Quellen in `data/sources.yaml`
|
||||
- Seed: 24 Fälle, 39 Entitäten, 2 Promises (`data/seed/`)
|
||||
- Tests: 30 pytest grün (Docker-Postgres auf 5433 nötig)
|
||||
- Frontend: Login, Hierarchie, Suche, Cluster VdL/Ukraine, Falldetail, Review-Queue, Quellen, Fall-Editor
|
||||
|
||||
Seed-Cluster (API-verifiziert nach `python -m app.cli seed`):
|
||||
|
||||
- **vdl (4):** `ombudsman-sms-transparenz`, `bmvg-beratervertraege`, `vdl-dissertation-2015` (disputed), `orgenesis-interessenkonflikt` (allegation)
|
||||
- **ukraine (5):** `eca-ukraine-facility-opinion`, `eu-mfa-ukraine`, `nabu-mandate`, `opensanctions-ukraine-peps`, `zeitenwende-militaerhilfe` (disputed/partial)
|
||||
|
||||
Status bewusst konservativ: Ombudsmann-Fall ist `confirmed` für **Transparenzmangel**, nicht für Korruption.
|
||||
|
||||
## Starten
|
||||
|
||||
Toolchain: `mise.toml` (Python 3.12, Node 22, uv). Venv: `backend/.venv`.
|
||||
|
||||
```bash
|
||||
mise install
|
||||
mise run setup # Postgres :5433, migrate, admin, sources, seed
|
||||
mise run dev # API :8000 + UI :3000
|
||||
mise run test
|
||||
```
|
||||
|
||||
Oder: `docker compose up --build` (Backend-`DATABASE_URL` in Compose zeigt auf Host `postgres`).
|
||||
|
||||
- UI http://localhost:3000
|
||||
- API http://localhost:8000/docs
|
||||
|
||||
## Nächste sinnvolle Schritte (in dieser Reihenfolge)
|
||||
|
||||
1. **Live-Ingest:** `python -m app.cli ingest` (oder `--slug …`). Feeds können 404 sein → in `data/sources.yaml` `status: paused` oder URL korrigieren, dann `sync-sources`.
|
||||
2. **Review-Queue** unter `/admin/review` sichten; nichts auto-publizieren.
|
||||
3. **Seed-URL-Triage:** einige Seed-Links sind Katalog-/Themenseiten, nicht Primärdokumente (z. B. Orgenesis → Politico-Homepage, NABU-Feed). 404 ersetzen oder Fall nicht seeden.
|
||||
4. Frontend nicht im Browser E2E verifiziert (kein Browser-Tool in der Build-Session). Manuell: Login → Cluster VdL/Ukraine → Fallseite mit Zitaten → Editor lehnt Claim ohne Quelle ab.
|
||||
5. Optional später: Neo4j-Read-Model, mehr Adapter, härtere `confirmed`-Regel auf Claim-Ebene.
|
||||
|
||||
## Wichtige Pfade
|
||||
|
||||
| Pfad | Rolle |
|
||||
|---|---|
|
||||
| `docs/ethics.md` | Verbindliche Status-/Neutralitätsregeln |
|
||||
| `backend/app/core/ethics.py` | Durchsetzung |
|
||||
| `backend/app/services/cases.py` | Case-Write inkl. Validation |
|
||||
| `backend/app/ingest/runner.py` | RSS/API, Dedup, Keyword-Priority |
|
||||
| `data/sources.yaml` | Kanonischer Quellenkatalog |
|
||||
| `data/seed/cases.yaml` | 24 Seed-Fälle |
|
||||
| `data/seed/cases/_template.yaml` | Vorlage neuer Fälle |
|
||||
|
||||
## Nicht tun
|
||||
|
||||
- Öffentlich ohne Auth ausliefern
|
||||
- LLM-Summaries in die KB schreiben
|
||||
- HTML-Crawler gegen Medien
|
||||
- Architektur (Django/Neo4j/Scraping) neu diskutieren, solange v1-Ziele gelten
|
||||
- `confirmed` für ungerichtete Personen-Schuld setzen
|
||||
@@ -6,27 +6,28 @@ Jeder Claim braucht mindestens eine Quelle. Status (`confirmed` / `allegation` /
|
||||
|
||||
## Schnellstart
|
||||
|
||||
Voraussetzung lokal: [mise](https://mise.jdx.dev/) und Docker (für PostgreSQL).
|
||||
|
||||
```bash
|
||||
cp .env.example .env # Werte anpassen, vor allem JWT_SECRET und ADMIN_PASSWORD
|
||||
docker compose up --build
|
||||
mise trust # einmalig
|
||||
mise install # Python 3.12, Node 22, uv; danach hook: mise run install
|
||||
mise run setup # Postgres :5433, Migrate, Admin, Quellen, Seed
|
||||
mise run dev # API :8000 + UI :3000
|
||||
```
|
||||
|
||||
- UI: http://localhost:3000
|
||||
- API: http://localhost:8000/docs
|
||||
- Login: `ADMIN_USERNAME` / `ADMIN_PASSWORD` (Default `admin` / `changeme`)
|
||||
|
||||
Lokal ohne Docker-Frontend (Postgres bleibt in Compose, Port **5433** wegen Kollision mit lokaler Postgres):
|
||||
Häufige Tasks: `mise tasks` — u. a. `test`, `seed`, `sync`, `ingest`, `backend`, `frontend`.
|
||||
Postgres-Port lokal **5433** (5432 ist oft schon belegt). Compose intern weiter 5432.
|
||||
|
||||
Nur Docker (Backend und Frontend im Compose-Stack):
|
||||
|
||||
```bash
|
||||
docker compose up -d postgres
|
||||
cd backend && python3 -m venv .venv && .venv/bin/pip install -e ".[dev]"
|
||||
.venv/bin/alembic upgrade head
|
||||
.venv/bin/python -m app.cli bootstrap-admin
|
||||
.venv/bin/python -m app.cli sync-sources
|
||||
.venv/bin/python -m app.cli seed
|
||||
.venv/bin/uvicorn app.main:app --reload --port 8000
|
||||
|
||||
cd ../frontend && npm install && npm run dev
|
||||
cp .env.example .env
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
## Was das System tut
|
||||
@@ -65,10 +66,10 @@ Pflicht: mindestens ein Claim mit URL und `accessed_at`. `confirmed` nur mit Amt
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
cd backend && .venv/bin/pytest -q
|
||||
mise run test
|
||||
```
|
||||
|
||||
Benötigt Docker-Postgres auf `localhost:5433`.
|
||||
Benötigt Docker-Postgres auf `localhost:5433` (`mise run postgres`).
|
||||
|
||||
## Export
|
||||
|
||||
|
||||
Generated
-60
@@ -120,9 +120,6 @@
|
||||
"cpu": [
|
||||
"arm"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -139,9 +136,6 @@
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -158,9 +152,6 @@
|
||||
"cpu": [
|
||||
"ppc64"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -177,9 +168,6 @@
|
||||
"cpu": [
|
||||
"riscv64"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -196,9 +184,6 @@
|
||||
"cpu": [
|
||||
"s390x"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -215,9 +200,6 @@
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -234,9 +216,6 @@
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"libc": [
|
||||
"musl"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -253,9 +232,6 @@
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"libc": [
|
||||
"musl"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -272,9 +248,6 @@
|
||||
"cpu": [
|
||||
"arm"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -297,9 +270,6 @@
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -322,9 +292,6 @@
|
||||
"cpu": [
|
||||
"ppc64"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -347,9 +314,6 @@
|
||||
"cpu": [
|
||||
"riscv64"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -372,9 +336,6 @@
|
||||
"cpu": [
|
||||
"s390x"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -397,9 +358,6 @@
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -422,9 +380,6 @@
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"libc": [
|
||||
"musl"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -447,9 +402,6 @@
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"libc": [
|
||||
"musl"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -586,9 +538,6 @@
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -605,9 +554,6 @@
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"libc": [
|
||||
"musl"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -624,9 +570,6 @@
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"libc": [
|
||||
"glibc"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
@@ -643,9 +586,6 @@
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"libc": [
|
||||
"musl"
|
||||
],
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
min_version = "2024.9.5"
|
||||
|
||||
[tools]
|
||||
python = "3.12"
|
||||
node = "22"
|
||||
uv = "0.11"
|
||||
|
||||
[env]
|
||||
PROJECT_NAME = "quellenatlas"
|
||||
# Repo liegt auf einem anderen Volume als der uv-Cache → Copy statt Hardlink
|
||||
UV_LINK_MODE = "copy"
|
||||
NEXT_PUBLIC_API_URL = { default = "http://localhost:8000" }
|
||||
_.file = [".env.example", { path = ".env", redact = true }]
|
||||
# Virtualenv im Backend (Makefile/README nutzen backend/.venv)
|
||||
_.python.venv = { path = "backend/.venv", create = true }
|
||||
|
||||
[hooks]
|
||||
postinstall = "mise run install"
|
||||
|
||||
[tasks.default]
|
||||
description = "Verfügbare Tasks anzeigen"
|
||||
run = "mise tasks"
|
||||
|
||||
[tasks.install]
|
||||
description = "Python- (backend) und Node-Abhängigkeiten (frontend) installieren"
|
||||
alias = "i"
|
||||
run = """
|
||||
set -euo pipefail
|
||||
if [ -x backend/.venv/bin/python ]; then
|
||||
ver=$(backend/.venv/bin/python -c 'import sys; print("%d.%d" % sys.version_info[:2])')
|
||||
if [ "$ver" != "3.12" ]; then
|
||||
echo "backend/.venv ist Python $ver – neu anlegen für 3.12"
|
||||
rm -rf backend/.venv
|
||||
fi
|
||||
fi
|
||||
if [ ! -x backend/.venv/bin/python ]; then
|
||||
uv venv backend/.venv --python 3.12
|
||||
fi
|
||||
uv pip install -e ".[dev]" --python .venv --directory backend
|
||||
npm install --prefix frontend
|
||||
"""
|
||||
|
||||
[tasks.postgres]
|
||||
description = "PostgreSQL 16 via Compose auf Port 5433"
|
||||
alias = "db"
|
||||
run = """
|
||||
set -euo pipefail
|
||||
if [ ! -f .env ]; then
|
||||
cp .env.example .env
|
||||
echo "Angelegt: .env (aus .env.example) – JWT_SECRET und ADMIN_PASSWORD anpassen"
|
||||
fi
|
||||
docker compose up -d postgres
|
||||
for i in $(seq 1 30); do
|
||||
if docker compose exec -T postgres pg_isready -U quellenatlas >/dev/null 2>&1; then
|
||||
echo "Postgres bereit auf localhost:5433"
|
||||
exit 0
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
echo "Postgres nicht bereit" >&2
|
||||
exit 1
|
||||
"""
|
||||
|
||||
[tasks.migrate]
|
||||
description = "Alembic auf head"
|
||||
dir = "backend"
|
||||
run = "alembic upgrade head"
|
||||
|
||||
[tasks.bootstrap]
|
||||
description = "Admin-User aus ADMIN_USERNAME / ADMIN_PASSWORD anlegen"
|
||||
dir = "backend"
|
||||
run = "python -m app.cli bootstrap-admin"
|
||||
|
||||
[tasks.sync]
|
||||
description = "Quellenkatalog aus data/sources.yaml synchronisieren"
|
||||
dir = "backend"
|
||||
run = "python -m app.cli sync-sources"
|
||||
|
||||
[tasks.seed]
|
||||
description = "Seed-YAML (Jurisdiktionen, Kategorien, Entitäten, Fälle, Zusagen)"
|
||||
dir = "backend"
|
||||
run = "python -m app.cli seed"
|
||||
|
||||
[tasks.ingest]
|
||||
description = "RSS/API-Ingest in Review-Queue (nie Cases). Optional: mise run ingest -- --slug …"
|
||||
dir = "backend"
|
||||
run = "python -m app.cli ingest"
|
||||
|
||||
[tasks.setup]
|
||||
description = "Lokal startklar: Postgres, Migrate, Admin, Quellen, Seed"
|
||||
depends = ["install", "postgres"]
|
||||
run = """
|
||||
set -euo pipefail
|
||||
mise run migrate
|
||||
mise run bootstrap
|
||||
mise run sync
|
||||
mise run seed
|
||||
echo
|
||||
echo "UI http://localhost:3000"
|
||||
echo "API http://localhost:8000/docs"
|
||||
echo "Start: mise run dev"
|
||||
"""
|
||||
|
||||
[tasks.backend]
|
||||
description = "FastAPI mit Reload auf :8000"
|
||||
alias = "api"
|
||||
dir = "backend"
|
||||
raw = true
|
||||
run = "uvicorn app.main:app --reload --port 8000"
|
||||
|
||||
[tasks.frontend]
|
||||
description = "Next.js Dev-Server auf :3000"
|
||||
alias = "web"
|
||||
dir = "frontend"
|
||||
raw = true
|
||||
run = "npm run dev"
|
||||
|
||||
[tasks.dev]
|
||||
description = "Postgres + API + UI lokal (ohne Docker-App-Container)"
|
||||
depends = ["postgres"]
|
||||
run = "mise run --raw -j 2 ::: backend frontend"
|
||||
|
||||
[tasks.test]
|
||||
description = "pytest (braucht Postgres auf localhost:5433)"
|
||||
alias = "t"
|
||||
dir = "backend"
|
||||
run = "pytest -q"
|
||||
|
||||
[tasks.up]
|
||||
description = "Gesamtes Stack in Docker Compose bauen und starten"
|
||||
run = "docker compose up --build"
|
||||
|
||||
[tasks.down]
|
||||
description = "Docker Compose stoppen"
|
||||
run = "docker compose down"
|
||||
Reference in new issue
Block a user