monlet/server/README.md

Monlet Server

FastAPI server: heartbeat/event ingestion, current check state, incident lifecycle, stale/dead detector, notification outbox, Prometheus metrics.

Stack

• Python 3.14+
• uv
• FastAPI, Pydantic v2
• SQLAlchemy 2.x async + asyncpg
• Alembic (sync psycopg)
• structlog, prometheus-client
• PostgreSQL 16+

Local development

cd server
UV_PROJECT_ENVIRONMENT=.venv UV_CACHE_DIR=../.cache/uv uv sync --extra dev

Run a Postgres instance (Docker or local), set MONLET_DATABASE_URL, then:

UV_PROJECT_ENVIRONMENT=.venv UV_CACHE_DIR=../.cache/uv uv run alembic upgrade head
UV_PROJECT_ENVIRONMENT=.venv UV_CACHE_DIR=../.cache/uv uv run fastapi dev monlet_server/main.py

Environment

Var	Default	Purpose
MONLET_AUTH_TOKEN	required	UI/operator Bearer token. Space/comma-separated list; any accepted. changeme is rejected.
MONLET_DATABASE_URL	postgresql+asyncpg://monlet:monlet@127.0.0.1:5432/monlet	Async DSN
MONLET_LOG_LEVEL	INFO	Log level
MONLET_HOST	0.0.0.0	HTTP bind host for direct server runs
MONLET_PORT	8000	HTTP bind port for direct server runs
MONLET_BODY_LIMIT_BYTES	1048576	Request body limit (ADR-0005)
MONLET_OUTPUT_MAX_BYTES	8192	Per-event output limit
MONLET_MAX_BATCH_EVENTS	200	Maximum events per ingestion batch
MONLET_STALE_AFTER_SEC	20	Stale threshold
MONLET_DEAD_AFTER_SEC	30	Dead threshold
MONLET_DETECTOR_TICK_SEC	5	Detector tick
MONLET_ENABLE_DETECTOR	true	Toggle detector task
MONLET_DB_POOL_SIZE	10	Base async DB pool size
MONLET_DB_MAX_OVERFLOW	10	Extra DB connections above pool size
MONLET_DB_POOL_TIMEOUT_SEC	30	Seconds to wait for a DB connection
MONLET_DB_POOL_RECYCLE_SEC	1800	Recycle DB connections by age
MONLET_EVENTS_RETENTION_MONTHS	36	Months of events partitions to keep; 0 disables drop
MONLET_EVENTS_FUTURE_PARTITIONS	3	Future month partitions to pre-create
MONLET_EVENT_DEDUP_RETENTION_DAYS	30	Idempotency window for event_id
MONLET_EVENT_DEDUP_PRUNE_BATCH_SIZE	5000	Dedup rows pruned per maintenance tick
MONLET_MAX_PENDING_AGENTS	10000	Maximum pending agent keys before heartbeat rejects with 429
MONLET_PENDING_AGENT_TTL_DAYS	7	Pending key cleanup TTL; 0 disables pruning
MONLET_PENDING_AGENT_PRUNE_BATCH_SIZE	1000	Pending cleanup rows per detector tick
MONLET_PARTITION_MAINTENANCE_INTERVAL_SEC	3600	Partition create/drop worker interval
MONLET_OUTBOX_RETENTION_MAX_ROWS	50000	Maximum retained terminal outbox rows; 0 disables pruning
MONLET_OUTBOX_PRUNE_BATCH_SIZE	5000	Outbox rows pruned per maintenance tick
MONLET_ENABLE_NOTIFIER_WORKER	true	Toggle outbox worker
MONLET_NOTIFIER_TICK_SEC	5	Worker poll interval
MONLET_NOTIFIER_BATCH_SIZE	20	Rows claimed per tick
MONLET_NOTIFIER_CONCURRENCY	4	Parallel notification deliveries per tick
MONLET_NOTIFIER_MAX_ATTEMPTS	8	Terminal-failure threshold (ADR-0005)
MONLET_NOTIFIER_LEASE_SEC	300	Recovery lease for stuck sending rows
MONLET_NOTIFIER_HTTP_TIMEOUT_SEC	10	Outbound HTTP timeout
MONLET_NOTIFIER_INCLUDE_OUTPUT	true	Include redacted check output in notifications
MONLET_NOTIFIER_MAX_OUTPUT_BYTES	1024	Maximum notification output bytes after redaction
MONLET_NOTIFICATION_TIME_ZONE	UTC	IANA timezone for human-readable notification timestamps
MONLET_NOTIFIER_DEBUG_ENABLED	true	Log-only notifier
MONLET_NOTIFIER_TELEGRAM_ENABLED	false	Telegram notifier on/off
MONLET_NOTIFIER_TELEGRAM_TOKEN	``	Telegram bot token
MONLET_NOTIFIER_TELEGRAM_CHAT_ID	``	Telegram chat id
MONLET_NOTIFIER_WEBHOOK_ENABLED	false	Generic webhook notifier on/off
MONLET_NOTIFIER_WEBHOOK_URL	``	Webhook URL
MONLET_NOTIFIER_WEBHOOK_TOKEN	``	Optional Bearer token for webhook
MONLET_NOTIFIER_ALERTMANAGER_ENABLED	false	Alertmanager notifier on/off
MONLET_NOTIFIER_ALERTMANAGER_URL	``	Alertmanager base URL (e.g. http://am:9093)

Tests

Tests use testcontainers-python to spin up a real PostgreSQL container; Docker must be available.

UV_PROJECT_ENVIRONMENT=.venv UV_CACHE_DIR=../.cache/uv uv run pytest -q

Docker

Image installs dependencies inside the container only; host venv is never mounted.

docker build -t monlet-server .

Endpoints

All routes under /api/v1 (per api/openapi.yaml). /metrics and /api/v1/health, /api/v1/ready are open.

• UI/operator endpoints require Authorization: Bearer <MONLET_AUTH_TOKEN>.
• Agent ingestion endpoints accept the per-agent key generated and stored by the agent. A new key creates a pending agent; events are ignored until an operator accepts it.