# Backup and restore

All persistent state lives in PostgreSQL (`agents`, `checks`, `events`, `incidents`, `notification_outbox`). The server is stateless; the agent local spool is best-effort and not part of the canonical state (ADR-0006).

## What to back up

- The Monlet PostgreSQL database (logical or physical).
- The server `MONLET_AUTH_TOKEN` and `.env` (kept in your secret store, not in backups of the DB).
- The agent `config.toml` per host (kept in your config management).

## Logical backup (recommended for small deployments)

```sh
PGPASSWORD=monlet pg_dump \
  -h $PGHOST -U monlet -d monlet \
  --format=custom --no-owner --no-privileges \
  --file=monlet-$(date -u +%Y%m%dT%H%M%SZ).dump
```

Restore into an empty database:

```sh
createdb -h $PGHOST -U postgres monlet
PGPASSWORD=monlet pg_restore \
  -h $PGHOST -U monlet -d monlet \
  --no-owner --no-privileges \
  monlet-YYYYMMDDTHHMMSSZ.dump
```

The schema is owned by Alembic. Restoring a logical dump from the same Monlet version is safe. After restore, run `alembic upgrade head` only if you restored from an older version.

## Retention

- `events` grows unbounded in v1. Decide a retention policy that matches your storage budget (typical: 7–30 days). Stage 6+ may add server-side trimming; until then, run a manual cron:

  ```sql
  DELETE FROM events WHERE observed_at < now() - interval '14 days';
  ```

  `checks.last_event_id` is not a FK, so trimming `events` does not break current state.

- `incidents` and `notification_outbox` are small. Trim outbox `state IN ('sent','failed','discarded')` rows older than 7 days if storage is tight.

## Disaster recovery

- Restore the database.
- Restart the server. Agents will resume sending; spooled events on each agent are replayed and de-duplicated by `event_id` (ADR-0006).
- The agent spool is bounded (10 000 events / 50 MiB FIFO, ADR-0005). During a long outage the oldest events get dropped to make room for new ones, so observations from the start of the outage are lost first. Plan recovery time accordingly — for an agent producing ~1 event/s per check, ~10 000 events is roughly 2–3 hours of buffering for a handful of checks.
- If agents were also lost, only events that were not yet persisted to the spool file are lost in addition.