Monlet Agent

Go agent (Stage 2 MVP). Reads agent.toml, runs configured commands on schedule, captures status/output, pushes contract-valid heartbeat and events batches to the Monlet server with Bearer auth, and survives outages via a local on-disk spool.

Build and run

cd agent
GOTOOLCHAIN=local GOMODCACHE=$(pwd)/../.cache/go/pkg/mod GOCACHE=$(pwd)/../.cache/go/build GOBIN=$(pwd)/../.cache/go/bin go test ./...
GOTOOLCHAIN=local GOMODCACHE=$(pwd)/../.cache/go/pkg/mod GOCACHE=$(pwd)/../.cache/go/build GOBIN=$(pwd)/../.cache/go/bin go run ./cmd/monlet-agent -config config.example.toml

Do not use go install or go env -w. Module cache, build cache, and tool binaries stay under repo .cache/.

Configuration

See config.example.toml. Required sections: top-level state_dir, [server] (url, token), at least one [[checks]] (id, command, interval, timeout). agent_id is optional — if empty, the agent generates and persists a UUID under state_dir/agent_id.

Key	Default	Notes
`mode`	`hybrid`	`prometheus_only` (only `/metrics`), `push_only` (only heartbeat+events), `hybrid` (both). `prometheus_only` does not require `server.url`/`server.token`.
`server.heartbeat_interval`	`30s`	ADR-0005
`server.batch_interval`	`10s`	events flush cadence
`metrics.listen`	`127.0.0.1:9465`	low-cardinality only
`checks[].notification_owner`	`server`	`server`, `prometheus`, `none`

Behaviour

Per-check scheduler with no-overlap: a tick is skipped if the previous run is still in flight (monlet_agent_check_skipped_total).
Timeout via exec.CommandContext with Setpgid so the process group is killed on expiry; timeout → unknown.
Exit code mapping: 0=ok, 1=warning, 2=critical, 3+=unknown (ADR-0005).
output (stdout+stderr) is truncated by UTF-8 byte length to 8 KiB with marker ...[truncated N bytes].
event_id is UUIDv7 generated on check completion and preserved across spool replay (ADR-0006 idempotency).
Spool: per-event JSON files in state_dir/spool/, FIFO, limits 10 000 events / 50 MiB, drop oldest on overflow (ADR-0005).
Retry backoff for heartbeat and events: 1s → 30s, exp ×2, jitter ±20%. 4xx are dropped (not retried) to avoid hot loop.
Bearer auth header on both endpoints (ADR-0007).

Prometheus metrics

Exposed on metrics.listen at /metrics. Labels are static (check_id, status, kind):

monlet_agent_check_runs_total{check_id,status}
monlet_agent_check_skipped_total{check_id}
monlet_agent_check_duration_seconds{check_id} (histogram)
monlet_agent_check_status{check_id} (0=ok, 1=warning, 2=critical, 3=unknown)
monlet_agent_check_exit_code{check_id}
monlet_agent_check_last_run_timestamp_seconds{check_id}
monlet_agent_check_last_success_timestamp_seconds{check_id}
monlet_agent_check_interval_seconds{check_id}
monlet_agent_checks_configured
monlet_agent_spool_events, monlet_agent_spool_bytes
monlet_agent_send_attempts_total{kind}, monlet_agent_send_failures_total{kind} (kind ∈ heartbeat, events)
monlet_agent_events_accepted_total, monlet_agent_events_deduplicated_total
monlet_agent_last_heartbeat_timestamp_seconds
monlet_agent_build_info{version,mode} (constant 1)

systemd

See systemd/monlet-agent.service.example.

3.3 KiB Raw Blame History Unescape Escape