feat(monitoring): add backup and systemd metrics, alerts, and dashboards

- backup role emits last-success/run metrics into node_exporter's textfile collector
- node_exporter enables systemd and textfile collectors
- prometheus deploys BackupStale/BackupTimerSilent/SystemdUnitFailed alert rules
- grafana gains Backups - Restic and Systemd dashboards
- fix prometheus.yml params rendering (quote values so {job=...} stays a string)
This commit is contained in:
Clément Désiles
2026-09-09 12:07:50 +02:00
parent 8756eb1183
commit f6c4d36bbe
16 changed files with 335 additions and 7 deletions
+13
View File
@@ -21,6 +21,19 @@ Keep localhost-only — Prometheus scrapes it locally on the same host. To add
collectors (e.g. `--collector.systemd`, `--collector.textfile.directory`), extend
`node_exporter_listen_address` or override the args in host_vars.
## Collectors
The role enables, on top of the defaults:
- `--collector.zfs` — ZFS pool/dataset metrics.
- `--collector.systemd` — unit state (`node_systemd_unit_state`), restart
counts (`node_systemd_restart_count_total`) and timer trigger times
(`node_systemd_timer_last_trigger_seconds` / `..._next_trigger_seconds`).
Powers the systemd dashboard (failed units, timers that stopped firing).
- `--collector.textfile.directory={{ node_exporter_textfile_dir }}` — re-exposes
`*.prom` files dropped into that directory. The backup role writes
`backup_*` metrics there; other cron/timer jobs can do the same.
## Pairing with Prometheus
```yaml
+5
View File
@@ -4,3 +4,8 @@
# Address the exporter listens on. Keep localhost-only — Prometheus scrapes it
# locally on the same host; no need to expose on the network.
node_exporter_listen_address: "127.0.0.1:9100"
# Directory the textfile collector reads `*.prom` files from. Jobs (e.g. the
# backup role) drop metrics here; node_exporter re-exposes them to Prometheus.
# Must match backup_metrics_dir in roles/backup/defaults/main.yml.
node_exporter_textfile_dir: /var/lib/node_exporter/textfile_collector
+8
View File
@@ -10,6 +10,14 @@
name: "{{ node_exporter_package }}"
state: present
- name: Create textfile collector directory
ansible.builtin.file:
path: "{{ node_exporter_textfile_dir }}"
state: directory
owner: root
group: root
mode: "0755"
- name: Deploy node_exporter configuration
ansible.builtin.template:
src: env.conf.j2
+1 -1
View File
@@ -1,2 +1,2 @@
# Managed by Ansible - DO NOT EDIT MANUALLY
{{ node_exporter_args_var }}="--web.listen-address={{ node_exporter_listen_address }} --collector.zfs"
{{ node_exporter_args_var }}="--web.listen-address={{ node_exporter_listen_address }} --collector.zfs --collector.systemd --collector.textfile.directory={{ node_exporter_textfile_dir }}"