feat(monitoring): add backup and systemd metrics, alerts, and dashboards
- backup role emits last-success/run metrics into node_exporter's textfile collector
- node_exporter enables systemd and textfile collectors
- prometheus deploys BackupStale/BackupTimerSilent/SystemdUnitFailed alert rules
- grafana gains Backups - Restic and Systemd dashboards
- fix prometheus.yml params rendering (quote values so {job=...} stays a string)
This commit is contained in:
@@ -21,6 +21,19 @@ Keep localhost-only — Prometheus scrapes it locally on the same host. To add
|
||||
collectors (e.g. `--collector.systemd`, `--collector.textfile.directory`), extend
|
||||
`node_exporter_listen_address` or override the args in host_vars.
|
||||
|
||||
## Collectors
|
||||
|
||||
The role enables, on top of the defaults:
|
||||
|
||||
- `--collector.zfs` — ZFS pool/dataset metrics.
|
||||
- `--collector.systemd` — unit state (`node_systemd_unit_state`), restart
|
||||
counts (`node_systemd_restart_count_total`) and timer trigger times
|
||||
(`node_systemd_timer_last_trigger_seconds` / `..._next_trigger_seconds`).
|
||||
Powers the systemd dashboard (failed units, timers that stopped firing).
|
||||
- `--collector.textfile.directory={{ node_exporter_textfile_dir }}` — re-exposes
|
||||
`*.prom` files dropped into that directory. The backup role writes
|
||||
`backup_*` metrics there; other cron/timer jobs can do the same.
|
||||
|
||||
## Pairing with Prometheus
|
||||
|
||||
```yaml
|
||||
|
||||
@@ -4,3 +4,8 @@
|
||||
# Address the exporter listens on. Keep localhost-only — Prometheus scrapes it
|
||||
# locally on the same host; no need to expose on the network.
|
||||
node_exporter_listen_address: "127.0.0.1:9100"
|
||||
|
||||
# Directory the textfile collector reads `*.prom` files from. Jobs (e.g. the
|
||||
# backup role) drop metrics here; node_exporter re-exposes them to Prometheus.
|
||||
# Must match backup_metrics_dir in roles/backup/defaults/main.yml.
|
||||
node_exporter_textfile_dir: /var/lib/node_exporter/textfile_collector
|
||||
|
||||
@@ -10,6 +10,14 @@
|
||||
name: "{{ node_exporter_package }}"
|
||||
state: present
|
||||
|
||||
- name: Create textfile collector directory
|
||||
ansible.builtin.file:
|
||||
path: "{{ node_exporter_textfile_dir }}"
|
||||
state: directory
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
|
||||
- name: Deploy node_exporter configuration
|
||||
ansible.builtin.template:
|
||||
src: env.conf.j2
|
||||
|
||||
@@ -1,2 +1,2 @@
|
||||
# Managed by Ansible - DO NOT EDIT MANUALLY
|
||||
{{ node_exporter_args_var }}="--web.listen-address={{ node_exporter_listen_address }} --collector.zfs"
|
||||
{{ node_exporter_args_var }}="--web.listen-address={{ node_exporter_listen_address }} --collector.zfs --collector.systemd --collector.textfile.directory={{ node_exporter_textfile_dir }}"
|
||||
|
||||
Reference in New Issue
Block a user