Clément Désiles
f6c4d36bbe
- backup role emits last-success/run metrics into node_exporter's textfile collector
- node_exporter enables systemd and textfile collectors
- prometheus deploys BackupStale/BackupTimerSilent/SystemdUnitFailed alert rules
- grafana gains Backups - Restic and Systemd dashboards
- fix prometheus.yml params rendering (quote values so {job=...} stays a string)
18 lines
695 B
YAML
18 lines
695 B
YAML
# Systemd state alerts. Managed by Ansible (roles/prometheus).
|
|
#
|
|
# node_exporter's systemd collector exposes node_systemd_unit_state. A unit in
|
|
# the failed state is exactly the kind of thing that needs paging at 03:00.
|
|
groups:
|
|
- name: systemd
|
|
rules:
|
|
- alert: SystemdUnitFailed
|
|
expr: node_systemd_unit_state{state="failed"} == 1
|
|
for: 5m
|
|
labels:
|
|
severity: critical
|
|
annotations:
|
|
summary: "systemd unit {{ $labels.name }} is in failed state ({{ $labels.instance }})"
|
|
description: >-
|
|
Unit {{ $labels.name }} (type {{ $labels.type }}) has been failed
|
|
for more than 5 minutes on {{ $labels.instance }}.
|