Prometheus alerters

From wikinotes

Alerters are services that prometheus communicates with to issue alerts (ex: email, pagerduty).

Documentation

official docs https://prometheus.io/docs/alerting/latest/overview/

Notes

prometheus alertmanager

Debugging

When an alert is fired, you'll see something like this in your syslog (this one is an error)

Feb 20 00:54:53 yubaba prometheus[59312]: level=error ts=2022-02-20T00:54:53.073Z caller=notifier.go:527 component=notifier alertmanager=http://127.0.0.1:9093/api/v2/alerts count=1 msg="Error sending alert" err="Post \"http://127.0.0.1:9093/api/v2/alerts\": context deadline exceeded"