Blackbox exporter
From wikinotes
Prometheus exporter that probes endpoints over HTTP/HTTPS/DNS/TCP/ICMP.
Documentation
Locations
9115
default port ${PREFIX}/etc/blackbox_exporter.yml
config
Install
pacman -S prometheus-blackbox-exporter # Archlinux pkg install blackbox_exporter # FreeBSD
Configuration
- You define your own test-modules (endpoints test-conditions) in
blackbox_exporter.yml
.- You configure test-modules to test endpoints with in your
prometheus.yml
.Define Test-Modules
# /usr/local/etc/blackbox_exporter.yml modules: https_2xx: prober: http timeout: 5s http: valid_status_codes: [2] # any 2xx status fail_if_not_ssl: true no_follow_redirects: true preferred_ip_protocol: ip4 http_401: prober: http timeout: 5s http: valid_status_codes: [401] method: GET fail_if_not_ssl: true no_follow_redirects: true preferred_ip_protocol: ip4Here we configure two test-modules.
The first tests for a HTTP-2XX return-code, the second for an HTTP-401.See the sample config where all options/types are documented.
Use Modules
# /usr/local/etc/prometheus.yml global: scrape_interval: 120s scrape_configs: - job_name: https_2xx params: module: [https_2xx] static_configs: - targets: - https://foo.com - https://bar.net relabel_configs: - source_labels: [__address__] target_label: __param_target - source_labels: [__param_target] target_label: instance - target_label: __address__ replacement: 127.0.0.1:9115This config creates a scrape-job that uses our
http_2xx
module to test bothhttps://foo.com
andhttps://bar.net
.
Testing
/usr/bin/env /usr/local/bin/blackbox_exporter \ --web.listen-address=127.0.0.1:9115 \ --config.file=/usr/local/etc/blackbox_exporter.yml
Usage
Routes
When debugging, it's worth it to look at the debug logs.
Your relabeling rules may cause you to query the incorrect domain!# query metrics for target/module curl -v 'localhost:9115/probe\?module=http_2xx\&target=https://willpittman.net\&debug=true' w3m 'http://localhost:9115/probe\?module=http_2xx\&target=https://willpittman.net\&debug=true'Metrics
The docs suggest
probe_success
but I don't have it on FreeBSD-13.
Instead, I have:
up
promhttp_metric_handler_requests_total
up{instance="https://willpittman.net"} absent_over_time(promhttp_metric_handler_requests_total{instance="https://willpittman.net", code="200"}[1s])
Debugging
The best method of testing seems to be from a web-browser (not w3m).
The query target seems fixed againstprometheus.io
, if that works it should work for most simple cases.- open http://${IP}:9115/probe/&target=google.com&method=http_2xx - click 'Logs' in a failed recent probe - the request logs are at the top