Skip to content

Blackbox Exporter guide#1134

Closed
lucperkins wants to merge 1 commit intoprometheus:masterfrom
lucperkins:lperkins/blackbox-exporter-guide
Closed

Blackbox Exporter guide#1134
lucperkins wants to merge 1 commit intoprometheus:masterfrom
lucperkins:lperkins/blackbox-exporter-guide

Conversation

@lucperkins
Copy link
Contributor

@lucperkins lucperkins commented Jul 31, 2018

cc @brian-brazil @juliusv @RichiH

This is now ready for review.

@lucperkins lucperkins changed the title [WIP: not yet reviewable] Blackbox Exporter guide Blackbox Exporter guide Aug 1, 2018
Signed-off-by: lucperkins <lucperkins@gmail.com>
@lucperkins
Copy link
Contributor Author

Friendly ping on this. Currently ready for review.

@brian-brazil
Copy link
Contributor

I likely won't get to this until Tuesday. One initial comment is that building out a custom Python app including a full framework is unnecessary. Use what you have already, such as the node exporter.

method: GET
```

With this configuration, the Blackbox Exporter will use the built-in `http_2xx` module, which periodically probes HTTP endpoints (the interval is set in your Prometheus configuration).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"the built-in http_2xx module" -> The name http_2xx is actually arbitary and user-chosen in the configuration, so there's no built-in module http_2xx. It would be more correct to say that this configuration defines a http_2xx module that uses the built-in http prober.

## Installing and running the Node Exporter

The Prometheus Node Exporter is a single static binary that you can install [via tarball](#tarball-installation). Once you've downloaded it from the Prometheus [downloads page](/download#node_exporter) extract it, and run it:
The Prometheus Node Exporter is a single static binary that you can install via tarball. Once you've downloaded it from the Prometheus [downloads page](/download#node_exporter) extract it, and run it:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like this is an unrelated change that made it into this PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's an unrelated change but I felt it was small enough to include here (it fixes a broken link). I can put it in a separate PR if you'd like.

@juliusv
Copy link
Member

juliusv commented Aug 31, 2018

One initial comment is that building out a custom Python app including a full framework is unnecessary. Use what you have already, such as the node exporter.

  • If we just want to probe for a 200 response status code, I agree.
  • Another option would be to make the example more complex and return a specific string that we probe for in the HTTP response (you know, like checking that some API call contains expected fields) - that would be a better reason for introducing custom code, but it would also make the example less basic.
  • As a third option (and one I generally like), we could omit running the probe target altogether and just probe public targets like https//google.com, https://prometheus.io/, and https://twitter.com.

@RichiH
Copy link
Member

RichiH commented Aug 31, 2018

A fourth option, which is probably out of scope here, would be to extend blackbox_exporter to tackle more complex checks.

More within scope would be the keep a truly basic howto (Julius #3) and extend to a more complex custom API checker thing later (Julius #2)

@lucperkins
Copy link
Contributor Author

@juliusv I do like the simplicity of probing a public target like that. I'll push a new draft that takes that approach and we'll see what Richie and Brian have to say 😄

@brian-brazil
Copy link
Contributor

The risk with public targets is that the content may change, or we could be limited by anti-abuse stuff. You will also run into v6 issues on all those endpoints. Something local is safest.

module: [http_2xx]
static_configs:
- targets:
- localhost:2112/health
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be safest to quote this, it's less likely the user will have issues later as they adjust it

replacement: localhost:9115
```

This instructs Prometheus to scrape the Blackbox Exporter's `/probe` endpoint and passes the `targets` list, which includes the Python web server running on port 2112, to the Blackbox Exporter. With this configuration in place, start Prometheus:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

target is a single item


## Installing, configuring, and running the Blackbox Exporter

Like Prometheus, the Blackbox Exporter is a single static binary that you can install via tarball. [Download the latest release](/download#blackbox_exporter) for your platform and untar it:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd setup and play with the blackbox exporter first, then talk about Prometheus

cd blackbox_exporter-*.*-amd64
```

The untarred folder contains a `blackbox.yml` file for configuring the exporter. Delete the current contents and replace it with this:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already the default, what is the benefit of the user changing this?


With this configuration, the Blackbox Exporter will use the built-in `http_2xx` module, which periodically probes HTTP endpoints (the interval is set in your Prometheus configuration).

NOTE: In the Blackbox Exporter configuration, you don't need to specify host or port information for the specific endpoints that you want to probe, only which modules you'd like to use. Host and port information is specifed in the [Prometheus configuration](#prometheus-configuration).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They're targets, not host:port. HTTP can have more, icmp is just host.

`http_2xx` | `localhost:2112/health` | Success | Logs
`http_2xx` | `localhost:2112/health` | Success | Logs

In the Prometheus [expression browser](/docs/visualization/browser), accessible at http://localhost:9090/graph, you can a variety of Blackbox-Exporter-specific metrics, which have the `probe_` prefix, for example, the [`probe_success{instance="localhost:2112/health",job="blackbox"}`](http://localhost:9090/graph?g0.range_input=1h&g0.expr=probe_success{instance%3D%localhost%3A2112%2Fhealth%22}), which indicates whether the most recent probe was successful.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to talk more about probe_success, many users miss the importance of this

Maybe setup an alert?

I'd also mention the debug URL parameter.

aylei pushed a commit to aylei/docs that referenced this pull request Oct 28, 2019
…#1134)

Description for shared block cache was added in prometheus#1114, but it fail to mention the change is for 3.0 only.
@roidelapluie
Copy link
Member

@lucperkins Are you still interested by this?

@lucperkins
Copy link
Contributor Author

@roidelapluie I'm no longer with the CNCF, so it's unlikely that I will ever complete this.

@lucperkins lucperkins closed this Jul 22, 2020
@roidelapluie
Copy link
Member

@lucperkins Can I take over your branch and take it to the finish line?

@lucperkins
Copy link
Contributor Author

@roidelapluie Absolutely! You have my full endorsement 💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants