Provide a way for the plugin writer to silence warnings from downloaders

[dralley]

closes: #9317
https://pulp.plan.io/issues/9317

[pulpbot]

Attached issue: https://pulp.plan.io/issues/9317

[mdellweg]

This does not cover 404s because there is no need to retry. Right?

[dralley]

@mdellweg It covers 404s as well. That counts as a "failure" it just happens immediately and doesn't attempt to retry. Basically if quiet=True you won't see anything in the logs about these downloads.

The purpose is we sometimes try to download files that are optional and not guaranteed to be there, so we don't want to log errors in the event that they are not.

Here's a plugin example: pulp/pulp_rpm#2112

[mdellweg]

That sounds great. Attempting to download metadata that might be there can be quite common.

[bmbouter]

I want to think this over some. This is a leakly abstraction because base has no notion of backoff events, that's a pure HTTPDownloader thing.

[dralley]

We could use *args and **kwargs instead. I didn't want to do that, because it might tempt passing them on to _run(), which would be improper.

[bmbouter]

We keep running into this pattern actually. Maybe we should just put *args and **kwargs onto BaseDownloader

[bmbouter]

How can we integrate this with the existing raise_for_status feature. We've been having plugin writers already using that which silences the 400+ errors already.

[bmbouter]

Like this actually? Maybe your Pr should replace this usage?

[dralley]

@bmbouter I think raising exceptions is necessary for the backoffs to occur properly though, so we can't touch that. Backoff expects exceptions to be thrown when it fails and checks those exceptions to decide what to do next.

It actually covers a separate use case because we don't want to touch the mechanism of the download at all, just avoid logging it.

[bmbouter]

I think I'm still slow from coming back from paternity. When I read "The purpose is we sometimes try to download files that are optional and not guaranteed to be there, so we don't want to log errors in the event that they are not." I think to myself that is why we made the silence_errors_for_response_status_codes argument here. What you've made here just seems a lot nicer, but there is so much overlap with what we have already I want to somehow not have two things providing piecemeal value. @dralley wdyt?

[dralley]

@bmbouter You know, I think this actually does work, but I'm still kind of confused about the mechanism. I'll spend a few minutes digging into the code and then close this PR if it turns out we don't need it.

edit: I think this is probably OK.