Cache permanent redirects

[HebaruSan]

Motivation

This is a follow-up to #3387. Passing the token after a redirect solved the error spam, but the Inflator will still attempt to retrieve out of date URLs even after the server tells us it is out of date.

Changes

Now we save HTTP redirects that we receive in a Dictionary and use them to skip the old URL if it is needed again. This will somewhat reduce our GitHub API usage, leaving more slack to index more mods.

[DasSkelett]

We should make sure to cache only actually permanent redirects, i.e. those returned with a status code of 301 and 308.
For example, we also see some dynamically generated, authenticated temporary redirects for release assets:

1917 [1] WARN CKAN.Net (null) - Redirecting https://github.com/UmbraSpaceIndustries/USI-LS/releases/download/1.4.0/USI-LS_1.4.0.0.zip 
with 302 to 
https://github-releases.githubusercontent.com/34215491/25592580-43ab-11eb-91ba-f787d066b019?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=<IDontKnowIfThisShouldBeSharedPublicly>%2F20210608%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210608T143049Z&X-Amz-Expires=300&X-Amz-Signature=<IDontKnowIfThisShouldBeSharedPublicly>&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=34215491&response-content-disposition=attachment%3B%20filename%3DUSI-LS_1.4.0.0.zip&response-content-type=application%2Foctet-stream

[DasSkelett]

The permanentRedirects dict needs to be static so it's shared between all instances of RedirectingTimeoutWebClient, otherwise it's not of much use, because the web clients are very short lived (created and disposed within Download()/DownloadText().

[HebaruSan]

We should make sure to cache only actually permanent redirects, i.e. those returned with a status code of 301 and 308.

The permanentRedirects dict needs to be static so it's shared between all instances of RedirectingTimeoutWebClient

Oof, good catches. Both done in force push.
(I wanted to use the HttpStatusCode.PermanentRedirect enum element, but that isn't defined in our version of .NET, so I used ints instead.)

[DasSkelett]

I'm not sure if restricting the Authorization header carry over logic to 301s and 308s as well is what we want here, in theory it also applies to other redirects. In practice though, we only get 301s where it's relevant, the only other redirect I see is the 302 from above, which also leads to another host and has separate authorization data baked into the URL.
So I think we're fine, if not, GitHub will tell us loudly :P

[HebaruSan]

So I think we're fine, if not, GitHub will tell us loudly :P

Ohhh. No, let me fix that. I forgot that there was more logic going on here than just the caching...

[DasSkelett]

Thanks, looks very good now, and works fine in my hacky testing.