New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nzb download fails due to HTTP/1.1 307 Temporary Redirect #420

Closed
thmsndk opened this Issue Jul 27, 2017 · 7 comments

Comments

Projects
None yet
3 participants
@thmsndk

thmsndk commented Jul 27, 2017

One of my indexers changed some of their setup resulting in HTTP 307 redirects when trying to download the nzb file.

As a result I see the following lines in my log:

  • URL XXXXX @ YYYY failed: HTTP/1.1 307 Temporary Redirect
  • Download XXXXX @ YYYY failed

If I take the url for the .nzb and paste it into my browser, it correctly redirects and I get a HTTP 200 with the file.

It would appear nzbget does not handle redirects?

I'm currently running on v18, but a friend of mine running v19 has the same experience :)

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 27, 2017

Member

It does support redirects but it doesn't support HTTP 1.1. The web-server misbehaves if it responds with HTTP 1.1 on requests received via HTTP 1.0.

Member

hugbug commented Jul 27, 2017

It does support redirects but it doesn't support HTTP 1.1. The web-server misbehaves if it responds with HTTP 1.1 on requests received via HTTP 1.0.

@hugbug hugbug added the support label Jul 27, 2017

@king-cat

This comment has been minimized.

Show comment
Hide comment
@king-cat

king-cat Jul 27, 2017

He's talking about my server. Do you have insight on this @hugbug ? We are using a load balancer now in front of the cache servers and that seems to be when this issue started.

king-cat commented Jul 27, 2017

He's talking about my server. Do you have insight on this @hugbug ? We are using a load balancer now in front of the cache servers and that seems to be when this issue started.

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 27, 2017

Member

I thought nzbget doesn't like 1.1 in response. But after checking the code I can tell it doesn't check the version in response header. Moreover the spec says that the version in response header is to indicate which version server supports. Therefore 1.1 is OK here. However in the response server can only use features available in the version used in request.

Response code 307 is from version 1.1 and should not be used here. In that regard the server indeed behaves bad. NZBGet supports redirecting with codes 301 and 302. I'll add support for code 307 as this is no brainer but that doesn't help all existing clients.

If you can configure balancer to use code 301 or 302 that should fix the issue.

Member

hugbug commented Jul 27, 2017

I thought nzbget doesn't like 1.1 in response. But after checking the code I can tell it doesn't check the version in response header. Moreover the spec says that the version in response header is to indicate which version server supports. Therefore 1.1 is OK here. However in the response server can only use features available in the version used in request.

Response code 307 is from version 1.1 and should not be used here. In that regard the server indeed behaves bad. NZBGet supports redirecting with codes 301 and 302. I'll add support for code 307 as this is no brainer but that doesn't help all existing clients.

If you can configure balancer to use code 301 or 302 that should fix the issue.

@king-cat

This comment has been minimized.

Show comment
Hide comment
@king-cat

king-cat Jul 27, 2017

Looking in my server and load balancer logs I do not see a 307 returned anywhere. Additionally after I did some research on the 307 code specifically on google it looks like 307 is an "internal redirect" on the client. It's something that the browser does for some reason. I'd be happy to facilitate fixing this from the server side but I don't know how. No where is it doing 307 redirects explicitly. It only redirects http to https via 301 permanent redirect. (it will also do 302 redirects if someone tries to go to a page that requires authentication on the front-end but this should not affect any api-client like nzbget)

king-cat commented Jul 27, 2017

Looking in my server and load balancer logs I do not see a 307 returned anywhere. Additionally after I did some research on the 307 code specifically on google it looks like 307 is an "internal redirect" on the client. It's something that the browser does for some reason. I'd be happy to facilitate fixing this from the server side but I don't know how. No where is it doing 307 redirects explicitly. It only redirects http to https via 301 permanent redirect. (it will also do 302 redirects if someone tries to go to a page that requires authentication on the front-end but this should not affect any api-client like nzbget)

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 27, 2017

Member

Who can try with "wget"? I believe it shows all redirects.

Member

hugbug commented Jul 27, 2017

Who can try with "wget"? I believe it shows all redirects.

@king-cat

This comment has been minimized.

Show comment
Hide comment
@king-cat

king-cat Jul 27, 2017

Ok first of all, I did a wget with a getnzb link from an RSS feed and I did indeed see a 307 redirect. It was still something wget generated but has to do with HSTS headers that tell the client to always use HTTPS. The links themselves were incorrectly being output as http:// causing a redirect for every download as well as generating that hsts 307 internal redirect.

So @hugbug was correct, this was a bug in our system, however, redirecting http to https via an HSTS header is something that nzbget should support I would imagine so it might be something to look into regardless.

For the record I have fixed this bug and all future download links will properly be displayed as https forgoing the need for a redirect at all.

Thanks for the heads up on this @thmsndk

king-cat commented Jul 27, 2017

Ok first of all, I did a wget with a getnzb link from an RSS feed and I did indeed see a 307 redirect. It was still something wget generated but has to do with HSTS headers that tell the client to always use HTTPS. The links themselves were incorrectly being output as http:// causing a redirect for every download as well as generating that hsts 307 internal redirect.

So @hugbug was correct, this was a bug in our system, however, redirecting http to https via an HSTS header is something that nzbget should support I would imagine so it might be something to look into regardless.

For the record I have fixed this bug and all future download links will properly be displayed as https forgoing the need for a redirect at all.

Thanks for the heads up on this @thmsndk

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 27, 2017

Member

So @hugbug was correct, this was a bug in our system, however, redirecting http to https via an HSTS header is something that nzbget should support I would imagine so it might be something to look into regardless.

NZBGet would need to store somewhere HSTS data for all sites and apply it on each request. That would be too much work for such a small http client. The idea of HSTS is to protect users who type http-addresses into browsers address string. That's not a use case for nzbget.

Adding support for 307 code however is easy and shouldn't do any harm.

Member

hugbug commented Jul 27, 2017

So @hugbug was correct, this was a bug in our system, however, redirecting http to https via an HSTS header is something that nzbget should support I would imagine so it might be something to look into regardless.

NZBGet would need to store somewhere HSTS data for all sites and apply it on each request. That would be too much work for such a small http client. The idea of HSTS is to protect users who type http-addresses into browsers address string. That's not a use case for nzbget.

Adding support for 307 code however is easy and shouldn't do any harm.

hugbug added a commit that referenced this issue Aug 4, 2017

#420: support for redirect codes 303, 307 and 308
in web-client for fetching of rss feeds and nzb-files

@hugbug hugbug closed this Aug 4, 2017

@hugbug hugbug added the improvement label Aug 4, 2017

@hugbug hugbug added this to the v20 milestone Aug 4, 2017

hugbug added a commit that referenced this issue Oct 9, 2017

#420: support for redirect codes 303, 307 and 308
in web-client for fetching of rss feeds and nzb-files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment