New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RSS Hammering #416

Closed
king-cat opened this Issue Jul 24, 2017 · 11 comments

Comments

Projects
None yet
4 participants
@king-cat

king-cat commented Jul 24, 2017

Hi, I run an indexer and I was looking through my logs and I noticed a tremendous amount of requests from many different users using nzbget with RSS feeds on my site. It looks like NZBGet is making dozens of requests in a single minute for the same RSS feed URL. Is this expected behavior? Here's a snippet of my log file:

x.x.x.x - - [23/Jul/2017:21:24:26 -0400] "GET /rss?t=-2&dl=1&i=[UID]&r=[APIKEY]&del=1 HTTP/1.0" 404 162 "-" "nzbget/18.0" "-" 
x.x.x.x - - [23/Jul/2017:21:24:45 -0400] "GET /rss?t=-2&dl=1&i=[UID]&r=[APIKEY]&del=1 HTTP/1.0" 404 162 "-" "nzbget/18.0" "-" 
x.x.x.x - - [23/Jul/2017:21:24:28 -0400] "GET /rss?t=-2&dl=1&i=[UID]&r=[APIKEY]&del=1 HTTP/1.0" 404 162 "-" "nzbget/18.0" "-" 
x.x.x.x - - [23/Jul/2017:21:24:24 -0400] "GET /rss?t=-2&dl=1&i=[UID]&r=[APIKEY]&del=1 HTTP/1.0" 404 162 "-" "nzbget/18.0" "-" 
x.x.x.x - - [23/Jul/2017:21:24:14 -0400] "GET /rss?t=-2&dl=1&i=[UID]&r=[APIKEY]&del=1 HTTP/1.0" 404 162 "-" "nzbget/18.0" "-" 
x.x.x.x - - [23/Jul/2017:21:24:11 -0400] "GET /rss?t=-2&dl=1&i=[UID]&r=[APIKEY]&del=1 HTTP/1.0" 404 162 "-" "nzbget/18.0" "-" 
x.x.x.x - - [23/Jul/2017:21:24:04 -0400] "GET /rss?t=-2&dl=1&i=[UID]&r=[APIKEY]&del=1 HTTP/1.0" 404 162 "-" "nzbget/18.0" "-" 
x.x.x.x - - [23/Jul/2017:21:23:59 -0400] "GET /rss?t=-2&dl=1&i=[UID]&r=[APIKEY]&del=1 HTTP/1.0" 404 162 "-" "nzbget/18.0" "-" 
x.x.x.x - - [23/Jul/2017:21:24:09 -0400] "GET /rss?t=-2&dl=1&i=[UID]&r=[APIKEY]&del=1 HTTP/1.0" 404 162 "-" "nzbget/18.0" "-" 
x.x.x.x - - [23/Jul/2017:21:24:19 -0400] "GET /rss?t=-2&dl=1&i=[UID]&r=[APIKEY]&del=1 HTTP/1.0" 404 162 "-" "nzbget/18.0" "-" 
x.x.x.x - - [23/Jul/2017:21:24:07 -0400] "GET /rss?t=-2&dl=1&i=[UID]&r=[APIKEY]&del=1 HTTP/1.0" 404 162 "-" "nzbget/18.0" "-" 
x.x.x.x - - [23/Jul/2017:21:24:21 -0400] "GET /rss?t=-2&dl=1&i=[UID]&r=[APIKEY]&del=1 HTTP/1.0" 404 162 "-" "nzbget/18.0" "-" 

They all have a 404 status because they triggered my rate limiting protections. The above log is for a single user but I'm seeing this behavior from over a dozen different users so that makes me think it's an error in nzbget or a misconfiguration somewhere.

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 24, 2017

Member

Currently it tries until it get valid data. The issue is it doesn't differentiate between broken connection and a response with an error code like this.

I'll improve this to accept 404 as a valid response which doesn't require retry. Any other error codes I should treat this way?

Member

hugbug commented Jul 24, 2017

Currently it tries until it get valid data. The issue is it doesn't differentiate between broken connection and a response with an error code like this.

I'll improve this to accept 404 as a valid response which doesn't require retry. Any other error codes I should treat this way?

@hugbug hugbug added the improvement label Jul 24, 2017

@hugbug hugbug added this to the v19.x milestone Jul 24, 2017

@king-cat

This comment has been minimized.

Show comment
Hide comment
@king-cat

king-cat Jul 24, 2017

king-cat commented Jul 24, 2017

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 24, 2017

Member

only query the RSS feeds every 15 minutes

That's what nzbget is doing. However an error during fetch is considered a problem and it retries again and again...

As a workaround (until a fix is available in nzbget) you could return an empty rss feed to make nzbget happy.

Member

hugbug commented Jul 24, 2017

only query the RSS feeds every 15 minutes

That's what nzbget is doing. However an error during fetch is considered a problem and it retries again and again...

As a workaround (until a fix is available in nzbget) you could return an empty rss feed to make nzbget happy.

@Nothing4You

This comment has been minimized.

Show comment
Hide comment
@Nothing4You

Nothing4You Jul 24, 2017

@king-cat it'd make more sense for your setup to use one of the 4xx error codes, especially 429 Too Many Requests (see https://tools.ietf.org/html/rfc6585#page-3) to stay in line with the meaning of http status codes.

Imo nzbget shouldn't just skip a full fetch interval because it might have caught a server overload and it's working fine like a minute later. For something like 429 it could use e.g. an exponentially increasing (with upper limit) wait timer between following requests. Might make sense for other errors aswell.
An example of what i'm using for a different project could be min(5*2**error_count, 86400) starting with 10s delay up to 24h delay.

Edit:
Obviously, if available, a Retry-After header should be honored instead of custom wait intervals as specified in the linked rfc.

Nothing4You commented Jul 24, 2017

@king-cat it'd make more sense for your setup to use one of the 4xx error codes, especially 429 Too Many Requests (see https://tools.ietf.org/html/rfc6585#page-3) to stay in line with the meaning of http status codes.

Imo nzbget shouldn't just skip a full fetch interval because it might have caught a server overload and it's working fine like a minute later. For something like 429 it could use e.g. an exponentially increasing (with upper limit) wait timer between following requests. Might make sense for other errors aswell.
An example of what i'm using for a different project could be min(5*2**error_count, 86400) starting with 10s delay up to 24h delay.

Edit:
Obviously, if available, a Retry-After header should be honored instead of custom wait intervals as specified in the linked rfc.

@king-cat

This comment has been minimized.

Show comment
Hide comment
@king-cat

king-cat Jul 24, 2017

Good suggestion. I will switch the 503 to 429 for now. As a service provider, I would rather nzbget skip an interval rather than hammer my site, even if it's down. It seems like a lazy way to go about it. I don't know how sonarr handles it, but i don't see such hammers from them so it might be worth it to investigate their logic and copy it. They have been exemplary as far as service usage goes.

king-cat commented Jul 24, 2017

Good suggestion. I will switch the 503 to 429 for now. As a service provider, I would rather nzbget skip an interval rather than hammer my site, even if it's down. It seems like a lazy way to go about it. I don't know how sonarr handles it, but i don't see such hammers from them so it might be worth it to investigate their logic and copy it. They have been exemplary as far as service usage goes.

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 24, 2017

Member

Skiping an interval is also the easiest thing to do on nzbget side. With the default update interval of 15 minutes I think that's an acceptable solution.

Member

hugbug commented Jul 24, 2017

Skiping an interval is also the easiest thing to do on nzbget side. With the default update interval of 15 minutes I think that's an acceptable solution.

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 24, 2017

Member

Done.

That's how it works:

  • if connection to server cannot be established multiple attempts are done in a short time (options UrlRetries (3 by default) and UrlInterval (10 seconds by default). If all retries fail the fetch attempt is failed. If the server responded with an error code the fetch attempt is failed immediately. That part is not new.
  • on fetch failure NZBGet waits 1 minute before another fetch attempt, if failed again it waits 2 minutes, then 4 and so on up to feed interval limit (by default 15 minutes).
  • program reload resets wait timer meaning the feed will be fetched immediately after reload, then in 1 minute, 2 minutes, etc.
Member

hugbug commented Jul 24, 2017

Done.

That's how it works:

  • if connection to server cannot be established multiple attempts are done in a short time (options UrlRetries (3 by default) and UrlInterval (10 seconds by default). If all retries fail the fetch attempt is failed. If the server responded with an error code the fetch attempt is failed immediately. That part is not new.
  • on fetch failure NZBGet waits 1 minute before another fetch attempt, if failed again it waits 2 minutes, then 4 and so on up to feed interval limit (by default 15 minutes).
  • program reload resets wait timer meaning the feed will be fetched immediately after reload, then in 1 minute, 2 minutes, etc.

@hugbug hugbug closed this Jul 25, 2017

hugbug added a commit that referenced this issue Jul 26, 2017

hugbug added a commit that referenced this issue Oct 9, 2017

@denalain

This comment has been minimized.

Show comment
Hide comment
@denalain

denalain Dec 13, 2017

i have just installed nzbget on a synology ds218play ( I have tried nzbget-19.1-bin-linux.run and nzbget-20.0-testing-r2159-bin-linux.run ). And I have a problem with rss fetch.

Here's the log:

Wed Dec 13 21:36:54 2017 ERROR Download rssfeed failed
Wed Dec 13 21:36:54 2017 DETAIL Scheduling update for feed rssfeed in 4 minute(s)
Wed Dec 13 21:40:54 2017 DETAIL Downloading rssfeed
Wed Dec 13 21:40:54 2017 WARNING URL rssfeed failed: HTTP/1.1 429 Too Many Requests

I just keep getting 429 Too Many requests.

The feed Interval is set to 15 minutes.

To prevent from beeing kicked, I have set the URL retries to 1 and UrlInterval to 60 seconds.

Manual rss fetch gets the same error

denalain commented Dec 13, 2017

i have just installed nzbget on a synology ds218play ( I have tried nzbget-19.1-bin-linux.run and nzbget-20.0-testing-r2159-bin-linux.run ). And I have a problem with rss fetch.

Here's the log:

Wed Dec 13 21:36:54 2017 ERROR Download rssfeed failed
Wed Dec 13 21:36:54 2017 DETAIL Scheduling update for feed rssfeed in 4 minute(s)
Wed Dec 13 21:40:54 2017 DETAIL Downloading rssfeed
Wed Dec 13 21:40:54 2017 WARNING URL rssfeed failed: HTTP/1.1 429 Too Many Requests

I just keep getting 429 Too Many requests.

The feed Interval is set to 15 minutes.

To prevent from beeing kicked, I have set the URL retries to 1 and UrlInterval to 60 seconds.

Manual rss fetch gets the same error

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Dec 13, 2017

Member

It works as it should. See my previous comment.

Member

hugbug commented Dec 13, 2017

It works as it should. See my previous comment.

@denalain

This comment has been minimized.

Show comment
Hide comment
@denalain

denalain Dec 13, 2017

I have just tried version 16.4-21 installed via synocommunity on a ds209+II. rss fetching works fine. No timeout, no error.

on version 19.1 and 20.0 the retries work as you mentioned as they should. But there is a problem with the fetch, because on version 16.4 I do not have a problem.

denalain commented Dec 13, 2017

I have just tried version 16.4-21 installed via synocommunity on a ds209+II. rss fetching works fine. No timeout, no error.

on version 19.1 and 20.0 the retries work as you mentioned as they should. But there is a problem with the fetch, because on version 16.4 I do not have a problem.

@denalain

This comment has been minimized.

Show comment
Hide comment
@denalain

denalain Dec 13, 2017

appologies. I have installed version 16.4 and I also receive 429 Too Many Requests. So the problems seems to be linked to ds218play. Any idea, why?

denalain commented Dec 13, 2017

appologies. I have installed version 16.4 and I also receive 429 Too Many Requests. So the problems seems to be linked to ds218play. Any idea, why?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment