Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v4.1.5 timing out at 10 seconds on 'RedownloadMediaWorker' pull jobs #26131

Open
mejofi opened this issue Jul 23, 2023 · 1 comment
Open

v4.1.5 timing out at 10 seconds on 'RedownloadMediaWorker' pull jobs #26131

mejofi opened this issue Jul 23, 2023 · 1 comment
Labels
bug Something isn't working status/to triage This issue needs to be triaged

Comments

@mejofi
Copy link

mejofi commented Jul 23, 2023

Steps to reproduce the problem

RedownloadMediaWorker pull jobs timing out on downloading media at 10 seconds, before the remote endpoint throws a 503 at 15 seconds

Expected behaviour

Wait for the remote endpoint, register the 503, retry again later

Actual behaviour

Times out at 10 seconds, moves to the dead jobs queue

Detailed description

It looks like the Jortage pool is currently having some issues, which results in RedownloadMediaWorker pull jobs throwing a HTTP::TimeoutError at 10 seconds, 5 seconds before the Fastly CDN that fronts it throws a 503, at 15 seconds.

Mastodon instance

ngmx.com

Mastodon version

v4.1.5

Technical details

Versions;

$ ruby --version
ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux]

$ node --version
v16.20.1

curl test;

$ time curl -I https://pool.jortage.com/robots.txt
HTTP/2 503 
server: Varnish
retry-after: 0
content-type: text/html; charset=utf-8
accept-ranges: bytes
date: Sun, 23 Jul 2023 16:07:55 GMT
via: 1.1 varnish
x-served-by: cache-ams21070-AMS
x-cache: MISS
x-cache-hits: 0
x-timer: S1690128460.720579,VS0,VE15282
strict-transport-security: max-age=31557600
alt-svc: h3=":443";ma=86400,h3-29=":443";ma=86400,h3-27=":443";ma=86400
content-length: 444


real	0m15,366s
user	0m0,026s
sys	0m0,013s

Sidekiq errors in the 'pull' queue, for RedownloadMediaWorker, throwing a HTTP::TimeoutError after 10 seconds with a message like Read timed out after 10 seconds on <URL>.

@mejofi mejofi added bug Something isn't working status/to triage This issue needs to be triaged labels Jul 23, 2023
@p37307
Copy link

p37307 commented Jul 24, 2023

Is it related to this patch that was merged after v4.1.5 came out.
#26116
#26055

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working status/to triage This issue needs to be triaged
Projects
None yet
Development

No branches or pull requests

2 participants