Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

504 Gateway Time-out, The server didn't respond in time. #173

Closed
ChloeMackey opened this issue Sep 18, 2018 · 19 comments

Comments

@ChloeMackey
Copy link

commented Sep 18, 2018

Anyone else getting that message when trying to access their subscriptions?

@Zero3K

This comment has been minimized.

Copy link

commented Sep 18, 2018

I can confirm that it is happening for me. I thought it was fixed.

@omarroth

This comment has been minimized.

Copy link
Owner

commented Sep 18, 2018

It's a problem that occurs when the load reaches a certain threshold. So it's not so much permanently fixed as the server is upgraded to handle the increased usage.

@EsmailELBoBDev2

This comment has been minimized.

Copy link
Contributor

commented Sep 18, 2018

@omarroth i getting same error when i try open https://invidio.us/feed/subscriptions

@dimqua

This comment has been minimized.

Copy link
Contributor

commented Sep 18, 2018

Me too.

@Discookie

This comment has been minimized.

Copy link

commented Sep 19, 2018

I think it's also an issue that the site doesn't have proper error pages:

  • for 503/504 one that would explain that the servers are overloaded
  • for 500 one that would point them to make an issue here
  • etc etc.

I know some errors (like 404) do have their own pages, but I think all but the worst errors should be caught before the user sees them.

@d3d1rty

This comment has been minimized.

Copy link

commented Sep 19, 2018

It's a problem that occurs when the load reaches a certain threshold. So it's not so much permanently fixed as the server is upgraded to handle the increased usage.

Any idea of what the associated costs would be with upgrading the server to handle it?

@omarroth

This comment has been minimized.

Copy link
Owner

commented Sep 19, 2018

Right now I'm looking into optimizing rather than upgrading, but if/when I were to upgrade, it would probably be about a ~$20 increase.

@omarroth

This comment has been minimized.

Copy link
Owner

commented Sep 20, 2018

I just upgraded the server, and now am able to access my feed. If others find the same, then we can probably close this for now. I'm still looking into optimizing it as much as I can, but in the short-term this appears to be fixed.

@Discookie

This comment has been minimized.

Copy link

commented Sep 20, 2018

Looks fixed on my end, even though it still takes 20-odd seconds to load.

Also made a separate issue for proper error pages, instead of burying it here.

@omarroth

This comment has been minimized.

Copy link
Owner

commented Sep 20, 2018

Thanks. For folks that would like to see how Invidious queries the DB, I'd take a look here, and the table schema is here. I'd be very interested and thankful for thoughts as to how it could be improved.

@Discookie

This comment has been minimized.

Copy link

commented Sep 21, 2018

I don't see why the index channel_videos_ucid_idx is using btree instead of hash. It would gain quite a bit of performance, and I don't think you use < or > comparisons on it.

Another marginal gain in query speed would be to store all video ID's as 64-bit integers.

The id stores a variant of b64 characters - together these characters add up to 66 bits. But YouTube just doesn't care about the last 2 bits, so all of these open the same video:
https://www.youtube.com/watch?v=dvz9lsaIKLY
https://www.youtube.com/watch?v=dvz9lsaIKLZ
https://www.youtube.com/watch?v=dvz9lsaIKLa
https://www.youtube.com/watch?v=dvz9lsaIKLb

Channel IDs could be similarly shortened, although they are 132 bits long, and I don't know whether they ignore any bits or not.

@omarroth

This comment has been minimized.

Copy link
Owner

commented Sep 21, 2018

Thank you! That definitely helps a lot.

The comment about video IDs is interesting, but unfortunately the examples provided (except the first) return a Does not exist. My understanding is that YouTube uses some variant of hashids internally.

@Discookie

This comment has been minimized.

Copy link

commented Sep 21, 2018

Yeah, if you query them, they don't return anything, but if you open them on Youtube, they redirect to the first link. (This seems like a separate issue that needs to be fixed?)
A common example is linking the Rickroll video with the ending of XcR, XcS, etc. to trick you into clicking on an unvisited link :P

@omarroth

This comment has been minimized.

Copy link
Owner

commented Sep 22, 2018

Opening the links on YouTube results in a Does not exist as well. If a videoID is longer than 11 characters then YouTube will strip the extraneous ones, and Invidious does that as well. For example:

Unfortunately, I'm still encountering this issue, if there's any other suggestions I would be very eager to hear them.

@omarroth

This comment has been minimized.

Copy link
Owner

commented Oct 9, 2018

I'm pushing out 35e63fa right now which should speed up subscriptions significantly, and we can hopefully close this. There's some more info on how it works in yesterday's release.

I expect the migration to take ~1.5 hours. Everything except subscriptions should still be available during this time. Thanks everyone.

EDIT: Migration should be complete. I'd recommend folks check their feed to make sure everything is working properly.

Of note is that since the feed is cached, it may take a bit to update, even if there are notifications for new videos. I'd like to fix that as well, and if feeds are now displaying properly, then I'll close this in favor of #148.

@d3d1rty

This comment has been minimized.

Copy link

commented Oct 9, 2018

Just checked and the issue seems to be corrected. Much faster response. I had previously adjusted my settings to only show unseen notifications, so I reset it so that it shows everything with 25 results in the feed. First attempt was successful, super fast response, so I adjusted the results to 50 and then 100 videos in the feed. I ran 3 attempts per setting (25, 50, and 100) and recorded the results below.

Response Times:

  • 25 videos: 70 ms, 119 ms, 96 ms
  • 50 videos: 76 ms, 166 ms, 385 ms
  • 100 videos: 225 ms, 133 ms, 143 ms
@omarroth omarroth added the bug label Oct 13, 2018
@omarroth

This comment has been minimized.

Copy link
Owner

commented Nov 2, 2018

I'd be very interested to investigate, would you mind opening a new issue?

@omarroth

This comment has been minimized.

Copy link
Owner

commented Nov 7, 2018

It seems no one has encountered this for about a month, so I think I'll close this. If anyone encounters this again feel free to open a new issue, or I'll be happy to reopen.

@omarroth omarroth closed this Nov 7, 2018
@jeroen7s

This comment has been minimized.

Copy link

commented Dec 12, 2018

now i'm having this issue when checking out my subscriptions
EDIT: I do have almost 500 subscriptions though, which might contribute to the problem :/
EDIT2: its suddenly gone now, page loads in about 1.5 seconds

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.