Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nginx to {connect,login}Sever connection issues due to localhost IPv4 vs IPv6 #416

Closed
cpcallen opened this issue Sep 9, 2020 · 0 comments · Fixed by #418
Closed

nginx to {connect,login}Sever connection issues due to localhost IPv4 vs IPv6 #416

cpcallen opened this issue Sep 9, 2020 · 0 comments · Fixed by #418
Labels
bug /connect Client-side issues related to the /connect world & log views
Projects
Launch: Alpha
Milestone
Launch: Alpha

Comments

@cpcallen
Copy link
Collaborator

cpcallen commented Sep 9, 2020 •
edited
Loading

The change made in 953cb29 to add proxy_next_upstream_tries 1; to our nginx config files has an unanticipated side effect: 50% of connections to loginServer and connectSever fail, because localhost is defined in /etc/hosts twice:

Debian 10:

127.0.0.1       localhost
::1             localhost ip6-localhost ip6-loopback

macOS Catalina:

127.0.0.1       localhost
::1             localhost

It seems that the backends consistently listen on 127.0.0.1, but nginx round-robins between 127.0.0.1 and ::1, and if it happens to pick the latter it fails.

Previously it would automatically retry these failed connections (and succeed on the second attempt) because proxy_next_upstream_tries defaulted to 2, but now they generate a 500 error (which in turn generates another error because loginServer and connectServer do not serve /static/503.html on their respective domains.
@cpcallen cpcallen added bug /connect Client-side issues related to the /connect world & log views labels Sep 9, 2020
@cpcallen cpcallen added this to the Launch: Alpha milestone Sep 9, 2020
@cpcallen cpcallen added this to To do in Launch: Alpha via automation Sep 9, 2020
@cpcallen cpcallen mentioned this issue Sep 9, 2020
Merged
cpcallen added a commit that referenced this issue Sep 9, 2020
@cpcallen
Use IPv4 to connect to backends
81161fe 
No specific reason to prefer IPv4 over IPv6, but since loginServer and
connectServer are bound to 127.0.0.1 and not ::1 then go with that.

Fixes #416.
@cpcallen cpcallen mentioned this issue Sep 9, 2020
Merged
Launch: Alpha automation moved this from To do to Done Sep 10, 2020
cpcallen added a commit that referenced this issue Sep 10, 2020
@cpcallen
Use IPv4 to connect to backends
3570fa0 
No specific reason to prefer IPv4 over IPv6, but since loginServer and
connectServer are bound to 127.0.0.1 and not ::1 then go with that.

Fixes #416.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug /connect Client-side issues related to the /connect world & log views
Projects
Launch: Alpha
  
Done
Milestone
Launch: Alpha
Development

Successfully merging a pull request may close this issue.

More nginx config improvments google/CodeCity
1 participant
@cpcallen