New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

waitress 1.0 doesn't work on linux without reachable DNS #149

Closed
romor opened this Issue Dec 10, 2016 · 12 comments

Comments

Projects
None yet
2 participants
@romor

romor commented Dec 10, 2016

I use waitress on a Debian server without Internet connection. Waitress does not start if the DNS server is not reachable:

Traceback (most recent call last):
  File "/usr/local/bin/pserve", line 9, in <module>
    load_entry_point('pyramid==1.6.1', 'console_scripts', 'pserve')()
  File "/usr/local/lib/python3.4/dist-packages/pyramid-1.6.1-py3.4.egg/pyramid/scripts/pserve.py", line 60, in main
    return command.run()
  File "/usr/local/lib/python3.4/dist-packages/pyramid-1.6.1-py3.4.egg/pyramid/scripts/pserve.py", line 399, in run
    serve()
  File "/usr/local/lib/python3.4/dist-packages/pyramid-1.6.1-py3.4.egg/pyramid/scripts/pserve.py", line 378, in serve
    server(app)
  File "/usr/local/lib/python3.4/dist-packages/PasteDeploy-1.5.2-py3.4.egg/paste/deploy/loadwsgi.py", line 189, in server_wrapper
    **context.local_conf)
  File "/usr/local/lib/python3.4/dist-packages/PasteDeploy-1.5.2-py3.4.egg/paste/deploy/util.py", line 55, in fix_call
    val = callable(*args, **kw)
  File "/usr/local/lib/python3.4/dist-packages/waitress-1.0.0-py3.4.egg/waitress/__init__.py", line 20, in serve_paste
    serve(app, **kw)
  File "/usr/local/lib/python3.4/dist-packages/waitress-1.0.0-py3.4.egg/waitress/__init__.py", line 11, in serve
    server = _server(app, **kw)
  File "/usr/local/lib/python3.4/dist-packages/waitress-1.0.0-py3.4.egg/waitress/server.py", line 78, in create_server
    sockinfo=sockinfo)
  File "/usr/local/lib/python3.4/dist-packages/waitress-1.0.0-py3.4.egg/waitress/server.py", line 176, in __init__
    self.effective_host, self.effective_port = self.getsockname()
  File "/usr/local/lib/python3.4/dist-packages/waitress-1.0.0-py3.4.egg/waitress/server.py", line 292, in getsockname
    self.socketmod.NI_NUMERICSERV)
socket.gaierror: [Errno -3] Temporary failure in name resolution

I am not sure whether this is related to issue #131, but at least this is still open in version 1.0.0. Version 0.9.0 does not show this problem.

@bertjwregeer

This comment has been minimized.

Show comment
Hide comment
@bertjwregeer

bertjwregeer Dec 12, 2016

Member
  • What are you using for the host/port or listen argument?
  • What is your /etc/nsswitch.conf file set to?

glibc also returns the wrong error code when it should simply return the IP address when calling getnameinfo according to this bug report: https://bugzilla.redhat.com/show_bug.cgi?id=221583

Please verify that you don't have dns after files in nsswitch, because otherwise glibc will do the wrong thing.

Member

bertjwregeer commented Dec 12, 2016

  • What are you using for the host/port or listen argument?
  • What is your /etc/nsswitch.conf file set to?

glibc also returns the wrong error code when it should simply return the IP address when calling getnameinfo according to this bug report: https://bugzilla.redhat.com/show_bug.cgi?id=221583

Please verify that you don't have dns after files in nsswitch, because otherwise glibc will do the wrong thing.

@romor

This comment has been minimized.

Show comment
Hide comment
@romor

romor Dec 12, 2016

  • I start pserve using a config file, which only specifies port = 80
  • /etc/nsswitch.conf corresponds to the default settings from Debian Jessie: hosts: files dns

romor commented Dec 12, 2016

  • I start pserve using a config file, which only specifies port = 80
  • /etc/nsswitch.conf corresponds to the default settings from Debian Jessie: hosts: files dns
@bertjwregeer

This comment has been minimized.

Show comment
Hide comment
@bertjwregeer

bertjwregeer Dec 12, 2016

Member

@romor do you have a /etc/resolv.conf? If you do, does it contain a "nameserver" entry?

Member

bertjwregeer commented Dec 12, 2016

@romor do you have a /etc/resolv.conf? If you do, does it contain a "nameserver" entry?

@romor

This comment has been minimized.

Show comment
Hide comment
@romor

romor Dec 12, 2016

Yes, I have a /etc/resolv.conf and it contains a nameserver entry.

But, as mentioned, the system is not always connected to Internet so the DNS server may not be reachable while starting Pyramid.

romor commented Dec 12, 2016

Yes, I have a /etc/resolv.conf and it contains a nameserver entry.

But, as mentioned, the system is not always connected to Internet so the DNS server may not be reachable while starting Pyramid.

@bertjwregeer

This comment has been minimized.

Show comment
Hide comment
@bertjwregeer

bertjwregeer Dec 12, 2016

Member

That shouldn't matter. That call (getnameinfo) should either return the hostname OR an ip address. That function call is instead raising an error that according to the man page for getnameinfo should not be raised.

Even if DNS is missing or doesn't resolve, it should return the IP address. Do you have an exact version number for Debian so that I can attempt to replicate the issue and figure out an appropriate work-around?

Member

bertjwregeer commented Dec 12, 2016

That shouldn't matter. That call (getnameinfo) should either return the hostname OR an ip address. That function call is instead raising an error that according to the man page for getnameinfo should not be raised.

Even if DNS is missing or doesn't resolve, it should return the IP address. Do you have an exact version number for Debian so that I can attempt to replicate the issue and figure out an appropriate work-around?

@romor

This comment has been minimized.

Show comment
Hide comment
@romor

romor Dec 12, 2016

I just recently installed it using the net-installer (debian-8.6.0-amd64-netinst.iso) and all packages are up-to-date.

uname -v yields #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19)

For testing purposes I just patched server.py:292to:

    def getsockname(self):
        try:
            host, port = self.socketmod.getnameinfo(
                self.socket.getsockname(),
                self.socketmod.NI_NUMERICSERV)
        except:
            host, port = ("localhost", None)
        return (host, port)

This worked then...

romor commented Dec 12, 2016

I just recently installed it using the net-installer (debian-8.6.0-amd64-netinst.iso) and all packages are up-to-date.

uname -v yields #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19)

For testing purposes I just patched server.py:292to:

    def getsockname(self):
        try:
            host, port = self.socketmod.getnameinfo(
                self.socket.getsockname(),
                self.socketmod.NI_NUMERICSERV)
        except:
            host, port = ("localhost", None)
        return (host, port)

This worked then...

@bertjwregeer

This comment has been minimized.

Show comment
Hide comment
@bertjwregeer

bertjwregeer Dec 12, 2016

Member

Sure, you're swallowing the error and returning pre-canned data.

It looks like glibc "fixed" this here: https://bugzilla.redhat.com/show_bug.cgi?id=204122

This of course does the wrong thing, I never explicitly ask for NI_NAMEREQD, and according to the man page this means I should get an IP address back instead of host name.

Apparently that breaks Postfix, so now the behaviour doesn't match the man page, which is what bug https://bugzilla.redhat.com/show_bug.cgi?id=221583 is all about (which wasn't fixed, ultimately).

The fix for waitress is to try and then except the gaierror, verify it is the name lookup error, and then re-try with self.socketmod.NI_NUMERICHOST or'ed with self.socketmod.NI_NUMERICSERV, this should bypass DNS resolution, and return the IP address directly, thereby doing the right thing.

Member

bertjwregeer commented Dec 12, 2016

Sure, you're swallowing the error and returning pre-canned data.

It looks like glibc "fixed" this here: https://bugzilla.redhat.com/show_bug.cgi?id=204122

This of course does the wrong thing, I never explicitly ask for NI_NAMEREQD, and according to the man page this means I should get an IP address back instead of host name.

Apparently that breaks Postfix, so now the behaviour doesn't match the man page, which is what bug https://bugzilla.redhat.com/show_bug.cgi?id=221583 is all about (which wasn't fixed, ultimately).

The fix for waitress is to try and then except the gaierror, verify it is the name lookup error, and then re-try with self.socketmod.NI_NUMERICHOST or'ed with self.socketmod.NI_NUMERICSERV, this should bypass DNS resolution, and return the IP address directly, thereby doing the right thing.

@bertjwregeer

This comment has been minimized.

Show comment
Hide comment
@bertjwregeer

bertjwregeer Dec 12, 2016

Member

Pin to 0.9.x for right now. I'll try and get a new release out sometime this week with the aforementioned changes.

Member

bertjwregeer commented Dec 12, 2016

Pin to 0.9.x for right now. I'll try and get a new release out sometime this week with the aforementioned changes.

@romor

This comment has been minimized.

Show comment
Hide comment
@romor

romor Dec 12, 2016

Thanks for your help.

romor commented Dec 12, 2016

Thanks for your help.

@bertjwregeer

This comment has been minimized.

Show comment
Hide comment
@bertjwregeer

bertjwregeer Jan 17, 2017

Member

@romor Sorry for not getting back to you earlier, could you try this PR and verify this fixes the issue: #153

Thanks!

Member

bertjwregeer commented Jan 17, 2017

@romor Sorry for not getting back to you earlier, could you try this PR and verify this fixes the issue: #153

Thanks!

@romor

This comment has been minimized.

Show comment
Hide comment
@romor

romor Jan 17, 2017

This fix works for me. Thank you.

romor commented Jan 17, 2017

This fix works for me. Thank you.

@bertjwregeer

This comment has been minimized.

Show comment
Hide comment
@bertjwregeer

bertjwregeer Feb 4, 2017

Member

I've released Waitress 1.0.2 with the bugfix: https://pypi.python.org/pypi/waitress/1.0.2

Member

bertjwregeer commented Feb 4, 2017

I've released Waitress 1.0.2 with the bugfix: https://pypi.python.org/pypi/waitress/1.0.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment