New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python app loading indefinitely - bug intruduced in 4.0.37 #1215

Closed
naox opened this Issue Jun 17, 2014 · 11 comments

Comments

Projects
None yet
2 participants
@naox

naox commented Jun 17, 2014

This bug is not present in passenger 4.0.36. This bug is present in 4.0.37 and all later versions including newest 4.0.45

django aplication has error that would causes error page to be generated in passenger 4.0.36:

ImproperlyConfigured at /
The included urlconf dermclinic_zz.urls doesn't have any patterns in it
Request Method: GET
Request URL:    http://..../
Django Version: 1.6.5
Exception Type: ImproperlyConfigured
Exception Value:    
The included urlconf dermclinic.urls doesn't have any patterns in it
Exception Location: /home/dclinic/site-packages/Django-1.6.5-py2.7.egg/django/core/urlresolvers.py in url_patterns, line 369

passenger:wsgi.py

import os,sys,site
os.environ['LD_LIBRARY_PATH'] = '/usr/local/lib'

site.addsitedir('/home/dclinic/site-packages')
sys.path.insert(0,os.getcwd())
sys.path.insert(0,os.path.join(os.getcwd(), '/site-packages')) 

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "dermclinic_pl.settings")
import django.core.handlers.wsgi

from paste.exceptions.errormiddleware import ErrorMiddleware
application = django.core.handlers.wsgi.WSGIHandler()
application = ErrorMiddleware(application, debug=True)

however in passenger 4.0.37 and later error page is not displayed and what is worse:

  1. webpage seems to load "forever" (after >120s seconds connection is terminated however)
  2. Nothing is logged to apache main load about any error with this app

@FooBarWidget FooBarWidget added this to the 4.0.46 milestone Jun 18, 2014

@naox

This comment has been minimized.

Show comment
Hide comment
@naox

naox Jun 24, 2014

Since I have python aplication on standby that allows observation of this bug I've tested commits made between 4.0.36 and 4.0.37 release-4.0.36...release-4.0.37
I've determined that this bug was intruduced in commit 3b7f4af
and to be more exact in helper-scripts/wsgi-loader.py 3b7f4af#diff-2
I've put helper-scripts/wsgi-loader.py from before this commit into newest 4.0.45 and bug is not occuring

naox commented Jun 24, 2014

Since I have python aplication on standby that allows observation of this bug I've tested commits made between 4.0.36 and 4.0.37 release-4.0.36...release-4.0.37
I've determined that this bug was intruduced in commit 3b7f4af
and to be more exact in helper-scripts/wsgi-loader.py 3b7f4af#diff-2
I've put helper-scripts/wsgi-loader.py from before this commit into newest 4.0.45 and bug is not occuring

@naox naox changed the title from python app loading indefenetly - bug intruduced in 4.0.37 to python app loading indefinitely - bug intruduced in 4.0.37 Jun 24, 2014

@FooBarWidget

This comment has been minimized.

Show comment
Hide comment
@FooBarWidget

FooBarWidget Jun 24, 2014

Member

Excellent information. Thank you!

Member

FooBarWidget commented Jun 24, 2014

Excellent information. Thank you!

@FooBarWidget

This comment has been minimized.

Show comment
Hide comment
@FooBarWidget

FooBarWidget Jun 24, 2014

Member

I'll assign high priority to this issue for the next release.

Member

FooBarWidget commented Jun 24, 2014

I'll assign high priority to this issue for the next release.

@FooBarWidget

This comment has been minimized.

Show comment
Hide comment
@FooBarWidget

FooBarWidget Jun 26, 2014

Member

Ok I found the problem. It's because django.template.base goes through all WSGI environment values, and calls each value that is a callable. This invokes Passenger's "socket hijacking" feature. I've fixed this in b7e8607.

Member

FooBarWidget commented Jun 26, 2014

Ok I found the problem. It's because django.template.base goes through all WSGI environment values, and calls each value that is a callable. This invokes Passenger's "socket hijacking" feature. I've fixed this in b7e8607.

@naox

This comment has been minimized.

Show comment
Hide comment
@naox

naox Jun 26, 2014

Unfortunetly your fix, new wsgi-loader.py from yesterday intruduced a new bug.
Now at least one of aplications running on my servers stoped working and it only display standard apache 500 error (immediately, without any delay). In apache error log I see:

App 754 stderr: [ pid=754, time=2014-06-26 18:47:35,125 ]: WSGI application raised an exception!
App 754 stderr: Traceback (most recent call last):
App 754 stderr:   File "/usr/passenger-4.0.45/helper-scripts/wsgi-loader.py", line 140, in main_loop
App 754 stderr:     socket_hijacked = self.process_request(env, input_stream, client)
App 754 stderr:   File "/usr/passenger-4.0.45/helper-scripts/wsgi-loader.py", line 270, in process_request
App 754 stderr:     result = self.app(env, start_response)
App 754 stderr: TypeError: 'module' object is not callable
App 754 stderr: [ pid=754, time=2014-06-26 18:47:35,261 ]: WSGI application raised an exception!
App 754 stderr: Traceback (most recent call last):
App 754 stderr:   File "/usr/passenger-4.0.45/helper-scripts/wsgi-loader.py", line 140, in main_loop
App 754 stderr:     socket_hijacked = self.process_request(env, input_stream, client)
App 754 stderr:   File "/usr/passenger-4.0.45/helper-scripts/wsgi-loader.py", line 270, in process_request
App 754 stderr:     result = self.app(env, start_response)
App 754 stderr: TypeError: 'module' object is not callable

Problem is gone when I again switched to wsgi-loader.py before commit 3b7f4af but only after I restated aplication. It showed very low memory usage.

/home/login/rails/app#default:
App root: /home/login/rails/app
Requests in queue: 0

  • PID: 21970 Sessions: 0 Processed: 12 Uptime: 8m 3s
    CPU: 0% Memory : 10M Last used: 1m 56s ago

Aplication is written in Flask.

I guess you can reopen ticket

naox commented Jun 26, 2014

Unfortunetly your fix, new wsgi-loader.py from yesterday intruduced a new bug.
Now at least one of aplications running on my servers stoped working and it only display standard apache 500 error (immediately, without any delay). In apache error log I see:

App 754 stderr: [ pid=754, time=2014-06-26 18:47:35,125 ]: WSGI application raised an exception!
App 754 stderr: Traceback (most recent call last):
App 754 stderr:   File "/usr/passenger-4.0.45/helper-scripts/wsgi-loader.py", line 140, in main_loop
App 754 stderr:     socket_hijacked = self.process_request(env, input_stream, client)
App 754 stderr:   File "/usr/passenger-4.0.45/helper-scripts/wsgi-loader.py", line 270, in process_request
App 754 stderr:     result = self.app(env, start_response)
App 754 stderr: TypeError: 'module' object is not callable
App 754 stderr: [ pid=754, time=2014-06-26 18:47:35,261 ]: WSGI application raised an exception!
App 754 stderr: Traceback (most recent call last):
App 754 stderr:   File "/usr/passenger-4.0.45/helper-scripts/wsgi-loader.py", line 140, in main_loop
App 754 stderr:     socket_hijacked = self.process_request(env, input_stream, client)
App 754 stderr:   File "/usr/passenger-4.0.45/helper-scripts/wsgi-loader.py", line 270, in process_request
App 754 stderr:     result = self.app(env, start_response)
App 754 stderr: TypeError: 'module' object is not callable

Problem is gone when I again switched to wsgi-loader.py before commit 3b7f4af but only after I restated aplication. It showed very low memory usage.

/home/login/rails/app#default:
App root: /home/login/rails/app
Requests in queue: 0

  • PID: 21970 Sessions: 0 Processed: 12 Uptime: 8m 3s
    CPU: 0% Memory : 10M Last used: 1m 56s ago

Aplication is written in Flask.

I guess you can reopen ticket

@FooBarWidget FooBarWidget reopened this Jun 26, 2014

@FooBarWidget

This comment has been minimized.

Show comment
Hide comment
@FooBarWidget

FooBarWidget Jun 26, 2014

Member

Let me make a Flask app and test this.

Member

FooBarWidget commented Jun 26, 2014

Let me make a Flask app and test this.

@naox

This comment has been minimized.

Show comment
Hide comment
@naox

naox Jun 27, 2014

I've send you details for test eviroment and flask aplication to your @phusion.nl email

naox commented Jun 27, 2014

I've send you details for test eviroment and flask aplication to your @phusion.nl email

@FooBarWidget

This comment has been minimized.

Show comment
Hide comment
@FooBarWidget

FooBarWidget Jun 30, 2014

Member

Here's the first problem: the import statement in passenger_wsgi.py is not correct.

from app import app as application

app/app refers to the module object for app/app.py, so it's no wonder that it's not callable. You need to get the app attribute within the app.app module. So this works:

from app.app import app as application

However, I now get a different error, something related to Flask template rendering. I am not very familiar with Flask so I will need more investigation time.

Member

FooBarWidget commented Jun 30, 2014

Here's the first problem: the import statement in passenger_wsgi.py is not correct.

from app import app as application

app/app refers to the module object for app/app.py, so it's no wonder that it's not callable. You need to get the app attribute within the app.app module. So this works:

from app.app import app as application

However, I now get a different error, something related to Flask template rendering. I am not very familiar with Flask so I will need more investigation time.

@FooBarWidget

This comment has been minimized.

Show comment
Hide comment
@FooBarWidget

FooBarWidget Jun 30, 2014

Member

This is the error that I get. Not sure what it means. I even get it when running the app under the normal Flask server, so it's probably a configuration error on my side. Can you check whether your app works under Passenger after fixing the import?

127.0.0.1 - - [30/Jun/2014 12:27:38] "GET / HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/handlers.py", line 928, in emit
    smtp = smtplib.SMTP(self.mailhost, port, timeout=self._timeout)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/smtplib.py", line 251, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/smtplib.py", line 311, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/smtplib.py", line 286, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 553, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
gaierror: [Errno 8] nodename nor servname provided, or not known
Logged from file app.py, line 1423
Member

FooBarWidget commented Jun 30, 2014

This is the error that I get. Not sure what it means. I even get it when running the app under the normal Flask server, so it's probably a configuration error on my side. Can you check whether your app works under Passenger after fixing the import?

127.0.0.1 - - [30/Jun/2014 12:27:38] "GET / HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/handlers.py", line 928, in emit
    smtp = smtplib.SMTP(self.mailhost, port, timeout=self._timeout)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/smtplib.py", line 251, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/smtplib.py", line 311, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/smtplib.py", line 286, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 553, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
gaierror: [Errno 8] nodename nor servname provided, or not known
Logged from file app.py, line 1423
@naox

This comment has been minimized.

Show comment
Hide comment
@naox

naox Jun 30, 2014

After fixing import in wsgi-loader.py aplication works on newest passenger_wsgi.py Without this fix however it worked on wsgi-loader.py from version 4.0.36. However if it was not suposse to work in first place - I guess thats ok.

I also have very little idea about flask (I did not made this app) so I don't know about this "gaierror" error. If you think issue sould be investigated further then you can use virtual machine I've sent you details for earlier - it is clean ubuntu with passenger standalone, stock passenger.

naox commented Jun 30, 2014

After fixing import in wsgi-loader.py aplication works on newest passenger_wsgi.py Without this fix however it worked on wsgi-loader.py from version 4.0.36. However if it was not suposse to work in first place - I guess thats ok.

I also have very little idea about flask (I did not made this app) so I don't know about this "gaierror" error. If you think issue sould be investigated further then you can use virtual machine I've sent you details for earlier - it is clean ubuntu with passenger standalone, stock passenger.

@FooBarWidget

This comment has been minimized.

Show comment
Hide comment
@FooBarWidget

FooBarWidget Jun 30, 2014

Member

It looks like it is a configuration problem after all. It's trying to send email but it was not correctly configured.

Seeing that everything in Passenger has been fixed, I'm closing this issue. :)

Member

FooBarWidget commented Jun 30, 2014

It looks like it is a configuration problem after all. It's trying to send email but it was not correctly configured.

Seeing that everything in Passenger has been fixed, I'm closing this issue. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment