Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gc refcount related crash when loading web UI under Python 2.6.6 #49

Closed
daubman opened this issue Dec 18, 2012 · 7 comments
Closed

gc refcount related crash when loading web UI under Python 2.6.6 #49

daubman opened this issue Dec 18, 2012 · 7 comments
Labels
bug

Comments

@daubman
Copy link
Contributor

@daubman daubman commented Dec 18, 2012

Running under Python 2.6.6 (Centos 6.3) I get the following crash as soon as I try and load the web UI. This seems to work fine using Python 2.7.3 as an altinstall on the same host. I have seen this on two different Centos 6.3 hosts...

--snip--
$ /usr/bin/locust -f some_locust_script.py
/usr/lib/python2.6/site-packages/locust/rpc/init.py:7: UserWarning: WARNING: Using pure Python socket RPC implementation instead of zmq. This will not affect you if your not running locust in distributed mode, but if you are, we recommend you to install the python packages: pyzmq and gevent-zeromq
warnings.warn("WARNING: Using pure Python socket RPC implementation instead of zmq. This will not affect you if your not running locust in distributed mode, but if you are, we recommend you to install the python packages: pyzmq and gevent-zeromq")
[2012-12-18 15:14:52,705] INFO/locust.main: Starting web monitor on port 8089
[2012-12-18 15:14:52,705] INFO/locust.main: Starting Locust 0.6.1
Modules/gcmodule.c:348: visit_decref: Assertion "gc->gc.gc_refs != 0" failed.
refcount was too small
object : <weakref at 0x15b1db8; to 'gevent.core.http_request' at 0x15b1d60>
type : weakref
refcount: 1
address : 0x15b1db8
Aborted
--snip--

@cgbystrom
Copy link
Member

@cgbystrom cgbystrom commented Dec 18, 2012

That sounds really strange. We've run Locust fine for countless hours under different 2.6.x versions.

  1. Are you using a vanilla CentOS 6.3 version?
  2. Did you use the default Python interpreter that CentOS was shipped with or did you build it yourself?

Since I haven't experienced this myself I did quick google and for some this issue is caused by too aggressive compilation options when building the Python interpreter.

@Jahaja
Copy link
Member

@Jahaja Jahaja commented Dec 18, 2012

I was able to reproduce the problem using a minimal centos 6.3 installation on a VM. As well as using the python version that's shipped with it.

It seems to be related to gevent. Updating gevent to 1.0rc2 seems to solve the problem. So that should be a work-around for now.

@cgbystrom
Copy link
Member

@cgbystrom cgbystrom commented Dec 19, 2012

I was able reproduce it as well. @Jahaja's fix works!

Will close this issue as there is a short term workaround and it will be solved when gevent hits 1.0 (Locust will switch).
If you still have issues, please comment.

@daubman
Copy link
Contributor Author

@daubman daubman commented Jan 8, 2013

Would you please provide some more details on the fix, I have tried this and it does not appear to help - here's what I've done:

$ uname -a
Linux ip-10-46-247-69 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011 x86_64 x86_64 x86_64 GNU/Linux
$ cat /etc/redhat-release
CentOS release 6.3 (Final)

$ which python
/usr/bin/python
$ python --version
Python 2.6.6

$ yum install c-ares-devel
$ yum install libev-devel
$ yum install zeromq-devel
$ pip install https://github.com/downloads/SiteSupport/gevent/gevent-1.0rc2.tar.gz
$ pip search gevent
gevent - Python network library that uses greenlet and libevent for easy and scalable concurrency
INSTALLED: 1.0rc2
LATEST: 0.13.8
...
$ pip install https://github.com/locustio/locust/archive/master.zip
$ pip install gevent-zeromq
$ pip install pyzmq
$ locust -f my_locust_file.py --master
[2013-01-08 16:07:53,945] INFO/locust.main: Starting web monitor on port 8089
[2013-01-08 16:07:53,947] ERROR/stderr: Traceback (most recent call last):
[2013-01-08 16:07:53,947] ERROR/stderr: File "/usr/bin/locust", line 9, in
[2013-01-08 16:07:53,948] ERROR/stderr:
[2013-01-08 16:07:53,948] ERROR/stderr: load_entry_point('locustio==0.6.1', 'console_scripts', 'locust')()
[2013-01-08 16:07:53,948] ERROR/stderr: File "/usr/lib/python2.6/site-packages/locust/main.py", line 363, in main
[2013-01-08 16:07:53,948] ERROR/stderr:
[2013-01-08 16:07:53,948] ERROR/stderr: runners.locust_runner = MasterLocustRunner(locust_classes, options.hatch_rate, options.num_clients, num_requests=options.num_requests, host=options.host, master_host=options.master_host)
[2013-01-08 16:07:53,948] ERROR/stderr: File "/usr/lib/python2.6/site-packages/locust/runners.py", line 241, in init
[2013-01-08 16:07:53,948] ERROR/stderr:
[2013-01-08 16:07:53,949] ERROR/stderr: self.greenlet.spawn(self.client_listener).link_exception()
[2013-01-08 16:07:53,949] ERROR/stderr: File "/usr/lib64/python2.6/site-packages/gevent/greenlet.py", line 370, in link_exception
[2013-01-08 16:07:53,949] ERROR/stderr:
[2013-01-08 16:07:53,949] ERROR/stderr: self.link(receiver=receiver, SpawnedLink=SpawnedLink)
[2013-01-08 16:07:53,949] ERROR/stderr: File "/usr/lib64/python2.6/site-packages/gevent/greenlet.py", line 355, in link
[2013-01-08 16:07:53,949] ERROR/stderr:
[2013-01-08 16:07:53,949] ERROR/stderr: self.rawlink(SpawnedLink(receiver))
[2013-01-08 16:07:53,950] ERROR/stderr: File "/usr/lib64/python2.6/site-packages/gevent/greenlet.py", line 23, in init
[2013-01-08 16:07:53,950] ERROR/stderr:
[2013-01-08 16:07:53,950] ERROR/stderr: raise TypeError("Expected callable: %r" % (callback, ))
[2013-01-08 16:07:53,950] ERROR/stderr: TypeError
[2013-01-08 16:07:53,950] ERROR/stderr: :
[2013-01-08 16:07:53,950] ERROR/stderr: Expected callable: None
[2013-01-08 16:07:53,950] ERROR/stderr:

Also, from this post, it appears locust's use of gevent is not compatible with 1.0rc2:
https://groups.google.com/forum/?fromgroups=#!topic/gevent/JHbtDU5HtL8

@heyman heyman reopened this Jan 9, 2013
heyman added a commit that referenced this issue Jan 9, 2013
@heyman
Copy link
Member

@heyman heyman commented Jan 9, 2013

I've just pushed some changes which I think fixes the problems. Could you try to see if works and then I can push a new version to PyPI.

Thanks for reporting!

@jvenkataramanan
Copy link

@jvenkataramanan jvenkataramanan commented Jan 9, 2013

Yes this change seems to fix all the issues. Thanks for getting to it so fast!

@heyman
Copy link
Member

@heyman heyman commented Jan 10, 2013

That's great to hear! The fix is now in a new 0.6.2 release on PyPI.

@heyman heyman closed this Jan 10, 2013
pancaprima added a commit to pancaprima/locust that referenced this issue May 14, 2018
Merge Develop to Master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.