New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python issue with multiprocessing Queue #155

Closed
svenfuchs opened this Issue Feb 17, 2013 · 22 comments

Comments

Projects
None yet
@svenfuchs
Member

svenfuchs commented Feb 17, 2013

The issue below was reported to support@travis-ci.org.

/cc @joshk

+++

Hello,
I'm not sure this is the right place for issues with Travis, but if it is I
have a problem with a python package
https://travis-ci.org/lsbardel/python-stdnet

which depends on an other package which uses python multiprocessing Queue.
The error seems to be that for POSIX semaphores to work, the users need r/w
access to shared memory (/dev/shm).

On my laptop (Ubuntu) it looks like this:

ls -ld /dev/shm
lrwxrwxrwx 1 root root 8 Feb 8 23:20 /dev/shm -> /run/shm

This is a post on stack overflow which may be of interest
http://stackoverflow.com/questions/2009278/python-multiprocessing-permission-denied

Thanks

This is the stack trace.

running install_egg_info
290Writing
/home/travis/virtualenv/python3.2/lib/python3.2/site-packages/python_stdnet-0.8a1.egg-info
291$ ls -la /dev/shm
292total 16
293drwxr-xr-x 2 root root 4096 Jan 9 08:26 .
294drwxr-xr-x 5 root root 12288 Feb 17 17:21 ..
295$ python -m runtests --verbosity 2
296Traceback (most recent call last):
297 File "/usr/lib/python3.2/runpy.py", line 160, in _run_module_as_main
298 "__main__", fname, loader, pkg_name)
299 File "/usr/lib/python3.2/runpy.py", line 73, in _run_code
300 exec(code, run_globals)
301 File "/home/travis/build/lsbardel/python-stdnet/runtests.py", line 76,
in <module>
302 start()
303 File "/home/travis/build/lsbardel/python-stdnet/runtests.py", line 61,
in start
304 profile.Profile())
305 File
"/home/travis/virtualenv/python3.2/lib/python3.2/site-packages/pulsar/apps/__init__.py",
line 284, in __init__
306 self()
307 File
"/home/travis/virtualenv/python3.2/lib/python3.2/site-packages/pulsar/apps/__init__.py",
line 304, in __call__
308 ioqueue=self.ioqueue)
309 File
"/home/travis/virtualenv/python3.2/lib/python3.2/site-packages/pulsar/apps/__init__.py",
line 343, in ioqueue
310 self.local.queue = self.get_ioqueue()
311 File
"/home/travis/virtualenv/python3.2/lib/python3.2/site-packages/pulsar/apps/tasks/__init__.py",
line 229, in get_ioqueue
312 self.local.ioqueue = self.cfg.task_queue_factory()
313 File "/usr/lib/python3.2/multiprocessing/queues.py", line 63, in
__init__
314 self._rlock = Lock()
315 File "/usr/lib/python3.2/multiprocessing/synchronize.py", line 147, in
__init__
316 SemLock.__init__(self, SEMAPHORE, 1, 1)
317 File "/usr/lib/python3.2/multiprocessing/synchronize.py", line 75, in
__init__
318 sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
319OSError: [Errno 13] Permission denied
320
321The command "python -m runtests --verbosity 2" exited with 255.
322
323Done. Your build exited with 1.
@lsbardel

This comment has been minimized.

Show comment
Hide comment
@lsbardel

lsbardel Feb 18, 2013

Hi,
I reported the issueto to support@travis-ci.org.
On a related note,
when I test the package which causes the issue with the multiprocessing module, https://travis-ci.org/quantmind/pulsar, the error does not occur.
In other words it occurs only when pulsar package is a dependency and therefore installed by virtualenv into the site-packages directory.

Hi,
I reported the issueto to support@travis-ci.org.
On a related note,
when I test the package which causes the issue with the multiprocessing module, https://travis-ci.org/quantmind/pulsar, the error does not occur.
In other words it occurs only when pulsar package is a dependency and therefore installed by virtualenv into the site-packages directory.

@guillermo-carrasco

This comment has been minimized.

Show comment
Hide comment
@guillermo-carrasco

guillermo-carrasco Feb 18, 2013

Hi!

I have the same problem in our repository, here. Until it is solved, I bypassed it with this commit (the build continue failing, but I thing is our fault now).

Maybe this bug report from ubuntu is of your interest?

Thanks!

Hi!

I have the same problem in our repository, here. Until it is solved, I bypassed it with this commit (the build continue failing, but I thing is our fault now).

Maybe this bug report from ubuntu is of your interest?

Thanks!

@lsbardel

This comment has been minimized.

Show comment
Hide comment
@lsbardel

lsbardel Feb 18, 2013

Cool,
that has fixed the problem

Cool,
that has fixed the problem

@bitprophet

This comment has been minimized.

Show comment
Hide comment
@bitprophet

bitprophet Mar 1, 2013

I can confirm that I have this issue as well (example).

@guillermo-carrasco's workaround works for me as well, which was expected given the apparent nature of the problem. Would be great if the Travis worker initialization took care of this for users who may not be expecting the issue.

I can confirm that I have this issue as well (example).

@guillermo-carrasco's workaround works for me as well, which was expected given the apparent nature of the problem. Would be great if the Travis worker initialization took care of this for users who may not be expecting the issue.

@pwaller

This comment has been minimized.

Show comment
Hide comment

pwaller commented Mar 6, 2013

Cross-referencing travis-ci/travis-ci#943.

@turicas

This comment has been minimized.

Show comment
Hide comment
@turicas

turicas Mar 7, 2013

I'm trying to use travis on pypelinin and, as one of its daemons creates lots of processes with the multiprocessing library, I'm being affected by this bug too - but the solution of adding "sudo rm -rf /dev/shm && sudo ln -s /run/shm /dev/shm" to the install directive is not working for me.

For instance, you can see the log of before and after the addition of this line (same problem).

turicas commented Mar 7, 2013

I'm trying to use travis on pypelinin and, as one of its daemons creates lots of processes with the multiprocessing library, I'm being affected by this bug too - but the solution of adding "sudo rm -rf /dev/shm && sudo ln -s /run/shm /dev/shm" to the install directive is not working for me.

For instance, you can see the log of before and after the addition of this line (same problem).

@pwaller

This comment has been minimized.

Show comment
Hide comment
@pwaller

pwaller Mar 7, 2013

@turicas: The latter is not obviously the same problem to me - python's queue doesn't raise an exception anymore.

pwaller commented Mar 7, 2013

@turicas: The latter is not obviously the same problem to me - python's queue doesn't raise an exception anymore.

@turicas

This comment has been minimized.

Show comment
Hide comment
@turicas

turicas Mar 7, 2013

@pwaller, you're right, but anyway, it's failing in tests that are not failing anywhere else (I executed the same commands travis does in other machines). As I know the tests, probably the problems are related to the creation of new processes on the system or some kind of permission... the problem is that I need to debug it in a detailed way to see why the tests are not passing in travis' VMs and passing everywhere else.

turicas commented Mar 7, 2013

@pwaller, you're right, but anyway, it's failing in tests that are not failing anywhere else (I executed the same commands travis does in other machines). As I know the tests, probably the problems are related to the creation of new processes on the system or some kind of permission... the problem is that I need to debug it in a detailed way to see why the tests are not passing in travis' VMs and passing everywhere else.

@guillermo-carrasco

This comment has been minimized.

Show comment
Hide comment
@guillermo-carrasco

guillermo-carrasco Mar 8, 2013

Hi @turicas ,

My builds seems to keep raising this error, here.

Thanks a lot!

Hi @turicas ,

My builds seems to keep raising this error, here.

Thanks a lot!

@turicas

This comment has been minimized.

Show comment
Hide comment
@turicas

turicas Mar 8, 2013

@guillermo-carrasco, did you try this change to your .travis.yml?

turicas commented Mar 8, 2013

@guillermo-carrasco, did you try this change to your .travis.yml?

@guillermo-carrasco

This comment has been minimized.

Show comment
Hide comment
@guillermo-carrasco

guillermo-carrasco Mar 8, 2013

Yep, I proposed this change above in this thread. I thought that the bug was fixed.

Thank you anyway! 👍

Yep, I proposed this change above in this thread. I thought that the bug was fixed.

Thank you anyway! 👍

@turicas

This comment has been minimized.

Show comment
Hide comment
@turicas

turicas Mar 8, 2013

@guillermo-carrasco, oh, now I see. :) But my problem remains. =/

turicas commented Mar 8, 2013

@guillermo-carrasco, oh, now I see. :) But my problem remains. =/

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Mar 8, 2013

Member

I'm sorry guys, I need to find the right place to add this to our cookbooks then create a new VM.

If someone can help with the cookbooks addition I can build the new VM and make it live.

Thanks

On 9/03/2013, at 9:43 AM, Álvaro Justen notifications@github.com wrote:

@guillermo-carrasco, oh, now I see. :) But my problem remains. =/


Reply to this email directly or view it on GitHub.

Member

joshk commented Mar 8, 2013

I'm sorry guys, I need to find the right place to add this to our cookbooks then create a new VM.

If someone can help with the cookbooks addition I can build the new VM and make it live.

Thanks

On 9/03/2013, at 9:43 AM, Álvaro Justen notifications@github.com wrote:

@guillermo-carrasco, oh, now I see. :) But my problem remains. =/


Reply to this email directly or view it on GitHub.

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Mar 11, 2013

Member

This should be fixed on the new Python VMs made live over the weekend.

I am closing this issue for now, but please reopen it if the problem still persists.

<3<3<3

Member

joshk commented Mar 11, 2013

This should be fixed on the new Python VMs made live over the weekend.

I am closing this issue for now, but please reopen it if the problem still persists.

<3<3<3

@joshk joshk closed this Mar 11, 2013

@guillermo-carrasco

This comment has been minimized.

Show comment
Hide comment
@guillermo-carrasco

guillermo-carrasco Mar 11, 2013

HI @joshk ,

I've re-ran a build that reproduces this error, and seems to keep failing. Can you take a look please?

Thanks a lot! :-D

HI @joshk ,

I've re-ran a build that reproduces this error, and seems to keep failing. Can you take a look please?

Thanks a lot! :-D

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Mar 12, 2013

Member

I have been fighting this for over a day now.

Reopening until I fix it.

Member

joshk commented Mar 12, 2013

I have been fighting this for over a day now.

Reopening until I fix it.

@joshk joshk reopened this Mar 12, 2013

@bitprophet

This comment has been minimized.

Show comment
Hide comment
@bitprophet

bitprophet Mar 14, 2013

@joshk Have nothing to add besides lots of ❤️ ❤️ ❤️ ❤️ - thanks for poking at this :)

@joshk Have nothing to add besides lots of ❤️ ❤️ ❤️ ❤️ - thanks for poking at this :)

@joshk

This comment has been minimized.

Show comment
Hide comment
@joshk

joshk Mar 14, 2013

Member

Thanks Jeff, we are still investigating further and hope to have more info soon. :)

Sent from my Commodore 64

On 15/03/2013, at 11:40 AM, Jeff Forcier notifications@github.com wrote:

@joshk Have nothing to add besides lots of - thanks for poking at this :)


Reply to this email directly or view it on GitHub.

Member

joshk commented Mar 14, 2013

Thanks Jeff, we are still investigating further and hope to have more info soon. :)

Sent from my Commodore 64

On 15/03/2013, at 11:40 AM, Jeff Forcier notifications@github.com wrote:

@joshk Have nothing to add besides lots of - thanks for poking at this :)


Reply to this email directly or view it on GitHub.

myint added a commit to myint/language-check that referenced this issue May 12, 2013

mgedmin added a commit to zopefoundation/ZEO that referenced this issue May 22, 2013

mgalardini added a commit to combogenomics/DuctApe that referenced this issue May 24, 2013

@tfarago tfarago referenced this issue Aug 27, 2013

Closed

Speedup #34

njsmith added a commit to njsmith/pyrerp that referenced this issue Sep 25, 2013

ivanov added a commit to ivanov/ipython that referenced this issue Sep 26, 2013

ivanov added a commit to ivanov/ipython that referenced this issue Oct 23, 2013

Bengt added a commit to cigroup-ol/metaopt that referenced this issue Nov 4, 2013

gatoatigrado pushed a commit to gatoatigrado/vimap that referenced this issue Nov 5, 2013

@mlopes

This comment has been minimized.

Show comment
Hide comment
@mlopes

mlopes Nov 13, 2013

Hi,

Still having the same problem with my code when using multiprocessing.

Cheers,
Marco Lopes

mlopes commented Nov 13, 2013

Hi,

Still having the same problem with my code when using multiprocessing.

Cheers,
Marco Lopes

@roidrage

This comment has been minimized.

Show comment
Hide comment
@roidrage

roidrage Nov 13, 2013

Member

Sorry, not yet. As this problem seems unfixable by our cookbooks and pops up at runtime, we'll look into a build time fix.

Member

roidrage commented Nov 13, 2013

Sorry, not yet. As this problem seems unfixable by our cookbooks and pops up at runtime, we'll look into a build time fix.

@mlopes

This comment has been minimized.

Show comment
Hide comment
@mlopes

mlopes Nov 13, 2013

Meanwhile, this @turicas suggested fix (turicas/pypelinin@28f624c) fixed it for me.

mlopes commented Nov 13, 2013

Meanwhile, this @turicas suggested fix (turicas/pypelinin@28f624c) fixed it for me.

mrtheb added a commit to mrtheb/kafka-python that referenced this issue Nov 14, 2013

secat pushed a commit to secat/kafka-python that referenced this issue Nov 21, 2013

wari added a commit to wari/pytddmon that referenced this issue Nov 26, 2013

Travis multi-processing workaround
Based on the commit:
turicas/pypelinin@28f624c

as pointed by travis-ci/travis-cookbooks#155

Note: May not work however.

wari added a commit to wari/pytddmon that referenced this issue Nov 26, 2013

Travis multi-processing workaround
Based on the commit:
turicas/pypelinin@28f624c

as pointed by travis-ci/travis-cookbooks#155

Note: May not work however.
@wari

This comment has been minimized.

Show comment
Hide comment
@wari

wari Nov 26, 2013

Thanks to this issue (and reported workaround), subprocess.check_output and subprocess.check_call works as intended for me.

wari commented Nov 26, 2013

Thanks to this issue (and reported workaround), subprocess.check_output and subprocess.check_call works as intended for me.

ryanshow pushed a commit to Perkville/fabric that referenced this issue Dec 3, 2013

dimazest added a commit to dimazest/poultry that referenced this issue Dec 11, 2013

qdamian added a commit to qdamian/depict that referenced this issue Dec 30, 2013

prymitive added a commit to prymitive/upaas-admin that referenced this issue Jan 3, 2014

mrocklin added a commit to pytoolz/toolz that referenced this issue Jan 7, 2014

lpsinger added a commit to lpsinger/healpy that referenced this issue Feb 10, 2014

@meatballhat meatballhat closed this Feb 8, 2017

sonicdoe added a commit to sonicdoe/grunt-node-gyp that referenced this issue Feb 12, 2017

algitbot pushed a commit to alpinelinux/aports that referenced this issue Mar 31, 2017

travis: bind /run and symlink /run/shm -> /dev/shm
This fixes problem with e.g. Python multiprocessing Queue - for POSIX
semaphores to work, the users need r/w access to shared memory
(/dev/shm).

See travis-ci/travis-cookbooks#155
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment