Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiprocessing.Queue created by sub-process fails when used in sub-sub-process ("bad file descriptor" in q.get()) #49405

Closed
bbutler mannequin opened this issue Feb 5, 2009 · 6 comments
Labels
stdlib type-crash

Comments

@bbutler
Copy link
Mannequin

@bbutler bbutler mannequin commented Feb 5, 2009

BPO 5155
Files
  • queue_test.py: Test script
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2009-06-30.17:14:17.766>
    created_at = <Date 2009-02-05.01:52:52.621>
    labels = ['library', 'type-crash']
    title = 'Multiprocessing.Queue created by sub-process fails when used in sub-sub-process ("bad file descriptor" in q.get())'
    updated_at = <Date 2009-06-30.17:14:17.764>
    user = 'https://bugs.python.org/bbutler'

    bugs.python.org fields:

    activity = <Date 2009-06-30.17:14:17.764>
    actor = 'jnoller'
    assignee = 'jnoller'
    closed = True
    closed_date = <Date 2009-06-30.17:14:17.766>
    closer = 'jnoller'
    components = ['Library (Lib)']
    creation = <Date 2009-02-05.01:52:52.621>
    creator = 'bbutler'
    dependencies = []
    files = ['12944']
    hgrepos = []
    issue_num = 5155
    keywords = []
    message_count = 6.0
    messages = ['81183', '81184', '84947', '89196', '89902', '89944']
    nosy_count = 3.0
    nosy_names = ['OG7', 'jnoller', 'bbutler']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = None
    status = 'closed'
    superseder = None
    type = 'crash'
    url = 'https://bugs.python.org/issue5155'
    versions = ['Python 2.6', 'Python 3.0']

    @bbutler
    Copy link
    Mannequin Author

    @bbutler bbutler mannequin commented Feb 5, 2009

    (Note: This issue only happens on linux - on Windows things work fine.)

    Please see the attached script.

    The script creates a process (TestProcess()). That process runs and
    creates a sub-process (TestSubProcess()). When TestSubProcess tries to
    access a Queue created by TestProcess, the .get() call fails with the
    following stack trace:

    Process Process-1:1:
    Traceback (most recent call last):
      File "/opt/python2.6/lib/python2.6/multiprocessing/process.py", line
    231, in _bootstrap
        self.run()
      File "/opt/python2.6/lib/python2.6/multiprocessing/process.py", line
    88, in run
        self._target(*self._args, **self._kwargs)
      File "queue_test.py", line 7, in TestSubProcess
        item = q.get()                        # Fails with IOError if queue
    created in TestProcess
      File "/opt/python2.6/lib/python2.6/multiprocessing/queues.py", line
    91, in get
        res = self._recv()
    IOError: [Errno 9] Bad file descriptor

    If the same queue is created by main (top level), TestSubProcess
    accesses it with no errors. This behavior can be stimulated by
    uncommenting the q = multiprocessing.Queue() line in TestProcess.

    Tested using:
    Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit
    (Intel)] on win32
    (Installed using binary installer, windows XPSP2)

    *No problems*

    Python 2.6.1 (r261:67515, Jan 6 2009, 15:23:19) [GCC 4.2.4 (Ubuntu
    4.2.4-1ubuntu3)] on linux2
    (Compiled from source due to ubuntu 8.04 (Hardy) not having python2.6 yet)

    *Fails on q.get() in TestSubProcess*

    @bbutler bbutler mannequin added stdlib type-crash labels Feb 5, 2009
    @bbutler bbutler mannequin changed the title Multiprocessing.Queue created by subprocess fails when used in sub-sub-process Multiprocessing.Queue created by sub-process fails when used in sub-sub-process Feb 5, 2009
    @bbutler bbutler mannequin changed the title Multiprocessing.Queue created by sub-process fails when used in sub-sub-process Multiprocessing.Queue created by sub-process fails when used in sub-sub-process ("bad file descriptor" in q.get()) Feb 5, 2009
    @bbutler
    Copy link
    Mannequin Author

    @bbutler bbutler mannequin commented Feb 5, 2009

    Also reproduced in Python3.0 on OSX 10.5

    @bbutler
    Copy link
    Mannequin Author

    @bbutler bbutler mannequin commented Mar 31, 2009

    Quick note: Have tracked this down to what appears to be buggy behaviour
    on the part of os.pipe() when called from within a Process.

    First invocation of os.pipe() in a Process returns (0,x) - stdin(?)
    resulting in the 'bad file descriptor' error.

    Interactive prompt test follows:

    >>> import multiprocessing as MP, os
    >>> def r():  print os.pipe(), os.pipe(), os.pipe()
    ... 
    >>> r()
    (9, 10) (11, 12) (13, 14)
    >>> MP.Process(target=r).start()
    (0, 15) (16, 17) (18, 19)
    >>> MP.Process(target=r).start()
    (0, 15) (16, 17) (18, 19)

    @OG7
    Copy link
    Mannequin

    @OG7 OG7 mannequin commented Jun 10, 2009

    bpo-5313 seems to be the culprit.

    @jnoller
    Copy link
    Mannequin

    @jnoller jnoller mannequin commented Jun 30, 2009

    Patch attached to bpo-5313, please review.

    @jnoller
    Copy link
    Mannequin

    @jnoller jnoller mannequin commented Jun 30, 2009

    Committed in r73708 on trunk

    @jnoller jnoller mannequin closed this as completed Jun 30, 2009
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    stdlib type-crash
    Projects
    None yet
    Development

    No branches or pull requests

    0 participants