Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SetPartitioner (using gevent) fails when lock acquisition is aborted #247

Closed
emulbreh opened this issue Sep 16, 2014 · 3 comments
Closed
Labels

Comments

@emulbreh
Copy link

SetPartitioner._abort_lock_acquisition() doesn't pass async=True to _child_watching(), which causes the rawlinked function (_allocate_transition()) to be executed directly from the event loop callback:

Traceback (most recent call last):
  File "/.../env/src/gevent/gevent/event.py", line 293, in _notify_links
    link(self)
  File "/.../env/lib/python2.7/site-packages/kazoo/recipe/partitioner.py", line 273, in _allocate_transition
    self._identifier, list(self._party), self._set)
  File "/.../env/lib/python2.7/site-packages/kazoo/recipe/party.py", line 63, in __len__
    return len(self._get_children())
  File "/.../env/lib/python2.7/site-packages/kazoo/recipe/party.py", line 66, in _get_children
    return self.client.retry(self.client.get_children, self.path)
  File "/.../env/lib/python2.7/site-packages/kazoo/retry.py", line 122, in __call__
    return func(*args, **kwargs)
  File "/.../env/lib/python2.7/site-packages/kazoo/client.py", line 945, in get_children
    return self.get_children_async(path, watch, include_data).get()
  File "/.../env/src/gevent/gevent/event.py", line 233, in get
    result = self.hub.switch()
  File "/.../env/src/gevent/gevent/hub.py", line 344, in switch
    switch_out()
  File "/.../env/src/gevent/gevent/hub.py", line 348, in switch_out
    raise AssertionError('Impossible to call blocking function in the event loop callback')
AssertionError: Impossible to call blocking function in the event loop callback
(<bound method SetPartitioner._allocate_transition of <kazoo.recipe.partitioner.SetPartitioner object at 0x10a74a9d0>>,
 <gevent.event.AsyncResult object at 0x10a92a250>) failed with AssertionError

#240 is my attempt to fix this problem. Maybe it would be cleaner to default to async=True or even remove the parameter.

@timcherry
Copy link

Running into the same issue.

In cases where we don't set async=True we need to be extra careful not to use blocking calls in the callback, as gevent calls rawlink callbacks from inside the hub greenlet.

The workaround, as stated by @emulbreh, is to set async=True. It may be a better long term solution to use a Condition() here similar to the SequentialThreadingHandler.

@bbangert bbangert added the Bug label Sep 23, 2014
@ninowalker
Copy link

👍 Seeing this as well.

@StephenSorriaux
Copy link
Member

Closing this issue since it was fixed with #283. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants