Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why switch_out() may be called twice in a row? #1343

Open
dmugtasimov opened this issue Jan 13, 2019 · 0 comments
Open

Why switch_out() may be called twice in a row? #1343

dmugtasimov opened this issue Jan 13, 2019 · 0 comments
Labels
Type: Question User support and/or waiting for responses

Comments

@dmugtasimov
Copy link

dmugtasimov commented Jan 13, 2019

In a simple setup like this all works fine:

from gevent import monkey; monkey.patch_all()

import time
from collections import deque

import gevent


def func():
    while True:
        print('{:.3f} {} iteration'.format(time.time(), id(gevent.getcurrent())))
        gevent.sleep(0.5)


class CustomGreenlet(gevent.Greenlet):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        self.switch_log = deque(maxlen=100)

    def switch(self, *args, **kwargs):
        print('{:.3f} {} switch in'.format(time.time(), id(gevent.getcurrent())))
        self.switch_log.append(0)
        return super().switch(*args, **kwargs)

    def switch_out(self):
        print('{:.3f} {} switch out'.format(time.time(), id(gevent.getcurrent())))
        assert self.switch_log, 'Switch out without switch in is detected'
        assert self.switch_log[-1] == 0, 'Switch out without switch in is detected'
        self.switch_log.append(1)


greenlets = [CustomGreenlet(func) for _ in range(100)]

for greenlet in greenlets:
    greenlet.start()

gevent.joinall(greenlets)

But in a real world applications that involve actual IO and a lot of other stuff I got assert self.switch_log[-1] == 0, 'Switch out without switch in is detected' fired.

2019-01-13 16:06:26,455 ERROR gid:48 ERROR during iteration (in 0.000431776s): TaskResultsConsumer
Traceback (most recent call last):
  File "/home/dmugtasimov/gitrep/packy/packy-agent/packy_agent/worker/loops/base/loop.py", line 38, in iteration_wrapper
    self.iteration()
  File "/home/dmugtasimov/gitrep/packy/packy-agent/packy_agent/worker/loops/base/consumer.py", line 27, in iteration
    item = self.inbound_queue.get(timeout=self.timeout)
  File "src/gevent/queue.py", line 329, in gevent._queue.Queue.get
  File "src/gevent/queue.py", line 344, in gevent._queue.Queue.get
  File "src/gevent/queue.py", line 321, in gevent._queue.Queue.__get_or_peek
  File "src/gevent/_waiter.py", line 151, in gevent.__waiter.Waiter.get
  File "src/gevent/_greenlet_primitives.py", line 60, in gevent.__greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 60, in gevent.__greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_greenlet_primitives.py", line 63, in gevent.__greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "/home/dmugtasimov/gitrep/packy/packy-agent/packy_agent/utils/gevent.py", line 34, in switch_out
    assert self.switch_log[-1][0] == SwitchType.IN, 'Switch out without switch in is detected'
AssertionError: Switch out without switch in is detected

How should I approach to debug this? Is it a bug or feature? Should I ignore consecutive switch_out() calls to the same Greenlet? If so, then which ones?

@jamadden jamadden added the Type: Question User support and/or waiting for responses label Dec 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Question User support and/or waiting for responses
Projects
None yet
Development

No branches or pull requests

2 participants