Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetcher concurrency issue #28

Closed
popravich opened this issue Apr 11, 2016 · 1 comment
Closed

Fetcher concurrency issue #28

popravich opened this issue Apr 11, 2016 · 1 comment

Comments

@popravich
Copy link
Member

There is a problem with Fetcher, I don't know if its bad design or simply "ported as is" issue, anyway:
fetched_records and next_record methods use _wait_empty_future to halt until results are read,
but this would only work if you had only one consumer task running. If you'd need to run several
asyncio tasks for consuming and processing messages you'll definetly end up with
_wait_empty_future being overriden with other one from other task and thus leave first task
waiting for future's result until loop is closed.

https://github.com/aio-libs/aiokafka/blob/master/aiokafka/fetcher.py#L604
Basically there is no check if _wait_empty_future already set (not None)

@fabregas
Copy link
Contributor

please see #29

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants