Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use minheap as timeout container and fix collision of multiple SelectConnection timeouts with same deadline and callback #947

Merged

Conversation

vitaly-krugl
Copy link
Member

@vitaly-krugl vitaly-krugl commented Feb 5, 2018

(Note that the failed coverage test results from pika's encrypted credentials configuration issue.)

  1. NOTE: On Windows, hash(frozenset(value.items())) used by the legacy implementation of SelectConnection's _PollerBase.add_timeout readily results in timer id collision if multiple timers have very close deadlines and same callback. See failed TestViabilityOfMultipleTimeoutsWithSameDeadlineAndCallback test results in https://ci.appveyor.com/project/gmr/pika/build/1.0.409. This is aided by Windows time.time() having lower resolution than on Linux. However, on a faster machine, the same problem would also likely show up on linux, and hashes aren't guaranteed to be unique for all input set. And I recall getting it on my dev MacBookPro a long time ago.
  2. A minheap is a more natural and efficient container for managing timeouts.

@vitaly-krugl vitaly-krugl changed the title DO NOT MERGE Implemented BlockingConnection test TestViabilityOfMultipleTimeoutsWithSameDeadlineAndCallback DO NOT MERGE Fix collision of multiple SelectConnection timeouts with same deadline and callback Feb 5, 2018
@vitaly-krugl vitaly-krugl force-pushed the fix-multiple-timers-same-deadline branch 2 times, most recently from 970c2d3 to 5bd1a04 Compare February 5, 2018 06:59
@vitaly-krugl vitaly-krugl force-pushed the fix-multiple-timers-same-deadline branch from 5495577 to 676e7e9 Compare February 5, 2018 07:17
@vitaly-krugl vitaly-krugl force-pushed the fix-multiple-timers-same-deadline branch from 8b7f4ff to 84b1061 Compare February 7, 2018 08:41
@vitaly-krugl vitaly-krugl force-pushed the fix-multiple-timers-same-deadline branch from 84b1061 to 90a32c8 Compare February 7, 2018 09:01
@vitaly-krugl vitaly-krugl changed the title DO NOT MERGE Fix collision of multiple SelectConnection timeouts with same deadline and callback DO NOT MERGE Use heap as timeout container and fix collision of multiple SelectConnection timeouts with same deadline and callback Feb 7, 2018
@vitaly-krugl vitaly-krugl changed the title DO NOT MERGE Use heap as timeout container and fix collision of multiple SelectConnection timeouts with same deadline and callback DO NOT MERGE Use minheap as timeout container and fix collision of multiple SelectConnection timeouts with same deadline and callback Feb 7, 2018
@vitaly-krugl vitaly-krugl changed the title DO NOT MERGE Use minheap as timeout container and fix collision of multiple SelectConnection timeouts with same deadline and callback Use minheap as timeout container and fix collision of multiple SelectConnection timeouts with same deadline and callback Feb 10, 2018
@vitaly-krugl
Copy link
Member Author

cc @lukebakken

Ready for review

…OLoop to the poller in order to prevent cyclical references that interfere with garbage collection.
@vitaly-krugl
Copy link
Member Author

I tried to use weakref and it resulted in test failures. I will be working through it shortly

@lukebakken lukebakken added this to the 1.0.0 milestone Feb 12, 2018
…ods, since instance methods have temporary references
@vitaly-krugl
Copy link
Member Author

@lukebakken - this is ready for review. Thanks!

@vitaly-krugl
Copy link
Member Author

vitaly-krugl commented Feb 15, 2018

@lukebakken, do you anticipate an opportunity to review this in the near future? My next PR #956 that implements the thread-safe callback request feature in BlockingConnection and SelectConnection builds on this PR. Thx!

@lukebakken
Copy link
Member

@vitaly-krugl I'll get it today. I didn't notice that "DO NOT MERGE" had been removed from the title.

Copy link
Member

@lukebakken lukebakken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great and tests pass locally and on Travis

@vitaly-krugl vitaly-krugl merged commit 0e13941 into pika:master Feb 16, 2018
@vitaly-krugl
Copy link
Member Author

Thank you for reviewing, @lukebakken!

lukebakken pushed a commit that referenced this pull request Apr 16, 2018
…adline

Use minheap as timeout container and fix collision of multiple SelectConnection timeouts with same deadline and callback

(cherry picked from commit 0e13941)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants