New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up channel mapping #227

Merged
merged 2 commits into from Feb 9, 2019

Conversation

Projects
None yet
1 participant
@frankmcsherry
Copy link
Member

frankmcsherry commented Feb 8, 2019

This PR adjusts how the worker manages its mapping from usize channel identifiers to Vec<usize> operator addresses that require activation. Where it used to be stored in a Vec<Vec<usize>>, which would eventually grow embarrassingly large, it is now in a HashMap<usize, Vec<usize>>. This also allows us to clean up entries when we close a dataflow. The way we track which entries should be cleaned up is a bit grotty at the moment, and perhaps in the future the Canary type used in some of the communication allocators could be used here, so that we can clean up a mapping when the Pull handle is removed.

Tests pass, and unless there are bugs there should be no adverse performance implications.

frankmcsherry added some commits Feb 8, 2019

@frankmcsherry frankmcsherry merged commit 55d9940 into master Feb 9, 2019

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@frankmcsherry frankmcsherry deleted the channel_cleanup branch Feb 9, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment