Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup Plasma Async Callback #7452

Merged
merged 81 commits into from
Mar 16, 2020
Merged

Conversation

ijrsvt
Copy link
Contributor

@ijrsvt ijrsvt commented Mar 4, 2020

Why are these changes needed?

Moves all instance variables related to PlasmaEventHandlers into Python threads. Before, the C++ callback interfaced with the PlasmaEventHandler's waiting dictionary. Also moves the callback into Raylet.

Related issue number

Checks

@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/22737/
Test FAILed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/22908/
Test FAILed.

@ijrsvt ijrsvt changed the title Remove Async Callback Race Condition Cleanup Plasma Async Callback Mar 9, 2020
@ijrsvt ijrsvt requested review from edoakes and simon-mo March 9, 2020 20:44
Copy link
Contributor

@simon-mo simon-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! This makes almost all python handler run in threadsafe event loop.

Copy link
Contributor

@edoakes edoakes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

python/ray/_raylet.pyx Outdated Show resolved Hide resolved
Co-Authored-By: Edward Oakes <ed.nmi.oakes@gmail.com>
@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/22989/
Test FAILed.

@simon-mo simon-mo merged commit 46953c5 into ray-project:master Mar 16, 2020
stephanie-wang pushed a commit to stephanie-wang/ray that referenced this pull request Mar 17, 2020
stephanie-wang added a commit that referenced this pull request Mar 19, 2020
* enable

* Turn on eager eviction

* Shorten tests and drain ReferenceCounter

* Don't force kill actor handles that have gone out of scope, lint

* Fix locks

* Cleanup Plasma Async Callback (#7452)

* [rllib][tune] fix some nans (#7611)

* Change /tmp to platform-specific temporary directory (#7529)

* [Serve] UI Improvements (#7569)

* bugfix about test_dynres.py (#7615)

Co-authored-by: senlin.zsl <senlin.zsl@antfin.com>

* Java call Python actor method use actor.call (#7614)

* bug fix about useage of absl::flat_hash_map::erase and absl::flat_hash_set::erase (#7633)

Co-authored-by: senlin.zsl <senlin.zsl@antfin.com>

* [Java] Make both `RayActor` and `RayPyActor` inheriting from `BaseActor` (#7462)

* [Java] Fix the issue that the cached value in `RayObject` is serialized (#7613)

* Add failure tests to test_reference_counting (#7400)

* Fix typo in asyncio documentation (#7602)

* Fix segfault

* debug

* Force kill actor

* Fix test
@ijrsvt ijrsvt deleted the AsyncPlasmaRefactor branch August 26, 2020 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants