This PR enables transitively pinning (TP) objects from particular roots for Immix/StickyImmix #897
Conversation
|
I think we should stop using red/black roots in the PR, and in future discussion just to avoid any confusion. As we discussed, we will call it |
udesou
changed the title
…andle node roots that should be kept it place but their children may move
… nursery GC, allowing moving on full heap but disabling it for the nursery GC
|
The To workaround this, we can let the
Cherry-pick this commit, instead: wks@1617f54 It fixed more cases and added comments. |
|
binding-refs |
qinsoon
added
the
PR-testing
|
The Julia tests failed, as the Julia PR did not include the change about its tests so it timed out. I don't know what happened for the Ruby tests. |
@wks I'm also not familiar with that error. Do you have any idea why the ruby tests failed to run? |
When running binding tests for mmtk-core, I used a Python script to replace the mmtk dependency with a local directory. See: https://github.com/mmtk/mmtk-core/blob/master/.github/scripts/replace-mmtk-dep.py This script works by deleting the However, there is a problem. In my [dependencies.mmtk]
features = ["is_mmtk_object", "object_pinning"]
# Uncomment one of the following lines to choose where to find mmtk-core.
#git = "https://github.com/mmtk/mmtk-core.git" # Use mmtk-core from the official repository.
#path = "../../mmtk-core" # Use mmtk-core from a local repository.
git = "https://github.com/udesou/mmtk-core" # Use mmtk-core with rb-roots feature
branch = "feature/rb-roots"By removing the branch = "feature/rb-roots"
path = "/home/wks/projects/mmtk-github/mmtk-core"
It is a bug in https://github.com/mmtk/mmtk-core/blob/master/.github/scripts/replace-mmtk-dep.py |
I see. But the local directory would take the |
Yes. I made a PR to fix it. (#947) Now the script simply remove all of "git", "branch", "version" and "registry". But it should be easier to just cherry-pick #947 into this PR and run the CI again. |
Remove all keys that may specify the location of a dependency since they may conflict.
|
Done. Thank you @wks! EDIT: Something still isn't right. Maybe a network error. I'll try rerunning the tests after the other ones finish. |
warning: spurious network error (3 tries remaining): [28] Timeout was reached (Operation too slow. Less than 10 bytes/sec transferred the last 30 seconds)
warning: spurious network error (2 tries remaining): [28] Timeout was reached (Operation too slow. Less than 10 bytes/sec transferred the last 30 seconds)
warning: spurious network error (1 tries remaining): [28] Timeout was reached (Operation too slow. Less than 10 bytes/sec transferred the last 30 seconds)Seems to be a network issue of the cargo registry. Probably just rerun it later. |
|
I ran benchmarks for the PR with OpenJDK/Immix/2.5x MC min heap (plotty link). It measures the impact of the code changes and extra buckets from the PR, but it does not measure the transitively pinning or the pinning roots (as the OpenJDK binding does not use them). The results show there is no slowdown in this PR for introducing transitively pinning. Mutator time GC time |
Related PR: mmtk/mmtk-core#897 Co-authored-by: Eduardo Souza <ledusou@gmail.com>
…#99) Merge after mmtk/mmtk-core#897 to reflect the API change. --------- Co-authored-by: Yi Lin <qinsoon@gmail.com>
This PR adds support for (partially) moving objects in Julia and should be merged with mmtk/julia#27 and mmtk/mmtk-core#897. - It adds support for pinning/unpinning objects and checking if an object is pinned (the implementation uses a pin bit). (`mmtk_pin_object`, `mmtk_unpin_object` and `mmtk_is_pinned`) - It adds support for providing transitively pinned (`tpinned`) roots . - It implements the `copy` function in `object_model.rs`. Note that arrays with inlined data must be treated specially, as their `a->data` pointer needs to be updated after copying. - It uses Julia's GC bits to store forwarding information and the object's header to store the forwarding pointer. - Currently, all stack roots are transitively pinned. Note that we also need to traverse the `tls->live_tasks` to make sure that any stack root from these tasks are transitively pinned. - `scan_julia_object` had to be adapted to cover a few corner cases: - when an array object contains a pointer to the owner of the data, `a->data` needs to be updated in case the owner moves. - the `using` field inside a `jl_module_t` may also be inlined inside the module, and if that's the case, we need to make sure that field is updated if the module moves. - when marking finalizers, traversing the list of malloced arrays, and the list of live tasks at the end of GC, we need to updated these lists with objects that have possibly been moved. - Added a few debug assertions to capture scanning of misaligned objects and roots. NB: I've only tested moving immix; sticky immix is still non-moving. --------- Co-authored-by: Luis Eduardo de Souza Amorim <eduardo@shrew.moma> Co-authored-by: Luis Eduardo de Souza Amorim <eduardo@bear.moma> Co-authored-by: mmtkgc-bot <mmtkgc.bot@gmail.com>
This PR adds support for (partially) moving objects in Julia and should be merged with mmtk/julia#27 and mmtk/mmtk-core#897. - It adds support for pinning/unpinning objects and checking if an object is pinned (the implementation uses a pin bit). (`mmtk_pin_object`, `mmtk_unpin_object` and `mmtk_is_pinned`) - It adds support for providing transitively pinned (`tpinned`) roots . - It implements the `copy` function in `object_model.rs`. Note that arrays with inlined data must be treated specially, as their `a->data` pointer needs to be updated after copying. - It uses Julia's GC bits to store forwarding information and the object's header to store the forwarding pointer. - Currently, all stack roots are transitively pinned. Note that we also need to traverse the `tls->live_tasks` to make sure that any stack root from these tasks are transitively pinned. - `scan_julia_object` had to be adapted to cover a few corner cases: - when an array object contains a pointer to the owner of the data, `a->data` needs to be updated in case the owner moves. - the `using` field inside a `jl_module_t` may also be inlined inside the module, and if that's the case, we need to make sure that field is updated if the module moves. - when marking finalizers, traversing the list of malloced arrays, and the list of live tasks at the end of GC, we need to updated these lists with objects that have possibly been moved. - Added a few debug assertions to capture scanning of misaligned objects and roots. NB: I've only tested moving immix; sticky immix is still non-moving. --------- Co-authored-by: Luis Eduardo de Souza Amorim <eduardo@shrew.moma> Co-authored-by: Luis Eduardo de Souza Amorim <eduardo@bear.moma> Co-authored-by: mmtkgc-bot <mmtkgc.bot@gmail.com> (cherry picked from commit 1622162) # Conflicts: # julia/mmtk_julia.c # mmtk/Cargo.lock # mmtk/Cargo.toml # mmtk/api/mmtk.h # mmtk/src/julia_scanning.rs # mmtk/src/lib.rs # mmtk/src/scanning.rs
ImmovableClosurehas been created to trace and transitively pin roots, i.e., roots in which no object in its transitive closure is allowed to move.RootsWorkFactoryhas two new functionscreate_process_tp_edge_roots_workandcreate_process_tp_node_roots_workto create work to process these roots.GCWorkContextexpects anTPProcessEdgestype, which performs the transitively pinning trace (and is unimplemented for unsupported plans).create_process_node_roots_workcreates work to process roots in theNodeRootsTracebucket, which is executed afterTPClosure, but any work derived from it is put into the regularClosurebucket, meaning that the binding shouldn't need to pin root nodes.sticky_immix_non_moving_nursery), but the solution here is to use theTPClosureon the modified buffer constructed from the write barrier (since during nursery collection we do not know if these objects should be transitively pinned or not).