[2/n][transfer-to-object] Add pruning of object marker table at epoch boundaries #13420
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
3 Ignored Deployments
|
7 tasks
tzakian
force-pushed
the
tzakian/tto-pruning
branch
from
dc35499
to
a0f15c9
Compare
lanvidr
reviewed
Aug 16, 2023
crates/sui-core/src/authority.rs
Outdated
| @@ -2150,6 +2162,7 @@ impl AuthorityState { | |||
| .epoch_start_state() | |||
| .protocol_version(), | |||
| ); | |||
| self.clear_object_per_epoch_marker_table()?; | |||
There was a problem hiding this comment.
I wonder if it would be better to instead include it as part of the pruner process instead of critical path of reconfiguration?
There was a problem hiding this comment.
this is the most appropriate place imo - the pruner is handles the less trivial cases that require processing checkpoints.
mystenmark
approved these changes
Aug 17, 2023
crates/sui-core/src/authority.rs
Outdated
| @@ -2150,6 +2162,7 @@ impl AuthorityState { | |||
| .epoch_start_state() | |||
| .protocol_version(), | |||
| ); | |||
| self.clear_object_per_epoch_marker_table()?; | |||
There was a problem hiding this comment.
this is the most appropriate place imo - the pruner is handles the less trivial cases that require processing checkpoints.
crates/sui-core/src/authority.rs
Outdated
| @@ -2077,6 +2077,18 @@ impl AuthorityState { | |||
| .enqueue_certificates(certs, epoch_store) | |||
| } | |||
|
|
|||
| // NB: This must only be called at time of reconfiguration. | |||
| fn clear_object_per_epoch_marker_table(&self) -> SuiResult<()> { | |||
There was a problem hiding this comment.
please take a reference to the execution lock to ensure that no one can later inadvertently move the call to outside of reconfig
tzakian
force-pushed
the
tzakian/tto-rust-sdk-support
branch
from
3470a04
to
12abec6
Compare
tzakian
requested review from
a team,
randall-Mysten and
ronny-mysten
as code owners
tzakian
force-pushed
the
tzakian/tto-pruning
branch
from
a0f15c9
to
4a5886e
Compare
tzakian
force-pushed
the
tzakian/tto-rust-sdk-support
branch
from
12abec6
to
da17e9f
Compare
tzakian
force-pushed
the
tzakian/tto-pruning
branch
from
4a5886e
to
847a40a
Compare
tzakian
force-pushed
the
tzakian/tto-rust-sdk-support
branch
2 times, most recently
from
6f3189d
to
9251d81
Compare
tzakian
force-pushed
the
tzakian/tto-pruning
branch
from
847a40a
to
9ada62a
Compare
tzakian
force-pushed
the
tzakian/tto-rust-sdk-support
branch
from
9251d81
to
27bdffd
Compare
Base automatically changed from
tzakian/tto-rust-sdk-support
to
tzakian/tto-typescript-sdk-support
September 6, 2023 22:07
tzakian
force-pushed
the
tzakian/tto-typescript-sdk-support
branch
2 times, most recently
from
f3659c0
to
4460991
Compare
tzakian
force-pushed
the
tzakian/tto-pruning
branch
from
9b049b0
to
95b1bc4
Compare
tzakian
force-pushed
the
tzakian/tto-json-rpc
branch
from
afb0582
to
d45bc8d
Compare
tzakian
force-pushed
the
tzakian/tto-pruning
branch
from
95b1bc4
to
8e93bcd
Compare
tzakian
force-pushed
the
tzakian/tto-json-rpc
branch
from
d45bc8d
to
1a9dafc
Compare
tzakian
force-pushed
the
tzakian/tto-pruning
branch
from
8e93bcd
to
be1a9d6
Compare
tnowacki
reviewed
Sep 22, 2023
tzakian
force-pushed
the
tzakian/tto-json-rpc
branch
from
1a9dafc
to
6022a17
Compare
tzakian
force-pushed
the
tzakian/tto-pruning
branch
from
be1a9d6
to
1d7ce25
Compare
tzakian
force-pushed
the
tzakian/tto-json-rpc
branch
from
6022a17
to
5626740
Compare
tzakian
force-pushed
the
tzakian/tto-pruning
branch
from
1d7ce25
to
d72ffa7
Compare
tzakian
force-pushed
the
tzakian/tto-json-rpc
branch
from
5626740
to
be7473e
Compare
tzakian
force-pushed
the
tzakian/tto-pruning
branch
from
d72ffa7
to
33ca886
Compare
tzakian
added a commit
that referenced
this pull request
Sep 22, 2023
## Description This PR implements the core transfer-to-object functionality. In particular it implements the ability to "receive" an object that was sent to the address (object ID) of another object using one of the `transfer` or `transfer_public` functions in the `transfer` module. More detail is given on the programming model in the attached issue so I will not go into that. SDK support for receiving objects has been added in the two PRs stacked on this one: * #12987 Adds the `Receiving` type to the json-rpc types, and adds support receiving objects in the Typescript SDK. * #12988 Adds support for receiving objects in the Rust SDK * #13420 Adds pruning of the `per_epoch_object_marker` table at epoch boundaries ## Test Plan I've written a number of tests for this that I believe cover things: * Execution-correctness tests for this in the transactional tests * Tests for effect computation in the new sui-core `transfer_to_object.rs` tests (e.g., receive-then-unwrap, receive-unwrap-wrap, etc). * Tests for lock-freeness of receiving arguments (i.e., that the object identified by the `Receiving` argument is not locked at signing) in the sui-core `transfer_to_object.rs` tests * Tests that dependencies are correctly registered, and notified in the transaction manager for `Receiving` arguments to transactions (see new tests in the `transaction_manager_tests.rs` file). A more detailed listing of the tests: * PTBs - Receive object and return to PTB - Do not do anything with the returned (non-drop) value [`receive_return_object_dont_touch.move`] - Call transfer object on it [`receive_return_object_then_transfer.move`] - Basic "can receive and then do something in the function" [`basic_receive.move`] - Duplicate "Receive" arguments to the PTB [`duplicate_receive_argument.move`] - Pass but don't use `Receiving` argument, then later use it in PTB. - By immut ref [`pass_receiver_immut_then_reuse.move`] - By mut ref [`pass_receiver_mut_then_reuse.move`] - By value and returning it [`pass_through_then_receive.move`] - Various combinations of receivership being passed [`receive_by_ref.move`] (checking borrow/borrow_mut, and restore rules for PTB execution) - Receive object of different type [`receive_invalid_type.move`] - Receive object with non-address owner ownership [`receive_object_owner.move`] - Reuse of input receiving argument [`take_receiver_then_try_to_reuse.move`] * Type malleability [`receive_invalid_param_ty.move`] - Pass receiver into a non-receiver type - primitive type - struct type with same layout - struct type with different layout - Pass non-receiver into a receiver type - primitive type - struct type with same layout - struct type with different layout * Resource conservation/Effects calculation (both transactional tests and sui-core tests for explicit effects checks) - Do various things with object after receiving it: - Immediately place it as a dynamic field [`receive_dof_and_mutate.move`] - Immediately add a dynamic field to it [`receive_add_dof_and_mutate.move`] - Immediately add a dynamic field to it, add as a dynamic field to parent object, then mutate both [`receive_add_dof_and_mutate.move`] - Immediately transfer it [`receive_and_send_back.move`] - Immediately delete it [`receive_and_deleted.move`] - Immediately wrap it [`receive_and_wrap.move`] - Immediately abort [`receive_and_abort.move`] - Don't use it [`receive_by_value_flow_through.move`] - Receive multiple times in a row making sure effects stay in-sync as expected [`receive_multiple_times_in_row.move`] - Shared objects - Make sure we can receive if object is transferred to an object which is already shared [`shared_parent/basic_receive.move`] - Make sure we can receive if object is transferred to an object which is then shared [`shared_parent/transfer_then_share.move`] - Non-usage of receiving object argument off a shared parent object [`shared_parent/drop_receiving.move`] - Receive object off of shared parent, add as dynamic field of shared parent and then mutate through the parent [`shared_parent/receive_dof_and_mutate.move`] - Send and receive the same object to the same shared parent multiple times [`shared_parent/receive_multiple_times_in_row.move`] - MVCC -- Test that we calculate contained UIDs correctly when we receive an object. This is tested in [`mvcc/receive_object_dof.move`] and [`mvcc/receive_object_split_changes_dof.move`] - Sui core tests checking explicit parts of the calculated effects to make sure they match what we expect: - Immediately unwrap then transfer inner object [`transfer_to_object_tests.rs/test_tto_unwrap_transfer`] - Immediately unwrap then delete inner object as well [`transfer_to_object_tests.rs/test_tto_unwrap_delete`] - Immediately unwrap then add inner object as dynamic field [`transfer_to_object_tests.rs/test_tto_unwrap_add_as_dynamic_field`] - Immediately unwrap, then wrap again -- this is part of the above since adding a dynamic field wraps the object - Basic object receive [`transfer_to_object_tests/test_tto_transfer`] - Pass but don't ise Receiving argument [`transfer_to_object_tests/test_tto_unused_receiver`] - Pass by different references [`transfer_to_object_tests/test_tto_pass_receiving_by_refs`] - Receive and immediately delete [`transfer_to_object_tests/test_tto_delete`] - Receive, wrap, and then transfer wrapped object [`transfer_to_object_tests/test_tto_wrap`] * Sui Core for object locking and transaction dependendency calculation in effects - Test that receiving object arguments are not locked, and that different orders of execution for two certs that want to receive the same argument (but only one is valid) can both be run in either order, and both return the same execution effects in either order [`transfer_to_object_tests/test_tt_not_locked`] - Test that transaction dependencies are added correctly: - Basic test that we add transaction dependendency if we execute successfully and receive the object [`transfer_to_object_tests/test_tto_valid_dependencies`] - Similar case for if we delete the object immediately [`transfer_to_object_tests/test_tto_valid_dependencies_delete_on_receive`] - That we don't register the transaction dependendency if we don't receive the object [`transfer_to_object_tests/test_tto_dependencies_dont_receive`] - That we don't register the transaction dependendency if we don't receive the object and we abort [`transfer_to_object_tests/test_tto_dependencies_dont_receive_but_abort`] - That we register the dependendency if we received the object, even if we then went on to abort in the transaction [`transfer_to_object_tests/test_tto_dependencies_receive_and_abort`] - Dynamic object field spoofing: make sure we don't accidentally register a dynamic object field load of an object that we want to receive at a different version as a receivership of that object (i.e., don't register the transaction dependendency) [`transfer_to_object_tests/receive_and_dof_interleave`] ## Additional tests - PTBs - `MakeMoveVec`: - create but don't use [receive_many_move_vec.move] - pass vec by value but don't receive [receive_many_move_vec.move] - pass vec by ref then use value to receive in later command [receive_many_move_vec.move] - Pass vec by mut ref and pop/receive some, then receive rest in other call [receive_many_move_vec.move] - Pass vec by mut ref, only receive some [receive_many_move_vec.move] - Pass vec by value, only receive some [receive_many_move_vec.move] - Pass vec by value, receive all [receive_many_move_vec.move] - Pack receiving tickets into a struct (some/all) then receive transitively [receive_duo_struct.move] - Type mismatches: - Receiving and phony struct with same struct layout and right type args ([receive_invalid_param_ty.move]) - Receiving with mismatched type args [move_vec_receiving_types.move] - Receiving with multiple different type args [move_vec_receiving_types.move] - `TransferObjects` - Try to transfer receiving ticket [receive_ticket_coin_operations.move] - `SplitCoins` - Try to split a receiving ticket [receive_ticket_coin_operations.move] - `MergeCoins` - Try to merge a receiving ticket [receive_ticket_coin_operations.move] - MVCC [`receive_object_access_through_parent[dof/df].move`] - Transaction input checks (in sui-core tests) - Delete between cert and execution [tests in `test_tto_not_locked`in the sui-core tests - Cert denial if sending a transaction where `input_objects \intersect receiving_object != {}` [`test_tto_intersection_input_and_receiving_objects`] - Type-fixing for receiving arguments [pt_receive_type_fixing.move] --- If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process. ### Type of Change (Check all that apply) - [X] protocol change - [X] user-visible impact - [ ] breaking change for a client SDKs - [X] breaking change for FNs (FN binary must upgrade) - [X] breaking change for validators or node operators (must upgrade binaries) - [ ] breaking change for on-chain data layout - [ ] necessitate either a data wipe or data migration ### Release notes Added the ability to receive objects off of another object. This is currently only turned on in devnet. More information on transfer-to-object, receiving objects off of other objects, and current SDK support can be found in the GitHub issue which can be found here: #12658
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
All elements of the
object_per_epoch_markertable can be pruned at epoch boundaries, so this adds this in to the reconfiguration step that transitions between each epoch. I believe the way I have it here is the correct way, but I'm less familiar with this so please let me know if there's a better way to do this.Test Plan
Added e2e tests with reconfiguration enabled, and manually checked to make sure any entries were cleared out at epoch boundaries.