Skip to content

Commit

Permalink
[sui-framework/package] Publisher checks recognise upgrades (#12026)
Browse files Browse the repository at this point in the history
## Description

Change the behaviour of functions like:

```
sui::package::from_package<T>(self: &Publisher): bool
sui::package::from_module<T>(self: &Publisher): bool
```

to return `true` when `T` is from an upgraded version of the package
associated with `Publisher`. Previously it was impossible to check the
package ownership of a type introduced in an upgraded package (the
upgraded type's type name didn't match the original package's
`Publisher` and it was impossible to create a `Publisher` for the
upgraded package, because it requires an OTW which we can't create for
upgraded packages).

This support was added by introducing a new stdlib function:
`std::typename::get_original`, as of protocol version 11, which produces
a runtime representation of the type that uses original (aka runtime)
package IDs instead of defining package IDs (the default behaviour for
`std::typename::get` and any other case where a runtime representation
of the type is converted into an on-chain or storage representation of a
type).

## Test Plan

New transactional test cases for `std::type_name::get_original` and
`sui::package::from_package`:

```
$ cargo nextest run -- upgrade/type_names.move
$ cargo nextest run -- upgrade/publisher.move
```

---
If your changes are not user-facing and not a breaking change, you can
skip the following section. Otherwise, please indicate what changed, and
then add to the Release Notes section as highlighted during the release
process.

### Type of Change (Check all that apply)

- [x] user-visible impact
- [ ] breaking change for a client SDKs
- [x] breaking change for FNs (FN binary must upgrade)
- [x] breaking change for validators or node operators (must upgrade
binaries)
- [ ] breaking change for on-chain data layout
- [ ] necessitate either a data wipe or data migration

### Release notes

As of protocol version 11:

- `sui::package::from_package<T>(self: &Publisher)` and
`sui::package::from_module<T>(self: &Publisher)` return `true` when `T`
is a type introduced at any upgraded version of the package associated
with `self: &Publisher` (rather than just the types originally present
when that package was published).
- `std::type_name::get_original` is introduced as a new stdlib function
to return a representation of the type where package IDs have all been
normalized to "original" IDs (the ID of the first version of a package)
in contrast to the default behaviour of `std::type_name::get` which
distinguishes types introduced at different versions of a package by
assigning them the ID of the package that introduced them.
  • Loading branch information
amnn committed May 16, 2023
1 parent c9feb2c commit 731ed97
Show file tree
Hide file tree
Showing 17 changed files with 569 additions and 63 deletions.
22 changes: 22 additions & 0 deletions crates/sui-adapter-transactional-tests/tests/upgrade/publisher.exp
@@ -0,0 +1,22 @@
processed 5 tasks

init:
A: object(0,0)

task 1 'publish'. lines 6-17:
created: object(1,0), object(1,1), object(1,2)
mutated: object(0,0)
gas summary: computation_cost: 1000000, storage_cost: 8557600, storage_rebate: 0, non_refundable_storage_fee: 0

task 2 'upgrade'. lines 19-35:
created: object(2,0)
mutated: object(0,0), object(1,2)
gas summary: computation_cost: 1000000, storage_cost: 7691200, storage_rebate: 2595780, non_refundable_storage_fee: 26220

task 3 'run'. lines 37-37:
mutated: object(0,0), object(1,1)
gas summary: computation_cost: 1000000, storage_cost: 2812000, storage_rebate: 2783880, non_refundable_storage_fee: 28120

task 4 'run'. lines 39-39:
mutated: object(0,0), object(1,1)
gas summary: computation_cost: 1000000, storage_cost: 2812000, storage_rebate: 2783880, non_refundable_storage_fee: 28120
@@ -0,0 +1,39 @@
// Copyright (c) Mysten Labs, Inc.
// SPDX-License-Identifier: Apache-2.0

//# init --addresses A0=0x0 A1=0x0 --accounts A

//# publish --upgradeable --sender A
module A0::m {
use sui::package;
use sui::tx_context::TxContext;

struct A {}
struct M has drop {}

fun init(m: M, ctx: &mut TxContext) {
package::claim_and_keep(m, ctx);
}
}

//# upgrade --package A0 --upgrade-capability 1,2 --sender A
module A1::m {
use sui::package::{Self, Publisher};
use sui::tx_context::TxContext;

struct A {}
struct B {}
struct M has drop {}

fun init(m: M, ctx: &mut TxContext) {
package::claim_and_keep(m, ctx);
}

entry fun test<T>(p: &Publisher) {
assert!(package::from_package<T>(p), 0)
}
}

//# run A1::m::test --type-args A0::m::A --args object(1,1) --sender A

//# run A1::m::test --type-args A1::m::B --args object(1,1) --sender A
@@ -0,0 +1,59 @@
processed 12 tasks

init:
A: object(0,0)

task 1 'publish'. lines 6-17:
created: object(1,0), object(1,1)
mutated: object(0,0)
gas summary: computation_cost: 1000000, storage_cost: 6171200, storage_rebate: 0, non_refundable_storage_fee: 0

task 2 'upgrade'. lines 19-30:
created: object(2,0)
mutated: object(0,0), object(1,1)
gas summary: computation_cost: 1000000, storage_cost: 6520800, storage_rebate: 2595780, non_refundable_storage_fee: 26220

task 3 'upgrade'. lines 32-62:
created: object(3,0)
mutated: object(0,0), object(1,1)
gas summary: computation_cost: 1000000, storage_cost: 9066800, storage_rebate: 2595780, non_refundable_storage_fee: 26220

task 4 'run'. lines 64-64:
created: object(4,0)
mutated: object(0,0)
gas summary: computation_cost: 1000000, storage_cost: 2728400, storage_rebate: 978120, non_refundable_storage_fee: 9880

task 5 'run'. lines 66-66:
created: object(5,0)
mutated: object(0,0)
gas summary: computation_cost: 1000000, storage_cost: 2728400, storage_rebate: 978120, non_refundable_storage_fee: 9880

task 6 'run'. lines 68-68:
created: object(6,0)
mutated: object(0,0)
gas summary: computation_cost: 1000000, storage_cost: 2728400, storage_rebate: 978120, non_refundable_storage_fee: 9880

task 7 'run'. lines 70-70:
created: object(7,0)
mutated: object(0,0)
gas summary: computation_cost: 1000000, storage_cost: 2728400, storage_rebate: 978120, non_refundable_storage_fee: 9880

task 8 'view-object'. lines 72-72:
Owner: Account Address ( A )
Version: 5
Contents: A0::m::Canary {id: sui::object::UID {id: sui::object::ID {bytes: fake(4,0)}}, addr: vector[57u8, 98u8, 48u8, 52u8, 101u8, 52u8, 50u8, 57u8, 50u8, 54u8, 101u8, 55u8, 48u8, 54u8, 54u8, 56u8, 99u8, 101u8, 56u8, 50u8, 97u8, 52u8, 99u8, 50u8, 51u8, 56u8, 56u8, 98u8, 101u8, 52u8, 101u8, 50u8, 100u8, 52u8, 99u8, 53u8, 98u8, 54u8, 54u8, 49u8, 98u8, 101u8, 48u8, 50u8, 52u8, 98u8, 53u8, 56u8, 101u8, 49u8, 53u8, 49u8, 56u8, 101u8, 55u8, 49u8, 99u8, 55u8, 54u8, 56u8, 57u8, 50u8, 51u8, 54u8]}

task 9 'view-object'. lines 74-74:
Owner: Account Address ( A )
Version: 6
Contents: A0::m::Canary {id: sui::object::UID {id: sui::object::ID {bytes: fake(5,0)}}, addr: vector[57u8, 98u8, 48u8, 52u8, 101u8, 52u8, 50u8, 57u8, 50u8, 54u8, 101u8, 55u8, 48u8, 54u8, 54u8, 56u8, 99u8, 101u8, 56u8, 50u8, 97u8, 52u8, 99u8, 50u8, 51u8, 56u8, 56u8, 98u8, 101u8, 52u8, 101u8, 50u8, 100u8, 52u8, 99u8, 53u8, 98u8, 54u8, 54u8, 49u8, 98u8, 101u8, 48u8, 50u8, 52u8, 98u8, 53u8, 56u8, 101u8, 49u8, 53u8, 49u8, 56u8, 101u8, 55u8, 49u8, 99u8, 55u8, 54u8, 56u8, 57u8, 50u8, 51u8, 54u8]}

task 10 'view-object'. lines 76-76:
Owner: Account Address ( A )
Version: 7
Contents: A0::m::Canary {id: sui::object::UID {id: sui::object::ID {bytes: fake(6,0)}}, addr: vector[57u8, 98u8, 48u8, 52u8, 101u8, 52u8, 50u8, 57u8, 50u8, 54u8, 101u8, 55u8, 48u8, 54u8, 54u8, 56u8, 99u8, 101u8, 56u8, 50u8, 97u8, 52u8, 99u8, 50u8, 51u8, 56u8, 56u8, 98u8, 101u8, 52u8, 101u8, 50u8, 100u8, 52u8, 99u8, 53u8, 98u8, 54u8, 54u8, 49u8, 98u8, 101u8, 48u8, 50u8, 52u8, 98u8, 53u8, 56u8, 101u8, 49u8, 53u8, 49u8, 56u8, 101u8, 55u8, 49u8, 99u8, 55u8, 54u8, 56u8, 57u8, 50u8, 51u8, 54u8]}

task 11 'view-object'. lines 78-78:
Owner: Account Address ( A )
Version: 8
Contents: A0::m::Canary {id: sui::object::UID {id: sui::object::ID {bytes: fake(7,0)}}, addr: vector[98u8, 55u8, 50u8, 55u8, 55u8, 53u8, 48u8, 100u8, 98u8, 50u8, 54u8, 48u8, 49u8, 52u8, 101u8, 53u8, 54u8, 101u8, 99u8, 102u8, 57u8, 50u8, 99u8, 100u8, 51u8, 50u8, 101u8, 52u8, 100u8, 48u8, 54u8, 56u8, 50u8, 51u8, 56u8, 56u8, 50u8, 49u8, 54u8, 50u8, 51u8, 99u8, 48u8, 52u8, 54u8, 57u8, 56u8, 100u8, 57u8, 54u8, 50u8, 57u8, 48u8, 56u8, 100u8, 49u8, 50u8, 48u8, 97u8, 53u8, 49u8, 54u8, 100u8, 48u8]}
@@ -0,0 +1,78 @@
// Copyright (c) Mysten Labs, Inc.
// SPDX-License-Identifier: Apache-2.0

//# init --addresses A0=0x0 A1=0x0 A2=0x0 --accounts A

//# publish --upgradeable --sender A
module A0::m {
use sui::object::UID;

struct Canary has key {
id: UID,
addr: vector<u8>,
}

struct A {}

}

//# upgrade --package A0 --upgrade-capability 1,1 --sender A
module A1::m {
use sui::object::UID;

struct Canary has key {
id: UID,
addr: vector<u8>,
}

struct A {}
struct B {}
}

//# upgrade --package A1 --upgrade-capability 1,1 --sender A
module A2::m {
use std::ascii;
use std::type_name;
use sui::object::{Self, UID};
use sui::transfer;
use sui::tx_context::{Self, TxContext};

struct Canary has key {
id: UID,
addr: vector<u8>,
}

struct A {}
struct B {}

entry fun canary<T>(use_original: bool, ctx: &mut TxContext) {
let type = if (use_original) {
type_name::get_original<T>()
} else {
type_name::get<T>()
};

let addr = ascii::into_bytes(type_name::get_address(&type));

transfer::transfer(
Canary { id: object::new(ctx), addr },
tx_context::sender(ctx),
)
}
}

//# run A2::m::canary --type-args A0::m::A --args true --sender A

//# run A2::m::canary --type-args A1::m::B --args true --sender A

//# run A2::m::canary --type-args A0::m::A --args false --sender A

//# run A2::m::canary --type-args A1::m::B --args false --sender A

//# view-object 4,0

//# view-object 5,0

//# view-object 6,0

//# view-object 7,0
6 changes: 3 additions & 3 deletions crates/sui-framework/docs/package.md
Expand Up @@ -341,7 +341,7 @@ but multiple per package (!).
<pre><code><b>public</b> <b>fun</b> <a href="package.md#0x2_package_claim">claim</a>&lt;OTW: drop&gt;(otw: OTW, ctx: &<b>mut</b> TxContext): <a href="package.md#0x2_package_Publisher">Publisher</a> {
<b>assert</b>!(<a href="types.md#0x2_types_is_one_time_witness">types::is_one_time_witness</a>(&otw), <a href="package.md#0x2_package_ENotOneTimeWitness">ENotOneTimeWitness</a>);

<b>let</b> type = <a href="_get">type_name::get</a>&lt;OTW&gt;();
<b>let</b> type = <a href="_get_original">type_name::get_original</a>&lt;OTW&gt;();

<a href="package.md#0x2_package_Publisher">Publisher</a> {
id: <a href="object.md#0x2_object_new">object::new</a>(ctx),
Expand Down Expand Up @@ -426,7 +426,7 @@ Check whether type belongs to the same package as the publisher object.


<pre><code><b>public</b> <b>fun</b> <a href="package.md#0x2_package_from_package">from_package</a>&lt;T&gt;(self: &<a href="package.md#0x2_package_Publisher">Publisher</a>): bool {
<b>let</b> type = <a href="_get">type_name::get</a>&lt;T&gt;();
<b>let</b> type = <a href="_get_original">type_name::get_original</a>&lt;T&gt;();

(<a href="_get_address">type_name::get_address</a>(&type) == self.<a href="package.md#0x2_package">package</a>)
}
Expand All @@ -453,7 +453,7 @@ Check whether a type belongs to the same module as the publisher object.


<pre><code><b>public</b> <b>fun</b> <a href="package.md#0x2_package_from_module">from_module</a>&lt;T&gt;(self: &<a href="package.md#0x2_package_Publisher">Publisher</a>): bool {
<b>let</b> type = <a href="_get">type_name::get</a>&lt;T&gt;();
<b>let</b> type = <a href="_get_original">type_name::get_original</a>&lt;T&gt;();

(<a href="_get_address">type_name::get_address</a>(&type) == self.<a href="package.md#0x2_package">package</a>)
&& (<a href="_get_module">type_name::get_module</a>(&type) == self.module_name)
Expand Down
15 changes: 14 additions & 1 deletion crates/sui-framework/packages/move-stdlib/sources/type_name.move
Expand Up @@ -21,12 +21,25 @@ module std::type_name {
name: String
}

/// Return a value representation of the type `T`.
/// Return a value representation of the type `T`. Package IDs
/// that appear in fully qualified type names in the output from
/// this function are defining IDs (the ID of the package in
/// storage that first introduced the type).
public native fun get<T>(): TypeName;
spec get {
pragma opaque;
}

/// Return a value representation of the type `T`. Package IDs
/// that appear in fully qualified type names in the output from
/// this function are original IDs (the ID of the first version of
/// the package, even if the type in question was introduced in a
/// later upgrade).
public native fun get_original<T>(): TypeName;
spec get_original {
pragma opaque;
}

/// Get the String representation of `self`
public fun borrow_string(self: &TypeName): &String {
&self.name
Expand Down
Expand Up @@ -93,7 +93,7 @@ module sui::package {
public fun claim<OTW: drop>(otw: OTW, ctx: &mut TxContext): Publisher {
assert!(types::is_one_time_witness(&otw), ENotOneTimeWitness);

let type = type_name::get<OTW>();
let type = type_name::get_original<OTW>();

Publisher {
id: object::new(ctx),
Expand All @@ -118,14 +118,14 @@ module sui::package {

/// Check whether type belongs to the same package as the publisher object.
public fun from_package<T>(self: &Publisher): bool {
let type = type_name::get<T>();
let type = type_name::get_original<T>();

(type_name::get_address(&type) == self.package)
}

/// Check whether a type belongs to the same module as the publisher object.
public fun from_module<T>(self: &Publisher): bool {
let type = type_name::get<T>();
let type = type_name::get_original<T>();

(type_name::get_address(&type) == self.package)
&& (type_name::get_module(&type) == self.module_name)
Expand Down Expand Up @@ -271,7 +271,7 @@ module sui::package {
#[test_only]
/// Test-only function to claim a Publisher object bypassing OTW check.
public fun test_claim<OTW: drop>(_: OTW, ctx: &mut TxContext): Publisher {
let type = type_name::get<OTW>();
let type = type_name::get_original<OTW>();

Publisher {
id: object::new(ctx),
Expand Down
2 changes: 1 addition & 1 deletion crates/sui-open-rpc/spec/openrpc.json
Expand Up @@ -1320,7 +1320,7 @@
"name": "Result",
"value": {
"minSupportedProtocolVersion": "1",
"maxSupportedProtocolVersion": "10",
"maxSupportedProtocolVersion": "11",
"protocolVersion": "6",
"featureFlags": {
"advance_epoch_start_time_in_safe_mode": true,
Expand Down
4 changes: 3 additions & 1 deletion crates/sui-protocol-config/src/lib.rs
Expand Up @@ -10,7 +10,7 @@ use tracing::{info, warn};

/// The minimum and maximum protocol versions supported by this build.
const MIN_PROTOCOL_VERSION: u64 = 1;
const MAX_PROTOCOL_VERSION: u64 = 10;
const MAX_PROTOCOL_VERSION: u64 = 11;

// Record history of protocol version allocations here:
//
Expand All @@ -36,6 +36,7 @@ const MAX_PROTOCOL_VERSION: u64 = 10;
// Version 10:increase bytecode verifier `max_verifier_meter_ticks_per_function` and
// `max_meter_ticks_per_module` limits each from 6_000_000 to 16_000_000. sui-system
// framework changes.
// Version 11: Introduce `std::type_name::get_original` to the system frameworks.

#[derive(Copy, Clone, Debug, Hash, Serialize, Deserialize, PartialEq, Eq, PartialOrd, Ord)]
pub struct ProtocolVersion(u64);
Expand Down Expand Up @@ -1150,6 +1151,7 @@ impl ProtocolConfig {
cfg.max_meter_ticks_per_module = Some(16_000_000);
cfg
}
11 => Self::get_for_version_impl(version - 1),
// Use this template when making changes:
//
// // modify an existing constant.
Expand Down

2 comments on commit 731ed97

@vercel
Copy link

@vercel vercel bot commented on 731ed97 May 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Successfully deployed to the following URLs:

offline-signer-helper – ./dapps/offline-signer-helper

offline-signer-helper-mysten-labs.vercel.app
offline-signer-helper.vercel.app
offline-signer-helper-git-main-mysten-labs.vercel.app

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4 Validators 500/s Owned Transactions Benchmark Results

Benchmark Report:
+-------------+-----+-----+--------+---------------+---------------+---------------+-----------------------+----------------------------+
| duration(s) | tps | cps | error% | latency (min) | latency (p50) | latency (p99) | gas used (MIST total) | gas used/hr (MIST approx.) |
+=======================================================================================================================================+
| 60          | 999 | 999 | 0      | 15            | 25            | 39            | 747,136,704,000       | 44,828,202,240,000         |
Stress Performance Report:
+-----------+-----+-----+
| metric    | p50 | p99 |
+=======================+
| cpu usage | 40  | 48  |

4 Validators 500/s Shared Transactions Benchmark Results

Benchmark Report:
+-------------+-----+-----+--------+---------------+---------------+---------------+-----------------------+----------------------------+
| duration(s) | tps | cps | error% | latency (min) | latency (p50) | latency (p99) | gas used (MIST total) | gas used/hr (MIST approx.) |
+=======================================================================================================================================+
| 60          | 996 | 996 | 0      | 14            | 264           | 432           | 882,482,434,800       | 52,948,946,088,000         |
Stress Performance Report:
+-----------+-----+-----+
| metric    | p50 | p99 |
+=======================+
| cpu usage | 43  | 55  |

20 Validators 50/s Owned Transactions Benchmark Results

Benchmark Report:
+-------------+-----+-----+--------+---------------+---------------+---------------+-----------------------+----------------------------+
| duration(s) | tps | cps | error% | latency (min) | latency (p50) | latency (p99) | gas used (MIST total) | gas used/hr (MIST approx.) |
+=======================================================================================================================================+
| 60          | 200 | 200 | 0      | 28            | 75            | 125           | 160,727,808,000       | 9,643,668,480,000          |
Stress Performance Report:
+-----------+-----+-----+
| metric    | p50 | p99 |
+=======================+
| cpu usage | 49  | 64  |

20 Validators 50/s Shared Transactions Benchmark Results

Benchmark Report:
+-------------+-----+-----+--------+---------------+---------------+---------------+-----------------------+----------------------------+
| duration(s) | tps | cps | error% | latency (min) | latency (p50) | latency (p99) | gas used (MIST total) | gas used/hr (MIST approx.) |
+=======================================================================================================================================+
| 60          | 197 | 197 | 0      | 73            | 574           | 842           | 190,281,354,000       | 11,416,881,240,000         |
Stress Performance Report:
+-----------+-----+-----+
| metric    | p50 | p99 |
+=======================+
| cpu usage | 51  | 72  |

Narwhal Benchmark Results

 SUMMARY:
-----------------------------------------
 + CONFIG:
 Faults: 0 node(s)
 Committee size: 4 node(s)
 Worker(s) per node: 1 worker(s)
 Collocate primary and workers: True
 Input rate: 50,000 tx/s
 Transaction size: 512 B
 Execution time: 0 s

 Header number of batches threshold: 32 digests
 Header maximum number of batches: 1,000 digests
 Max header delay: 2,000 ms
 GC depth: 50 round(s)
 Sync retry delay: 10,000 ms
 Sync retry nodes: 3 node(s)
 batch size: 500,000 B
 Max batch delay: 200 ms
 Max concurrent requests: 500,000 

 + RESULTS:
 Batch creation avg latency: 202 ms
 Header creation avg latency: -1 ms
 	Batch to header avg latency: -1 ms
 Header to certificate avg latency: 2 ms
 	Request vote outbound avg latency: 0 ms
 Certificate commit avg latency: 846 ms

 Consensus TPS: 0 tx/s
 Consensus BPS: 0 B/s
 Consensus latency: 0 ms

 End-to-end TPS: 0 tx/s
 End-to-end BPS: 0 B/s
 End-to-end latency: 0 ms
-----------------------------------------

Please sign in to comment.