Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[move-core] Add ident_str! macro to create const IdentStr's #8300

Merged
merged 5 commits into from
May 13, 2021

Conversation

phlip9
Copy link
Contributor

@phlip9 phlip9 commented Apr 29, 2021

Overview

A PR for something that's been bothering me for a while; namely, storing &'static strs instead of &'static IdentStrs for MoveResource::MODULE_NAME and MoveResource::STRUCT_NAME, which means lots of Lazy<Identifier> and unwraps everywhere in diem-types.

This diff adds a new macro ident_str! which lets us construct const &'static IdentStrs that are asserted valid at compile-time.

Note for reviewers: most of the diff is just swapping Identifier::new("..") and IdentStr::new("..") for ident_str!. The interesting parts to check out are the macro in language/move-core/types/src/identifier.rs and the consts in MoveResource changed to &'static IdentStr.

There's also an unsafe line in here:

macro_rules ident_str {
    ($ident:literal) => {{
        // ..

        // SAFETY: the following transmute is safe because
        // (1) it's equivalent to the unsafe-reborrow inside IdentStr::ref_cast()
        //     (which we can't use b/c it's not const).
        // (2) we've just asserted that IdentStr impls RefCast<From = str>, which
        //     already guarantees the transmute is safe (RefCast checks that
        //     IdentStr(str) is #[repr(transparent)]).
        // (3) both in and out lifetimes are 'static, so we're not widening the lifetime.
        // (4) we've just asserted that the IdentStr passes the is_valid check.
        //
        // Note: this lint is unjustified and no longer checked. See issue:
        // https://github.com/rust-lang/rust-clippy/issues/6372
        #[allow(clippy::transmute_ptr_to_ptr)]
        unsafe {
            ::std::mem::transmute::<&'static str, &'static $crate::identifier::IdentStr>(s)
        }
    }}
}

which is safe because we derive RefCast for IdentStr, which guarantees this kind of transmute is safe if the trait is derivable. Finally, the transmute is effectively the same as the unsafe pointer cast inside RefCast::ref_cast(s), except we can actually use transmute in a const context (though not in a const fn for some reason, which is why this is implemented as a macro and not an IdentStr method).

@phlip9 phlip9 added the refactoring Refactoring label Apr 29, 2021
@bors-libra bors-libra added this to In Review in bors Apr 29, 2021
// Only valid identifier strings are allowed.
// Note: Work-around hack to print an error message in a const block.
let is_valid = $crate::identifier::is_valid(s);
["String is not a valid Move identifier"][!is_valid as usize];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

woah...

@@ -680,41 +668,37 @@ pub(crate) fn remapping(script_bytes: &[u8]) -> Option<(&'static ModuleId, &'sta
static ACCOUNT_ADMINISTRATION_SCRIPTS: Lazy<ModuleId> = Lazy::new(|| {
ModuleId::new(
CORE_CODE_ADDRESS,
Identifier::new("AccountAdministrationScripts").unwrap(),
ident_str!("AccountAdministrationScripts").to_owned(),
)
});

static ACCOUNT_CREATION_SCRIPTS: Lazy<ModuleId> = Lazy::new(|| {
ModuleId::new(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be made const now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we can make ModuleId::new const b/c we can't make const Identifiers, as they need to allocate on the heap.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right... Unfortunate.

Copy link
Contributor

@bmwill bmwill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tnowacki It seems like you didn't have any major concerns with this PR? If not I'm going to go ahead and approve this so that Philip can get this landed. Unfortunately it already looks like its bitrotted a lot and will require a rebase.

Copy link
Contributor

@tnowacki tnowacki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay, but yes it seems fine to me! I think this does a good bit of relieving pain around the str version of Idents, especially for constants. Thanks!

@phlip9
Copy link
Contributor Author

phlip9 commented May 13, 2021

rebased

@phlip9
Copy link
Contributor Author

phlip9 commented May 13, 2021

/land

@bors-libra bors-libra moved this from In Review to Queued in bors May 13, 2021
bors-libra pushed a commit that referenced this pull request May 13, 2021
@bors-libra bors-libra moved this from Queued to Testing in bors May 13, 2021
@github-actions
Copy link

Cluster Test Result

Test runner setup time spent 390 secs
Cluster setup failed: `Nodes did not become healthy after deployment`
Logs: http://kibana.ct-1-k8s-testnet.aws.hlw3truzy4ls.com/app/kibana#/discover?_g=(time:(from:'2021-05-13T03:40:53Z',to:'2021-05-13T03:55:52Z'))
Dashboard: http://grafana.ct-1-k8s-testnet.aws.hlw3truzy4ls.com/d/performance/performance?from=1620877253000&to=1620878152000
Validator 1 logs: http://kibana.ct-1-k8s-testnet.aws.hlw3truzy4ls.com/app/kibana#/discover?_g=(time:(from:'2021-05-13T03:40:53Z',to:'2021-05-13T03:55:52Z'))&_a=(columns:!(log),query:(language:kuery,query:'kubernetes.pod_name:"val-1"'),sort:!(!('@timestamp',desc)))

❗ Cluster Test failed - non-zero exit code for cti

Repro cmd:

./scripts/cti --tag land_4b237180 --cluster-test-tag land_d01fbab7 -E BATCH_SIZE=15 -E UPDATE_TO_TAG=land_d01fbab7 --report report.json --suite land_blocking_compat

@bors-libra
Copy link
Contributor

💔 Test Failed - ci-test

@bors-libra bors-libra moved this from Testing to In Review in bors May 13, 2021
@zihaoccc
Copy link
Contributor

/land

@bors-libra bors-libra moved this from In Review to Queued in bors May 13, 2021
@bors-libra bors-libra moved this from Queued to Testing in bors May 13, 2021
@github-actions
Copy link

Cluster Test Result

Test runner setup time spent 264 secs
Compatibility test results for land_88bef8ec ==> land_9428e916 (PR)
1. All instances running land_88bef8ec, generating some traffic on network
2. First full node land_88bef8ec ==> land_9428e916, to validate new full node to old validator node traffic
3. First Validator node land_88bef8ec ==> land_9428e916, to validate storage compatibility
4. First batch validators (14) land_88bef8ec ==> land_9428e916, to test consensus and traffic between old full nodes and new validator node
5. First batch full nodes (14) land_88bef8ec ==> land_9428e916
6. Second batch validators (15) land_88bef8ec ==> land_9428e916, to upgrade rest of the validators
7. Second batch of full nodes (15) land_88bef8ec ==> land_9428e916, to finish the network upgrade, time spent 674 secs
all up : 857 TPS, 5299 ms latency, 6050 ms p99 latency, no expired txns, time spent 250 secs
Logs: http://kibana.ct-1-k8s-testnet.aws.hlw3truzy4ls.com/app/kibana#/discover?_g=(time:(from:'2021-05-13T16:13:17Z',to:'2021-05-13T16:36:01Z'))
Dashboard: http://grafana.ct-1-k8s-testnet.aws.hlw3truzy4ls.com/d/performance/performance?from=1620922397000&to=1620923761000
Validator 1 logs: http://kibana.ct-1-k8s-testnet.aws.hlw3truzy4ls.com/app/kibana#/discover?_g=(time:(from:'2021-05-13T16:13:17Z',to:'2021-05-13T16:36:01Z'))&_a=(columns:!(log),query:(language:kuery,query:'kubernetes.pod_name:"val-1"'),sort:!(!('@timestamp',desc)))

Repro cmd:

./scripts/cti --tag land_88bef8ec --cluster-test-tag land_9428e916 -E BATCH_SIZE=15 -E UPDATE_TO_TAG=land_9428e916 --report report.json --suite land_blocking_compat

🎉 Land-blocking cluster test passed! 👌

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants