Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Compiler-V2] Unused type parameter checking #12806

Merged
merged 11 commits into from
Apr 12, 2024

Conversation

fEst1ck
Copy link
Contributor

@fEst1ck fEst1ck commented Apr 5, 2024

Description

  • Implement a checker for unused parameters in struct definitions, as an environment pipeline
  • Improve the error messages for duplicate type parameters for functions and structs, by adding the location of the duplication.

Fixed #11709

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Other (specify)

How Has This Been Tested?

Existing tests and checking/naming/unused_type_parameter_struct.move

Copy link

trunk-io bot commented Apr 5, 2024

⏱️ 12h 27m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-smoke-tests 1h 48m 🟩🟩🟩
rust-unit-tests 1h 39m 🟥🟥🟥🟩
execution-performance / single-node-performance 1h 31m 🟩🟩🟩🟩
rust-move-unit-coverage 1h 17m 🟩🟩🟥🟥
windows-build 1h 7m 🟩🟩🟩🟩
rust-move-tests 45m 🟩🟩🟩🟩
forge-e2e-test / forge 43m 🟩🟩🟩
rust-images / rust-all 42m 🟩🟩🟩
forge-compat-test / forge 35m 🟩🟩🟩
cli-e2e-tests / run-cli-tests 22m 🟩🟩🟩
rust-lints 21m 🟩🟩🟩🟩
run-tests-main-branch 21m 🟩🟩🟩🟩🟩
rust-targeted-unit-tests 20m 🟩
check 13m 🟩🟩🟩
rust-build-cached-packages 12m 🟩🟩🟩
check-dynamic-deps 12m 🟩🟩🟩🟩🟩 (+1 more)
general-lints 8m 🟩🟩🟩🟩
node-api-compatibility-tests / node-api-compatibility-tests 3m 🟩🟩🟩
semgrep/ci 2m 🟩🟩🟩🟩🟩 (+1 more)
file_change_determinator 1m 🟩🟩🟩🟩🟩
file_change_determinator 1m 🟩🟩🟩🟩🟩
file_change_determinator 56s 🟩🟩🟩🟩🟩
file_change_determinator 55s 🟩🟩🟩🟩
execution-performance / file_change_determinator 55s 🟩🟩🟩🟩
permission-check 22s 🟩🟩🟩🟩🟩 (+1 more)
permission-check 19s 🟩🟩🟩🟩🟩 (+1 more)
permission-check 15s 🟩🟩🟩🟩🟩
permission-check 13s 🟩🟩🟩🟩
permission-check 11s 🟩🟩🟩🟩🟩
determine-docker-build-metadata 7s 🟩🟩🟩🟩

settingsfeedbackdocs ⋅ learn more about trunk.io

@fEst1ck fEst1ck changed the title [Compiler-V2] Unused params checking [Compiler-V2] Unused parameterss checking Apr 5, 2024
@brmataptos brmataptos changed the title [Compiler-V2] Unused parameterss checking [Compiler-V2] Unused parameter checking Apr 5, 2024
@brmataptos brmataptos changed the title [Compiler-V2] Unused parameter checking [Compiler-V2] Unused type parameter checking Apr 5, 2024
Copy link

codecov bot commented Apr 5, 2024

Codecov Report

Attention: Patch coverage is 70.42254% with 21 lines in your changes are missing coverage. Please review.

Project coverage is 62.6%. Comparing base (c807593) to head (25c99a8).
Report is 26 commits behind head on main.

❗ Current head 25c99a8 differs from pull request most recent head 004acec. Consider uploading reports for the commit 004acec to get more accurate results

Files Patch % Lines
...d_party/move/move-model/src/builder/exp_builder.rs 0.0% 16 Missing ⚠️
...move/move-compiler-v2/src/unused_params_checker.rs 88.8% 5 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main   #12806       +/-   ##
===========================================
- Coverage    69.2%    62.6%     -6.7%     
===========================================
  Files        2296      822     -1474     
  Lines      438064   184232   -253832     
===========================================
- Hits       303482   115435   -188047     
+ Misses     134582    68797    -65785     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Comment on lines 59 to 62
let mut set = BTreeSet::new();
set.insert(*i);
set
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let mut set = BTreeSet::new();
set.insert(*i);
set
},
BTreeSet::from([*i])
},

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could also use BTreeSet::from_iter(iter::once(*i))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. Thanks!

set
},
Type::Fun(from, to) => {
let mut used_in_from = used_type_parameters_in_ty(from);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe nice to have functional style code here instead: used_type_parameters_in_ty(from).union(&used_type_parameters_in_ty(to)).collect() or similar? It seems like we have mixed functional (with the flat_map above) and imperative styles, which seems a bit inconsistent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't do it this way because this would require copies in general. (But copying u16 is ok in this case).

.collect()
}

/// Returns the indices of type parameters used in the given type.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these indices unique across all possible type parameters in the program, or within a struct declaration, or something else?

When I read this comment, it made me think the indices are w.r.t the given type, but that could be problematic. Maybe we want to clarify what the "scope" of these indices are (eg., unique across all compilation targets): ideally, this would be done where TypeParameter is declared through a comment.

Copy link
Contributor Author

@fEst1ck fEst1ck Apr 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The indices are not unique cross programs, structs, etc. Only the caller of the method may know the scope, but we can be sure that the indices within a type are in the same scope

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add a comment of this nature for this function documentation? For example, if have a vector inside a struct, the "given type" when calling this method recursively would be the vector, but the type param indexes refer to the original type struct.

set.insert(*i);
set
},
Type::Fun(from, to) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this code even reachable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently no. I include this case in case we allow them in the future.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, which test case triggers this (or attempts to)? I see the ones for other features, but not this one. I am not sure we have a plan to ever have functions as fields of structs.

In any case, I think we should move it to the unreachable! case because I don't understand the interaction of phantom types with function types in structs, and we should not have arbitrary implementations.

// Copyright © Aptos Foundation
// SPDX-License-Identifier: Apache-2.0

//! Implements an environment pipeline which checks for unused type parameters in struct definitions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to add if there are any pre-conditions (such as passes and checks that this assumes), so that we can re-order stuff.

Eg, we may require recursive struct checks to be already done, otherwise, we may have infinite recursion here.

Copy link
Contributor Author

@fEst1ck fEst1ck Apr 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added pre-conditions. (But we don't need recursive struct checks in this case.)

fn used_type_parameters_in_ty(ty: &Type) -> BTreeSet<u16> {
match ty {
Type::Primitive(_) => BTreeSet::new(),
Type::Tuple(tys) | Type::Struct(_, _, tys) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can tuple types be reachable in this context?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently no. I include this case in case we allow them in the future.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, as well as in all other cases that are not currently reachable, we should instead move them to the unreachable! case below. I think we should not speculate non-existent features and invent an unnecessary implementation.

used_in_from.append(&mut used_type_parameters_in_ty(to));
used_in_from
},
Type::Vector(ty) | Type::Reference(_, ty) => used_type_parameters_in_ty(ty),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are reference types reachable in this context?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently no. I include this case in case we allow them in the future.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, let's please move them to unreachable!, so if we ever have this in the future, we are forced to think about its semantics and implications.

struct S2<T1, phantom T2> {
f: S3<T2>,
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add some more tests for coverage here (or in a different test file)?

  • use vectors, maybe some more depth of structs, other stuff asked I asked for reachability if they are reachable?
  • add recursive structs with type params (to perhaps catch any unwanted re-orderings of passes).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@fEst1ck
Copy link
Contributor Author

fEst1ck commented Apr 9, 2024

All comments addressed. PTAL @vineethk

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite compat success on aptos-node-v1.10.1 ==> 004acec34f3fffb667edac5407ee23885afb08de

Compatibility test results for aptos-node-v1.10.1 ==> 004acec34f3fffb667edac5407ee23885afb08de (PR)
1. Check liveness of validators at old version: aptos-node-v1.10.1
compatibility::simple-validator-upgrade::liveness-check : committed: 6437 txn/s, latency: 4978 ms, (p50: 4800 ms, p90: 8500 ms, p99: 10200 ms), latency samples: 238200
2. Upgrading first Validator to new version: 004acec34f3fffb667edac5407ee23885afb08de
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1868 txn/s, latency: 15515 ms, (p50: 19500 ms, p90: 22200 ms, p99: 22600 ms), latency samples: 91560
3. Upgrading rest of first batch to new version: 004acec34f3fffb667edac5407ee23885afb08de
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1714 txn/s, latency: 16566 ms, (p50: 19300 ms, p90: 22500 ms, p99: 23500 ms), latency samples: 89140
4. upgrading second batch to new version: 004acec34f3fffb667edac5407ee23885afb08de
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3685 txn/s, latency: 8714 ms, (p50: 9800 ms, p90: 12600 ms, p99: 12900 ms), latency samples: 143740
5. check swarm health
Compatibility test for aptos-node-v1.10.1 ==> 004acec34f3fffb667edac5407ee23885afb08de passed
Test Ok

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on 004acec34f3fffb667edac5407ee23885afb08de

two traffics test: inner traffic : committed: 8125 txn/s, latency: 4834 ms, (p50: 4500 ms, p90: 5700 ms, p99: 10700 ms), latency samples: 3502140
two traffics test : committed: 100 txn/s, latency: 1771 ms, (p50: 1800 ms, p90: 2100 ms, p99: 2300 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.209, avg: 0.202", "QsPosToProposal: max: 0.273, avg: 0.234", "ConsensusProposalToOrdered: max: 0.445, avg: 0.431", "ConsensusOrderedToCommit: max: 0.336, avg: 0.321", "ConsensusProposalToCommit: max: 0.767, avg: 0.752"]
Max round gap was 1 [limit 4] at version 1758989. Max no progress secs was 4.685833 [limit 15] at version 1758989.
Test Ok

Copy link
Contributor

@vineethk vineethk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@fEst1ck fEst1ck merged commit 5dc7037 into aptos-labs:main Apr 12, 2024
52 of 53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

[Feature Request][compiler-v2] Implement unused struct type parameter warning
3 participants