Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FVM] signature verification refactoring #2171

Merged
merged 22 commits into from
May 10, 2022
Merged

Conversation

tarakby
Copy link
Contributor

@tarakby tarakby commented Mar 21, 2022

  • remove the user-domain tag completely from the repo, as this is not a protocol constant anymore.
  • remove not needed SignatureVerifier interface to verify signatures.
  • separate the verification functions of runtime and transaction signatures that support different algorithms for a better clarity and to avoid future errors.
  • add extra checks for transaction verification inputs (although they are supposed to be valid as they are read from the state).
  • make the public function SignWithTag unable to sign other that the transaction tag for safety.
  • adjust existing tests, add new tests
  • avoid using fmt.Sprintf for key deduplication (performance)

@github-actions
Copy link
Contributor

github-actions bot commented Mar 21, 2022

FVM Benchstat comparison

This branch with compared with the base branch onflow:master commit 99d9259

The command (for i in {1..N}; do go test ./fvm --bench . --tags relic -shuffle=on; done) was used.

Bench tests were run a total of 7 times on each branch.

Results

old.txtnew.txt
time/opdelta
RuntimeNFTBatchTransfer-2173ms ± 7%172ms ± 5%~(p=0.710 n=7+7)
RuntimeTransaction/reference_tx-246.6ms ± 6%44.7ms ± 6%~(p=0.209 n=7+7)
RuntimeTransaction/convert_int_to_string-246.8ms ± 5%48.9ms ± 6%~(p=0.053 n=7+7)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-250.7ms ± 7%50.2ms ± 7%~(p=0.805 n=7+7)
RuntimeTransaction/get_signer_address-248.2ms ±10%47.8ms ± 5%~(p=0.805 n=7+7)
RuntimeTransaction/get_public_account-251.8ms ±12%50.3ms ± 5%~(p=0.165 n=7+7)
RuntimeTransaction/get_account_and_get_balance-2912ms ±13%899ms ± 6%~(p=0.805 n=7+7)
RuntimeTransaction/get_account_and_get_available_balance-2751ms ± 9%710ms ± 4%~(p=0.053 n=7+7)
RuntimeTransaction/get_account_and_get_storage_used-267.8ms ± 7%66.3ms ± 6%~(p=0.710 n=7+7)
RuntimeTransaction/get_account_and_get_storage_capacity-2694ms ± 9%663ms ±15%~(p=0.209 n=7+7)
RuntimeTransaction/get_signer_vault-258.1ms ± 4%58.3ms ± 8%~(p=1.000 n=7+7)
RuntimeTransaction/get_signer_receiver-274.2ms ± 3%72.9ms ± 4%~(p=0.485 n=6+6)
RuntimeTransaction/transfer_tokens-2286ms ± 2%298ms ± 6%~(p=0.051 n=6+7)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-256.9ms ± 7%57.6ms ± 8%~(p=0.902 n=7+7)
RuntimeTransaction/load_and_save_long_string_on_signers_address-2116ms ± 3%121ms ± 5%~(p=0.101 n=6+7)
RuntimeTransaction/create_new_account-21.74s ± 3%1.76s ± 1%~(p=0.240 n=6+6)
RuntimeTransaction/call_empty_contract_function-253.5ms ± 8%52.4ms ± 5%~(p=0.366 n=6+7)
RuntimeTransaction/emit_event-275.5ms ± 9%73.9ms ± 5%~(p=0.534 n=7+6)
RuntimeTransaction/borrow_array_from_storage-2186ms ±12%178ms ± 6%~(p=0.259 n=7+7)
RuntimeTransaction/copy_array_from_storage-2162ms ± 8%170ms ± 9%~(p=0.165 n=7+7)
 
computationdelta
RuntimeTransaction/reference_tx-2202 ± 0%202 ± 0%~(all equal)
RuntimeTransaction/convert_int_to_string-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-2502 ± 0%502 ± 0%~(all equal)
RuntimeTransaction/get_signer_address-2302 ± 0%302 ± 0%~(all equal)
RuntimeTransaction/get_public_account-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_balance-2802 ± 0%802 ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_available_balance-22.40k ± 0%2.40k ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_storage_used-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_storage_capacity-21.10k ± 0%1.10k ± 0%~(all equal)
RuntimeTransaction/get_signer_vault-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/get_signer_receiver-2602 ± 0%602 ± 0%~(all equal)
RuntimeTransaction/transfer_tokens-23.50k ± 0%3.50k ± 0%~(all equal)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-2602 ± 0%602 ± 0%~(all equal)
RuntimeTransaction/load_and_save_long_string_on_signers_address-2602 ± 0%602 ± 0%~(all equal)
RuntimeTransaction/create_new_account-2202 ± 0%202 ± 0%~(all equal)
RuntimeTransaction/call_empty_contract_function-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/emit_event-2602 ± 0%602 ± 0%~(all equal)
RuntimeTransaction/borrow_array_from_storage-22.60k ± 0%2.60k ± 0%~(all equal)
RuntimeTransaction/copy_array_from_storage-22.60k ± 0%2.60k ± 0%~(all equal)
 
interactionsdelta
RuntimeTransaction/reference_tx-245.6k ± 0%45.6k ± 0%~(all equal)
RuntimeTransaction/convert_int_to_string-245.6k ± 0%45.6k ± 0%~(all equal)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-245.6k ± 0%45.6k ± 0%~(all equal)
RuntimeTransaction/get_signer_address-245.6k ± 0%45.6k ± 0%~(all equal)
RuntimeTransaction/get_public_account-245.6k ± 0%45.6k ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_balance-216.7M ± 0%16.7M ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_available_balance-25.32M ± 0%5.32M ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_storage_used-248.4k ± 0%48.4k ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_storage_capacity-25.32M ± 0%5.32M ± 0%~(all equal)
RuntimeTransaction/get_signer_vault-246.8k ± 0%46.8k ± 0%~(all equal)
RuntimeTransaction/get_signer_receiver-247.2k ± 0%47.2k ± 0%~(all equal)
RuntimeTransaction/transfer_tokens-248.0k ± 0%48.0k ± 0%~(all equal)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-249.0k ± 0%49.0k ± 0%~(all equal)
RuntimeTransaction/load_and_save_long_string_on_signers_address-254.0k ± 0%54.0k ± 0%~(all equal)
RuntimeTransaction/create_new_account-28.44M ± 0%8.44M ± 0%~(all equal)
RuntimeTransaction/call_empty_contract_function-245.8k ± 0%45.8k ± 0%~(all equal)
RuntimeTransaction/emit_event-245.8k ± 0%45.8k ± 0%~(all equal)
RuntimeTransaction/borrow_array_from_storage-252.0k ± 0%52.0k ± 0%~(all equal)
RuntimeTransaction/copy_array_from_storage-252.0k ± 0%52.0k ± 0%~(all equal)
 

@codecov-commenter
Copy link

codecov-commenter commented Mar 23, 2022

Codecov Report

Merging #2171 (88db7a1) into master (ba1f157) will decrease coverage by 1.32%.
The diff coverage is 23.81%.

@@            Coverage Diff             @@
##           master    #2171      +/-   ##
==========================================
- Coverage   57.47%   56.14%   -1.33%     
==========================================
  Files         645      653       +8     
  Lines       38428    39911    +1483     
==========================================
+ Hits        22085    22409     +324     
- Misses      13530    14630    +1100     
- Partials     2813     2872      +59     
Flag Coverage Δ
unittests 56.14% <23.81%> (-1.33%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
cmd/bootstrap/transit/cmd/version.go 14.28% <ø> (ø)
cmd/execution_builder.go 0.00% <0.00%> (ø)
cmd/util/ledger/reporters/export_reporter.go 0.00% <0.00%> (ø)
cmd/verification_builder.go 0.00% <0.00%> (ø)
engine/common/rpc/convert/convert.go 16.07% <0.00%> (+2.13%) ⬆️
engine/execution/state/state.go 22.12% <0.00%> (-0.53%) ⬇️
fvm/context.go 68.68% <ø> (-0.32%) ⬇️
fvm/scriptEnv.go 39.15% <ø> (-0.16%) ⬇️
fvm/transactionEnv.go 46.60% <ø> (+0.08%) ⬆️
model/flow/constants.go 60.00% <ø> (ø)
... and 48 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4bb600e...88db7a1. Read the comment docs.

@tarakby tarakby marked this pull request as ready for review March 23, 2022 02:36
@tarakby tarakby self-assigned this Mar 23, 2022
fvm/crypto/crypto.go Outdated Show resolved Hide resolved
fvm/transactionVerifier.go Show resolved Hide resolved
fvm/crypto/hash.go Outdated Show resolved Hide resolved
Comment on lines -270 to +279
observedSigs := make(map[string]bool)
type uniqueKey struct {
address flow.Address
index uint64
}
observedSigs := make(map[uniqueKey]bool)
for _, sig := range append(tx.PayloadSignatures, tx.EnvelopeSignatures...) {
keyStr := sig.UniqueKeyString()
if observedSigs[keyStr] {
if observedSigs[uniqueKey{sig.Address, sig.KeyIndex}] {
return DuplicatedSignatureError{Address: sig.Address, KeyIndex: sig.KeyIndex}
}
observedSigs[keyStr] = true
observedSigs[uniqueKey{sig.Address, sig.KeyIndex}] = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this change is mostly for performance reasons, could you please check we have tests for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines -49 to -50
const UserTagString = "FLOW-V0.0-user"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might be in use by go-sdk or others ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is in use, this is a breaking change for the flow-go-sdk/examples. We should keep that in mind.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll update the sdk too after merging this PR: onflow/flow-go-sdk#273

Comment on lines +228 to +230
return false, errors.NewUnknownFailure(fmt.Errorf(
hashAlgo.String(), "is not supported in transactions"))
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure about this, you mean if there is some problematic old keys we would stop the network? I think we should just error and add a watch on the error type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tagged you about this here. I suggested to panic, Janez suggested the Failure error. As mentioned in the comment, that case can only happen if there is a bug in the code. These keys are all account keys, they are supposed to be checked and added previously, and they are not supposed to be problematic.

What do you suggest to use here?

Comment on lines +218 to +220
if pk.Algorithm() != crypto.ECDSAP256 && pk.Algorithm() != crypto.ECDSASecp256k1 {
// TODO: check if we should panic
// This case only happens in production if there is a bug
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should probably run some analysis on the mainnet data to verify all data are okey

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are all keys already checked and added using AddAccountKey, so this check is only a sanity check, and if it fails then we have am internal bug somewhere.

This is the same as above. I suggested to panic and Janez suggest Failure error. What do you suggest?

Copy link
Contributor

@ramtinms ramtinms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. thanks for clearing things up.

Copy link
Contributor

@janezpodhostnik janezpodhostnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments.

model/flow/transaction.go Outdated Show resolved Hide resolved
Comment on lines -49 to -50
const UserTagString = "FLOW-V0.0-user"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is in use, this is a breaking change for the flow-go-sdk/examples. We should keep that in mind.

fvm/transactionVerifier.go Show resolved Hide resolved
fvm/crypto/hash.go Outdated Show resolved Hide resolved
Copy link
Contributor

@ramtinms ramtinms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@tarakby
Copy link
Contributor Author

tarakby commented May 9, 2022

bors merge

@bors
Copy link
Contributor

bors bot commented May 10, 2022

Build succeeded:

@bors bors bot merged commit 1d17d82 into master May 10, 2022
@bors bors bot deleted the tarak/5466-fvm-signature branch May 10, 2022 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants