Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set transaction size limit #4107

Merged
merged 21 commits into from Mar 24, 2021
Merged

Set transaction size limit #4107

merged 21 commits into from Mar 24, 2021

Conversation

Longarithm
Copy link
Member

No description provided.

@Longarithm Longarithm linked an issue Mar 15, 2021 that may be closed by this pull request
@Longarithm Longarithm changed the title Set transaction size limit #3274 Set transaction size limit Mar 15, 2021
@@ -72,6 +72,14 @@ impl Action {
_ => 0,
}
}
#[cfg(feature = "protocol_feature_tx_size_limit")]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest for the transaction size to use the size of the transaction serialized as borsh. This will remove potential tricky angles of attack, e.g. it is still possible to create a heavy transaction that adds a very large number of function access keys with long method names. Also, that removes ambiguity in interpreting what transaction size is, so e.g. we can say that block size is sum of the transaction sizes plus header data.

Ideally, we would add a trait to Borsh called BorshSize that implements borsh_size() -> u64 method which we can use similarly to BorshSerialize and BorshDeserialize, like this: #[derive(BorshSize)]. AFAIR there might some places in our code where we measure the size of the objects by first serializing them in Borsh and then counting bytes, so it might be useful there too. It shouldn't be difficult to add to borsh and will probably be largely copy-paste of https://github.com/near/borsh-rs/blob/master/borsh/src/ser/mod.rs

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah agree with what Max said here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!
Changed the way of size computation and added a separate issue for adding new trait.

runtime/runtime/src/verifier.rs Outdated Show resolved Hide resolved
core/primitives-core/src/config.rs Show resolved Hide resolved
@@ -57,6 +57,19 @@ pub fn validate_transaction(
return Err(InvalidTxError::InvalidSignature.into());
}

#[cfg(feature = "protocol_feature_tx_size_limit")]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check should only be done once. There is no reason to check transactions for which you already checked. For example, inside prepare_transactions it doesn't make sense to check again because this check should have been done when the transaction is received. To further reduce the overhead, you could change NetworkClientMessage::Transaction to include the transaction size. It probably even makes sense to implement custom borsh deserializer that would fail when the a larger-than-allowed transaction arrives on the network (this can potentially be done in a separate PR)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that at least one serialization is already happening inside SignedTransaction::new, to save Transaction hash. So I saved size there in the same fashion and started to use it, which shouldn't give significant overhead.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that makes sense

@@ -57,6 +57,19 @@ pub fn validate_transaction(
return Err(InvalidTxError::InvalidSignature.into());
}

#[cfg(feature = "protocol_feature_tx_size_limit")]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that makes sense

@bowenwang1996
Copy link
Collaborator

Please run nayduck

@bowenwang1996
Copy link
Collaborator

@Longarithm is this PR ready for review?

@Longarithm
Copy link
Member Author

Not yet, I got some Nayduck tests failing: http://nayduck.eastus.cloudapp.azure.com:3000/#/run/1401

@Longarithm
Copy link
Member Author

http://nayduck.eastus.cloudapp.azure.com:3000/#/run/1414 - runs for master, 26 failed tests in nightly
http://nayduck.eastus.cloudapp.azure.com:3000/#/run/1413 - runs for this branch, 24 failed tests in nightly
The only new test failing is pytest sanity/epoch_switches.py
Adding @mikhailOK

@Longarithm Longarithm marked this pull request as ready for review March 23, 2021 22:25
core/primitives/src/transaction.rs Outdated Show resolved Hide resolved
@Longarithm Longarithm merged commit 4088a98 into master Mar 24, 2021
@Longarithm Longarithm deleted the limit-transaction branch March 24, 2021 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Limit transaction size
6 participants