Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: Refactor PrepareProposal to produce blocks using the non-interactive defaults #692

Merged
merged 46 commits into from
Sep 20, 2022

Conversation

evan-forbes
Copy link
Member

@evan-forbes evan-forbes commented Sep 8, 2022

This PR refactors PrepareProposal to produce blocks according to the non-interactive defaults. It could probably be split up into a few PRs, and we might have to do that to make it easier to review.

a big part of #382
(we can optionally spin out some changes into a different PR, but this PR was blocked on the below issues, so I included the relevant changes here (pls see comments for details))
closes #693
closes #588

@evan-forbes evan-forbes added C: Celestia app consensus breaking modifies block validity rules in a way that will break consensus unless all nodes update their rules labels Sep 8, 2022
@evan-forbes evan-forbes self-assigned this Sep 8, 2022
@codecov-commenter
Copy link

codecov-commenter commented Sep 8, 2022

Codecov Report

Merging #692 (7d2ee33) into main (b98f9ab) will decrease coverage by 4.22%.
The diff coverage is 69.05%.

@@            Coverage Diff             @@
##             main     #692      +/-   ##
==========================================
- Coverage   46.38%   42.16%   -4.23%     
==========================================
  Files          33       42       +9     
  Lines        3251     4383    +1132     
==========================================
+ Hits         1508     1848     +340     
- Misses       1625     2387     +762     
- Partials      118      148      +30     
Impacted Files Coverage Δ
app/prepare_proposal.go 0.00% <0.00%> (ø)
app/process_proposal.go 0.00% <0.00%> (ø)
app/parse_txs.go 64.00% <64.00%> (ø)
pkg/wrapper/nmt_wrapper.go 86.20% <66.66%> (ø)
app/malleate_txs.go 71.87% <71.87%> (ø)
app/test_util.go 75.00% <75.00%> (ø)
pkg/shares/share_splitting.go 63.91% <75.00%> (+4.16%) ⬆️
app/estimate_square_size.go 81.64% <81.64%> (ø)
pkg/prove/proof.go 82.48% <100.00%> (ø)
pkg/shares/non_interactive_defaults.go 94.82% <100.00%> (ø)
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

app/test/fuzz_abci_test.go Outdated Show resolved Hide resolved
Comment on lines +121 to +123
// TODO: redo this tests, which is more difficult to do now that it requires the
// data to be processed by PrepareProposal func
// TestProcessMessagesWithReservedNamespaces(t *testing.T) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test was commented out as its still something we want, although I think it can be done in a different PR. The modified test above kind of tests this, but its difficult to perform the same test since we've added requirements to how we arrange the block. We basically have to call PrepareProposal in order for ProcessProposal to accept the block, which itself checks for these invalid txs.

Comment on lines 24 to 25
func TestMessageInclusionCheck(t *testing.T) {
signer := testutil.GenerateKeyringSigner(t, testAccName)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refactored this test to also test for the deleted test

Comment on lines 22 to 26
// GenerateManyRawWirePFD creates many raw WirePayForData transactions. Using
// negative numbers for count and size will randomize those values. count is
// capped at 5000 and size is capped at 3MB. Going over these caps will result
// in randomized values.
func GenerateManyRawWirePFD(t *testing.T, txConfig client.TxConfig, signer *types.KeyringSigner, count, size int) [][]byte {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have a lot of duplicate test code that we need to sort out

@rootulp rootulp mentioned this pull request Sep 12, 2022
@evan-forbes
Copy link
Member Author

evan-forbes commented Sep 12, 2022

It could probably be split up into a few PRs, and we might have to do that to make it easier to review.

we can try and do this. It might mean duplicate or unused code in some instances, or that we have to spend more time debugging getting that chunk of code working with the old implementation, but those things might be unavoidable in order to get a proper review.

We also have roughly 2.5 weeks until this needs to be finished and merged now that the audit has been postponed a bit, so we should have the necessary time.

will likely close this PR after opening new smaller ones.

@rootulp
Copy link
Collaborator

rootulp commented Sep 16, 2022

Blocked on #691

@evan-forbes evan-forbes changed the title Refactor PrepareProposal to produce blocks using the non-interactive defaults feat!: Refactor PrepareProposal to produce blocks using the non-interactive defaults Sep 19, 2022
@evan-forbes evan-forbes marked this pull request as ready for review September 19, 2022 22:34
@evan-forbes
Copy link
Member Author

I haven't made many changes other than to make compatible with main. Reopening after attempting to break into a few PRs. While possible to break into smaller chunks, its impossible to add tests to those chunks until all portions are functional.

Comment on lines +28 to +29
removedContiguousShares := 0
contigBytesCursor := 0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[follow-up PR] we may replace the contiguous terminology with compact

@@ -160,5 +160,5 @@ require (
replace (
github.com/cosmos/cosmos-sdk => github.com/celestiaorg/cosmos-sdk v1.2.0-sdk-v0.46.0
github.com/gogo/protobuf => github.com/regen-network/protobuf v1.3.3-alpha.regen.1
github.com/tendermint/tendermint => github.com/celestiaorg/celestia-core v1.5.0-tm-v0.34.20
github.com/tendermint/tendermint v0.34.20 => github.com/celestiaorg/celestia-core v1.5.0-tm-v0.34.20
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[question][non-blocking] out of curiosity, why is the exact tendermint version specified?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoops, good catch, this actually wasn't on purpose. While it technically shouldn't matter we might want to switch it back. I think the only time it did matter was when used to change the imports from tendermint/tendermint to celestiaorg/celestia-core

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merging w/o as this is optional, and I don't want to look at this PR anymore lol

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. :shipit: and we can always remove later

@evan-forbes evan-forbes merged commit e18de88 into main Sep 20, 2022
@evan-forbes evan-forbes deleted the evan/NID-PrepareProposalRefactor branch September 20, 2022 19:51
rach-id pushed a commit to rach-id/celestia-app that referenced this pull request Nov 16, 2022
…eractive defaults (celestiaorg#692)

Co-authored-by: Rootul Patel <rootulp@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
consensus breaking modifies block validity rules in a way that will break consensus unless all nodes update their rules
Projects
No open projects
Status: Done
3 participants