Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix!: e2e tests are flaky due to slow app state processing #745

Merged
merged 10 commits into from
Mar 5, 2024

Conversation

lklimek
Copy link
Collaborator

@lklimek lklimek commented Feb 29, 2024

Issue being fixed or feature implemented

TestApp_TxTooBig fails in GHA pipelines due to long execution time.
It is caused by JSON serialization within example kvstore application, which consumes significant amount of time and memory.

What was done?

  1. Instead of running the test on each node one by one, broadcast txs to all nodes first, and then check results in separate loop.
  2. Refactored kvstore to use streaming JSON serialization and deserialization when saving/restoring state.
  3. Changed GenesisDoc.AppState to []byte
  4. Updated e2e to use new format of initial app state
  5. In e2e dashcore and rotate networks, decreased block size and evidence size to consume less resources during test

How Has This Been Tested?

GHA

Breaking Changes

Genesis doc AppState MUST be a base64-encoded bytes array, not JSON string.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

@lklimek lklimek marked this pull request as draft February 29, 2024 11:48
@lklimek lklimek changed the title test(e2e): fix flaky TestApp_TxTooBig fix(kvstore): app state encoding performance fixed and genesis doc format changed Mar 4, 2024
@lklimek lklimek changed the title fix(kvstore): app state encoding performance fixed and genesis doc format changed fix!: genesis doc initial app state changed to base64 and refactored kvstore state processing Mar 4, 2024
@lklimek lklimek changed the title fix!: genesis doc initial app state changed to base64 and refactored kvstore state processing fix!: e2e tests are flaky due to TestApp_TxTooBig exec time, changes GenesisDoc.InitialAppState Mar 4, 2024
@lklimek lklimek changed the title fix!: e2e tests are flaky due to TestApp_TxTooBig exec time, changes GenesisDoc.InitialAppState fix!: e2e tests are flaky due to TestApp_TxTooBig exec time Mar 4, 2024
@lklimek lklimek changed the title fix!: e2e tests are flaky due to TestApp_TxTooBig exec time fix!: e2e tests are flaky due to slow app state processing Mar 4, 2024
@lklimek lklimek marked this pull request as ready for review March 4, 2024 17:14
@lklimek lklimek enabled auto-merge (squash) March 4, 2024 17:25
Copy link
Member

@shumkov shumkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@lklimek lklimek merged commit 420f7a3 into v0.14-dev Mar 5, 2024
15 checks passed
@lklimek lklimek deleted the fix/e2e-tests-big-block branch March 5, 2024 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants