Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[manta] Fix CI integration test false negatives #421

Merged
merged 4 commits into from Mar 4, 2022

Conversation

ghzlatarev
Copy link
Contributor

@ghzlatarev ghzlatarev commented Mar 1, 2022

Description

closes: #420

  • Our CI integration test frequently failed because it relied on parsing a specific finalized block, for example block 6. However in some cases blocks are finalized but not logged. Their finalization is implied from succeeding finalized blocks. So in those cases our old test would fail.
  • The fix is a simple tool to query the chain state directly instead of parsing logs. It succeeds if the block is more than or equal to the specified target block https://github.com/Manta-Network/Dev-Tools/tree/main/check-finalized-block

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

  • Targeted PR against correct branch (manta or dolphin) with right title (start with [Manta] or [Dolphin]),
  • Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
  • Wrote unit tests.
  • Updated relevant documentation in the code.
  • Added one line describing your change in <branch>/CHANGELOG.md
  • Re-reviewed Files changed in the Github PR explorer.
  • If runtime changes, need to update the version numbers properly:
    • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
    • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
    • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
    • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. If this number is updated, then the spec_version must also be updated
  • If needed, notify the committer this is a draft-release and a tag is needed after merging the PR.
  • Verify benchmarks & weights have been updated for any modified runtime logics
  • If needed, bump version for every crate.
  • If import a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
  • If needed, update our Javascript/Typescript APIs. These APIs are offcially used by exchanges or community developers.
  • If we're going to issue a new release, freeze the code one week early(it depends, but usually it's one week), ensure we have enough time for related testing.
  • Check if inheriting any upstream runtime storage migrations. If any, perform tests with try-runtime.

@ghzlatarev ghzlatarev requested review from grenade, stechu and Dengjianping and removed request for grenade March 1, 2022 11:04
@ghzlatarev ghzlatarev self-assigned this Mar 1, 2022
@ghzlatarev ghzlatarev added A-ci Area: Continuous Integration A-automation Area: Issues and PRs related to Automation labels Mar 1, 2022
@ghzlatarev ghzlatarev changed the title Fix CI integration test false negatives [not ready for review] Fix CI integration test false negatives Mar 1, 2022
@ghzlatarev ghzlatarev changed the title [not ready for review] Fix CI integration test false negatives Fix CI integration test false negatives Mar 2, 2022
@ghzlatarev ghzlatarev changed the title Fix CI integration test false negatives [manta] Fix CI integration test false negatives Mar 2, 2022
Copy link
Contributor

@Garandor Garandor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense that blocks don't necessarily get finalized one-by-one in log prints.
I like the resolution of just asking the node what it's last block is ( some awk-magic could also have done it but that's less maintainable )

LGTM

@ghzlatarev ghzlatarev requested a review from Garandor March 3, 2022 09:18
@stechu stechu merged commit dd20b83 into manta Mar 4, 2022
@stechu stechu deleted the ghzlatarev/fix-ci-false-negatives branch March 4, 2022 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-automation Area: Issues and PRs related to Automation A-ci Area: Continuous Integration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix CI tests false negatives
3 participants