Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

QA/v1: Run 200-nodes test for final report without latency emulation #2461

Closed
Tracked by #2100
hvanz opened this issue Feb 27, 2024 · 0 comments · Fixed by #2667
Closed
Tracked by #2100

QA/v1: Run 200-nodes test for final report without latency emulation #2461

hvanz opened this issue Feb 27, 2024 · 0 comments · Fixed by #2667
Assignees
@hvanz
Labels
qa Quality assurance
Milestone
2024-Q2

Comments

@hvanz
Copy link
Member

hvanz commented Feb 27, 2024 •
edited by sergio-mena

This is the regular 200-nodes test for the v1 release as described in https://github.com/cometbft/cometbft/blob/main/RELEASES.md#200-node-testnet.

Settings:

  • Run on v1.x branch.
  • Vote extensions enable height: 1
  • PBTS enable height: 1
  • Run two sets of experiments:
    1. for discovery the saturation point
    2. to gather metrics on all nodes using a load slight below the saturation point
  • Duration ~40 mins
  • NOT using latency emulation, so we can compare the results with the last QA process for v0.38

DoD:

  • output: QA report, 200-node test, with saturation point report section, and metrics report section (for sat point)
  • output: compare metrics between (v0.38.x, bft time) and (v1, PBTS)
@hvanz hvanz added the qa Quality assurance label Feb 27, 2024
@hvanz hvanz added this to the 2024-Q1 milestone Feb 27, 2024
This was referenced Feb 27, 2024
Closed
Closed
@hvanz hvanz changed the title QA/v1: Run 200-nodes test for final report QA/v1: Run 200-nodes test for final report without latency emulation Mar 4, 2024
@hvanz hvanz mentioned this issue Mar 4, 2024
Closed
@hvanz hvanz mentioned this issue Mar 21, 2024
Merged
4 tasks
@adizere adizere modified the milestones: 2024-Q1, 2024-Q2 Apr 1, 2024
github-merge-queue bot pushed a commit that referenced this issue Apr 10, 2024
@hvanz @cason @sergio-mena
docs(qa/v1): Report on saturation point and 200-nodes test (#2667)
53e6127 
Closes #2461, #2513

This report includes results for tests with and without latency
emulation.


[Rendered](https://github.com/cometbft/cometbft/blob/hvanz/qa-v1-report-2461/docs/references/qa/CometBFT-QA-v1.md)

---

#### PR checklist

- [ ] Tests written/updated
- [ ] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [ ] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [X] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec

---------

Co-authored-by: Daniel <daniel.cason@informal.systems>
Co-authored-by: Sergio Mena <sergio@informal.systems>
mergify bot pushed a commit that referenced this issue Apr 10, 2024
@hvanz @mergify
docs(qa/v1): Report on saturation point and 200-nodes test (#2667)
20cab9e 
Closes #2461, #2513

This report includes results for tests with and without latency
emulation.

[Rendered](https://github.com/cometbft/cometbft/blob/hvanz/qa-v1-report-2461/docs/references/qa/CometBFT-QA-v1.md)

---

#### PR checklist

- [ ] Tests written/updated
- [ ] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [ ] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [X] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec

---------

Co-authored-by: Daniel <daniel.cason@informal.systems>
Co-authored-by: Sergio Mena <sergio@informal.systems>
(cherry picked from commit 53e6127)

# Conflicts:
#	scripts/qa/reporting/prometheus_plotter.py
@mergify mergify bot mentioned this issue Apr 10, 2024
Merged
4 tasks
hvanz added a commit that referenced this issue Apr 12, 2024
@mergify @hvanz
docs(qa/v1): Report on saturation point and 200-nodes test (backport #…
c519126 
…2667) (#2764)

Closes #2461, #2513

This report includes results for tests with and without latency
emulation.


[Rendered](https://github.com/cometbft/cometbft/blob/hvanz/qa-v1-report-2461/docs/references/qa/CometBFT-QA-v1.md)

---

#### PR checklist

- [ ] Tests written/updated
- [ ] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [ ] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [X] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
<hr>This is an automatic backport of pull request #2667 done by
[Mergify](https://mergify.com).

---------

Co-authored-by: Hernán Vanzetto <15466498+hvanz@users.noreply.github.com>
Co-authored-by: hvanz <hernan.vanzetto@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hvanz hvanz

Labels
qa Quality assurance
Projects
CometBFT
Status: Done
Milestone
2024-Q2
Development

Successfully merging a pull request may close this issue.

docs(qa/v1): Report on saturation point and 200-nodes test cometbft/cometbft
2 participants
@adizere @hvanz