Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QA/v1: Run 200-nodes test for final report with latency emulation #2513

Closed
Tracked by #2100
hvanz opened this issue Mar 4, 2024 · 0 comments · Fixed by #2667
Closed
Tracked by #2100

QA/v1: Run 200-nodes test for final report with latency emulation #2513

hvanz opened this issue Mar 4, 2024 · 0 comments · Fixed by #2667
Assignees
Labels
qa Quality assurance
Milestone

Comments

@hvanz
Copy link
Member

hvanz commented Mar 4, 2024

This is essentially the same 200-nodes test for the v1 release as described in #2461 but using the latency emulation feature of the e2e framework.

Settings:

  • Run on v1.x branch.
  • Vote extensions enable height: 1
  • PBTS enable height: 1
  • Run two sets of experiments:
    1. for discovery of the saturation point (mainly because with latencies the saturation point is likely to be different)
    2. to gather metrics on all nodes using a load slight below the saturation point
  • Duration ~40 mins
  • With latency emulation enabled by randomly assigning "zones" to all nodes.

DoD:

  • On the final QA report for v1, a section for the 200-nodes test with latency emulation, so we can use a baseline for comparison with future releases, where the 200-nodes test without latency emulation will be deprecated.
  • The report should contain three subsections:
@hvanz hvanz added the qa Quality assurance label Mar 4, 2024
@hvanz hvanz added this to the 2024-Q1 milestone Mar 4, 2024
@adizere adizere modified the milestones: 2024-Q1, 2024-Q2 Apr 1, 2024
@adizere adizere linked a pull request Apr 3, 2024 that will close this issue
4 tasks
github-merge-queue bot pushed a commit that referenced this issue Apr 10, 2024
Closes #2461, #2513

This report includes results for tests with and without latency
emulation.


[Rendered](https://github.com/cometbft/cometbft/blob/hvanz/qa-v1-report-2461/docs/references/qa/CometBFT-QA-v1.md)

---

#### PR checklist

- [ ] Tests written/updated
- [ ] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [ ] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [X] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec

---------

Co-authored-by: Daniel <daniel.cason@informal.systems>
Co-authored-by: Sergio Mena <sergio@informal.systems>
mergify bot pushed a commit that referenced this issue Apr 10, 2024
Closes #2461, #2513

This report includes results for tests with and without latency
emulation.

[Rendered](https://github.com/cometbft/cometbft/blob/hvanz/qa-v1-report-2461/docs/references/qa/CometBFT-QA-v1.md)

---

#### PR checklist

- [ ] Tests written/updated
- [ ] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [ ] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [X] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec

---------

Co-authored-by: Daniel <daniel.cason@informal.systems>
Co-authored-by: Sergio Mena <sergio@informal.systems>
(cherry picked from commit 53e6127)

# Conflicts:
#	scripts/qa/reporting/prometheus_plotter.py
hvanz added a commit that referenced this issue Apr 12, 2024
…2667) (#2764)

Closes #2461, #2513

This report includes results for tests with and without latency
emulation.


[Rendered](https://github.com/cometbft/cometbft/blob/hvanz/qa-v1-report-2461/docs/references/qa/CometBFT-QA-v1.md)

---

#### PR checklist

- [ ] Tests written/updated
- [ ] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [ ] Updated relevant documentation (`docs/` or `spec/`) and code
comments
- [X] Title follows the [Conventional
Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec
<hr>This is an automatic backport of pull request #2667 done by
[Mergify](https://mergify.com).

---------

Co-authored-by: Hernán Vanzetto <15466498+hvanz@users.noreply.github.com>
Co-authored-by: hvanz <hernan.vanzetto@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
qa Quality assurance
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants