Add recreate benchmark script #8473

korthout · 2021-12-23T11:47:53Z

Description

One of the steps in the daily release process is to check whether there are new commits on the release branch. If there are any, the benchmark must be restarted. There did not exist any script for this yet, and the manual steps are not straight forward (e.g. newBenchmark will fail because the namespace is already in use).

To avoid these problems, a new recreateBenchmark.sh script is introduced that:

checks whether the namespace already exists and if not suggests solutions
rebuilds the project and the docker images
redeploys the benchmark

This uses a special check for the benchmark name to be a valid DNS label. See #8455 for more details

Related issues

NA

Definition of Done

Not all items need to be done depending on the issue and the pull request.

Code changes:

The changes are backwards compatibility with previous versions
If it fixes a bug then PRs are created to backport the fix to the last two minor versions. You can trigger a backport by assigning labels (e.g. backport stable/0.25) to the PR, in case that fails you need to create backports manually.

Testing:

There are unit/integration tests that verify all acceptance criterias of the issue
New tests are written to ensure backwards compatibility with further versions
The behavior is tested manually
The change has been verified by a QA run
The impact of the changes is verified by a benchmark

Documentation:

The documentation is updated (e.g. BPMN reference, configuration, examples, get-started guides, etc.)
New content is added to the release announcement

No need to spend time on running the checks. We also don't run the tests. That's what we have CI for.

deepthidevaki

LGTM. Some minor comments.

recreateBenchmark.sh

deepthidevaki · 2021-12-23T12:10:48Z

recreateBenchmark.sh

+docker build --build-arg DISTBALL=dist/target/camunda-cloud-zeebe-*.tar.gz --build-arg APP_ENV=dev -t "gcr.io/zeebe-io/zeebe:$benchmark" .
+docker push "gcr.io/zeebe-io/zeebe:$benchmark"
+
+cd "benchmarks/setup/$benchmark"


This folder may not exists, right? I don't think we commit release benchmark setup anywhere.

Good point. We should commit those somewhere, and this reminds me that it should also rebuild the benchmark project. So we should check that the folder exists

Rebuilding the benchmark project also doesn't require this folder specifically. So no need for the check.

I've update the script to rebuild the benchmark project.

This script allows you to easily recreate an existing benchmark that was created with createBenchmark.sh. It simply builds the docker images and updates the deployment. An additional check that the k8s namespace is available is added to this, as a quick check that you're using the correct namespace and are on the correct context. Or perhaps the user was actually intending to run ./createBenchmark.sh instead. Sadly, the make update does not restart the pods. We should consider solutions for that. However, as soon as the pods restart, the pods are updated because the pullPolicy is set to always. Therefore this applies a hard delete of all the broker and gateway pods to force the update. This is not a rolling update.

korthout · 2021-12-23T16:27:57Z

Thanks for the review @deepthidevaki. Can you have another look?

deepthidevaki

Thanks 🚀

deepthidevaki · 2021-12-24T08:14:32Z

recreateBenchmark.sh

+cd "$pwd/benchmarks/project"
+sed_inplace "s/:SNAPSHOT/:$benchmark/" docker-compose.yml
+# Use --no-cache to force rebuild the image for the benchmark application. Without this changes to zeebe-client were not picked up. This can take longer to build.
+docker-compose build --no-cache
+docker-compose push
+git restore -- docker-compose.yml
+


In most cases there won't be any need to re-build the starters and workers. But it make sense to do it always incase there are some changes. 👍

I agree. If it bothers us we could spend some time adding a flag for this, so it can be turned on and off at will. But I didn't want to spend too much time on it now. Perhaps another time.

korthout · 2021-12-24T09:08:44Z

bors merge

8473: Add recreate benchmark script r=korthout a=korthout ## Description  One of the steps in the daily release process is to check whether there are new commits on the release branch. If there are any, the benchmark must be restarted. There did not exist any script for this yet, and the manual steps are not straight forward (e.g. `newBenchmark` will fail because the namespace is already in use). To avoid these problems, a new `recreateBenchmark.sh` script is introduced that: - checks whether the namespace already exists and if not suggests solutions - rebuilds the project and the docker images - redeploys the benchmark This uses a special check for the benchmark name to be a valid DNS label. See #8455 for more details ## Related issues  NA Co-authored-by: Nico Korthout <nico.korthout@camunda.com>

ghost · 2021-12-24T09:37:18Z

Build failed:

continuous-integration/jenkins/branch

korthout · 2021-12-24T09:42:41Z

Flaky test ExporterDirectorDistributionTest.shouldDistributeExporterPositions. I'll open an issue.

bors merge

korthout · 2021-12-24T09:54:24Z

Opened #8475 for the flaky test

ghost · 2021-12-24T10:09:05Z

Build succeeded:

github-actions · 2021-12-24T10:10:06Z

Successfully created backport PR #8476 for stable/1.1.

github-actions · 2021-12-24T10:10:11Z

Successfully created backport PR #8477 for stable/1.2.

github-actions · 2021-12-24T10:10:16Z

Successfully created backport PR #8478 for release-1.3.0.

8478: [Backport release-1.3.0] Add recreate benchmark script r=korthout a=github-actions[bot] # Description Backport of #8473 to `release-1.3.0`. relates to #8455 Co-authored-by: Nico Korthout <nico.korthout@camunda.com>

8477: [Backport stable/1.2] Add recreate benchmark script r=korthout a=github-actions[bot] # Description Backport of #8473 to `stable/1.2`. relates to #8455 Co-authored-by: Nico Korthout <nico.korthout@camunda.com>

8476: [Backport stable/1.1] Add recreate benchmark script r=korthout a=github-actions[bot] # Description Backport of #8473 to `stable/1.1`. relates to #8455 Co-authored-by: Nico Korthout <nico.korthout@camunda.com>

8477: [Backport stable/1.2] Add recreate benchmark script r=korthout a=github-actions[bot] # Description Backport of #8473 to `stable/1.2`. relates to #8455 Co-authored-by: Nico Korthout <nico.korthout@camunda.com>

build: skip checks to create benchmark

f90dac7

No need to spend time on running the checks. We also don't run the tests. That's what we have CI for.

korthout added the backport release-1.3.0 label Dec 23, 2021

korthout force-pushed the korthout-recreate-benchmark branch from 734352a to 4a9076a Compare December 23, 2021 11:49

korthout added backport stable/1.1 labels Dec 23, 2021

deepthidevaki self-requested a review December 23, 2021 12:07

deepthidevaki requested changes Dec 23, 2021

View reviewed changes

korthout force-pushed the korthout-recreate-benchmark branch 3 times, most recently from d1340a5 to f633e3a Compare December 23, 2021 16:23

korthout force-pushed the korthout-recreate-benchmark branch from f633e3a to 3529bf2 Compare December 23, 2021 16:26

korthout requested a review from deepthidevaki December 23, 2021 16:26

deepthidevaki approved these changes Dec 24, 2021

View reviewed changes

ghost merged commit 1561ff3 into develop Dec 24, 2021

ghost deleted the korthout-recreate-benchmark branch December 24, 2021 10:09

github-actions bot mentioned this pull request Dec 24, 2021

[Backport stable/1.1] Add recreate benchmark script #8476

Merged

github-actions bot mentioned this pull request Dec 24, 2021

[Backport stable/1.2] Add recreate benchmark script #8477

Merged

github-actions bot mentioned this pull request Dec 24, 2021

[Backport release-1.3.0] Add recreate benchmark script #8478

Merged

npepinpe added Release: 1.1.10 labels Dec 29, 2021

korthout added the version:1.3.0 Marks an issue as being completely or in parts released in 1.3.0 label Jan 4, 2022

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add recreate benchmark script #8473

Add recreate benchmark script #8473

korthout commented Dec 23, 2021 •

edited

Loading

deepthidevaki left a comment

deepthidevaki Dec 23, 2021

korthout Dec 23, 2021

korthout Dec 23, 2021

korthout commented Dec 23, 2021

deepthidevaki left a comment

deepthidevaki Dec 24, 2021

korthout Dec 24, 2021

korthout commented Dec 24, 2021

ghost commented Dec 24, 2021

korthout commented Dec 24, 2021

korthout commented Dec 24, 2021

ghost commented Dec 24, 2021

github-actions bot commented Dec 24, 2021

github-actions bot commented Dec 24, 2021

github-actions bot commented Dec 24, 2021

Add recreate benchmark script #8473

Add recreate benchmark script #8473

Conversation

korthout commented Dec 23, 2021 • edited Loading

Description

Related issues

Definition of Done

deepthidevaki left a comment

Choose a reason for hiding this comment

deepthidevaki Dec 23, 2021

Choose a reason for hiding this comment

korthout Dec 23, 2021

Choose a reason for hiding this comment

korthout Dec 23, 2021

Choose a reason for hiding this comment

korthout commented Dec 23, 2021

deepthidevaki left a comment

Choose a reason for hiding this comment

deepthidevaki Dec 24, 2021

Choose a reason for hiding this comment

korthout Dec 24, 2021

Choose a reason for hiding this comment

korthout commented Dec 24, 2021

ghost commented Dec 24, 2021

korthout commented Dec 24, 2021

korthout commented Dec 24, 2021

ghost commented Dec 24, 2021

github-actions bot commented Dec 24, 2021

github-actions bot commented Dec 24, 2021

github-actions bot commented Dec 24, 2021

korthout commented Dec 23, 2021 •

edited

Loading