[Test] Add a feature to start workspaces in separate namespaces #22901

SkorikSergey · 2024-03-28T15:36:34Z

What does this PR do?

This PR:

adds a feature to start workspaces in separate namespaces (one workspace per namespace), by --s argument (e.g., ./load-test.sh --s)
all samples yaml files will be stored in samples folder
README.md file updated according to changes
also it is possible to get help info with this scripts option by ./load-test.sh --help)

Usage: ./load-test.sh [OPTIONS]
Options:
  -t <SECONDS>      Set the timeout in seconds (default: 120)
  -c <COUNT>         Set the number of workspaces to start (default: 3)
  -l                             Set the link to the devworkspace.yaml file
  -s                            Start workspaces in separate namespaces(one workspace per namespace)
  --help                     Display this help message

What issues does this PR fix or reference?

https://issues.redhat.com/browse/CRW-5633

How to test this PR?

Log in to Openshift cluster with DevSpaces deployed from terminal
Start load-test.sh script from test/e2e/performance/load-tests. Set number of started workspaces by -c parameter(like./load-test.sh -c 5)
To use devworkspace yaml file from lint use -l option(like./load-test.sh -l https://gist.githubusercontent.com/SkorikSergey/2eb152db36ed27a0736e0b367ee7f435/raw/dffdb84aaa8336977d9a1f25afe7749bb5c06e9a/simple-devworskpace.yaml)
As results there are average time of workspace starting and number of failed workspaces.

Logs

[sskoryk@fedora load-tests]$ ./load-test.sh --one-workspace-per-namespace
Parameter -c wasn't set, setting completitions count to 3.
Parameter -c was set to  3 .
Parameter -t wasn't set, setting timeout to 120 second.
Local devworkspace.yaml file will be used.
Clean up the environment
Delete test namespaces
Create test namespaces
namespace/load-test-namespace-1 created
namespace/load-test-namespace-2 created
namespace/load-test-namespace-3 created
devworkspace.workspace.devfile.io/load-test-dw-1 created
devworkspace.workspace.devfile.io/load-test-dw-3 created
devworkspace.workspace.devfile.io/load-test-dw-2 created
Wait for all workspaces are started
devworkspace.workspace.devfile.io/load-test-dw-2 condition met
devworkspace.workspace.devfile.io/load-test-dw-1 condition met
devworkspace.workspace.devfile.io/load-test-dw-3 condition met
mkdir: cannot create directory ‘logs’: File exists
Calculate average workspaces starting time
Devworkspace load-test-dw-1 in load-test-namespace-1 namespace starting time: 4 seconds
Devworkspace load-test-dw-2 in load-test-namespace-2 namespace starting time: 4 seconds
Devworkspace load-test-dw-3 in load-test-namespace-3 namespace starting time: 4 seconds
==================== Test results ====================
Average workspace starting time for 3 workspaces from 3 started: 4 seconds
0 workspaces failed. See failed workspace pod logs in the current folder for details.
Elapsed time: 12 seconds

PR Checklist

As the author of this Pull Request I made sure that:

Reviewers

Reviewers, please comment how you tested the PR when approving it.

tests/performance/load-tests/load-test.sh

ibuziuk · 2024-03-29T14:03:20Z

tests/performance/load-tests/samples/simple-with-pvc.yaml

@@ -0,0 +1,13 @@
+kind: DevWorkspace


the name is misleading should probably be simple-with-editor or the better name would be ephemeral-with-editor

I have added new samples. Review it, please.

ibuziuk · 2024-03-29T14:08:53Z

tests/performance/load-tests/load-test.sh

@@ -63,6 +77,17 @@ function parseArguments() {
  done
 }

+function cleanup() {
+  echo "Clean up the environment"
+  kubectl delete dw --all >/dev/null 2>&1


isn't it too risky, shoud we explicitly specify namespace that should be used

In this case it will be only current namespace but it will be better specify namespace. Thanks.

Done #22901 (comment)

ibuziuk · 2024-03-29T14:19:45Z

@SkorikSergey basically, I have concerns about implementation of the dw, dwt, namespace removal. Can we be sure that only those that were created by the script would be removed?

can we add some specific label / annotation to the namespace and implement more intricate checks

SkorikSergey · 2024-03-29T15:49:57Z

@SkorikSergey basically, I have concerns about implementation of the dw, dwt, namespace removal. Can we be sure that only those that were created by the script would be removed?

can we add some specific label / annotation to the namespace and implement more intricate checks

@ibuziuk I have updated cleanup func. Now only created test namespaces and dw will be deleted.

# Default value for test namespace name
test_namespace_name=load-test-namespace-
# Default value for test devworkspace name
dw_name=load-test-dw-

  if [ $start_separately = true ]; then
    echo "Delete test namespaces"
    kubectl delete namespace $(kubectl get namespace | grep $test_namespace_name | awk '{print $1}') >/dev/null 2>&1 || true
  else
    kubectl delete dw $(kubectl get dw | grep $dw_name | awk '{print $1}') -n $current_namespace >/dev/null 2>&1 || true
  fi

ibuziuk · 2024-03-29T17:35:42Z

@SkorikSergey let's make the cleanup process more robust and label all the k8s resources created during the load testing. After that we can use selector for cleanup e.g. oc delete all --selector

ibuziuk · 2024-04-02T10:01:36Z

tests/performance/load-tests/README.md

@@ -1,19 +1,20 @@
 # Overview
-This script tests how well OpenShift environment can handle running simultaneously many of workspaces. It evaluates the performance of the system under test by checking the average results across all pods and identifying failures that occur during the testing process. 
+This script tests the performance of an OpenShift environment by running multiple workspaces simultaneously. It evaluates the system under test by checking the average results across all pods and identifying any failures that occur during the testing process.


is it OpenShift specific, or would it work on any k8s?

I have checked it only on Openshift.

ack, it probably not going to work on vanilla k8s, but it is ok for the time-being

ibuziuk · 2024-04-02T10:03:56Z

tests/performance/load-tests/load-test.sh

+
+  if [ $start_separately = true ]; then
+    echo "Delete test namespaces"
+    kubectl delete namespace --selector=test-type=load-tests >/dev/null 2>&1 || true


can we add selector definition to the top of the script ?

ibuziuk

@SkorikSergey good job! please, add the badge for starting load test workspace on workspaces.openshift.com (can be done in a separate PR)

SkorikSergey added 4 commits March 28, 2024 15:45

start ws in separated namespaces

63c72d4

add samples folder

46fc0d9

update example yaml file url

17ef23a

delete tmp yaml file

37bf7e0

SkorikSergey requested review from dmytro-ndp and ibuziuk March 28, 2024 15:36

SkorikSergey self-assigned this Mar 28, 2024

SkorikSergey added status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community. and removed status/in-progress This issue has been taken by an engineer and is under active development. labels Mar 28, 2024

SkorikSergey marked this pull request as ready for review March 28, 2024 15:51

SkorikSergey requested a review from SDawley as a code owner March 28, 2024 15:51

SkorikSergey changed the title ~~[Test] Add a feature to starts workspaces in separate namespace~~ [Test] Add a feature to start workspaces in separate namespace Mar 28, 2024

dmytro-ndp reviewed Mar 28, 2024

View reviewed changes

tests/performance/load-tests/load-test.sh Outdated Show resolved Hide resolved

dmytro-ndp approved these changes Mar 28, 2024

View reviewed changes

dmytro-ndp reviewed Mar 29, 2024

View reviewed changes

tests/performance/load-tests/load-test.sh Outdated Show resolved Hide resolved

update README.md file

291dfc7

ibuziuk reviewed Mar 29, 2024

View reviewed changes

SkorikSergey added 3 commits March 29, 2024 16:56

add new samples

5360416

check that only created dw objects will be deleted

261bc1a

fix 'no workspaces started' step

3468f3f

SkorikSergey added 2 commits April 1, 2024 12:45

label test dw objects

a795809

update sample link in README file

3fae005

ibuziuk reviewed Apr 2, 2024

View reviewed changes

ibuziuk approved these changes Apr 2, 2024

View reviewed changes

add selector definition to the top of the script

7e70e80

SkorikSergey merged commit bc658b5 into main Apr 2, 2024
4 checks passed

SkorikSergey deleted the load-tests branch April 2, 2024 11:18

SkorikSergey changed the title ~~[Test] Add a feature to start workspaces in separate namespace~~ [Test] Add a feature to start workspaces in separate namespaces Apr 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Test] Add a feature to start workspaces in separate namespaces #22901

[Test] Add a feature to start workspaces in separate namespaces #22901

SkorikSergey commented Mar 28, 2024 •

edited

Loading

ibuziuk Mar 29, 2024 •

edited

Loading

SkorikSergey Mar 29, 2024

ibuziuk Mar 29, 2024

SkorikSergey Mar 29, 2024

SkorikSergey Mar 29, 2024

ibuziuk commented Mar 29, 2024

SkorikSergey commented Mar 29, 2024 •

edited

Loading

ibuziuk commented Mar 29, 2024

ibuziuk Apr 2, 2024

SkorikSergey Apr 2, 2024

ibuziuk Apr 2, 2024

ibuziuk Apr 2, 2024

SkorikSergey Apr 2, 2024

ibuziuk left a comment

[Test] Add a feature to start workspaces in separate namespaces #22901

[Test] Add a feature to start workspaces in separate namespaces #22901

Conversation

SkorikSergey commented Mar 28, 2024 • edited Loading

What does this PR do?

What issues does this PR fix or reference?

How to test this PR?

Logs

PR Checklist

Reviewers

ibuziuk Mar 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ibuziuk commented Mar 29, 2024

SkorikSergey commented Mar 29, 2024 • edited Loading

ibuziuk commented Mar 29, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ibuziuk left a comment

Choose a reason for hiding this comment

SkorikSergey commented Mar 28, 2024 •

edited

Loading

ibuziuk Mar 29, 2024 •

edited

Loading

SkorikSergey commented Mar 29, 2024 •

edited

Loading