Standalone Jenkinsfile

– ensure test file paths, and their suite names are unique (the latter was broken for cqlshlib and python dtests) – removal of properties and system-out in test xml happens in CassandraXMLJUnitResultFormatter – new jenkins agent labels and introduce agents sizes - ci_summary.html generation script, ref work submitted in apache/cassandra-builds #99 – fix for stress-test and fqltool-test running on small - ant generate-test-report is limited to only running on individual test types (ci_parser.py provides ci_summary.html for the overview now) - each cell has a single retry, and the retry will happen on a different agent - on ci-cassandra the summary stage happens on the builtin, bc copyArtifacts on 15k+ files takes many hours otherwise - test-burn only needs two splits - dependency-check is disabled from the lint target until CASSANDRA-19213 - add $DEBUG env var to in-tree scripts, turns on bash debug - fix FBUtilities' handling of gcp cos_containerd (kernel version comes with a trailing '+' character) patch by Aleks Volochnev, Mick Semb Wever; reviewed by Aleksandr Volochnev, Josh McKenzie, Maxim Muzafarov, Stefan Miklosovic for CASSANDRA-18594 Co-authored-by: Aleksandr Volochnev <a.volochnev@gmail.com> Co-authored-by: Mick Semb Wever <mck@apache.org> Co-authored-by: Josh McKenzie <jmckenzie@apache.org> Co-authored-by: Artem Chekunov <artem.v.chekunov@gmail.com>
apache · Apr 4, 2024 · f7c11bd · f7c11bd
1 parent b0150e8
commit f7c11bd
Show file tree

Hide file tree

Showing 23 changed files with 1,791 additions and 886 deletions.
diff --git a/.build/README.md b/.build/README.md
@@ -29,66 +29,6 @@ The following applies to all build scripts.
 
     build_dir=/tmp/cass_Mtu462n .build/docker/check-code.sh
 
-Running Sonar analysis (experimental)
--------------------------------------
-
-Run:
-
-    ant sonar
-
-Sonar analysis requires the SonarQube server to be available. If there
-is already some SonarQube server, it can be used by setting the
-following env variables:
-
-    SONAR_HOST_URL=http://sonar.example.com
-    SONAR_CASSANDRA_TOKEN=cassandra-project-analysis-token
-    SONAR_PROJECT_KEY=<key of the Cassandra project in SonarQube>
-
-If SonarQube server is not available, one can be started locally in
-a Docker container. The following command will create a SonarQube
-container and start the server:
-
-    ant sonar-create-server
-
-The server will be available at http://localhost:9000 with admin
-credentials admin/password. The Docker container named `sonarqube`
-is created and left running. When using this local SonarQube server,
-no env variables to configure url, token, or project key are needed,
-and the analysis can be run right away with `ant sonar`.
-
-After the analysis, the server remains running so that one can 
-inspect the results.
-
-To stop the local SonarQube server:
-
-    ant sonar-stop-server
-
-However, this command just stops the Docker container without removing
-it. It allows to start the container later with:
-
-    docker container start sonarqube
-
-and access previous analysis results. To drop the container, run:
-
-    docker container rm sonarqube
-
-When `SONAR_HOST_URL` is not provided, the script assumes a dedicated
-local instance of the SonarQube server and sets it up automatically,
-which includes creating a project, setting up the quality profile, and
-quality gate from the configuration stored in
-[sonar-quality-profile.xml](sonar%2Fsonar-quality-profile.xml) and
-[sonar-quality-gate.json](sonar%2Fsonar-quality-gate.json)
-respectively. To run the analysis with a custom quality profile, start
-the server using `ant sonar-create-server`, create a project manually,
-and set up a desired quality profile for it. Then, create the analysis
-token for the project and export the following env variables:
-
-    SONAR_HOST_URL="http://127.0.0.1:9000"
-    SONAR_CASSANDRA_TOKEN="<token>"
-    SONAR_PROJECT_KEY="<key of the Cassandra project in SonarQube>"
-
-The analysis can be run with `ant sonar`.
-
 
 Building Artifacts (tarball and maven)
 -------------------------------------
@@ -177,7 +117,7 @@ Running other types of tests with docker:
     .build/docker/run-tests.sh jvm-dtest-upgrade
     .build/docker/run-tests.sh dtest
     .build/docker/run-tests.sh dtest-novnode
-    .build/docker/run-tests.sh dtest-offheap
+    .build/docker/run-tests.sh dtest-latest
     .build/docker/run-tests.sh dtest-large
     .build/docker/run-tests.sh dtest-large-novnode
     .build/docker/run-tests.sh dtest-upgrade
@@ -198,3 +138,63 @@ Other python dtest types without docker:
 
     .build/run-python-dtests.sh dtest-upgrade-large
 
+
+Running Sonar analysis (experimental)
+-------------------------------------
+
+Run:
+
+    ant sonar
+
+Sonar analysis requires the SonarQube server to be available. If there
+is already some SonarQube server, it can be used by setting the
+following env variables:
+
+    SONAR_HOST_URL=http://sonar.example.com
+    SONAR_CASSANDRA_TOKEN=cassandra-project-analysis-token
+    SONAR_PROJECT_KEY=<key of the Cassandra project in SonarQube>
+
+If SonarQube server is not available, one can be started locally in
+a Docker container. The following command will create a SonarQube
+container and start the server:
+
+    ant sonar-create-server
+
+The server will be available at http://localhost:9000 with admin
+credentials admin/password. The Docker container named `sonarqube`
+is created and left running. When using this local SonarQube server,
+no env variables to configure url, token, or project key are needed,
+and the analysis can be run right away with `ant sonar`.
+
+After the analysis, the server remains running so that one can
+inspect the results.
+
+To stop the local SonarQube server:
+
+    ant sonar-stop-server
+
+However, this command just stops the Docker container without removing
+it. It allows to start the container later with:
+
+    docker container start sonarqube
+
+and access previous analysis results. To drop the container, run:
+
+    docker container rm sonarqube
+
+When `SONAR_HOST_URL` is not provided, the script assumes a dedicated
+local instance of the SonarQube server and sets it up automatically,
+which includes creating a project, setting up the quality profile, and
+quality gate from the configuration stored in
+[sonar-quality-profile.xml](sonar%2Fsonar-quality-profile.xml) and
+[sonar-quality-gate.json](sonar%2Fsonar-quality-gate.json)
+respectively. To run the analysis with a custom quality profile, start
+the server using `ant sonar-create-server`, create a project manually,
+and set up a desired quality profile for it. Then, create the analysis
+token for the project and export the following env variables:
+
+    SONAR_HOST_URL="http://127.0.0.1:9000"
+    SONAR_CASSANDRA_TOKEN="<token>"
+    SONAR_PROJECT_KEY="<key of the Cassandra project in SonarQube>"
+
+The analysis can be run with `ant sonar`.
diff --git a/.build/check-code.sh b/.build/check-code.sh
@@ -24,5 +24,5 @@ command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit 1
 [ -f "${CASSANDRA_DIR}/build.xml" ] || { echo >&2 "${CASSANDRA_DIR}/build.xml must exist"; exit 1; }
 
 # execute
-ant -f "${CASSANDRA_DIR}/build.xml" check dependency-check
+ant -f "${CASSANDRA_DIR}/build.xml" check # dependency-check # FIXME dependency-check now requires NVD key downloaded first
 exit $?