Jenkins Job to run OSDInteg in Parallel #3465
Conversation
Signed-off-by: Divya Madala <divyaasm@amazon.com>
Divyaasm
requested review from
dblock,
peterzhuamazon,
bbarani,
gaiksaya,
rishabh6788,
zelinh,
jordarlu,
prudhvigodithi and
tianleh
as code owners
Signed-off-by: Divya Madala <divyaasm@amazon.com>
Codecov Report
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more @@ Coverage Diff @@
## main #3465 +/- ##
==========================================
- Coverage 91.81% 91.44% -0.38%
==========================================
Files 181 181
Lines 5268 5305 +37
==========================================
+ Hits 4837 4851 +14
- Misses 431 454 +23 |
Signed-off-by: Divya Madala <divyaasm@amazon.com>
Signed-off-by: Divya Madala <divyaasm@amazon.com>
There was a problem hiding this comment.
Hi @Divyaasm , Good start to match OSD with OS behavior.
Add some comments.
Thanks.
| string( | ||
| name: 'TEST_MANIFEST', | ||
| description: 'Test manifest under the manifests folder, e.g. 2.0.0/opensearch-dashboards-2.0.0-test.yml.', | ||
| trim: true | ||
| ) | ||
| string( | ||
| name: 'BUILD_MANIFEST_URL', | ||
| description: 'The build manifest URL, e.g. https://ci.opensearch.org/ci/dbc/distribution-build-opensearch-dashboards/2.5.0/5367/linux/x64/tar/builds/opensearch-dashboards/manifest.yml', | ||
| description: 'The build manifest URL OpenSearch Dashboards, e.g. "https://ci.opensearch.org/ci/dbc/distribution-build-opensearch-dashboards/2.5.0/5367/linux/x64/tar/builds/opensearch-dashboards/manifest.yml".', |
There was a problem hiding this comment.
| description: 'The build manifest URL OpenSearch Dashboards, e.g. "https://ci.opensearch.org/ci/dbc/distribution-build-opensearch-dashboards/2.5.0/5367/linux/x64/tar/builds/opensearch-dashboards/manifest.yml".', | |
| description: 'The build manifest URL for OpenSearch Dashboards, e.g. https://ci.opensearch.org/ci/dbc/distribution-build-opensearch-dashboards/2.5.0/5367/linux/x64/tar/builds/opensearch-dashboards/manifest.yml', |
There was a problem hiding this comment.
Add the same comment change on OS as well. Thanks.
| } | ||
| } | ||
| } | ||
| stage('integ-test') { | ||
| // Required running on agent directly here to trigger docker stages in agent node, not trigger docker within docker container |
There was a problem hiding this comment.
Did not understand the wording here for "not trigger docker within docker container"
There was a problem hiding this comment.
That is copied from the OS part, where we have to make sure the run is happened directly on an agent not a docker container, so that the agent can trigger more docker container while it is within an agent, this cannot happen if you start in a docker container.
There was a problem hiding this comment.
Add the same comment change on OS as well. Thanks.
| echo "componentList: ${componentList}" | ||
|
|
||
| for (component_check in componentList) { | ||
| if (! componentDefaultList.contains(component_check)) { |
There was a problem hiding this comment.
| if (! componentDefaultList.contains(component_check)) { | |
| if (!componentDefaultList.contains(component_check)) { |
There was a problem hiding this comment.
Add the same comment change on OS as well. Thanks.
| docker.withRegistry('https://public.ecr.aws/') { | ||
| docker.image(docker_images["$distribution"]).inside(docker_args["$distribution"]) { | ||
| try { | ||
| stage("Run Integtest ${local_component}") { |
There was a problem hiding this comment.
nit: Everything here is integtest so we can avoid writing the same everywhere
| stage("Run Integtest ${local_component}") { | |
| stage("${local_component}") { |
There was a problem hiding this comment.
Add the same comment change on OS as well. Thanks.
| echo "Component Name: ${local_component}" | ||
| // Jenkins tend to not clean up workspace at times even though ws clean is called | ||
| // Due to docker is mounting the agent directory so it can communicated with the agent | ||
| // This sometimes causes the workspace to retain last run test-results and ends with build failures | ||
| // https://github.com/opensearch-project/opensearch-build/blob/6ed1ce3c583233eae4fe1027969d778cfc7660f7/src/test_workflow/test_recorder/test_recorder.py#L99 | ||
| sh("echo ${local_component} with index ${local_component_index} will sleep ${wait_seconds} seconds to reduce load && sleep ${wait_seconds}") |
There was a problem hiding this comment.
Back to back 2 echo statements are unnecessary. Remove one
There was a problem hiding this comment.
Add the same comment change on OS as well. Thanks.
Signed-off-by: Divya Madala <divyaasm@amazon.com>
| } | ||
| } | ||
| } | ||
| stage('integ-test') { | ||
| // Required running on agent directly here to trigger docker stages in agent node, not trigger docker within docker container |
There was a problem hiding this comment.
| // Required running on agent directly here to trigger docker stages in agent node, not trigger docker within docker container | |
| // Need to run this directly on agent node here in order to trigger stages with docker container and avoid docker within docker situation |
There was a problem hiding this comment.
Add the same comment change on OS as well. Thanks.
There was a problem hiding this comment.
Let's not touch the OS files in this PR. These are just comments nits that can come in new PR. Keep the change to minimum in case we need to revert
Signed-off-by: Divya Madala <divyaasm@amazon.com>
Description
The existing OSDIntegTest Jenkins Job is updated to run in parallel.
Issues Resolved
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.