Add soak-test CI job for insights-runtime-extractor

[IshwarKanse]

Summary

• Adds an periodic soak-test CI job to the insights-runtime-extractor main branch config
• The job provisions an AWS cluster via ipi-aws, clones the soak-test scripts from GitLab, deploys the extractor DaemonSet + Spring Boot workload, and runs 121 extraction iterations over ~60 minutes to detect memory leaks
• Configured with always_run: false and optional: true so it doesn't block PRs — must be triggered manually
• 90-minute timeout to accommodate the ~60-minute soak loop plus deployment overhead

Test plan

• CI config validation passes (automatic on PR)
• Manually trigger the soak-test job on a test PR to verify end-to-end execution
• Verify artifacts (memory snapshots, soak-test logs) are collected in $ARTIFACT_DIR

🤖 Generated with Claude Code

Summary by CodeRabbit

• Updates the OpenShift CI configuration for the insights-runtime-extractor main branch by adding a scheduled soak-test job under the tests list.
• The job provisions on AWS-based OpenShift (cluster_profile: openshift-org-aws) using the ipi-aws workflow, and is scheduled via cron 30 4 1-7,15-21 * 1 (Mondays at 04:30 UTC / 10:00 AM IST).
• On execution, it clones https://gitlab.cee.redhat.com/ikanse/soak-test.git into /tmp/soak-test (with GIT_SSL_NO_VERIFY=true) and runs ./ci/run-soak-test.sh.
• The job declares dependencies on INSIGHTS_RUNTIME_EXPORTER and INSIGHTS_RUNTIME_EXTRACTOR, runs from from: src, requests cpu: 100m, and sets the test timeout to 1h30m0s.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

Walkthrough

Adds a scheduled soak-test CI job for the insights runtime extractor using AWS IPI, runtime component dependencies, an external soak-test script, CPU settings, and a 1h30m timeout.

Changes

Soak-test CI integration

Layer / File(s)	Summary
Configure the soak-test job ci-operator/config/openshift/insights-runtime-extractor/openshift-insights-runtime-extractor-main.yaml	Adds a scheduled AWS soak-test job that clones and runs the external test repository, uses runtime exporter and extractor dependencies, and defines resource and timeout settings.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: jmesnil, smg247

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Ipv6 And Disconnected Network Test Compatibility	❓ Inconclusive	Need repo inspection to verify whether the new job depends on external network or IPv4 assumptions.	Inspect the changed CI config and referenced soak-test script to confirm any external connectivity or IPv4-only logic.

✅ Passed checks (14 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	Only ci-operator YAML/job definitions changed; no Ginkgo titles were added, and the new job names are static (e.g. soak-test).
Test Structure And Quality	✅ Passed	PR only changes CI job YAML/metadata; no Ginkgo test code or It/BeforeEach/Eventually blocks were added or modified.
Microshift Test Compatibility	✅ Passed	Only CI job YAML changed; no new Ginkgo test code was added, so MicroShift compatibility check is not applicable.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	Only CI/job YAML changed; no new in-repo Ginkgo e2e tests were added, so the SNO compatibility check is not applicable.
Topology-Aware Scheduling Compatibility	✅ Passed	PR only adds a CI soak-test job in YAML; no deployment manifest, controller, or topology-based scheduling logic was introduced.
Ote Binary Stdout Contract	✅ Passed	Only CI YAML changed; no OTE binary or suite setup code was modified, so the stdout contract check is not applicable.
No-Weak-Crypto	✅ Passed	PR only adds CI YAML for a soak-test job; scans found no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB or custom crypto/comparison logic.
Container-Privileges	✅ Passed	No changed manifest sets privileged, hostPID/hostNetwork/hostIPC, SYS_ADMIN, allowPrivilegeEscalation, or explicit root settings.
No-Sensitive-Data-In-Logs	✅ Passed	The new soak-test step only clones a repo and runs a script; it adds no logging of passwords, tokens, PII, or other sensitive data.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: adding a soak-test CI job for insights-runtime-extractor.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/openshift/insights-runtime-extractor/openshift-insights-runtime-extractor-main.yaml`:
- Around line 109-112: Update the soak-test clone command in the CI execution
block to remove GIT_SSL_NO_VERIFY, configure Git to trust the approved GitLab
CA, and check out a reviewed immutable commit rather than the mutable default
branch before executing the cloned scripts. Keep the existing script permission
and run steps unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: dd435cda-cf79-46f0-a7fb-150936dd695a

📥 Commits

Reviewing files that changed from the base of the PR and between 8a0fc1b and 5c8bb05.

📒 Files selected for processing (1)

• ci-operator/config/openshift/insights-runtime-extractor/openshift-insights-runtime-extractor-main.yaml

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: IshwarKanse
Once this PR has been reviewed and has the lgtm label, please assign jmesnil for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• ci-operator/config/openshift/insights-runtime-extractor/OWNERS
• ci-operator/jobs/openshift/insights-runtime-extractor/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[IshwarKanse]

/pj-rehearse pull-ci-openshift-insights-runtime-extractor-main-soak-test

[openshift-merge-bot]

@IshwarKanse: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[coderabbitai]

Actionable comments posted: 1

♻️ Duplicate comments (1)

ci-operator/config/openshift/insights-runtime-extractor/openshift-insights-runtime-extractor-main.yaml (1)

107-110: ⚠️ Potential issue | 🔴 Critical

The TLS-verification bypass remains present.

GIT_SSL_NO_VERIFY=true permits MITM/certificate-substitution attacks, and the command executes code cloned from the mutable default branch. This is the same unresolved security issue reported in the previous review; trust the approved GitLab CA and pin the clone to a reviewed commit.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/openshift/insights-runtime-extractor/openshift-insights-runtime-extractor-main.yaml`
around lines 107 - 110, Update the commands block for the soak-test clone to
remove GIT_SSL_NO_VERIFY, configure Git to trust the approved GitLab CA, and pin
the clone to the reviewed commit rather than the mutable default branch before
running run-soak-test.sh.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/openshift/insights-runtime-extractor/openshift-insights-runtime-extractor-main.yaml`:
- Around line 100-101: Update the soak-test scheduling configuration to prevent
the cron expression from triggering on every Monday and on the specified
day-of-month ranges; implement a runtime guard or use a scheduling mechanism
that reliably runs only on biweekly Mondays.

---

Duplicate comments:
In
`@ci-operator/config/openshift/insights-runtime-extractor/openshift-insights-runtime-extractor-main.yaml`:
- Around line 107-110: Update the commands block for the soak-test clone to
remove GIT_SSL_NO_VERIFY, configure Git to trust the approved GitLab CA, and pin
the clone to the reviewed commit rather than the mutable default branch before
running run-soak-test.sh.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 51ad10e2-8538-410c-bbe9-c3bb35454e46

📥 Commits

Reviewing files that changed from the base of the PR and between 881aa9a and ffb61e8.

⛔ Files ignored due to path filters (1)

• ci-operator/jobs/openshift/insights-runtime-extractor/openshift-insights-runtime-extractor-main-periodics.yaml is excluded by !ci-operator/jobs/**

📒 Files selected for processing (1)

• ci-operator/config/openshift/insights-runtime-extractor/openshift-insights-runtime-extractor-main.yaml

[IshwarKanse]

/pj-rehearse periodic-ci-openshift-insights-runtime-extractor-main-soak-test

[openshift-merge-bot]

@IshwarKanse: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[IshwarKanse]

/pj-rehearse

[openshift-merge-bot]

@IshwarKanse: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[IshwarKanse]

/pj-rehearse list

[openshift-merge-bot]

@IshwarKanse: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[IshwarKanse]

/pj-rehearse list

[openshift-merge-bot]

@IshwarKanse: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[IshwarKanse]

/pj-rehearse ack

[openshift-merge-bot]

@IshwarKanse: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[IshwarKanse]

/pj-rehearse list

[openshift-merge-bot]

@IshwarKanse: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[opokornyy]

/cc

[opokornyy]

Hey @IshwarKanse, do you think the soak test is worth running periodically? We already have some of the runtime-extractor tests defined here, and they're run periodically to catch regressions in the component. The results are then displayed on the Sippy dashboard.

[IshwarKanse]

@opokornyy The existing periodic tests validate functional correctness. The soak test specifically targets memory leak detection over extended runtime (~60 min, 121 iterations), which the existing short-lived tests don't cover.

That said, we can make it on-demand instead of periodic — use a cron entry that never triggers and run the job on demand via the Gangway API whenever we want to check for memory regressions.

This is an initial PR to get the job definition in place. Since I'm not in the OWNERS file for this config, I'm unable to trigger a rehearsal to validate the job. Once this PR is merged, I'll create a follow-up PR to:

1. Run the rehearsal job and verify the tests pass in OCP CI
2. Point to the upstream soak-test repo instead of my fork
3. Make any other adjustments based on the rehearsal results

[openshift-merge-bot]

[REHEARSALNOTIFIER]
@IshwarKanse: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
periodic-ci-openshift-insights-runtime-extractor-main-soak-test	N/A	periodic	Periodic changed

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.