Skip to content

CLDSRV-914: abort in-progress MPUs in mpuVersion test cleanup#6184

Merged
bert-e merged 2 commits into
development/9.3from
bugfix/CLDSRV-914-mpu-shadow-bucket-cleanup
Jun 4, 2026
Merged

CLDSRV-914: abort in-progress MPUs in mpuVersion test cleanup#6184
bert-e merged 2 commits into
development/9.3from
bugfix/CLDSRV-914-mpu-shadow-bucket-cleanup

Conversation

@tcarmet
Copy link
Copy Markdown
Contributor

@tcarmet tcarmet commented Jun 2, 2026

The mpuVersion functional test initiates multipart uploads that intentionally fail to complete (its error-handling cases), leaving in-progress uploads behind. The existing cleanup only removed object versions and delete markers, so the per-bucket MPU shadow bucket survived bucket deletion and lingered in shared metadata; on a consensus (RAFT) backend this leftover polluted the global bucket list and deterministically broke the metadata-routes "should retrieve list of buckets" test. This aborts any in-progress uploads for each test bucket before emptying and deleting them, so no shadow bucket is orphaned. The issue is pre-existing and not a 9.3 regression.

@claude
Copy link
Copy Markdown

claude Bot commented Jun 2, 2026

LGTM

Review by Claude Code

The error-handling cases initiate multipart uploads that intentionally
fail to complete, leaving an in-progress upload behind. bucketUtil.empty
only removes object versions and delete markers, so the per-bucket
mpuShadowBucket survived bucket deletion and lingered in shared metadata.
On a consensus (RAFT) backend this leftover polluted the global bucket
list and broke the routeMetadata "should retrieve list of buckets" test.

Abort any in-progress uploads for each test bucket in afterEach before
emptying and deleting them.
@tcarmet tcarmet force-pushed the bugfix/CLDSRV-914-mpu-shadow-bucket-cleanup branch from d0dc921 to 8a23ffe Compare June 2, 2026 23:28
@tcarmet tcarmet changed the base branch from bugfix/CLDSRV-911-incomplete-sdk-migration to development/9.3 June 2, 2026 23:28
@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Jun 2, 2026

Hello tcarmet,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request TBA
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Jun 2, 2026

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

@tcarmet tcarmet requested a review from a team June 2, 2026 23:29
@claude
Copy link
Copy Markdown

claude Bot commented Jun 2, 2026

LGTM

Review by Claude Code

@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 2, 2026

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
8478 2 8476 0
View the top 1 failed test(s) by shortest run time
"after each" hook for "should fail if trying to overwrite a delete marker"::MPU with x-scal-s3-version-id header With v4 signature "after each" hook for "should fail if trying to overwrite a delete marker"
Stack Traces | 0.013s run time
We encountered an internal error. Please try again.
View the full list of 1 ❄️ flaky test(s)
"after each" hook for "should fail if trying to overwrite a delete marker"::MPU with x-scal-s3-version-id header With default signature "after each" hook for "should fail if trying to overwrite a delete marker"

Flake rate in main: 100.00% (Passed 0 times, Failed 9 times)

Stack Traces | 0.014s run time
We encountered an internal error. Please try again.

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@tcarmet tcarmet requested a review from a team June 2, 2026 23:57
@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Jun 3, 2026

Incorrect fix version

The Fix Version/s in issue CLDSRV-914 contains:

  • 9.3.8

  • 9.4.0

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

  • 9.3.9

  • 9.4.0

Please check the Fix Version/s of CLDSRV-914, or the target
branch of this pull request.

checkError(err, 'MethodNotAllowed', 405);
});

putMPUVersion(s3, bucketName, objectName, vId)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

putMPUVersion is not awaited here — the test will always pass regardless of the promise outcome because the try block completes before the promise settles. The checkError assertion in .catch() is never verified during the test run. Pre-existing, but since this code was reformatted, consider adding await.

— Claude Code

@claude
Copy link
Copy Markdown

claude Bot commented Jun 3, 2026

  • unawaited promise at line 321: putMPUVersion call in the "should fail if version is a delete marker" test is not awaited, so the assertion in .catch() is never verified during the test run (pre-existing, but touched by this PR's formatting pass)

    The abortAllMPUs helper and its placement in afterEach before emptyMany/deleteMany look correct.

    Review by Claude Code

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Jun 3, 2026

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

@tcarmet
Copy link
Copy Markdown
Contributor Author

tcarmet commented Jun 3, 2026

/approve

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Jun 3, 2026

Integration data created

I have created the integration data for the additional destination branches.

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.8
  • development/9.0
  • development/9.1
  • development/9.2

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: approve

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Jun 3, 2026

Build failed

The build for commit did not succeed in branch w/9.4/bugfix/CLDSRV-914-mpu-shadow-bucket-cleanup

The following options are set: approve

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Jun 4, 2026

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

  • ✔️ development/9.3

  • ✔️ development/9.4

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.8
  • development/9.0
  • development/9.1
  • development/9.2

This pull request does not target the following hotfix branch(es) so they
will be left untouched:

  • hotfix/6.4.7
  • hotfix/7.10.49
  • hotfix/7.10.8
  • hotfix/7.70.11
  • hotfix/7.8.0
  • hotfix/7.4.8
  • hotfix/9.0.32
  • hotfix/7.10.27
  • hotfix/7.10.0
  • hotfix/7.10.15
  • hotfix/7.4.6
  • hotfix/7.4.5
  • hotfix/7.70.51
  • hotfix/9.0.7
  • hotfix/7.4.0
  • hotfix/9.2.24
  • hotfix/7.4.1
  • hotfix/7.70.73
  • hotfix/7.4.10
  • hotfix/7.2.0
  • hotfix/7.6.0
  • hotfix/7.4.4
  • hotfix/7.10.2
  • hotfix/7.4.2
  • hotfix/7.70.45
  • hotfix/8.8.45
  • hotfix/7.4.7
  • hotfix/7.10.1
  • hotfix/7.70.21
  • hotfix/7.4.9
  • hotfix/7.4.3
  • hotfix/7.7.0
  • hotfix/7.10.28
  • hotfix/7.10.3
  • hotfix/7.10.4
  • hotfix/7.9.0
  • hotfix/7.10.30

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

  • Any commit you add on the source branch will trigger a new cycle after the
    current queue is merged.
  • Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Jun 4, 2026

Queue build failed

The corresponding build for the queue failed:

  • Checkout the status page.
  • Identify the failing build and review the logs.
  • If no issue is found, re-run the build.
  • If an issue is identified, checkout the steps below to remove
    the pull request from the queue for further analysis and maybe rebase/merge.
Remove the pull request from the queue
  • Add a /wait comment on this pull request.
  • Click on login on the status page.
  • Go into the manage page.
  • Find the option called Rebuild the queue and click on it.
    Bert-E will loop again on all pull requests to put the valid ones
    in the queue again, while skipping the one with the /wait comment.
  • Wait for the new queue to merge, then merge/rebase your pull request
    with the latest changes to then work on a proper fix.
  • Once the issue is fixed, delete the /wait comment and
    follow the usual process to merge the pull request.

@bert-e
Copy link
Copy Markdown
Contributor

bert-e commented Jun 4, 2026

I have successfully merged the changeset of this pull request
into targetted development branches:

  • ✔️ development/9.3

  • ✔️ development/9.4

The following branches have NOT changed:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.8
  • development/9.0
  • development/9.1
  • development/9.2

Please check the status of the associated issue CLDSRV-914.

Goodbye tcarmet.

@bert-e bert-e merged commit 6214c48 into development/9.3 Jun 4, 2026
82 of 84 checks passed
@bert-e bert-e deleted the bugfix/CLDSRV-914-mpu-shadow-bucket-cleanup branch June 4, 2026 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants