CLDSRV-902: calculate final checksum from parts by leif-scality · Pull Request #6169 · scality/cloudserver

leif-scality · 2026-05-15T20:44:53Z

calculate final object checksum from parts

claude · 2026-05-15T20:46:57Z

LGTM

Review by Claude Code

codecov · 2026-05-15T20:48:12Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.29%. Comparing base (3f7ce59) to head (538c16c).
⚠️ Report is 2 commits behind head on development/9.4.
✅ All tests successful. No failed tests found.

Additional details and impacted files

Files with missing lines	Coverage Δ
lib/api/apiUtils/integrity/crcCombine.js	`100.00% <100.00%> (ø)`
lib/api/apiUtils/integrity/validateChecksums.js	`99.04% <100.00%> (+0.05%)`	⬆️

... and 1 file with indirect coverage changes

@@                 Coverage Diff                 @@
##           development/9.4    #6169      +/-   ##
===================================================
+ Coverage            85.21%   85.29%   +0.08%     
===================================================
  Files                  207      208       +1     
  Lines                13832    13917      +85     
===================================================
+ Hits                 11787    11871      +84     
- Misses                2045     2046       +1

Flag	Coverage Δ
file-ft-tests	`68.81% <5.88%> (-0.44%)`	⬇️
kmip-ft-tests	`28.13% <5.88%> (-0.14%)`	⬇️
mongo-v0-ft-tests	`69.95% <5.88%> (-0.46%)`	⬇️
mongo-v1-ft-tests	`70.02% <5.88%> (-0.46%)`	⬇️
multiple-backend	`36.60% <5.88%> (-0.19%)`	⬇️
sur-tests	`36.37% <5.88%> (+0.73%)`	⬆️
sur-tests-inflights	`37.37% <5.88%> (-0.20%)`	⬇️
unit	`72.01% <100.00%> (+0.17%)`	⬆️
utapi-v2-tests	`34.77% <5.88%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

bert-e · 2026-05-19T09:30:15Z

Hello leif-scality,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2026-05-19T09:30:20Z

Incorrect fix version

The Fix Version/s in issue CLDSRV-902 contains:

None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

9.4.0

Please check the Fix Version/s of CLDSRV-902, or the target
branch of this pull request.

jonathan-gramain · 2026-05-20T00:29:55Z

@@ -0,0 +1,185 @@
+'use strict';


not needed anymore

jonathan-gramain · 2026-05-20T00:49:00Z

+    const buf = Buffer.from(b64, 'base64');
+    let r = 0n;
+    for (let i = 0; i < buf.length; i += 1) {
+        r = (r << 8n) | BigInt(buf[i]);
+    }
+    return r;


You could convert to a BigInt one-shot like that (I think it's actually quite efficient, possibly more than converting one byte at a time):

Suggested change

const buf = Buffer.from(b64, 'base64');

let r = 0n;

for (let i = 0; i < buf.length; i += 1) {

r = (r << 8n) | BigInt(buf[i]);

}

return r;

const buf = Buffer.from(b64, 'base64');

return BigInt(`0x${buf.toString('hex')}`);

jonathan-gramain · 2026-05-20T00:53:50Z

+    const nBytes = dim / 8;
+    const buf = Buffer.alloc(nBytes);
+    let v = value;
+    for (let i = nBytes - 1; i >= 0; i -= 1) {
+        buf[i] = Number(v & 0xffn);
+        v >>= 8n;
+    }
+    return buf.toString('base64');


Likewise, this could be done one-shot over the whole bigint

Suggested change

const nBytes = dim / 8;

const buf = Buffer.alloc(nBytes);

let v = value;

for (let i = nBytes - 1; i >= 0; i -= 1) {

buf[i] = Number(v & 0xffn);

v >>= 8n;

}

return buf.toString('base64');

const unpaddedHex = value.toString(16);

const paddedHex = unpaddedHex.padStart(dim / 4, '0');

return Buffer.from(paddedHex, 'hex').toString('base64');

claude · 2026-05-21T08:52:42Z

LGTM

Review by Claude Code

…hecksum

…ders

claude · 2026-05-21T09:20:51Z

LGTM

Review by Claude Code

leif-scality · 2026-05-21T13:12:53Z

ping

bert-e · 2026-05-21T13:13:02Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
2 peers

leif-scality · 2026-05-21T13:18:42Z

/approve

bert-e · 2026-05-21T13:18:56Z

I have successfully merged the changeset of this pull request
into targetted development branches:

✔️ development/9.4

The following branches have NOT changed:

development/7.10
development/7.4
development/7.70
development/8.8
development/9.0
development/9.1
development/9.2
development/9.3

Please check the status of the associated issue CLDSRV-902.

Goodbye leif-scality.

The following options are set: approve

Base automatically changed from improvement/CLDSRV-903-prettier-lint-files to development/9.4 May 19, 2026 09:30

BourgoisMickael reviewed May 19, 2026

View reviewed changes

Comment thread lib/api/apiUtils/integrity/validateChecksums.js Outdated

BourgoisMickael reviewed May 19, 2026

View reviewed changes

Comment thread lib/api/apiUtils/integrity/crcCombine.js Outdated

jonathan-gramain approved these changes May 20, 2026

View reviewed changes

leif-scality force-pushed the improvement/CLDSRV-902-calculate-final-checksum-from-parts branch from 8754b9a to 2493909 Compare May 21, 2026 08:50

leif-scality added 2 commits May 21, 2026 11:12

CLDSRV-902: calculate final FULL_OBJECT (crc combine) and COMPOSITE c…

6600b33

…hecksum

CLDSRV-902: document validateXAmzChecksums and getChecksumDataFromHea…

538c16c

…ders

leif-scality requested a review from BourgoisMickael May 21, 2026 09:18

leif-scality force-pushed the improvement/CLDSRV-902-calculate-final-checksum-from-parts branch from 2493909 to 538c16c Compare May 21, 2026 09:18

BourgoisMickael approved these changes May 21, 2026

View reviewed changes

bert-e merged commit 538c16c into development/9.4 May 21, 2026
36 checks passed

bert-e deleted the improvement/CLDSRV-902-calculate-final-checksum-from-parts branch May 21, 2026 13:18

Conversation

leif-scality commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claude Bot commented May 15, 2026

Uh oh!

codecov Bot commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bert-e commented May 19, 2026

Hello leif-scality,

Uh oh!

bert-e commented May 19, 2026

Incorrect fix version

Uh oh!

Uh oh!

Uh oh!

jonathan-gramain May 20, 2026

Choose a reason for hiding this comment

Uh oh!

leif-scality May 21, 2026

Choose a reason for hiding this comment

Uh oh!

jonathan-gramain May 20, 2026

Choose a reason for hiding this comment

Uh oh!

leif-scality May 21, 2026

Choose a reason for hiding this comment

Uh oh!

jonathan-gramain May 20, 2026

Choose a reason for hiding this comment

Uh oh!

leif-scality May 21, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot commented May 21, 2026

Uh oh!

claude Bot commented May 21, 2026

Uh oh!

leif-scality commented May 21, 2026

Uh oh!

bert-e commented May 21, 2026

Waiting for approval

Uh oh!

leif-scality commented May 21, 2026

Uh oh!

bert-e commented May 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

leif-scality commented May 15, 2026 •

edited

Loading

codecov Bot commented May 15, 2026 •

edited

Loading