Add Crr Cascade capabilities to backbeat crr replication#2747
Add Crr Cascade capabilities to backbeat crr replication#2747SylvainSenechal wants to merge 2 commits into
Conversation
Hello sylvainsenechal,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| "@aws-sdk/client-sts": "^3.921.0", | ||
| "@aws-sdk/credential-providers": "^3.921.0", | ||
| "@scality/cloudserverclient": "^1.0.8", | ||
| "@scality/cloudserverclient": "file:../cloudserverclient/scality-cloudserverclient-v1.0.9.tgz", |
There was a problem hiding this comment.
@scality/cloudserverclient points to a local file path (file:../cloudserverclient/scality-cloudserverclient-v1.0.9.tgz). This will break CI and other developers' builds. Must be changed to a proper registry version or git-pinned tag before merge.
| "@scality/cloudserverclient": "file:../cloudserverclient/scality-cloudserverclient-v1.0.9.tgz", | |
| "@scality/cloudserverclient": "^1.0.9", |
— Claude Code
| const ObjectMDLocation = require('arsenal').models.ObjectMDLocation; | ||
| const { errors, jsutil, models, versioning } = require('arsenal'); | ||
| const ObjectMDLocation = models.ObjectMDLocation; | ||
| const { decode, checkCrrCascadeEvent } = versioning.VersionID; |
There was a problem hiding this comment.
Arsenal is pinned to 8.3.9, but checkCrrCascadeEvent and decode are imported from versioning.VersionID. Neither checkCrrCascadeEvent nor the getMicroVersionId() method (called on source/dest entries throughout this PR) appear to exist in arsenal 8.3.9 or in backbeat's own models. The arsenal dependency likely needs a version bump for this PR to work.
— Claude Code
| if (err.ObjNotFound || err.name === 'ObjNotFound') { | ||
| return cbOnce(err); | ||
| } | ||
| if (err.$metadata?.httpStatusCode === 409) { |
There was a problem hiding this comment.
Any 409 from the destination is assumed to be a cascade-stale scenario and the replication is silently marked COMPLETED. If cloudserver ever returns 409 for a different reason, the object would never be replicated. Consider checking for a more specific signal (e.g. a response body field or custom error code) rather than relying solely on the HTTP status code.
— Claude Code
|
There was a problem hiding this comment.
Can already check this pr, but should really be reviewed after all the other cascade prs, as changes in these pr would also mean changes here
There was a problem hiding this comment.
I think we can functional tests instead of just these,
But waiting for Arsenal/cloudserver to be merged, as it will be easier to make these tests (functional tests in backbeat rely on an image of cloudserver)
| "@aws-sdk/client-sts": "^3.921.0", | ||
| "@aws-sdk/credential-providers": "^3.921.0", | ||
| "@scality/cloudserverclient": "^1.0.8", | ||
| "@scality/cloudserverclient": "file:../cloudserverclient/scality-cloudserverclient-v1.0.9.tgz", |
There was a problem hiding this comment.
@scality/cloudserverclient is pinned to a local file path (file:../cloudserverclient/scality-cloudserverclient-v1.0.9.tgz). This will break CI and other developers' builds. Before merging, this needs to be changed to a proper npm version (e.g., ^1.0.9) or a git tag reference consistent with the other git-based deps in this project.
| "@scality/cloudserverclient": "file:../cloudserverclient/scality-cloudserverclient-v1.0.9.tgz", | |
| "@scality/cloudserverclient": "^1.0.9", |
— Claude Code
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files
@@ Coverage Diff @@
## development/9.5 #2747 +/- ##
===================================================
- Coverage 74.73% 74.71% -0.02%
===================================================
Files 199 199
Lines 13650 13641 -9
===================================================
- Hits 10201 10192 -9
Misses 3439 3439
Partials 10 10
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
| entry: destEntry.getLogInfo(), | ||
| }); | ||
| return doneOnce(null, partAlreadyAtDest); | ||
| default: { |
There was a problem hiding this comment.
When cascadeLoopDetected or cascadeDataComplete is returned as an error from a part, this error flows through the retry wrapper in _getAndPutPart. The BackbeatTask.retry() method checks err.retryable === undefined and then inspects err.code, err.name, and err.message to detect network errors. Since these cascade signals are plain objects ({}), none of those properties exist, so no mutation occurs today. But if a future change accidentally adds such a property to these sentinels, retry() would mutate the shared singleton (setting retryable = true) and corrupt all subsequent cascade checks. Consider using Object.freeze() on the sentinel objects at line 33-35 to prevent accidental mutation.
— Claude Code
|
2 participants
Issue: BB-767
Related PRs :
Arsenal : scality/Arsenal#2628
Cloudserver : scality/cloudserver#6179
CloudserverClient : scality/cloudserverclient#24
S3utils : scality/s3utils#395