Repository navigation
OSS Transfer v4.1.41
对象比对全面改进
- 不再限制 10,000 条差异:每条差异都保存在 Agent 数据目录下的文件中。按容量(默认 2 GiB)、保留天数(默认 30 天)、磁盘剩余空间(默认至少 1 GiB)和数量自动清理,可在 Agent 的
service.json中用可选的comparison块调整。 - 导出:ZIP(
differences.csv、retry-source-keys.incr、summary.json)或 JSON Lines。retry-source-keys.incr列出需要重新迁移的源 Key,可直接用于再次迁移;summary.json说明导出是否完整。 - 断点续比:取消后可以继续;Agent 重启或升级后自动从检查点继续。
- 更快:按 Key 顺序并发比对,每个对象两端的 HEAD 同时发出。测试环境中 321,845 个对象的整桶比对从 1 小时 55 分钟缩短到 9 分钟。
- 更准确:修复列举时间带毫秒、HEAD 时间只到秒导致的误判;列出不一致的字段;新增“ETag 不同”提示;读取失败时给出具体原因(HTTP 状态、错误码、请求 ID);两端都不存在的对象单独计数。
- 内容(SHA-256)模式:大对象按 8 MiB 分段读取,失败的分段从中断处重试,60 秒收不到数据的读取会中止并重试,并显示已读取的字节数。
- 控制台和 CLI:比对历史、按结果筛选、分页、导出 ZIP、继续比对;新增
oss-transfer-cli object list、object status --status、object export、object resume。 - 行为变化:只指定 Key 的
object compare现在会启动单对象比对,15 秒内完成时直接显示结果,否则显示比对 ID;--json输出改为比对记录。 - 兼容性:新 Master 可以管理旧版 Agent 上的比对(最多保存 10,000 条差异,不能继续比对)。如果在 Agent 的
service.json中添加了comparison块,降级到旧版前请先删除。
Object comparison overhaul
- No 10,000-difference limit: every difference is stored in files under the Agent's data directory. Results are cleaned up by size (2 GiB by default), age (30 days), free disk space (at least 1 GiB) and count; adjust these with the optional
comparisonblock in the Agent'sservice.json. - Export as ZIP (
differences.csv,retry-source-keys.incr,summary.json) or JSON Lines.retry-source-keys.incrlists the source keys to transfer again and can be used for another migration;summary.jsonstates whether the export is complete. - Resume: continue a canceled comparison; after an Agent restart or upgrade a comparison resumes from its checkpoint by itself.
- Faster: keys are compared concurrently in key order, with both HEADs of an object sent at once. A 321,845-object bucket in the test environment took 9 minutes instead of 1 hour 55 minutes.
- More accurate: fixes false differences when listings report milliseconds and HEAD only seconds; names the properties that differ; adds an "ETag differs" hint; read failures show the HTTP status, error code and request ID; objects missing on both sides are counted separately.
- Content (SHA-256) mode reads large objects in 8 MiB ranges, retries a failed range from where it stopped, aborts and retries a read that receives no data for 60 seconds, and shows the bytes read.
- Console and CLI: comparison history, status filters, paging, ZIP export and resume; new
oss-transfer-cli object list,object status --status,object exportandobject resume. - Behaviour change:
object comparewith only keys now starts a single-object comparison and prints the result when it finishes within 15 seconds, otherwise its ID;--jsonprints the comparison record. - Compatibility: a new Master manages comparisons on older Agents (at most 10,000 stored differences, no resume). If you added a
comparisonblock to an Agent'sservice.json, remove it before downgrading.
Full Changelog: v4.1.40...v4.1.41