OSS Transfer v4.1.45

修复

• 初始密码复制后登录失败：初始密码和 --reset-admin-password 重置后的密码改为只含字母和数字的 24 位字符串。以前的密码可能含 - 或 _，在终端中双击选择时只能选中一部分，导致首次登录提示密码错误。
• 手动运行脚本显示的密码不对：oss-transfer-run.sh 现在只读取本次启动的日志，把密码单独打印一行，后面不接任何文字；以前密码后紧跟“（首次登录后请修改）”，而且日志里有早先运行的记录时，可能显示失效的旧密码。管理员账号已存在时，脚本改为提示重置密码的命令。

新功能

• 京东云内网下载源，覆盖 Windows 和加入命令：Install.ps1 支持 -Source jdoss-internal。控制台 添加节点 窗口的原生部署指令上方新增 下载来源 选项，可在 Master、公网和 京东云内网（华北-北京） 之间切换；选择内网时，加入命令只访问 moonwx.s3-internal.cn-north-1.jdcloud-oss.com，不检测地区，也不回退到公网。
• oss-transfer-run.sh 随版本发布：手动运行脚本和安装脚本一样，出现在 GitHub Release 以及京东云 OSS 的 latest/ 和 v<版本号>/ 目录中，可以用 https://github.com/MoonWX/oss-transfer/releases/latest/download/oss-transfer-run.sh 或镜像、内网地址下载。

文档

• README 和各篇指南改为正式的书面语气，英文版按中文版逐页重写，并补充内网下载源和手动运行脚本的下载地址。

---

Fixes

• Sign-in failed with a copied initial password: initial passwords, and passwords set by --reset-admin-password, now consist of 24 letters and digits. Earlier passwords could contain - or _, where a terminal's double-click selection stops, so only part of the password was copied and the first sign-in was rejected.
• The manual run script showed the wrong password: oss-transfer-run.sh now reads only the log of the current start and prints the password alone on its line. It used to append a note directly after the password, and could show a stale password from an earlier run kept in the log. When an administrator already exists, the script prints the command for resetting the password instead.

New

• JD Cloud internal source for Windows and join commands: Install.ps1 accepts -Source jdoss-internal. In the console's Add node dialog, a Download source option above the native commands switches between the Master, the public sources and JD Cloud internal (Beijing). With the internal source, the join command contacts only moonwx.s3-internal.cn-north-1.jdcloud-oss.com, without location detection or a public fallback.
• oss-transfer-run.sh is published with each release: like the installers, the manual run script is part of the GitHub Release and of the latest/ and v<version>/ directories on JD Cloud OSS, for example at https://github.com/MoonWX/oss-transfer/releases/latest/download/oss-transfer-run.sh.

Documentation

• The README and guides are rewritten in a formal register; the English pages follow the Chinese pages one to one and cover the internal source and the manual run script downloads.

OSS Transfer v4.1.44

新功能

• 京东云内网下载源：Linux、macOS 和 FreeBSD 的安装脚本新增 --source jdoss-internal，经内网地址 https://moonwx.s3-internal.cn-north-1.jdcloud-oss.com/oss-transfer/ 下载发布文件。位于京东云华北-北京地域的主机不占用公网带宽，也不依赖访问公网。自动选择下载源时不会使用它。安装脚本本身也可以从内网地址获取，通过管道运行时用 bash -s -- 传入参数：

  curl -fsSL https://moonwx.s3-internal.cn-north-1.jdcloud-oss.com/oss-transfer/latest/install.sh | sudo bash -s -- --source jdoss-internal

---

New

• JD Cloud internal download source: the Linux, macOS and FreeBSD installers accept --source jdoss-internal, which downloads the release files through the internal endpoint https://moonwx.s3-internal.cn-north-1.jdcloud-oss.com/oss-transfer/. Hosts in the JD Cloud cn-north-1 (Beijing) region use no public bandwidth and need no access to the internet. Automatic source selection never picks it. The installer itself can come from the internal endpoint too; when piping it, pass options after bash -s --:

  curl -fsSL https://moonwx.s3-internal.cn-north-1.jdcloud-oss.com/oss-transfer/latest/install.sh | sudo bash -s -- --source jdoss-internal

OSS Transfer v4.1.43

修复

• 目标端密钥只有写入权限（没有 s3:GetObject）时，继续或重试任务不再卡住。恢复任务时，执行节点会用 HEAD 核对中断前已写入的对象；冲突策略为“覆盖同名对象或文件”且 HEAD 被拒绝（403）时，这些对象会被视为未确认并重新写入一遍，任务可以继续完成。其他两种冲突策略仍需 HEAD 才能判断，行为不变。

新功能

• 逐项检查存储桶权限：在“存储连接 → 测试连接 → 检查指定存储桶”中，会在该存储桶上实际执行迁移用到的请求（列举、HEAD、GET），并逐项显示结果和一句结论；勾选写入检查后，还会检查条件写入、读回和分片上传，写入的测试对象检查后删除。CLI 为 oss-transfer-cli storage test <连接名> --bucket <存储桶> [--write]，任一必需项未通过时退出码为 1。需要 Master 和执行节点都是 v4.1.43。
• 手动运行：没有 systemd、也不能用 Docker 的主机，可以用 deploy/manual/oss-transfer-run.sh 直接运行发布包中的程序，它负责后台运行、异常退出后重启和日志。“添加节点”窗口新增“手动运行”页签和“复制邀请链接”按钮，oss-transfer-agent --join-url 也会忽略复制时带上的 ?lang=… 等安装参数。

改进

• 加入命令恢复为 curl -fsSL '…' | bash。以普通用户运行时，安装脚本会自动通过 sudo 重新执行。
• 连接测试窗口中“检查指定存储桶”的输入框标签不再被遮挡。
• 文档：README 新增手动运行说明和对象比对的必要性与代价，规划文档按用途说明目标端各项权限，补充多张中英文截图，Linux 支持按 systemd、Bash 和 curl 的版本说明。

---

Fixes

• Resuming or retrying a task no longer gets stuck when the target key can write but lacks s3:GetObject. On recovery, the Agent checks the objects written before the interruption with HEAD. Under the conflict policy "Overwrite existing objects or files", objects whose HEAD is denied (403) are now treated as unconfirmed and written again, so the task can finish. The other two policies need HEAD to decide and behave as before.

New

• Per-permission bucket check: under Storage connections → Test connection → Check a bucket, the check runs the requests a migration makes on that bucket (list, HEAD, GET) and shows each result with a one-sentence verdict. With the write check selected, it also tries a conditional write, a read-back and a multipart upload, and deletes its test object afterwards. In the CLI: oss-transfer-cli storage test <connection> --bucket <bucket> [--write], which exits 1 when a required check fails. It needs the Master and the Agents on v4.1.43.
• Run manually: on hosts without systemd that cannot use Docker, deploy/manual/oss-transfer-run.sh runs the programs of the release archive, restarts them after an unexpected exit and keeps their logs. The Add node dialog has a new Run manually tab and a Copy invitation link button, and oss-transfer-agent --join-url ignores installer parameters such as ?lang=… copied with the link.

Improvements

• The join command is a plain curl -fsSL '…' | bash again; run by a user who is not root, the installer restarts itself through sudo.
• The label of the Check a bucket field in the connection test is no longer cut off.
• Documentation: the README covers running manually and why and at what cost to compare objects, the planning guide explains each target permission by use, new screenshots in Chinese and English, and Linux support is described by systemd, Bash and curl versions.

OSS Transfer v4.1.42

v4.1.42 以一位新用户从安装到完成迁移的完整体验为目标：按评估中发现的问题改进了安装、加入节点、存储连接、增量迁移、失败恢复、报告和卸载，并重写了文档。

文档

• 新增规划指南：拓扑与执行节点数量、端口与防火墙、源端和目标端所需权限（附最小权限策略示例）、速度与容量、大规模核对，以及增量迁移与割接流程。
• 新增安全指南：网络暴露、用 NGINX 提供 HTTPS、凭据如何存储和传递、密码与会话、加入邀请、以 root 运行的组件。
• README 快速开始写明每一步成功时应看到什么；安装、运维（按症状排查问题）、CLI、Docker Compose 等页面全面更新，中英文内容一致。

存储连接

• 自动寻址（默认）：Endpoint 是 IP、localhost、不含点的主机名、以 .internal/.local/.lan 结尾或带端口时用 Path-style，否则用 Virtual-hosted。MinIO、SeaweedFS、Ceph 等自建存储不再需要手动选择。明确选择过的连接不受影响。
• 连接检查在列举存储桶后，还会用实际寻址方式访问一个存储桶，所以寻址错误在保存时就能发现。只授权到单个存储桶的密钥只会得到警告，可用“检查指定存储桶”或 storage test --bucket 检查。
• 常见错误（密钥无效、签名不匹配、存储桶不存在、拒绝访问、无法解析域名、连接被拒绝或重置、超时、TLS）先显示一句可操作的说明，原始错误可展开查看或用 --verbose 查看。

迁移可靠性

• 存储重启或网络抖动时，连接被拒绝或重置、请求中途断开、超时、DNS 失败和 5xx 会重试约 10 次（1～2 分钟），不再立即判为失败；读取源对象时连接中断，会从断开处续读。约 40 秒的存储重启不再导致对象失败。
• 扫描后在源端被删除的对象记为“已跳过（源端已删除）”，不再计为失败（仅对象存储源）。
• 扫描后发生变化的源对象，重试时读取其当前版本。
• 中断后未清理的分片上传会在之后自动中止。
• 下载到本地前的预检会实际写入测试文件，并报告可用空间。
• 任务详情直接给出失败的主要原因和离线节点。
• 执行节点的 service.log 会记录每次子任务的开始和结束、失败对象（每次最多 20 个，其余计数）、存储重试和与 Master 的连接变化，便于排查。

增量迁移：覆盖有变化的对象

• 新冲突策略“覆盖有变化的对象”（CLI --conflict changed）：目标不存在时复制；大小不同、两端都是普通 MD5 ETag 且不同，或源端修改时间晚于目标端时覆盖；其余跳过。不读取对象内容。
• 使用“跳过同名对象或文件”且有对象被跳过时，任务详情会提示：这些对象只按名称判断，可用对象比对核对。
• 任务详情新增“比对源和目标”，直接打开预填好的对象比对。

报告

• 停止后继续、重试失败对象之后，任务报告会列出每个对象的最终结果，不再出现大量“结果未知”。
• task report 会显示报告是否完整、各类结果的数量和不完整的原因；报告不完整时退出码为 3。--output - 可把报告写到标准输出。

加入节点与 Docker

• 添加节点的 Docker 选项可直接用于 macOS/Windows 的 Docker Desktop，无需 root。每个邀请使用独立的容器名和数据卷（oss-transfer-agent-<编号>）。
• 当 Master 地址是本机回环地址时会给出警告，并建议 host.docker.internal 或局域网地址。
• 原生加入命令在 root 和普通用户（sudo）下都能直接运行。
• 版本更新页为 Docker 执行节点提供可复制的更新命令；版本已一致时提供“重建命令”，可用来增加目录挂载。
• 移除节点前会提示：保存在该节点上的任务报告明细和比对结果将无法再读取，请先导出。

安装、升级与卸载

• 新增 install.sh --uninstall，可配合 --role、--keep-data、--yes 使用；也支持安装脚本创建的 Docker Compose 安装。Windows 的手动卸载步骤见安装文档。
• 重复安装同一版本时默认跳过，并以“无需更改”结束；需要时加 --force。
• 安装前会显示安装计划（角色、版本、目录、服务名）。如果要安装的版本比现有数据所用的版本旧，会先警告；--keep-data 卸载后会给出带 --version 的完整重装命令。
• 被移除的执行节点会自行停止，不再反复重启。
• oss-transfer-cli update master 不再需要 --version，会自动安装最新正式版。

控制台与 CLI

• 控制台：
  • 新密码至少 8 个字符，不能与用户名、当前密码或初始密码相同，也不能是常见密码、单一重复字符或连续的数字、字母。
  • 新增首次使用清单。
  • 异常记录默认只显示未解决的记录，并适配手机屏幕。
  • 版本号统一显示为 v4.1.42。
  • 时间筛选会注明按哪个时区解释。
• CLI：
  • 退出码统一：0 成功，1 操作失败，2 命令行用法错误，3 结果不完整，130 被中断。
  • 状态、策略等代码翻译为可读文字（--json 仍输出原始代码），数字不再使用科学计数法。
  • 写操作只输出一句结果摘要，错误不再重复显示。
  • 连不上 Master 时会给出检查地址、端口和服务的提示；重复执行 stop/resume 时提示“无需处理”。

兼容性与升级

• 请先升级 Master，再升级执行节点。
• 混合版本期间：
  • 旧版执行节点会收到自动寻址解析后的结果。
  • “覆盖有变化的对象”只能分配给 v4.1.42 执行节点。
  • 重试时读取源对象当前版本，需要 Master 和执行节点都是 v4.1.42；否则行为与 v4.1.41 相同。
• 已知限制：
  • SSE-KMS/SSE-C 加密对象的 ETag 不是 MD5，“覆盖有变化的对象”可能会重复复制这些对象。
  • 源端的删除不会同步到目标端。
  • Install.ps1 暂不支持 -Uninstall。

---

v4.1.42 is about the whole experience of a new user, from installation to a finished and verified migration. Based on a new-user evaluation, it improves installation, adding nodes, storage connections, incremental migrations, failure recovery, reports and uninstallation, and rewrites the documentation.

Documentation

• New planning guide: topology and Agent count, ports and firewall, the permissions needed on the source and target (with minimal policy examples), speed and capacity, large-scale verification, and an incremental migration and cut-over runbook.
• New security guide: network exposure, HTTPS with NGINX, how credentials are stored and passed, passwords and sessions, join invitations, and what runs as root.
• The README quick start says what success looks like at each step. The install, operations (troubleshooting by symptom), CLI and Docker Compose pages are rewritten; English and Chinese match.

Storage connections

• Automatic addressing (default): path-style when the endpoint is an IP address, localhost, a name without a dot, ends with .internal/.local/.lan, or has a port; virtual-hosted otherwise. Self-hosted storage such as MinIO, SeaweedFS and Ceph needs no manual choice. Connections with an explicit choice are unchanged.
• After listing buckets, the connection check also reaches one bucket with the actual addressing, so addressing mistakes show up when you save. Keys limited to one bucket only get a warning; check that bucket with "Check a bucket" or storage test --bucket.
• Common errors are shown first as an actionable sentence: invalid key, signature mismatch, missing bucket, access denied, DNS, refused or reset connection, timeout and TLS. The raw error can be expanded, or shown with --verbose.

Reliability

• When storage restarts or the network blips, the transfer retries about 10 times over 1-2 minutes before an object fails. This covers refused or reset connections, connections dropped mid-request, timeouts, DNS failures and 5xx responses. A source read cut off mid-object continues from where it stopped. A storage restart of about 40 seconds no longer fails objects.
• Objects deleted at the source after the scan are skipped as "deleted at source", not failed (object storage sources only).
• Source objects that changed after the scan are read again at their current version when retried.
• Multipart uploads left behind by an interruption are aborted later automatically.
• The precheck before downloading to a local directory writes a probe file and reports the free space.
• The task detail names the main cause of failures and any offline nodes.
• The Agent's service.log records each child task attempt's start and end, failed objects (the first 20 per attempt plus a count of the rest), storage retries and Master connection changes.

Incremental migrations: Overwrite changed objects

• New conflict policy "Overwrite changed objects" (CLI --conflict changed). It copies a missing object, and overwrites when the sizes differ, when both ETags are plain MD5 ETags and differ, or when the source was modified after the target. Everything else is skipped, and object content is never read.
• When a task with "Skip existing objects or files" skipped objects, the task detail explains that they were matched by name only and can be checked with object comparison.
• The task detail has a new "Compare source and target" button that opens a prefilled object comparison.

Reports

• After stop and continue, or after retrying failed objects, task reports list every object's final result instead of many "unknown" rows.
• task report shows whether the report is complete, the counts per result and the reasons it is incomplete, and exits 3 when it is incomplete. --output - writes the report to standard output.

Adding nodes and Docker

• The Docker option of Add node works on Docker Desktop for macOS and Windows without root. Each invitation uses its own container and volume name (oss-transfer-agent-<id>).
• A loopback Master address shows a warning that suggests host.docker.internal or a LAN address.
• The native join command runs as root or, through sudo, as a regular user.
• Software updates shows copyable update commands for Docker Agents, and "Recreate commands" when the versions already match, for example to add a directory mount.
• Removing a node warns that the task report details and comparison results stored on it will no longer be available, so export them first.

Install, upgrade and uninstall

• New install.sh --uninstall, with --role, --keep-data and --yes. It also removes Docker Compose installations made by the installer. Windows manual removal steps are in the install guide.
• Reinstalling the same version is skipped and ends with "Nothing changed" unless you add --force.
• The installer shows an installation plan (roles, version, directories, service name) before it starts. It warns before installing a version older than the one that last used the data. After --uninstall --keep-data it prints the complete reinstall command, including --version.
• A removed Agent stops by itself instead of restarting in a loop.
• oss-transfer-cli update master no longer needs --version; it installs the latest stable release.

Console and CLI

• Console:
  • A new password needs at least 8 characters and must differ from the user name and from the current and initial passwords. It also must not be a common password, one repeated character, or a run of consecutive digits or letters.
  • New first-run checklist.
  • Exception records show unresolved records by default and fit phone screens.
  • Versions are shown as v4.1.42.
  • Time filters name the time zone used.
• CLI:
  • Consistent exit codes: 0 success, 1 operation failed, 2 usage error, 3 incomplete result, 130 interrupted.
  • States and policies are shown as readable words, while --json keeps the codes. Numbers are no longer printed in scientific notation.
  • Write commands print a one-line summary, and errors are no longer repeated.
  • When the CLI cannot reach the Master it suggests checking the address, the port and the service. Repeating task stop or task resume reports "No action needed".

Compatibility and upgrading

• Upgrade the Master first, then the Agents.
• While versions are mixed:
  • Older Agents receive automatic addressing already resolved.
  • "Overwrite changed objects" only runs on v4.1.42 Agents.
  • Reading the current version of a changed source on retry needs both the Master and the Agent on v4.1.42; otherwise the behaviour is that of v4.1.41.
• Known limits:
  • ETags of SSE-KMS/SSE-C encrypted objects are not MD5s, so "Overwrite changed objects" may copy such objects again.
  • Deletions at the source are not propagated to the target.
  • Install.ps1 has no -Uninstall yet.

OSS Transfer v4.1.41

对象比对全面改进

• 不再限制 10,000 条差异：每条差异都保存在 Agent 数据目录下的文件中。按容量（默认 2 GiB）、保留天数（默认 30 天）、磁盘剩余空间（默认至少 1 GiB）和数量自动清理，可在 Agent 的 service.json 中用可选的 comparison 块调整。
• 导出：ZIP（differences.csv、retry-source-keys.incr、summary.json）或 JSON Lines。retry-source-keys.incr 列出需要重新迁移的源 Key，可直接用于再次迁移；summary.json 说明导出是否完整。
• 断点续比：取消后可以继续；Agent 重启或升级后自动从检查点继续。
• 更快：按 Key 顺序并发比对，每个对象两端的 HEAD 同时发出。测试环境中 321,845 个对象的整桶比对从 1 小时 55 分钟缩短到 9 分钟。
• 更准确：修复列举时间带毫秒、HEAD 时间只到秒导致的误判；列出不一致的字段；新增“ETag 不同”提示；读取失败时给出具体原因（HTTP 状态、错误码、请求 ID）；两端都不存在的对象单独计数。
• 内容（SHA-256）模式：大对象按 8 MiB 分段读取，失败的分段从中断处重试，60 秒收不到数据的读取会中止并重试，并显示已读取的字节数。
• 控制台和 CLI：比对历史、按结果筛选、分页、导出 ZIP、继续比对；新增 oss-transfer-cli object list、object status --status、object export、object resume。
• 行为变化：只指定 Key 的 object compare 现在会启动单对象比对，15 秒内完成时直接显示结果，否则显示比对 ID；--json 输出改为比对记录。
• 兼容性：新 Master 可以管理旧版 Agent 上的比对（最多保存 10,000 条差异，不能继续比对）。如果在 Agent 的 service.json 中添加了 comparison 块，降级到旧版前请先删除。

Object comparison overhaul

• No 10,000-difference limit: every difference is stored in files under the Agent's data directory. Results are cleaned up by size (2 GiB by default), age (30 days), free disk space (at least 1 GiB) and count; adjust these with the optional comparison block in the Agent's service.json.
• Export as ZIP (differences.csv, retry-source-keys.incr, summary.json) or JSON Lines. retry-source-keys.incr lists the source keys to transfer again and can be used for another migration; summary.json states whether the export is complete.
• Resume: continue a canceled comparison; after an Agent restart or upgrade a comparison resumes from its checkpoint by itself.
• Faster: keys are compared concurrently in key order, with both HEADs of an object sent at once. A 321,845-object bucket in the test environment took 9 minutes instead of 1 hour 55 minutes.
• More accurate: fixes false differences when listings report milliseconds and HEAD only seconds; names the properties that differ; adds an "ETag differs" hint; read failures show the HTTP status, error code and request ID; objects missing on both sides are counted separately.
• Content (SHA-256) mode reads large objects in 8 MiB ranges, retries a failed range from where it stopped, aborts and retries a read that receives no data for 60 seconds, and shows the bytes read.
• Console and CLI: comparison history, status filters, paging, ZIP export and resume; new oss-transfer-cli object list, object status --status, object export and object resume.
• Behaviour change: object compare with only keys now starts a single-object comparison and prints the result when it finishes within 15 seconds, otherwise its ID; --json prints the comparison record.
• Compatibility: a new Master manages comparisons on older Agents (at most 10,000 stored differences, no resume). If you added a comparison block to an Agent's service.json, remove it before downgrading.

Full Changelog: v4.1.40...v4.1.41

OSS Transfer v4.1.40

Full Changelog: v4.1.39...v4.1.40

OSS Transfer v4.1.39

Full Changelog: v4.1.38...v4.1.39

OSS Transfer v4.1.38

Full Changelog: v4.1.37...v4.1.38

OSS Transfer v4.1.37

Full Changelog: v4.1.36...v4.1.37

OSS Transfer v4.1.36

Full Changelog: v4.1.35...v4.1.36