SOW v0.6.0

SOW v0.6.0

Downloads · 下载 · Full comparison

English

SOW 0.6.0 improves repository updates, file verification, and recovery after an interruption.
It keeps the current command set, configuration format, and public repository layout.
A Managed workspace already on 0.5.0 needs no database migration.

This note covers the changes since 0.5.0. Use the installation guide for upgrade steps
and the download page for release availability.

Changes for repository operators

Area	Change in 0.6.0	Effect on your workflow
Plain RPM updates	Keep current and immediately previous checksum-named metadata	Clients can finish fetching metadata referenced by the previous index
Managed builds	Record one publication time for metadata and signatures	Recovery preserves the original time; unchanged builds keep their existing output
File verification	Check file identity throughout authenticated reads	Replaced or rewritten files are detected, including rewrites that restore the modification time
Signing keys	Read complete OpenPGP key bundles	All supplied trusted keys are considered; malformed input is rejected
State and recovery	Preserve locks, consistent reads, and committed results	Maintenance errors do not make completed changes appear to have rolled back
Large repositories	Reuse verified data and limit parallel work	Builds, imports, checks, and publication avoid repeated work

Plain RPM updates keep the previous indexes

A client first downloads repomd.xml, then requests the metadata files named inside it.
An update between those requests could previously remove the old metadata files before the client fetched them.

Plain sow create now keeps the current and immediately previous checksum-named RPM metadata sets.
An unchanged run does not advance this window. Older metadata can be removed after the next changed index.

The small .sow-plain-rpm-history.json file records the retained references.
Keep it when copying a directory for further builds. It is not an operation journal; interrupted Plain builds still recover by rerunning create.

This window does not retain removed package files or cover every client cache lifetime.
For caches that span several updates, keep the required packages and metadata in your publication layer.
The YUM migration guide explains publication order, signatures, and cached clients.

Plain timestamps remain explicit. --metadata-timestamp still defaults to 0.
Keep the greatest published timestamp when replacing another generator or restoring older package content.

Managed builds keep one publication time

A new Managed build records one UTC time, with whole-second precision, before it publishes the affected Dist indexes.
The same value supplies APT Release dates, RPM repomd.xml data timestamps, and metadata signature times.
RPM revision remains the Generation number.

The chosen time respects previous build times and the selected signing key's activation time.
An unusable or not-yet-valid key causes an error. Keep the host clock correct.

Recovery reuses the recorded time. It rejects a changed time before publishing further Dist entry points.
A no-op build creates no new Generation and does not refresh timestamps or signatures.

This behavior applies to new metadata-building operations, including automatic builds from add and rm, and new Dists.
Layout migration and older journals keep their previous timestamp rules.
Standalone export rpm-leaf output still uses RPM data timestamp 0.

An rm --check preview remains read-only. It predicts membership, paths, actions, and the resulting revision and Generation.
Execution chooses its own publication time. Time-dependent bytes, hashes, and sizes can therefore differ from the preview.

Verify the file that was actually read

Metadata and package checks now bind the digest to the opened file's identity before and after reading.
The same checks apply to fast validation and reused verification results.
They detect file replacement and same-size rewrites, including changes that restore the original modification time.

Concurrent hard-link creation can change file attributes without changing package bytes.
SOW handles this case and retains the identity from the successful verification.
This keeps read-only hard-link exports compatible with repository checks.

Path traversal also checks that opened entries match the entries inspected earlier.
Parent directories need traversal permission; they no longer need directory-listing permission merely to reach a known child.
Package files still need read access, and controlled paths still reject unsafe symlinks and file types.

Read complete signing-key bundles

A trusted-key file can contain several public keys or consecutive armored key blocks.
SOW now reads the complete input instead of stopping after the first block.
It rejects malformed, truncated, or unsupported key material instead of silently skipping it.

A signing key reference must still identify exactly one primary key.
Put bundles in trusted_keys; use one signing identity for each metadata or package-signing setting.
The signing reference describes both cases.

Keep state and recovery results consistent

SQLite identity checks now preserve locks held by existing connections.
Composite repository queries read related state from one database snapshot.
Eligible WAL databases undergo validation once per open, avoiding repeated full validation during one connection's work.

A database commit can succeed before a later checkpoint or cleanup fails.
SOW now preserves the committed command result in that case.
Inspect the result, sow status, and sow log before retrying; a nonzero exit does not prove rollback.

Dist status also reports unfinished repository maintenance, including local GC and log pruning.
Matching Desired and Built package counts no longer hide this pending work.
Removal logs preserve policy-driven outcomes as well as explicitly selected removals.

For an already-started development-era C2 layout transition, repo migrate --abort keeps the trusted transition marker.
It can also finish before the first journal or base manifest exists.
If a journal binds a plan, SOW verifies that plan against its recorded base manifest and rejects conflicting control records.

A nonempty temporary migration stage moves intact into .sow/<repo>/abandoned/.
The command reports the retained path or cleanup problem in stderr and JSON warnings.
Stage-cleanup warnings do not prevent abandonment. The public tree remains unchanged.
Once selected, the Repository name remains available in migration failure responses.

A fresh C2 repository still cannot start a migration.
An older abort that already removed its trusted marker requires a matching backup or a new workspace.
See repo migrate for the supported recovery paths.

Reduce repeated work

The release reduces I/O without adding another full repository scan or recovery state machine:

• Reuse authenticated DEB bytes for parsing and checking within the same operation.
• Reuse current payload and Dist verification results where their file identities still match.
• Reuse verified RPM export output instead of reading the same copy again.
• Avoid duplicate writes while staging imports and remove an unused initial output-tree hash.
• Query memberships for the selected packages and reuse package data already loaded for the operation.
• Keep package-staging workers independent and bounded by the requested concurrency.

sow check still reads and hashes each unique physical package on every run.
These changes do not turn it into a fingerprint-only check.

R2 inventory checks now use at most eight concurrent HEAD requests per page.
The queued work and completed results stay bounded, including cancellation and failures.
Upload retries no longer restart the idle timeout merely by seeking in the source file.
Actual transfer progress is required. R2 garbage collection remains report-only.

Build, packaging, and validation

Source builds require Go 1.27.2 or newer. Quality checks use staticcheck 0.8.2.
Upstream verification for the patched RPM parser also works when Go selected a downloaded toolchain.

The release targets remain Linux and macOS on amd64 and arm64.
Packaging produces four archives, two RPMs, two DEBs, and SHA256SUMS.
Archives and packages retain their license files and third-party notices.
Deterministic source delivery excludes local agent instructions from the delivered product source.

Regression tests cover file replacement, restored modification times, concurrent hard links, and complete key bundles.
They also cover commit cancellation, post-commit failures, metadata-time recovery, migration abort, Plain index retention, and bounded R2 requests.
The release also removes unused code and brittle Go runtime I/O instrumentation.

Local release checks covered the macOS and Linux suites, race checks, APT/DNF integration, and S3-compatible transport.
They also covered 50,000-package RPM metadata generation, packaging, and an upgrade from the published 0.5.0 binary.
Use your actual package clients and publication target for deployment acceptance.

Upgrade from 0.5.0

Database schema v13, configuration schema sow/v3, and the public pool/ + dists/ layout remain unchanged.

1. Stop workspace writers.
2. Back up the complete workspace, preserving hard links.
3. Install SOW 0.6.0.
4. Run these commands for each Repository:

sow version
sow build -C /srv/sow -r REPOSITORY
sow check -C /srv/sow -r REPOSITORY

Replace REPOSITORY and /srv/sow with your values.
A current 0.5.0 repository needs no schema migration or forced rebuild; build reports a no-op.
Publish only when you intend to deliver a new Built Generation to a configured target.

For 0.3 or 0.4 workspaces, run sow repo migrate REPOSITORY before building.
See the complete upgrade procedure.

---

中文

SOW 0.6.0 改进仓库更新、文件校验和中断恢复，保持现有命令、配置格式与公开仓库布局不变。
已使用 0.5.0 的 Managed 工作区无需迁移数据库。

本文记录自 0.5.0 以来的变化。升级步骤见安装说明，
制品发布状态见下载页。

仓库维护者需要了解的变化

范围	0.6.0 的变化	对操作的影响
Plain RPM 更新	保留当前与紧邻上一轮以校验和命名的元数据	客户端可以继续下载上一版索引引用的元数据
Managed 构建	元数据与签名共用一个记录的发布时间	恢复保留原时间；没有变化的构建保留已有输出
文件校验	在认证读取期间持续核对文件身份	检测文件替换与内容改写，包括恢复原修改时间的改写
签名密钥	读取完整 OpenPGP 密钥集合	检查全部受信密钥，拒绝格式错误的输入
状态与恢复	保持锁、读取一致性和已提交结果	维护错误不会使已完成的变更看起来已经回滚
大型仓库	复用已验证数据，限制并发工作量	减少构建、导入、检查和发布中的重复工作

Plain RPM 更新保留上一轮索引

客户端先下载 repomd.xml，再请求其中列出的元数据文件。
如果这两步之间发生仓库更新，旧元数据可能在客户端下载前就被清理。

Plain sow create 现在保留当前与紧邻上一轮以校验和命名的 RPM 元数据。
没有变化的运行不会推进该窗口。下一次索引变化后，更早的元数据可以被清理。

小文件 .sow-plain-rpm-history.json 记录保留引用。
复制目录后还要继续构建时，一并保留该文件。它不是操作日志；Plain 构建中断后仍通过重新执行 create 恢复。

此窗口不保留已删除的包文件，也不覆盖所有客户端缓存期限。
缓存跨越多轮更新时，在发布层保留所需的软件包与元数据。
YUM 迁移指南说明发布顺序、签名与旧客户端缓存的处理方法。

Plain 时间戳仍由调用者指定，--metadata-timestamp 默认值仍为 0。
替换其他生成工具或恢复旧软件包内容时，保留已发布的最大时间戳。

Managed 构建保留一个发布时间

新的 Managed 构建在发布受影响 Dist 的索引前，记录一个精确到秒的 UTC 时间。
APT Release 日期、RPM repomd.xml 的 data 时间戳和元数据签名共用该值。
RPM revision 仍使用构建快照编号。

所选时间遵守已有构建时间与所选签名密钥启用时间的下限。
密钥不可用或尚未生效时，命令返回错误。请保持主机时钟正确。

恢复操作复用已经记录的时间。如果时间发生漂移，SOW 会在发布后续 Dist 入口前拒绝继续。
空操作构建不创建新的构建快照，也不刷新时间戳或签名。

此规则适用于新的元数据构建操作，包括 add、rm 自动触发的构建和新建 Dist。
布局迁移与旧日志保留此前的时间戳规则。
独立的 export rpm-leaf 输出仍使用 RPM data 时间戳 0。

rm --check 预览仍为只读操作，预测成员、路径、动作，以及结果的修订号和构建快照编号。
实际执行会选择自己的发布时间，因此含时间信息的字节、哈希与大小可能不同于预览。

校验实际读取的文件

元数据与软件包校验现在将摘要绑定到打开的文件身份，并在读取前后核对。
快速校验和复用校验结果也采用相同检查。
这能发现文件替换、等长改写，以及改写后恢复原修改时间的情况。

并发创建硬链接可能改变文件属性，但不改变软件包内容。
SOW 正确处理这种情况，并保留成功校验时的文件身份。
只读硬链接导出因此可以与仓库检查并行使用。

路径遍历还会核对实际打开的条目与此前检查的条目是否相同。
访问已知子路径时，父目录只需具备遍历权限，不再额外要求目录列表读取权限。
软件包仍需可读，受控路径仍拒绝不安全的符号链接和文件类型。

读取完整的签名密钥集合

一个受信密钥文件可以包含多个公钥或连续的 ASCII 装甲密钥块。
SOW 现在读取完整输入，不再遇到第一个块后就停止。
格式错误、被截断或不受支持的密钥内容会被拒绝，不会被静默跳过。

签名 key 引用仍须对应一个主密钥。
将密钥集合放入 trusted_keys，每项元数据或软件包签名设置使用一个签名身份。
具体规则见签名参考。

保持状态与恢复结果一致

SQLite 身份检查现在保留现有连接持有的锁。
组合仓库查询从同一个数据库快照读取相关状态。
符合条件的 WAL 数据库在每次打开时验证一次，避免同一连接内反复执行完整验证。

数据库提交可能已经成功，后续检查点或清理才发生错误。
SOW 现在保留这种情况下已经提交的命令结果。
重试前查看结果、sow status 和 sow log；非零退出码不代表事务已回滚。

Dist 状态也会显示未完成的仓库维护，包括本地 GC 和日志裁剪。
即使期望成员与已构建成员数量相同，也不会隐藏这些待处理操作。
移除操作日志同时保留策略造成的结果与显式选中的移除结果。

对于已经启动的开发期 C2 布局迁移，repo migrate --abort 会保留可信迁移标记。
即使首份日志或基准清单尚未生成，也能完成中止。
日志已经绑定迁移计划时，SOW 会按记录的基准清单核对计划，并拒绝矛盾的控制记录。

非空迁移暂存目录会被完整移入 .sow/<repo>/abandoned/。
命令通过 stderr 和 JSON warnings 报告保留路径或清理问题。
暂存清理告警不会阻止中止操作，公开目录保持不变。
仓库选择成功后，即使迁移失败，响应中也会保留仓库名称。

尚未启动迁移的 C2 仓库仍不支持直接迁移。
如果旧版中止操作已经移除了可信标记，需要恢复匹配备份或创建新工作区。
支持的恢复路径见 repo migrate。

减少重复工作

本版减少 I/O，没有增加额外的全仓扫描或恢复状态机：

• 在同一次操作中，复用已认证的 DEB 字节完成解析与检查。
• 文件身份仍匹配时，复用当前软件包与 Dist 的校验结果。
• 复用已验证的 RPM 导出结果，避免再次读取同一份副本。
• 避免导入暂存阶段的重复写入，移除没有使用的初始输出目录树哈希。
• 只查询所选软件包的成员关系，并复用本次操作整理出的软件包数据。
• 保持软件包暂存任务相互独立，并按指定并发数限制工作量。

sow check 仍会在每次运行时读取并哈希每个独立的物理包文件，不会退化为仅检查文件指纹。

R2 清单检查现在每页最多并发执行八个 HEAD 请求。
排队任务与已完成结果的数量都有上限，取消与失败时也一样。
上传重试不再仅因重新定位源文件就重启空闲超时，必须有实际传输进展。
R2 垃圾回收仍只报告候选对象，不删除远端对象。

构建、打包与验证

源码构建要求 Go 1.27.2 或更新版本，质量检查使用 staticcheck 0.8.2。
即使 Go 自动选择了下载的工具链，修补后的 RPM 解析模块也能完成上游来源验证。

发布目标仍为 Linux 与 macOS 的 amd64、arm64。
打包产物包括四个归档、两个 RPM、两个 DEB 和 SHA256SUMS。
归档与安装包继续包含许可证文件和第三方许可声明。
可复现源码交付会将本地代理说明排除在产品源码之外。

回归覆盖增加了文件替换、恢复原修改时间、并发硬链接、完整密钥集合、提交取消、提交后失败、
元数据时间恢复、迁移中止、Plain 索引保留和 R2 有界请求。
本版还移除未使用的代码，以及依赖 Go 运行时内部实现的脆弱 I/O 插桩。

本地发布前核验覆盖 macOS/Linux 测试、race 检查、APT/DNF 集成、S3 兼容传输、
5 万包 RPM 元数据生成、打包，以及从已发布 0.5.0 二进制升级。
部署验收仍使用实际的包管理客户端和发布目标。

从 0.5.0 升级

数据库 schema v13、配置 sow/v3 与公开 pool/ + dists/ 布局均保持不变。

1. 停止工作区写入。
2. 备份完整工作区，并保留硬链接。
3. 安装 SOW 0.6.0。
4. 对每个仓库执行：

sow version
sow build -C /srv/sow -r REPOSITORY
sow check -C /srv/sow -r REPOSITORY

将 REPOSITORY 与 /srv/sow 换成实际值。
已处于最新状态的 0.5.0 仓库无需迁移数据库或强制重建，build 会报告空操作。
需要把新的构建快照交付到配置目标时，再执行发布。

从 0.3 或 0.4 工作区升级时，先执行 sow repo migrate REPOSITORY，再构建。
详见完整升级步骤。