2001

Brings the tool up to the currently deployed build, and folds self-regulation in so there is no longer a second script to run alongside it.

copy_file_range is now off by default

--copy-file-range opts back in. A size sweep to 200 GiB found a fixed per-call cost of roughly 9 ms, which at 256 KiB is the entire transfer — 27x slower than a plain userspace copy — and amortizes away by 4 MiB.

That default suits a workload whose large files have already migrated and whose remainder is a small-file tail. It may be wrong for you: on CephFS copy_file_range can become an OSD-to-OSD copy that never crosses the client's link, which matters far more than per-call overhead for a client on a slow or distant path. Measurement here showed both methods moving 2.02x filesize across the client link, so no offload was happening, but that is one configuration. The README explains how to measure it yourself.

Self-regulation, folded in

The transcoder can now throttle itself against a latency signal it could not otherwise see: what the filesystem's real clients experience, rather than what the transcoder experiences. Previously a separate process drove it over signals and grepped its log to confirm each one landed.

Configured with a complete PromQL expression, contract being one value in milliseconds:

regulate_prometheus_url = https://prometheus.example.net/api/v1/query
regulate_query = 1e3 * sum(increase(mds_latency_sum{...}[1m])) / sum(increase(mds_latency_count{...}[1m]))
regulate_pause_ms = 150

Nothing assumes Ceph's exporter, or Ceph. {volume} is optionally substituted from mds_namespace and regex-escaped. Fully usable with no Prometheus — leave the URL unset and the thread never starts. A query that fails, returns nothing, or returns more than one series holds current settings rather than adjusting, because a monitoring outage is not evidence about the filesystem. Units are checked at startup and called out, since seconds instead of milliseconds fails silently in both directions.

Other changes

• Per-path replace lock. replace_lock was a module-global mutex around the whole stat/rename/utime sequence, so one worker process-wide could be in the MDS-bound phase at a time — measured effective concurrency 0.96 on a 15-thread job. Now striped by path hash, preserving the per-path guarantee.
• --source-pool restricts a run to files currently in one pool, so draining one pool into another does not also sweep up everything on the default pool.
• --prune-small-subtrees skips subtrees using the MDS's own ceph.dir.rbytes/rfiles — one getfattr in place of a walk. The rbytes ceiling is the safety property, since a total bounds the loss however the bytes are distributed where a mean cannot. --prune-budget-bytes caps the aggregate. Off by default.
• --help no longer crashes. argparse renders the epilog through text % dict(prog=...), and the epilog embeds the config example, one of whose comments contained a literal %.
• Crossover advisory. Warns when --min-size sits below the size at which the move starts saving space. The source scheme is a parameter, not an assumption — comparing against 3x replication only is wrong when the source is R2 or another EC profile.
• mds_namespace, not the path. A mount point's name is not necessarily its filesystem's name. Autofs is triggered before reading, since os.stat alone does not mount.

Documentation and tests

README now documents the runtime config file, including that an edit re-applies every key present in it — which has already reset a running job's thread count. Also carries a comparison against RADOS-layer pool migration, and what transcoding costs in object count rather than bytes.

45 unit tests, up from none. Two are deliberately controls that fail against code which does nothing, so the rest cannot pass vacuously.