You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reduce container image build time by excluding workspace packages that are not needed to produce or validate the tracker runtime image from all cargo commands in the Containerfile.
Background
Issue #1853 removed --benches --examples --all-targets from all cargo commands. However, --workspace still causes two workspace members that are unrelated to the tracker runtime to be compiled on every container build:
workspace-coupling (contrib/dev-tools/analysis/workspace-coupling) — a local analysis tool with unique dependencies (regex, serde_json) not shared by any other package. It has no relationship to the tracker runtime image.
torrust-tracker-torrent-repository-benchmarking — a benchmark harness with 17 inline unit tests. It is not depended on by any other workspace member.
What the CI log revealed
A recent CI run (after #1853 merged) showed the following in the build-tracker-image step:
workspace-coupling is compiled twice: once in the cook stage (with stub source, ~5 min), and again in the build stage (with real source, after an ~840s gap). The 840s gap is the compilation cost of workspace-coupling's unique transitive dependencies (regex-automata, regex-syntax, and serde_json internals) from scratch — these were not pre-cooked because the cook layer was built with stubs, and the real dep graph for those crates is only triggered when the actual source is compiled.
The total build step time was 19m03s; removing these two packages is expected to cut it significantly.
Scope
In Scope:
Add --exclude workspace-coupling --exclude torrust-tracker-torrent-repository-benchmarking to all cargo commands in the Containerfile (cargo chef cook × 2, cargo nextest archive × 4).
Determine whether cargo chef prepare should also receive --exclude flags, and if so, remove the corresponding COPY/stub lines from the recipe stage.
Measure the impact on CI build time with evidence from a full CI run after the change.
Implementing cross-workflow cache sharing (tracked in the dependency-layer-cache-reuse issue).
Changing which packages are part of the workspace [members] list.
Broad Containerfile restructuring unrelated to the exclusion change.
Implementation Plan
ID
Task
Notes
T1
Verify cargo nextest archive supports --exclude
Confirm the flag works for cargo-nextest; check docs and local test.
T2
Add --exclude flags to all Containerfile cargo commands
Six commands: cargo chef cook × 2 and cargo nextest archive × 4. Both excluded packages named explicitly.
T3
Decide on cargo chef prepare exclusion
Determine if --exclude is supported/needed for cargo chef prepare. If yes, remove the COPY/stub entries for the two packages from the recipe stage. If no, document why the COPY/stub lines must stay.
T4
Run full CI build and record timing evidence
CI log showing build time after exclusion. Compare against pre-fix baseline (19m03s build step, 38m total).
Goal
Reduce container image build time by excluding workspace packages that are not needed to produce or validate the tracker runtime image from all cargo commands in the Containerfile.
Background
Issue #1853 removed
--benches --examples --all-targetsfrom all cargo commands. However,--workspacestill causes two workspace members that are unrelated to the tracker runtime to be compiled on every container build:workspace-coupling(contrib/dev-tools/analysis/workspace-coupling) — a local analysis tool with unique dependencies (regex,serde_json) not shared by any other package. It has no relationship to the tracker runtime image.torrust-tracker-torrent-repository-benchmarking— a benchmark harness with 17 inline unit tests. It is not depended on by any other workspace member.What the CI log revealed
A recent CI run (after #1853 merged) showed the following in the
build-tracker-imagestep:workspace-couplingis compiled twice: once in the cook stage (with stub source, ~5 min), and again in the build stage (with real source, after an ~840s gap). The 840s gap is the compilation cost ofworkspace-coupling's unique transitive dependencies (regex-automata,regex-syntax, andserde_jsoninternals) from scratch — these were not pre-cooked because the cook layer was built with stubs, and the real dep graph for those crates is only triggered when the actual source is compiled.The total build step time was 19m03s; removing these two packages is expected to cut it significantly.
Scope
In Scope:
--exclude workspace-coupling --exclude torrust-tracker-torrent-repository-benchmarkingto all cargo commands in the Containerfile (cargo chef cook× 2,cargo nextest archive× 4).cargo chef prepareshould also receive--excludeflags, and if so, remove the correspondingCOPY/stub lines from the recipe stage.Out of Scope:
[members]list.Implementation Plan
cargo nextest archivesupports--exclude--excludeflags to all Containerfile cargo commandscargo chef cook× 2 andcargo nextest archive× 4. Both excluded packages named explicitly.cargo chef prepareexclusion--excludeis supported/needed forcargo chef prepare. If yes, remove the COPY/stub entries for the two packages from the recipe stage. If no, document why the COPY/stub lines must stay.References