Skip to content

Exclude irrelevant workspace members from container build #1868

@josecelano

Description

@josecelano

Goal

Reduce container image build time by excluding workspace packages that are not needed to produce or validate the tracker runtime image from all cargo commands in the Containerfile.

Background

Issue #1853 removed --benches --examples --all-targets from all cargo commands. However, --workspace still causes two workspace members that are unrelated to the tracker runtime to be compiled on every container build:

  • workspace-coupling (contrib/dev-tools/analysis/workspace-coupling) — a local analysis tool with unique dependencies (regex, serde_json) not shared by any other package. It has no relationship to the tracker runtime image.
  • torrust-tracker-torrent-repository-benchmarking — a benchmark harness with 17 inline unit tests. It is not depended on by any other workspace member.

What the CI log revealed

A recent CI run (after #1853 merged) showed the following in the build-tracker-image step:

#60 [dependencies 3/4] cargo chef cook --tests --workspace --all-features ...
#60 278.9    Compiling workspace-coupling v0.0.1 ...
#60 304.1     Finished in 5m 04s

#61 [dependencies 4/4] cargo nextest archive ... (warmup)
#61 71.63     Finished in 1m 10s

#64 [build 3/3] cargo nextest archive --tests --workspace --all-features ...
#64 253.4    Compiling torrust-tracker v3.0.0-develop
#64 1094.3   Compiling workspace-coupling v3.0.0-develop   <- 840s after tracker
#64 1144.0     Finished in 19m 03s

workspace-coupling is compiled twice: once in the cook stage (with stub source, ~5 min), and again in the build stage (with real source, after an ~840s gap). The 840s gap is the compilation cost of workspace-coupling's unique transitive dependencies (regex-automata, regex-syntax, and serde_json internals) from scratch — these were not pre-cooked because the cook layer was built with stubs, and the real dep graph for those crates is only triggered when the actual source is compiled.

The total build step time was 19m03s; removing these two packages is expected to cut it significantly.

Scope

In Scope:

  • Add --exclude workspace-coupling --exclude torrust-tracker-torrent-repository-benchmarking to all cargo commands in the Containerfile (cargo chef cook × 2, cargo nextest archive × 4).
  • Determine whether cargo chef prepare should also receive --exclude flags, and if so, remove the corresponding COPY/stub lines from the recipe stage.
  • Measure the impact on CI build time with evidence from a full CI run after the change.

Out of Scope:

  • Removing tests from the container build (tracked in Evaluate test execution policy in container image build #1854).
  • Implementing cross-workflow cache sharing (tracked in the dependency-layer-cache-reuse issue).
  • Changing which packages are part of the workspace [members] list.
  • Broad Containerfile restructuring unrelated to the exclusion change.

Implementation Plan

ID Task Notes
T1 Verify cargo nextest archive supports --exclude Confirm the flag works for cargo-nextest; check docs and local test.
T2 Add --exclude flags to all Containerfile cargo commands Six commands: cargo chef cook × 2 and cargo nextest archive × 4. Both excluded packages named explicitly.
T3 Decide on cargo chef prepare exclusion Determine if --exclude is supported/needed for cargo chef prepare. If yes, remove the COPY/stub entries for the two packages from the recipe stage. If no, document why the COPY/stub lines must stay.
T4 Run full CI build and record timing evidence CI log showing build time after exclusion. Compare against pre-fix baseline (19m03s build step, 38m total).

References

Metadata

Metadata

Assignees

Labels

Type

No fields configured for Task.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions