Skip to content

Scoped build evicts out-of-scope commands from the saved index #125

Description

@typeless

A scoped build — putup -B build <target> — that runs at least one job saves a new index built only from the scoped graph, dropping every out-of-scope command, its output file entries, its stat-cache data, and its implicit edges. The saved index no longer describes the whole project.

Pre-existing and independent of the data-loss fix in #124 (which stops the deletion but not the eviction). Found during the review of #122.

Repro (verified)

# two independent dirs alpha, beta; full build leaves build/alpha/a.out, build/beta/b.out
$ echo 'int a(void){return 42;}' > alpha/a.c        # touch alpha so a scoped build runs a job
$ putup -B build alpha                              # scoped: runs alpha's rule, saves index
$ cp beta/b.c build/beta/b.out                      # restore beta's output bit-identical
$ putup -B build -v                                 # full build, beta's output present + correct
[build]   New command: 
[build] cp b.c ../build/beta/b.out                   # <-- beta re-run despite correct output

With #124 in place the scoped build no longer deletes build/beta/b.out, so the restore step isn't needed to see it: a full build after any job-running scoped build re-runs all out-of-scope commands.

Consequences

  1. Spurious mass rebuild: the next full build re-runs every out-of-scope command, because their identities are gone from the index.
  2. Orphan window: scoped build evicts beta from the index → user then deletes beta's rule → next full build has no record of beta → build/beta/b.out is orphaned forever. Before Stop scoped builds from deleting out-of-scope outputs #124 this was masked because the scoped build deleted the output immediately; Stop scoped builds from deleting out-of-scope outputs #124 un-masks it (litter, not data loss).

Cause

The scoped build's build_index (src/cli/cmd_build.cpp, serialize_graph_nodes / serialize_command_nodes) serializes only the scoped graph. preserve_old_implicit_edges drops out-of-scope edges (no identity match in the new index). The minimal repro is masked only because the Nothing to do (up to date) fast path returns before the index save.

Suggested direction

When the parse was scoped, merge forward the out-of-scope commands + their output file entries + edges from the old index into the new one at build_index time. Node ids are positional (id == array_index), so this needs id remapping — preserve_old_implicit_edges' identity re-resolution and get_or_create_dir/create_implicit_file are the in-repo precedents. This makes "the saved index describes the whole project" a structural invariant enforced at the single construction site, rather than a convention every consumer must remember. Larger and riskier than #124's gate, hence a separate issue.

Root cause shared with #122/#124: a scoped parse masquerades as the whole project.

Update (from #124 review — verified)

The orphan is permanent, not a window: full build → delete an out-of-scope rule → make an in-scope change → scoped build (outputs preserved by #124, index saved with only in-scope commands) → full build. The removed command is now in neither the graph nor the old index, so its output is never cleaned. It also survives putup clean (index-driven — clean only removes tracked outputs); only distclean or manual deletion clears it. Eviction erases the only record that could ever clean the orphan.

Also confirmed: eviction causes redundant rebuilds — the next full build re-runs every out-of-scope command as "New command" even when its output is present and correct.

Related root-cause siblings: #122/#124 (out-of-scope), #126 (parse failure under -k). All stem from "a command absent from the identity map is treated as removed regardless of why"; the merge-forward here plus a parsed-Tupfile gate would make the whole family structural.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions