Skip to content

Commit

Permalink
Fix a race condition in scheduling code in `FileSystemValueCheckerInf…
Browse files Browse the repository at this point in the history
…erringAncestors`.

`FileSystemValueCheckerInferringAncestors` works on the premise that we schedule all leaf entries from the diff and work our way up the path from there. The code which starts parallel processing detects leaves by looking at how many children to process each entry has. That is only correct if no processing is happening.

Fix the race condition happening due to the fact that we schedule the processing in a stream detecting the leaves -- this means that the detection algorithm works in parallel with processing. Add a materialization step in between to detect all leaves before any processing starts.

PiperOrigin-RevId: 390417452
  • Loading branch information
alexjski authored and Copybara-Service committed Aug 12, 2021
1 parent d2ef250 commit 1652361
Showing 1 changed file with 4 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,10 @@ private ImmutableDiff processEntries(int nThreads)
ImmutableList<Future<?>> futures =
nodeStates.entrySet().stream()
.filter(e -> e.getValue().childrenToProcess.get() == 0)
// Materialize all leaves before scheduling them -- otherwise, we could race with the
// processing code which decrements childrenToProcess.
.collect(toImmutableList())
.stream()
.map(
e ->
executor.submit(
Expand Down

0 comments on commit 1652361

Please sign in to comment.