Detect stale runfiles on Windows when symlinks fall back to copies#29566
Open
olbapmar wants to merge 1 commit into
Open
Detect stale runfiles on Windows when symlinks fall back to copies#29566olbapmar wants to merge 1 commit into
olbapmar wants to merge 1 commit into
Conversation
On Windows without symlink privileges, `WindowsFileSystem.createSymbolicLink` silently falls back to `Files.copy`. The runfiles short-circuit in `RunfilesTreeUpdater` compares input/output manifest digests to decide the tree is up to date — this is sound for real symlinks (which auto-follow source content) but not for copies, where byte-equal manifests say nothing about content freshness. The result is silently stale runfiles until `bazel clean`. This adds `--experimental_detect_stale_runfiles` (default off). When enabled, the short-circuit probes the on-disk type of the first non-empty manifest entry: if it's a real symlink, the optimization applies; if it's a regular file (the copy fallback), the tree is re-synced. Also hardens the `Files.copy` fallback with `REPLACE_EXISTING` and `COPY_ATTRIBUTES`. - New flag: `--experimental_detect_stale_runfiles` - New method: `RunfilesTreeUpdater.runfilesUseRealSymlinks` - Unit tests for the new helper - Unskips `test_runfiles_updated_correctly_with_nobuild_runfile_links` on Windows
github-actions
Bot
added
area-Windows
Author
|
The failed test looks like a flake unrelated to my changes. Please correct me if I'm wrong and I'll double check. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add
--experimental_detect_stale_runfilesflag (default off). When enabled,RunfilesTreeUpdaterprobes the on-disk type of the first non-empty manifest entry before short-circuiting: if it's a real symlink the optimization applies; if it's a regular file (theFiles.copyfallback on Windows without symlink privileges), the tree is re-synced.Also hardens
WindowsFileSystem.createSymbolicLink's copy fallback withREPLACE_EXISTINGandCOPY_ATTRIBUTES.This also fixes a test that has been skipped for windows until now.
Motivation
On Windows without symlink privileges,
WindowsFileSystem.createSymbolicLinksilently falls back toFiles.copy. The runfiles short-circuit inRunfilesTreeUpdatercompares manifest digests to decide the tree is up to date, which is safe for real symlinks (which auto-follow source content), but not for copies, where byte-equal manifests say nothing about content freshness. The result is runfiles getting stale on every incremental build untilbazel clean.Build API Changes
New experimental flag:
--experimental_detect_stale_runfiles(boolean, default false). No breaking changes, as the behavior is identical to current master when the flag is off.Checklist
Release Notes
RELNOTES[NEW]: New
--experimental_detect_stale_runfilesflag detects stalerunfiles on Windows when symlink privileges are unavailable and the runfiles
tree is materialized as copies rather than symlinks.