Skip to content

Make debug repro packing resilient to repeated input paths - #2502

Open
Sergio0694 wants to merge 2 commits into
staging/3.0from
user/sergiopedri/fix-debug-repro-duplicate-paths
Open

Make debug repro packing resilient to repeated input paths#2502
Sergio0694 wants to merge 2 commits into
staging/3.0from
user/sergiopedri/fix-debug-repro-duplicate-paths

Conversation

@Sergio0694

Copy link
Copy Markdown
Member

Problem

All five CsWinRT build tools support a debug repro: with --debug-repro-directory set they package every input file plus a faithful .rsp into a self-contained .zip that can be replayed by passing the .zip back to the tool. Files are staged under a name derived from a SHA-256 of their full original path, and that name is then used as the key of the original-path map.

MSBuild item lists are not deduplicated, and several targets contribute to the same reference sets, so the same path routinely reaches a generator more than once. @(ReferencePathWithRefAssemblies) really does end up with e.g. Microsoft.CSharp.dll listed twice. originalPaths.Add(...) then threw, and the whole build failed:

error CSWINRTPROJECTIONGEN9999: The CsWinRT projection generator failed with an unhandled exception
('ArgumentException': 'Argument_AddingDuplicateWithKey, Microsoft.CSharp_5FB6655A486AE16885AE02CEFF80DE94.dll')
during the 'save-debug-repro' phase.

This is reproducible on staging/3.0 today by building any project with -p:CsWinRTGeneratorDebugReproDirectory=<dir>. It is how I found it: I could not capture an interop-debug-repro.zip to attach to an upstream issue, because asking for one broke the build.

Fix

Staging is now deduplicated per destination folder. The hashed name is path-derived, so a file already staged at a given destination necessarily came from the exact same source and staging it again is a no-op. The check is deliberately per-folder rather than "already in the map", because the same file can legitimately be staged under two folders of the same repro (e.g. as both the input assembly and a reference assembly), and both copies are load-bearing. The returned name list still mirrors the original argument list, so the packed .rsp stays faithful.

Two related issues surfaced while writing the tests and are fixed here too:

  • Read-only inputs. File.Copy propagates the read-only attribute, so a read-only input (normal in enlistment-style builds) left a read-only staged copy behind. Staging it twice failed with UnauthorizedAccessException, and Directory.Delete(recursive: true) on the staging directory failed as well. The attribute is now cleared on each staged copy.
  • Unpack mis-attribution. The single top-level file (the input/output assembly) was matched by entry name before the category subfolder was checked. Since entry names are the path-derived hashed names, a file staged both at the top level and inside a subfolder had its subfolder entry mis-claimed as the top-level file and dropped from its category — producing an archive that saved fine but failed to replay with CSWINRTWINMDGEN0003: Failed to parse argument 'ReferenceAssemblyPaths'. Category subfolders now win; the top-level entry is still validated by name.

As a side effect the interop generator's local CopyHashedFileToDirectory shrinks to just the reserved-.dll path validation and defers staging to the shared helper, which also removes the same latent skip bug from it.

Tests

Three new tests in src/Tests/WinMDGeneratorTest/Test_DebugRepro.cs, driving the real cswinrtwinmdgen end-to-end. Each saves an archive and replays it, so a repro that saves but cannot be replayed still fails. Each one fails on staging/3.0 with a distinct symptom:

Test Symptom without the fix
DebugRepro_IsSavedAndCanBeReplayed ArgumentException: Argument_AddingDuplicateWithKey
DebugRepro_WithRepeatedInputPaths_IsSavedAndCanBeReplayed ArgumentException: Argument_AddingDuplicateWithKey
DebugRepro_WithRepeatedReadOnlyInputPaths_IsSavedAndCanBeReplayed UnauthorizedAccessException on the staged copy

Reverting only the unpack-ordering change instead makes the first two fail with CSWINRTWINMDGEN0003 on replay, so that half is load-bearing too.

WinMDGeneratorTest has never been run in CI (it was added in #2443 and only ever built), so it is also wired into CsWinRT-Test-Steps.yml, x64-only, mirroring the source generator tests.

Validation

  • WinMDGeneratorTest: 17/17 pass with the fix, 14/17 without it (the 3 new ones fail).
  • Real end-to-end: building ObjectLifetimeTests.Lifted with -p:CsWinRTGeneratorDebugReproDirectory=<dir> fails on staging/3.0 and succeeds here, producing both interop-debug-repro.zip (21 MB) and projection-debug-repro.zip (16 MB).
  • Both of those archives were then replayed successfully through cswinrtinteropgen.exe / cswinrtprojectiongen.exe, each emitting its output assembly.

The CsWinRT build tools all package their inputs into a self-contained debug
repro '.zip' when '--debug-repro-directory' is set. Files are staged under a
name derived from a hash of their full original path, and the original path is
then recorded in a map keyed by that name.

MSBuild item lists are not deduplicated, and several targets contribute to the
same reference sets (e.g. '@(ReferencePathWithRefAssemblies)'), so the same path
can reach a generator more than once. That made 'originalPaths.Add' throw, and
the whole build failed with an unhandled 'CSWINRT*GEN9999' error during the
'save-debug-repro' phase. Staging is now deduplicated per destination folder
(the hashed name is path-derived, so a file already staged there came from the
exact same source), while the returned name list still mirrors the original
argument list.

Two related issues are fixed at the same time:

  - 'File.Copy' propagates the read-only attribute, so staging a read-only input
    (normal in enlistment-style builds) left a read-only copy behind: staging it
    twice failed with 'UnauthorizedAccessException', and the staging directory
    could not be deleted afterwards. The attribute is now cleared on each copy.
  - On unpack, the single top-level file (the input/output assembly) was matched
    by entry name before the category subfolder was checked. When the same file
    was staged both at the top level and inside a subfolder, the subfolder entry
    was mis-claimed and dropped from its category, producing an archive that
    saved but could not be replayed. Category subfolders now win.

The interop generator's local variant now only validates that a reserved .dll
was passed with a consistent path, and defers staging to the shared helper.

Adds regression tests to 'WinMDGeneratorTest' (each one fails without the fix)
and wires that project into CI, where it was not being run.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8e084ca2-e552-4f3c-82dd-cf30a5970335
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Split the reserved-.dll validation back into separate steps: return early for a
null input path, then compute the hashed name once, then check it. This keeps
the shape it had before and avoids the stacked conditions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8e084ca2-e552-4f3c-82dd-cf30a5970335
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant