spurious test failure: ui\panics\panic-short-backtrace-windows-x86_64.rs on x86_64-msvc-2

[matthiaskrgr]

The ui\panics\panic-short-backtrace-windows-x86_64.rs test seems to randomly fail on x86_64-msvc-2 sometimes.

https://pipelines.actions.githubusercontent.com/DhU3eyB11Ei4viT99vzh0CpKzg7REqx1Nup2tB9LzmxBcoHdbh/_apis/pipelines/1/runs/14339/signedlogcontent/31?urlExpires=2021-12-16T12%3A15%3A54.9212371Z&urlSigningMethod=HMACV1&urlSignature=%2FsTx%2F9cdlfikD4rjevyZM9XXK2s%2BqXIVMXzWYgCm%2BpI%3D

2021-12-16T11:34:35.6412816Z test [ui] ui\zero-sized\zero-size-type-destructors.rs ... ok
2021-12-16T11:34:35.6713965Z test [ui] ui\zero-sized\zero-sized-tuple-struct.rs ... ok
2021-12-16T11:34:35.7868498Z test [ui] ui\zero-sized\zero-sized-binary-heap-push.rs ... ok
2021-12-16T11:34:35.8159398Z test [ui] ui\zero-sized\zero-sized-linkedlist-push.rs ... ok
2021-12-16T11:34:36.5978797Z test [ui] ui\zero-sized\zero-sized-btreemap-insert.rs ... ok
2021-12-16T11:34:36.8111973Z test [ui] ui\wrong-hashset-issue-42918.rs ... ok
2021-12-16T11:34:36.8114707Z 
2021-12-16T11:34:36.8115216Z failures:
2021-12-16T11:34:36.8115506Z 
2021-12-16T11:34:36.8116378Z ---- [ui] ui\panics\panic-short-backtrace-windows-x86_64.rs stdout ----
2021-12-16T11:34:36.8117565Z $DIR\panic-short-backtrace-windows-x86_64.rs
2021-12-16T11:34:36.8118406Z diff of run.stderr:
2021-12-16T11:34:36.8118722Z 
2021-12-16T11:34:36.8119073Z 5	   2: c
2021-12-16T11:34:36.8119524Z 6	   3: b
2021-12-16T11:34:36.8119889Z 7	   4: a
2021-12-16T11:34:36.8120298Z -	   5: some Rust fn
2021-12-16T11:34:36.8121075Z 9	note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
2021-12-16T11:34:36.8122001Z 10	
2021-12-16T11:34:36.8122244Z 
2021-12-16T11:34:36.8122481Z 
2021-12-16T11:34:36.8123093Z The actual run.stderr differed from the expected run.stderr.
2021-12-16T11:34:36.8124799Z Actual run.stderr saved to D:\a\rust\rust\build\x86_64-pc-windows-msvc\test\ui\panics\panic-short-backtrace-windows-x86_64\panic-short-backtrace-windows-x86_64.run.stderr
2021-12-16T11:34:36.8126083Z 
2021-12-16T11:34:36.8126633Z error: 1 errors occurred comparing run output.
2021-12-16T11:34:36.8127238Z status: exit code: 101
2021-12-16T11:34:36.8142352Z command: PATH="D:\a\rust\rust\build\x86_64-pc-windows-msvc\stage2\lib\rustlib\x86_64-pc-windows-msvc\lib;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files (x86)\Windows Kits\10\bin\10.0.22000.0\x64;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64;D:\a\rust\rust\build\x86_64-pc-windows-msvc\stage0-bootstrap-tools\x86_64-pc-windows-msvc\release\deps;D:\a\rust\rust\build\x86_64-pc-windows-msvc\stage0\bin;D:\a\rust\rust\ninja;D:\a\rust\rust\msys2\mingw64\bin;C:\hostedtoolcache\windows\Python\3.10.0\x64\Scripts;C:\hostedtoolcache\windows\Python\3.10.0\x64;C:\msys64\usr\bin;D:\a\rust\rust\sccache;C:\Program Files\MongoDB\Server\5.0\bin;C:\aliyun-cli;C:\vcpkg;C:\cf-cli;C:\Program Files (x86)\NSIS;C:\tools\zstd;C:\Program Files\Mercurial;C:\hostedtoolcache\windows\stack\2.7.3\x64;C:\cabal\bin;C:\ghcup\bin;C:\tools\ghc-9.2.1\bin;C:\Program Files\dotnet;C:\mysql\bin;C:\Program Files\R\R-4.1.2\bin\x64;C:\SeleniumWebDrivers\GeckoDriver;C:\Program Files (x86)\sbt\bin;C:\Rust\.cargo\bin;C:\Program Files (x86)\GitHub CLI;C:\Program Files\Git\bin;C:\Program Files (x86)\pipx_bin;C:\hostedtoolcache\windows\go\1.15.15\x64\bin;C:\hostedtoolcache\windows\Python\3.7.9\x64\Scripts;C:\hostedtoolcache\windows\Python\3.7.9\x64;C:\hostedtoolcache\windows\Ruby\2.5.9\x64\bin;C:\tools\kotlinc\bin;C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.312-7\x64\bin;C:\npm\prefix;C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin;C:\ProgramData\kind;C:\Program Files\Eclipse Foundation\jdk-8.0.302.8-hotspot\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\ProgramData\Chocolatey\bin;C:\Program Files\Docker;C:\Program Files\PowerShell\7;C:\Program Files\Microsoft\Web Platform Installer;C:\Program Files\dotnet;C:\Program Files\Microsoft SQL Server\130\Tools\Binn;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn;C:\Program Files\nodejs;C:\Program Files\OpenSSL\bin;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\ProgramData\chocolatey\lib\pulumi\tools\Pulumi\bin;C:\Program Files\TortoiseSVN\bin;C:\Program Files\CMake\bin;C:\ProgramData\chocolatey\lib\maven\apache-maven-3.8.4\bin;C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;C:\Program Files\Git\cmd;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\tools\php;C:\Program Files (x86)\sbt\bin;C:\SeleniumWebDrivers\ChromeDriver;C:\SeleniumWebDrivers\EdgeDriver;C:\Program Files\Amazon\AWSCLIV2;C:\Program Files\Amazon\SessionManagerPlugin\bin;C:\Program Files\Amazon\AWSSAMCLI\bin;C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin;C:\Program Files (x86)\Microsoft BizTalk Server;C:\Program Files\LLVM\bin;C:\Users\runneradmin\.dotnet\tools;C:\Users\runneradmin\AppData\Local\Microsoft\WindowsApps" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\test\\ui\\panics\\panic-short-backtrace-windows-x86_64\\a.exe"
2021-12-16T11:34:36.8157035Z stdout:
2021-12-16T11:34:36.8157528Z ------------------------------------------
2021-12-16T11:34:36.8157943Z 
2021-12-16T11:34:36.8158403Z ------------------------------------------
2021-12-16T11:34:36.8158900Z stderr:
2021-12-16T11:34:36.8159387Z ------------------------------------------
2021-12-16T11:34:36.8160534Z thread 'main' panicked at 'd was called', D:\a\rust\rust\src/test\ui\panics\panic-short-backtrace-windows-x86_64.rs:48:5
2021-12-16T11:34:36.8161495Z stack backtrace:
2021-12-16T11:34:36.8162628Z    0: std::panicking::begin_panic
2021-12-16T11:34:36.8163069Z    1: d
2021-12-16T11:34:36.8163407Z    2: c
2021-12-16T11:34:36.8163739Z    3: b
2021-12-16T11:34:36.8164085Z    4: a
2021-12-16T11:34:36.8164744Z note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
2021-12-16T11:34:36.8165286Z 
2021-12-16T11:34:36.8165720Z ------------------------------------------
2021-12-16T11:34:36.8166060Z 
2021-12-16T11:34:36.8166280Z 
2021-12-16T11:34:36.8166496Z 
2021-12-16T11:34:36.8166864Z failures:
2021-12-16T11:34:36.8167600Z     [ui] ui\panics\panic-short-backtrace-windows-x86_64.rs
2021-12-16T11:34:36.8168457Z 
2021-12-16T11:34:36.8169168Z test result: FAILED. 12343 passed; 1 failed; 144 ignored; 0 measured; 0 filtered out; finished in 488.23s
2021-12-16T11:34:36.8169755Z 
2021-12-16T11:34:36.8170674Z Some tests failed in compiletest suite=ui mode=ui host=x86_64-pc-windows-msvc target=x86_64-pc-windows-msvc
2021-12-16T11:34:36.8212572Z 
2021-12-16T11:34:36.8212999Z 
2021-12-16T11:34:36.8229529Z command did not execute successfully: "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage0-tools-bin\\compiletest.exe" "--compile-lib-path" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage2\\bin" "--run-lib-path" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage2\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "--rustc-path" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage2\\bin\\rustc.exe" "--src-base" "D:\\a\\rust\\rust\\src/test\\ui" "--build-base" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\test\\ui" "--stage-id" "stage2-x86_64-pc-windows-msvc" "--suite" "ui" "--mode" "ui" "--target" "x86_64-pc-windows-msvc" "--host" "x86_64-pc-windows-msvc" "--llvm-filecheck" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\llvm\\build\\bin\\FileCheck.exe" "--nodejs" "C:\\Program Files\\nodejs\\node" "--npm" "C:\\Program Files\\nodejs\\npm" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0  -Lnative=D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\native\\rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0  -Lnative=D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\native\\rust-test-helpers" "--docck-python" "C:\\hostedtoolcache\\windows\\Python\\3.10.0\\x64\\python3.exe" "--lldb-python" "C:\\hostedtoolcache\\windows\\Python\\3.10.0\\x64\\python3.exe" "--gdb" "C:\\msys64\\usr\\bin\\gdb" "--llvm-version" "13.0.0-rust-1.59.0-nightly" "--llvm-components" "aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfogsym debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwp engine executionengine extensions filecheck frontendopenacc frontendopenmp fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interfacestub interpreter ipo irreader jitlink libdriver lineeditor linker lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts object objectyaml option orcjit orcshared orctargetprocess passes powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target textapi transformutils vectorize webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info xray" "--cc" "" "--cxx" "" "--cflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--channel" "nightly" "--color" "always"
2021-12-16T11:34:36.8246709Z expected success, got: exit code: 1
2021-12-16T11:34:36.8247081Z 
2021-12-16T11:34:36.8247310Z 
2021-12-16T11:34:36.8266331Z Build completed unsuccessfully in 0:38:37

[hkratz]

@rustbot label +O-windows

[camelid]

This is causing a lot of spurious failures, as evidenced by the numerous "mentioned this issue" items in this issue. Should this test be ignored temporarily until the underlying problem is identified and fixed? cc @rust-lang/infra

[mati865]

@rustbot modify labels: +O-windows-msvc

[wesleywiser]

I'm on PTO but I'll put this on my list of things to look into when I get back next week. In the mean time, disabling the test seems reasonable to me.

[camelid]

The test is now ignored, so there shouldn't be any more spurious failures. However, the underlying issue isn't fixed yet.

[Aaron1011]

The comment in the test seems potentially relevant:

rust/src/test/ui/panics/panic-short-backtrace-windows-x86_64.rs

Lines 17 to 20 in f1ce0e6

	// We need to normalize out frame 5 because without debug info, dbghelp.dll doesn't know where CGU
	// internal functions like `main` start or end and so it will return whatever symbol happens
	// to be located near the address.
	// normalize-stderr-test: "5: .*" -> "5: some Rust fn"

If dbghelp.dll is essentially returning an arbitrary symbol name when the test passes, perhaps it's (non-deterministically?) choosing to give up and not return any symbol in the cases where the test fails.

[hkratz]

We could modify the normalize-stderr-test regex replacement to just remove the whole "5: ..." line if it exists.