Describe the issue
Test error in Windows GPU CI Pipeline
https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=1328998&view=logs&j=5076e696-f193-5f12-2d8a-703dda41a79b&t=350ee1d5-2fb9-512b-c937-52f129a93c2e&l=14034
It seems that the test expected NaN (It is odd). Please take a look whether the test is designed properly, or add a random seed to avoid random failure.
Urgency
No response
Target platform
windows 11
Build script
See Windows GPU CI Pipeline
Error / output
[ RUN ] CudaKernelTest.InvertibleLayerNormGrad_MidSizeTensor
1: D:\a_work\1\s\onnxruntime\test\providers\compare_provider_test_utils.cc(120): error: Expected equality of these values:
1: ret.first
1: Which is: 4-byte object <01-00 00-00>
1: COMPARE_RESULT::SUCCESS
1: Which is: 4-byte object <00-00 00-00>
1: expected -nan(ind) (ffc00000), got -1.22133 (bf9c5467), diff: nan, tol=nan idx=0. 491512 of 491520 differ
1: Google Test trace:
1: D:\a_work\1\s\onnxruntime\test/common/random_generator.h(49): ORT test random seed: 3553885692
1: Stack trace:
1: 00007FF67ABF38AB: onnxruntime::test::TestInvertibleLayerNormGrad
1: 00007FF67ABF0760: onnxruntime::test::CudaKernelTest_InvertibleLayerNormGrad_MidSizeTensor_Test::TestBody
1: 00007FF67B5D8C92: testing::internal::HandleSehExceptionsInMethodIfSupportedtesting::Test,void
1: 00007FF67B5D8B56: testing::internal::HandleExceptionsInMethodIfSupportedtesting::Test,void
1: 00007FF67B5FE152: testing::Test::Run
1: 00007FF67B5FE284: testing::TestInfo::Run
1: 00007FF67B5FE444: testing::TestSuite::Run
1: ... Google Test internal frames ...
Visual Studio Version
No response
GCC / Compiler Version
No response
Describe the issue
Test error in Windows GPU CI Pipeline
https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=1328998&view=logs&j=5076e696-f193-5f12-2d8a-703dda41a79b&t=350ee1d5-2fb9-512b-c937-52f129a93c2e&l=14034
It seems that the test expected NaN (It is odd). Please take a look whether the test is designed properly, or add a random seed to avoid random failure.
Urgency
No response
Target platform
windows 11
Build script
See Windows GPU CI Pipeline
Error / output
[ RUN ] CudaKernelTest.InvertibleLayerNormGrad_MidSizeTensor
1: D:\a_work\1\s\onnxruntime\test\providers\compare_provider_test_utils.cc(120): error: Expected equality of these values:
1: ret.first
1: Which is: 4-byte object <01-00 00-00>
1: COMPARE_RESULT::SUCCESS
1: Which is: 4-byte object <00-00 00-00>
1: expected -nan(ind) (ffc00000), got -1.22133 (bf9c5467), diff: nan, tol=nan idx=0. 491512 of 491520 differ
1: Google Test trace:
1: D:\a_work\1\s\onnxruntime\test/common/random_generator.h(49): ORT test random seed: 3553885692
1: Stack trace:
1: 00007FF67ABF38AB: onnxruntime::test::TestInvertibleLayerNormGrad
1: 00007FF67ABF0760: onnxruntime::test::CudaKernelTest_InvertibleLayerNormGrad_MidSizeTensor_Test::TestBody
1: 00007FF67B5D8C92: testing::internal::HandleSehExceptionsInMethodIfSupportedtesting::Test,void
1: 00007FF67B5D8B56: testing::internal::HandleExceptionsInMethodIfSupportedtesting::Test,void
1: 00007FF67B5FE152: testing::Test::Run
1: 00007FF67B5FE284: testing::TestInfo::Run
1: 00007FF67B5FE444: testing::TestSuite::Run
1: ... Google Test internal frames ...
Visual Studio Version
No response
GCC / Compiler Version
No response