Ensure compatibility with the new custom kokoro win2019 image #32649

jtattermusch · 2023-03-17T13:56:48Z

Fix incompatibilities identified when running adhoc runs on the new custom win2019 image.
After merging this, it should be possible to switch to the new image without breaking any tests.

for most fixes I added a comment that explains why they're necessary.
the new image won't have VS2015 installed, so I'm switching the protoc artifact build to VS2017

This PR will need to be backported to older release branches to ensure the windows tests continue working on those branches as well (IMHO I haven't made any changes that would be difficult to backport and I tried to keeps the diff as small as possible to avoid issues when backporting).

After we switch to the new image (and all the windows tests are green), we can incrementally move the builds that are still using VS2017 to VS2019.

jtattermusch · 2023-03-20T14:39:39Z

The ObjC tests and Android tests seem unrelated to this PR (and they happen on master as well).
I'm going to merge to get ready for switching our kokoro windows workers over to win2019 (see cl/517400022)

…2649) Fix incompatibilities identified when running adhoc runs on the new custom win2019 image. After merging this, it should be possible to switch to the new image without breaking any tests. - for most fixes I added a comment that explains why they're necessary. - the new image won't have VS2015 installed, so I'm switching the protoc artifact build to VS2017 This PR will need to be backported to older release branches to ensure the windows tests continue working on those branches as well (IMHO I haven't made any changes that would be difficult to backport and I tried to keeps the diff as small as possible to avoid issues when backporting). After we switch to the new image (and all the windows tests are green), we can incrementally move the builds that are still using VS2017 to VS2019.

…ge (#32649) to 1.53.x (#32660) Backports #32649.

(part of removing support for VS2017) Also see #32649 Also see #32615 The switch to grpc-win2019 windows workers has already happened: (cl/517400022). Once this PR lands, I'll backport to 1.53.x branch as well (since that release removes the VS2017 support).

(part of removing support for VS2017) Also see grpc#32649 Also see grpc#32615 The switch to grpc-win2019 windows workers has already happened: (cl/517400022). Once this PR lands, I'll backport to 1.53.x branch as well (since that release removes the VS2017 support).

Followup for #32649 (which disabled the tests mentioned below). Also sets correct path for tests build by ninja on windows, so that they don't get skipped. Once merged, I'll backport to 1.54.x and 1.53.x The original issue with tests being skipped. ``` + python3 workspace_c_windows_dbg_native/tools/run_tests/run_tests.py -t -j 8 -x run_tests/c_windows_dbg_native/sponge_log.xml --report_suite_name c_windows_dbg_native -l c -c dbg --iomgr_platform native --bq_result_table aggregate_results --measure_cpu_costs 2023-03-20 07:56:53,523 START: tools\run_tests\helper_scripts\build_cxx.bat 2023-03-20 08:04:51,388 PASSED: tools\run_tests\helper_scripts\build_cxx.bat [time=477.9sec, retries=0:0; cpu_cost=0.0; estimated=1.0] 2023-03-20 08:04:52,434 detected port server running version 21 2023-03-20 08:04:52,672 my port server is version 21 2023-03-20 08:04:52,703 SUCCESS: All tests passed WARNING: binary not found, skipping cmake/build/Debug/bad_server_response_test.exe WARNING: binary not found, skipping cmake/build/Debug/connection_refused_test.exe WARNING: binary not found, skipping cmake/build/Debug/goaway_server_test.exe WARNING: binary not found, skipping cmake/build/Debug/invalid_call_argument_test.exe WARNING: binary not found, skipping cmake/build/Debug/multiple_server_queues_test.exe WARNING: binary not found, skipping cmake/build/Debug/no_server_test.exe WARNING: binary not found, skipping cmake/build/Debug/pollset_windows_starvation_test.exe WARNING: binary not found, skipping cmake/build/Debug/public_headers_must_be_c89.exe ```

Followup for grpc#32649 (which disabled the tests mentioned below). Also sets correct path for tests build by ninja on windows, so that they don't get skipped. Once merged, I'll backport to 1.54.x and 1.53.x The original issue with tests being skipped. ``` + python3 workspace_c_windows_dbg_native/tools/run_tests/run_tests.py -t -j 8 -x run_tests/c_windows_dbg_native/sponge_log.xml --report_suite_name c_windows_dbg_native -l c -c dbg --iomgr_platform native --bq_result_table aggregate_results --measure_cpu_costs 2023-03-20 07:56:53,523 START: tools\run_tests\helper_scripts\build_cxx.bat 2023-03-20 08:04:51,388 PASSED: tools\run_tests\helper_scripts\build_cxx.bat [time=477.9sec, retries=0:0; cpu_cost=0.0; estimated=1.0] 2023-03-20 08:04:52,434 detected port server running version 21 2023-03-20 08:04:52,672 my port server is version 21 2023-03-20 08:04:52,703 SUCCESS: All tests passed WARNING: binary not found, skipping cmake/build/Debug/bad_server_response_test.exe WARNING: binary not found, skipping cmake/build/Debug/connection_refused_test.exe WARNING: binary not found, skipping cmake/build/Debug/goaway_server_test.exe WARNING: binary not found, skipping cmake/build/Debug/invalid_call_argument_test.exe WARNING: binary not found, skipping cmake/build/Debug/multiple_server_queues_test.exe WARNING: binary not found, skipping cmake/build/Debug/no_server_test.exe WARNING: binary not found, skipping cmake/build/Debug/pollset_windows_starvation_test.exe WARNING: binary not found, skipping cmake/build/Debug/public_headers_must_be_c89.exe ```

…2649) Fix incompatibilities identified when running adhoc runs on the new custom win2019 image. After merging this, it should be possible to switch to the new image without breaking any tests. - for most fixes I added a comment that explains why they're necessary. - the new image won't have VS2015 installed, so I'm switching the protoc artifact build to VS2017 This PR will need to be backported to older release branches to ensure the windows tests continue working on those branches as well (IMHO I haven't made any changes that would be difficult to backport and I tried to keeps the diff as small as possible to avoid issues when backporting). After we switch to the new image (and all the windows tests are green), we can incrementally move the builds that are still using VS2017 to VS2019.

(part of removing support for VS2017) Also see grpc#32649 Also see grpc#32615 The switch to grpc-win2019 windows workers has already happened: (cl/517400022). Once this PR lands, I'll backport to 1.53.x branch as well (since that release removes the VS2017 support).

Followup for grpc#32649 (which disabled the tests mentioned below). Also sets correct path for tests build by ninja on windows, so that they don't get skipped. Once merged, I'll backport to 1.54.x and 1.53.x The original issue with tests being skipped. ``` + python3 workspace_c_windows_dbg_native/tools/run_tests/run_tests.py -t -j 8 -x run_tests/c_windows_dbg_native/sponge_log.xml --report_suite_name c_windows_dbg_native -l c -c dbg --iomgr_platform native --bq_result_table aggregate_results --measure_cpu_costs 2023-03-20 07:56:53,523 START: tools\run_tests\helper_scripts\build_cxx.bat 2023-03-20 08:04:51,388 PASSED: tools\run_tests\helper_scripts\build_cxx.bat [time=477.9sec, retries=0:0; cpu_cost=0.0; estimated=1.0] 2023-03-20 08:04:52,434 detected port server running version 21 2023-03-20 08:04:52,672 my port server is version 21 2023-03-20 08:04:52,703 SUCCESS: All tests passed WARNING: binary not found, skipping cmake/build/Debug/bad_server_response_test.exe WARNING: binary not found, skipping cmake/build/Debug/connection_refused_test.exe WARNING: binary not found, skipping cmake/build/Debug/goaway_server_test.exe WARNING: binary not found, skipping cmake/build/Debug/invalid_call_argument_test.exe WARNING: binary not found, skipping cmake/build/Debug/multiple_server_queues_test.exe WARNING: binary not found, skipping cmake/build/Debug/no_server_test.exe WARNING: binary not found, skipping cmake/build/Debug/pollset_windows_starvation_test.exe WARNING: binary not found, skipping cmake/build/Debug/public_headers_must_be_c89.exe ```

…2649) Fix incompatibilities identified when running adhoc runs on the new custom win2019 image. After merging this, it should be possible to switch to the new image without breaking any tests. - for most fixes I added a comment that explains why they're necessary. - the new image won't have VS2015 installed, so I'm switching the protoc artifact build to VS2017 This PR will need to be backported to older release branches to ensure the windows tests continue working on those branches as well (IMHO I haven't made any changes that would be difficult to backport and I tried to keeps the diff as small as possible to avoid issues when backporting). After we switch to the new image (and all the windows tests are green), we can incrementally move the builds that are still using VS2017 to VS2019.

…2019 image (#32649) (#33052) Fix incompatibilities identified when running adhoc runs on the new custom win2019 image. After merging this, it should be possible to switch to the new image without breaking any tests. - for most fixes I added a comment that explains why they're necessary. - the new image won't have VS2015 installed, so I'm switching the protoc artifact build to VS2017 This PR will need to be backported to older release branches to ensure the windows tests continue working on those branches as well (IMHO I haven't made any changes that would be difficult to backport and I tried to keeps the diff as small as possible to avoid issues when backporting). After we switch to the new image (and all the windows tests are green), we can incrementally move the builds that are still using VS2017 to VS2019. --------- Co-authored-by: Jan Tattermusch <jtattermusch@google.com> Co-authored-by: Esun Kim <veblush@google.com>

Fix incompatibilities identified when running adhoc runs on the new custom win2019 image. After merging this, it should be possible to switch to the new image without breaking any tests. - for most fixes I added a comment that explains why they're necessary. - the new image won't have VS2015 installed, so I'm switching the protoc artifact build to VS2017 This PR will need to be backported to older release branches to ensure the windows tests continue working on those branches as well (IMHO I haven't made any changes that would be difficult to backport and I tried to keeps the diff as small as possible to avoid issues when backporting). After we switch to the new image (and all the windows tests are green), we can incrementally move the builds that are still using VS2017 to VS2019.

(part of removing support for VS2017) Also see #32649 Also see #32615 The switch to grpc-win2019 windows workers has already happened: (cl/517400022). Once this PR lands, I'll backport to 1.53.x branch as well (since that release removes the VS2017 support).

Followup for #32649 (which disabled the tests mentioned below). Also sets correct path for tests build by ninja on windows, so that they don't get skipped. Once merged, I'll backport to 1.54.x and 1.53.x The original issue with tests being skipped. ``` + python3 workspace_c_windows_dbg_native/tools/run_tests/run_tests.py -t -j 8 -x run_tests/c_windows_dbg_native/sponge_log.xml --report_suite_name c_windows_dbg_native -l c -c dbg --iomgr_platform native --bq_result_table aggregate_results --measure_cpu_costs 2023-03-20 07:56:53,523 START: tools\run_tests\helper_scripts\build_cxx.bat 2023-03-20 08:04:51,388 PASSED: tools\run_tests\helper_scripts\build_cxx.bat [time=477.9sec, retries=0:0; cpu_cost=0.0; estimated=1.0] 2023-03-20 08:04:52,434 detected port server running version 21 2023-03-20 08:04:52,672 my port server is version 21 2023-03-20 08:04:52,703 SUCCESS: All tests passed WARNING: binary not found, skipping cmake/build/Debug/bad_server_response_test.exe WARNING: binary not found, skipping cmake/build/Debug/connection_refused_test.exe WARNING: binary not found, skipping cmake/build/Debug/goaway_server_test.exe WARNING: binary not found, skipping cmake/build/Debug/invalid_call_argument_test.exe WARNING: binary not found, skipping cmake/build/Debug/multiple_server_queues_test.exe WARNING: binary not found, skipping cmake/build/Debug/no_server_test.exe WARNING: binary not found, skipping cmake/build/Debug/pollset_windows_starvation_test.exe WARNING: binary not found, skipping cmake/build/Debug/public_headers_must_be_c89.exe ```

jtattermusch requested a review from veblush March 17, 2023 13:56

github-actions bot added lang/C# lang/Python labels Mar 17, 2023

jtattermusch assigned veblush Mar 17, 2023

jtattermusch added the release notes: no Indicates if PR should not be in release notes label Mar 17, 2023

grpc-checks bot added bloat/none per-call-memory/neutral per-channel-memory/neutral labels Mar 17, 2023

veblush approved these changes Mar 17, 2023

View reviewed changes

jtattermusch added 10 commits March 20, 2023 10:51

use noninteractive mode in src/csharp/install_dotnet_sdk.ps1

a587cc1

switch protoc artifact build to VS2017

9ac1367

only set VS150COMNTOOLS conditionally

c7cd5d1

fix tools/internal_ci/helper_scripts/install_python_interpreters.ps1

97694fe

fixup install_dotnet_sdk.ps1

0731173

fix windows c++ cmake 3.16+ distribtest

518cb5c

print windows image ID

e6086b0

Explicitly use VS2017 for windows c++ distribtest

c93bdaa

workaround "dotnet restore" issue on windows workers

25f5914

skip a few run_tests.py C tests on windows

5675390

jtattermusch force-pushed the win2019_experiments branch from 2356ae0 to 5675390 Compare March 20, 2023 09:56

jtattermusch merged commit 5873ba9 into grpc:master Mar 20, 2023

This was referenced Mar 20, 2023

Backport: Ensure compatibility with the new custom kokoro win2019 image (#32649) to 1.53.x #32660

Merged

Switch all VS2017 builds to VS2019 #32662

Merged

copybara-service bot added the imported Specifies if the PR has been imported to the internal repository label Mar 20, 2023

jtattermusch added a commit that referenced this pull request Mar 21, 2023

Backport: Ensure compatibility with the new custom kokoro win2019 ima…

6e1ebe7

…ge (#32649) to 1.53.x (#32660) Backports #32649.

jtattermusch mentioned this pull request Apr 4, 2023

[test-infra] Re-enable skipped windows tests #32658

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ensure compatibility with the new custom kokoro win2019 image #32649

Ensure compatibility with the new custom kokoro win2019 image #32649

jtattermusch commented Mar 17, 2023

jtattermusch commented Mar 20, 2023

Ensure compatibility with the new custom kokoro win2019 image #32649

Ensure compatibility with the new custom kokoro win2019 image #32649

Conversation

jtattermusch commented Mar 17, 2023

jtattermusch commented Mar 20, 2023