illink - Task host node exited prematurely

[ViktorHofer]

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=427703
Build error leg or test failing: Build / CoreCLR Product Build windows x64 checked / Build managed product components and packages
Pull request: #90110

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "illink\\.targets\\(.*\\): error MSB4217: Task host node exited prematurely\\. Diagnostic information may be found in files in the temporary files directory named MSBuild_\\*\\.failure\\.txt",
  "BuildRetry": true,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=427703
Error message validated: illink\.targets\(.*\): error MSB4217: Task host node exited prematurely\. Diagnostic information may be found in files in the temporary files directory named MSBuild_\*\.failure\.txt
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 10/5/2023 9:42:24 PM UTC

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=425842
Build error leg or test failing: Build / CoreCLR Product Build windows x64 checked / Build managed product components and packages
Pull request: #92928

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "illink\\.targets\\(.*\\): error MSB4217: Task host node exited prematurely\\. Diagnostic information may be found in files in the temporary files directory named MSBuild_\\*\\.failure\\.txt",
  "BuildRetry": true,
  "ExcludeConsoleLog": false
}

Author:	ViktorHofer
Assignees:	-
Labels:	area-Infrastructure-coreclr, blocking-clean-ci, untriaged, Known Build Error
Milestone:	-

[ghost]

Tagging subscribers to this area: @agocke, @sbomer, @vitek-karas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=425842
Build error leg or test failing: Build / CoreCLR Product Build windows x64 checked / Build managed product components and packages
Pull request: #92928

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "illink\\.targets\\(.*\\): error MSB4217: Task host node exited prematurely\\. Diagnostic information may be found in files in the temporary files directory named MSBuild_\\*\\.failure\\.txt",
  "BuildRetry": true,
  "ExcludeConsoleLog": false
}

Author:	ViktorHofer
Assignees:	-
Labels:	blocking-clean-ci, untriaged, Known Build Error, area-Tools-ILLink
Milestone:	-

[MichalStrehovsky]

Possibly related to #92290.

[ViktorHofer]

#92290 is about child nodes exiting prematurely. This one is about the TaskHostFactory feature that we now use in illink.targets which has a slightly different error message.

@ericstj I can't get the errorPattern to match. Any idea whom to contact to fix that?

[MichalStrehovsky]

#92290 is about child nodes exiting prematurely

I read both as "msbuild weirdly crashes". We should keep both bugs open but whoever investigates should be aware of possible duplicated effort because both are weird msbuild crashes that showed up around the same time.

[ViktorHofer]

Agreed. I already reached out to @rainersigwald offline. He is aware but needs more information to debug the msbuild child node and task host issues.

[ericstj]

@riarenas can you help us figure out why build analysis isn't linking up this issue with build?

You can see the failure it's trying to match in attempt 1 of https://dev.azure.com/dnceng-public/public/_build/results?buildId=425842&view=logs&j=39a07da7-60a1-56aa-621d-b74991bb2f3d&s=6884a131-87da-5381-61f3-d7acc3b91d76&t=577d5284-495b-5810-e593-0c14b147fbab&l=89

[riarenas]

I think the error pattern is missing the "(NETCORE_ENGINEERING_TELEMETRY=Build)" piece of the build error. I can play around with the error pattern in a couple of hours.

[riarenas]

@ericstj the problem was actually that we don't examine older attempts when trying to provide the validation match. We updated the validation build to one where the problem is still active and that fixed up the linking. Do you think it's valuable to try to validate against older attempts? We could see if we can manage to look that deep if it's valuable.

[ericstj]

I think for these build failures it would be good to try to match previous attempts, since in these cases engineers will often rerun the build -- since it's not OK to merge with a build failure. I don't see it as a high pri fix though, but one to have on the backlog.

[danmoseley]

Diagnostic information may be found in files in the temporary files directory named MSBuild_*.failure.txt

Can we somehow redirect, or copy after the fact, files merging this pattern into either the log or into the artifacts folder that will suck them into helix so we can see the stack?

[ViktorHofer]

32 hits so far (since so far), like about 60 per month

cc @rainersigwald @rokonec

[rainersigwald]

Diagnostic information may be found in files in the temporary files directory named MSBuild_*.failure.txt

Can we somehow redirect, or copy after the fact, files merging this pattern into either the log or into the artifacts folder that will suck them into helix so we can see the stack?

Yes, the environment variable MSBUILDDEBUGPATH controls where we try to put those last-chance exception handler stack dumps. However, since we try to read them from the entrypoint node when the process that crashes is a worker node, I don't think that'd help in this case.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=427703
Build error leg or test failing: Build / CoreCLR Product Build windows x64 checked / Build managed product components and packages
Pull request: #90110

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "illink\\.targets\\(.*\\): error MSB4217: Task host node exited prematurely\\. Diagnostic information may be found in files in the temporary files directory named MSBuild_\\*\\.failure\\.txt",
  "BuildRetry": true,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=427703
Error message validated: illink\.targets\(.*\): error MSB4217: Task host node exited prematurely\. Diagnostic information may be found in files in the temporary files directory named MSBuild_\*\.failure\.txt
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 10/5/2023 9:42:24 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
443683	dotnet/runtime	Build product	Log
442838	dotnet/runtime	Build product	Log	#93611
442126	dotnet/runtime	Build managed product components and packages	Log
441883	dotnet/runtime	Restore and Build Product	Log	#93636
2293729	dotnet-runtime	Build managed CoreCLR components, all libraries, hosts, and packs	Log
441666	dotnet/runtime	Build product	Log	#93373
441549	dotnet/runtime	Build product	Log	#93368
440846	dotnet/runtime	Build managed product components and packages	Log	#93595
439954	dotnet/runtime	Build product	Log	#92881
439839	dotnet/runtime	Build product	Log	#93178
439488	dotnet/runtime	Build product	Log	#93547
439350	dotnet/runtime	Restore and Build Product	Log
439083	dotnet/runtime	Build managed product components and packages	Log
438685	dotnet/runtime	Build product	Log	#93369
438454	dotnet/runtime	Build managed product components and packages	Log	#93373
438449	dotnet/runtime	Restore and Build Product	Log	#93507
438315	dotnet/runtime	Build product	Log	#92881
436912	dotnet/runtime	Build product	Log	#93198
436808	dotnet/runtime	Build product	Log	#93417
436375	dotnet/runtime	Build product	Log	#93368
435451	dotnet/runtime	Restore and Build Product	Log	#93377
2288204	dotnet-runtime	Build Repository	Log
433407	dotnet/runtime	Restore and Build Product	Log	#93301
433235	dotnet/runtime	Restore and Build Product	Log	#93254
432776	dotnet/runtime	Build managed product components and packages	Log
430978	dotnet/runtime	Restore and Build Product	Log	#93176
430652	dotnet/runtime	Build product	Log	#93115
430326	dotnet/runtime	Build product	Log	#93124
428307	dotnet/runtime	Build product	Log	#92279
2284100	dotnet-runtime	Build product	Log
427703	dotnet/runtime	Restore and Build Product	Log
427324	dotnet/runtime	Build managed product components and packages	Log	#92772

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	32

Author:	ViktorHofer
Assignees:	-
Labels:	blocking-clean-ci, area-Infrastructure, untriaged, Known Build Error
Milestone:	-

[jeffschwMSFT]

removing blocking-clean-ci as it has not been hit in 30 days

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

[MichalStrehovsky]

I think we can close this. It has not been hit since October, not just the last 30 days. I don't see anything actionable.