-
Notifications
You must be signed in to change notification settings - Fork 4.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Roslyn source generator crash on mono/linux/arm64 #81123
Comments
I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label. |
Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions Issue Details
|
I also hit this in an unrelated PR: #81070 The shared failure message seems to be:
It's affecting at least 3 Generator Roslyn tests:
Microsoft.Extensions.Logging.Generators.Roslyn3.11.TestsCallstack
System.Text.Json.SourceGeneration.Roslyn4.4.TestsCallstack
Microsoft.Extensions.Logging.Generators.Roslyn4.0.TestsCallstack
|
System.Text.RegularExpressions.Tests
native crash on mono/linux/arm64
This issue has heavy impact in many PRs. Looking for an area owner. |
This seems to happen in different suites, so disabling may turn into a whack-a-mole game. For example, I spotted one happening in System.Security.Cryptography tests. |
going to try to decode the crashed ip address in-process in the crash handler. #83219 |
I can reproduce a crash on osx with the Microsoft.Extensions.Logging.Abstractions/tests/Microsoft.Extensions.Logging.Generators.Tests/Microsoft.Extensions.Logging.Generators.Roslyn4.0.Tests.csproj |
Some of the failures appear to be races with delegate invocation.
This fails when ran in a loop on on an apple m1. |
So the problem seems to be related to the invoke trampolines which do: These trampolines are stored into delegate->invoke_impl by mono_delegate_trampoline (). Another thread makes an indirect call through delegate->invoke_impl and observes delegate->method_ptr being NULL. delegate->method_ptr is also set by mono_delegate_trampoline (), but even with a memory barrier between the stores, its not guaranteed that another thread will not observed the previous NULL value. |
We think The numbers on March 21 2023 before the change were:
|
Thanks @lambdageek and @vargaz. Once we confirm that fixes the issue, please consider backporting it to 7.0. We are seeing the failure happen there too. [Pending verifying benchmark results as @lambdageek told me offline]. |
#83688 is probably needed too. |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as outdated.
This comment was marked as outdated.
Most of the recent hits are either on the 7.0 branch, or are unrelated false positives (the query catches too many other native crashes) There is one hit from #84004 from System.Security.Cryptography.Tests.WorkItemExecution that seems real:
|
It still seems to crash, but the crashes are less frequent. |
The XML test lately failing are showing something like:
with reported native stack:
and managed stack:
These are fixed by #88892 |
System.Text.RegularExpressions.Tests
native crash onBuild Libraries Test Run release mono linux arm64 Debug
on unrelated #81066 .Build, and log:
Report
Summary
Known issue validation
Build: 🔎⚠️ Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: 🔎" line.
Result validation:
Validation performed at: 6/28/2023 10:04:05 PM UTC
The text was updated successfully, but these errors were encountered: