Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Mono drop crashing in C# symbol tests #13775

Closed
jaredpar opened this issue Apr 1, 2019 · 3 comments · Fixed by #13785

Comments

@jaredpar
Copy link
Contributor

@jaredpar jaredpar commented Apr 1, 2019

Steps to Reproduce

  1. Clone https://github.com/dotnet/roslyn
  2. Make sure mono is on your path
  3. Run git reset --hard be54c4a899c7d7bb6360f68243759f62c30e5408 (ensures your before we disabled the tests)
  4. Run ./build.sh --restore --build
  5. Run ./build.sh --test --testMono

The instability here caused us to temporarily disable C# symbol tests on Mono. Here is our tracking issue

dotnet/roslyn#34646

Current Behavior

Eventually this will crash with a SIGSEGV. May have to run it a few times. The actual test which causes the crash will vary from run to run.

Expected Behavior

Expectation is this test will succeed.

Mono Version

From our apt log

Setting up mono-devel (6.1.0.713-0nightly2+ubuntu1604b1

Stacktrace


=================================================================
	Native Crash Reporting
=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================
/proc/self/maps:
00400000-0087d000 r-xp 00000000 08:21 8522697                            /usr/bin/mono-sgen
00a7c000-00a83000 r--p 0047c000 08:21 8522697                            /usr/bin/mono-sgen
00a83000-00a89000 rw-p 00483000 08:21 8522697                            /usr/bin/mono-sgen
00a89000-00aa1000 rw-p 00000000 00:00 0 
01791000-070fd000 rw-p 00000000 00:00 0                                  [heap]
402e8000-402f8000 rwxp 00000000 00:00 0 
40443000-40506000 rwxp 00000000 00:00 0 
405b8000-405c8000 rwxp 00000000 00:00 0 
4091a000-4099a000 rwxp 00000000 00:00 0 
4099c000-40bbc000 rwxp 00000000 00:00 0 
40bc8000-40c68000 rwxp 00000000 00:00 0 
40d57000-40f2d000 rwxp 00000000 00:00 0 
40f38000-40fe8000 rwxp 00000000 00:00 0 
40ffe000-4100e000 rwxp 00000000 00:00 0 
41010000-410a0000 rwxp 00000000 00:00 0 
410bf000-411f2000 rwxp 00000000 00:00 0 
411fd000-4120d000 rwxp 00000000 00:00 0 
4152c000-4156c000 rwxp 00000000 00:00 0 
4174e000-4175e000 rwxp 00000000 00:00 0 
4177e000-4179e000 rwxp 00000000 00:00 0 
417ae000-4180e000 rwxp 00000000 00:00 0 
41864000-418f4000 rwxp 00000000 00:00 0 
41bae000-41c3e000 rwxp 00000000 00:00 0 
41ef4000-41f14000 rwxp 00000000 00:00 0 
41f2d000-41f3d000 rwxp 00000000 00:00 0 

=================================================================
	Native stacktrace:
=================================================================
	0x4bdb93 - mono : (null)
	0x4bdec1 - mono : (null)
	0x468ac1 - mono : (null)
	0x4b6b71 - mono : (null)
	0x615b61 - mono : mono_metadata_free_type
	0x7f96b4c00000 - Unknown

=================================================================
	Telemetry Dumper:
=================================================================
Pkilling 0x7f9669cea700 from 0x7f9626a72700
Pkilling 0x7f96698e8700 from 0x7f9626a72700
Pkilling 0x7f96694e6700 from 0x7f9626a72700
Pkilling 0x7f96690e4700 from 0x7f9626a72700
Pkilling 0x7f9698e72700 from 0x7f9626a72700
Pkilling 0x7f96b6db7780 from 0x7f9626a72700
Pkilling 0x7f9688253700 from 0x7f9626a72700
Pkilling 0x7f9692d22700 from 0x7f9626a72700
Pkilling 0x7f96ab9c4700 from 0x7f9626a72700
Pkilling 0x7f96ab5c2700 from 0x7f9626a72700
Pkilling 0x7f96b26f7700 from 0x7f9626a72700
Pkilling 0x7f969323b700 from 0x7f9626a72700
Pkilling 0x7f96889ff700 from 0x7f9626a72700
Pkilling 0x7f9669eeb700 from 0x7f9626a72700
Pkilling 0x7f966b9ff700 from 0x7f9626a72700
Pkilling 0x7f9699073700 from 0x7f9626a72700
Pkilling 0x7f9669ae9700 from 0x7f9626a72700
Pkilling 0x7f96696e7700 from 0x7f9626a72700
Pkilling 0x7f96692e5700 from 0x7f9626a72700
Pkilling 0x7f9639a73700 from 0x7f9626a72700
Pkilling 0x7f9688454700 from 0x7f9626a72700
Pkilling 0x7f9692f23700 from 0x7f9626a72700
Pkilling 0x7f96ab7c3700 from 0x7f9626a72700
Pkilling 0x7f96ab3c1700 from 0x7f9626a72700
Pkilling 0x7f96887fe700 from 0x7f9626a72700
Entering thread summarizer pause from 0x7f9626a72700
Finished thread summarizer pause from 0x7f9626a72700.
    Microsoft.CodeAnalysis.CSharp.UnitTests.Symbols.Metadata.PE.TypeForwarders.LookupMissingForwardedType [FINISHED] Time: 0.0628234s
    Microsoft.CodeAnalysis.CSharp.UnitTests.Symbols.Metadata.PE.TypeForwarders.ForwardToMissingAssembly [STARTING]
    Microsoft.CodeAnalysis.CSharp.UnitTests.Symbols.Metadata.PE.TypeForwarders.ForwardToMissingAssembly [FINISHED] Time: 0.1007015s

Waiting for dumping threads to resume
    Microsoft.CodeAnalysis.CSharp.UnitTests.Symbols.Metadata.PE.TypeForwarders.EmitForwarder_OpenGeneric [STARTING]
    Microsoft.CodeAnalysis.CSharp.UnitTests.Symbols.Metadata.PE.TypeForwarders.EmitForwarder_OpenGeneric [SKIP]
      Test supported only on CLR
    Microsoft.CodeAnalysis.CSharp.UnitTests.Symbols.Metadata.PE.TypeForwarders.EmitForwarder_OpenGeneric [FINISHED] Time: 0s
    Microsoft.CodeAnalysis.CSharp.UnitTests.Symbols.Metadata.PE.TypeForwarders.Test1 [STARTING]

=================================================================
	External Debugger Dump:
=================================================================
mono_gdb_render_native_backtraces not supported on this platform, unable to find gdb or lldb

=================================================================
	Basic Fault Adddress Reporting
=================================================================
Memory around native instruction pointer (0x615b61):0x615b51  a0 00 00 00 48 39 c3 48 0f 45 c2 5b c3 66 90 53  ....H9.H.E.[.f.S
0x615b61  f6 47 0b 01 48 89 fb 74 10 8b 47 10 85 c0 74 09  .G..H..t..G...t.
0x615b71  48 8b 7f 18 e8 a6 01 00 00 48 8d 05 1f f1 1b 00  H........H......
0x615b81  48 39 c3 72 12 48 8d 05 33 f3 1b 00 48 39 c3 73  H9.r.H..3...H9.s

=================================================================
	Managed Stacktrace:
=================================================================
=================================================================
/opt/code/eng/invoke-mono.sh: line 8:  3813 Aborted                 (core dumped) mono --debug "$@"
=== COMMAND LINE ===
"/opt/code/eng/invoke-mono.sh"  "/opt/code/.packages/xunit.runner.console/2.4.1-pre.build.4059/tools/net472/xunit.console.exe" "/opt/code/artifacts/bin/Microsoft.CodeAnalysis.CSharp.Symbol.UnitTests/Debug/net472/Microsoft.CodeAnalysis.CSharp.Symbol.UnitTests.dll" -noshadow -xml "/opt/code/artifacts/TestResults/Debug/Microsoft.CodeAnalysis.CSharp.Symbol.UnitTests_net472_x64.xml" -html "/opt/code/artifacts/TestResults/Debug/Microsoft.CodeAnalysis.CSharp.Symbol.UnitTests_net472_x64.html" -verbose > "/opt/code/artifacts/log/Debug/Microsoft.CodeAnalysis.CSharp.Symbol.UnitTests_net472_x64.log" 2>&1
@marek-safar

This comment has been minimized.

Copy link
Member

@marek-safar marek-safar commented Apr 1, 2019

@lambdageek this looks like recent regression

@marek-safar marek-safar added this to the 2019-04 (6.2.xx) milestone Apr 1, 2019
@lambdageek lambdageek self-assigned this Apr 1, 2019
@lambdageek

This comment has been minimized.

Copy link
Member

@lambdageek lambdageek commented Apr 1, 2019

Passed (although I only did a single run) on mono 6.1.0.769 (9ebf3a8).

Failed on 6.1.0.790 (acb8dd1).

Here's a part of a stack trace:
 thread #52, name = 'Domain unloader'
    frame #0: 0x00007fff5fb23356 libsystem_kernel.dylib`__wait4 + 10
    frame #1: 0x000000010aac8178 mono`mono_dump_native_crash_info at mini-posix.c:1111:3 [opt]
    frame #2: 0x000000010aac7efc mono`mono_dump_native_crash_info(signal="SIGABRT", mctx=, info=) at mini-posix.c:1153 [opt]
    frame #3: 0x000000010aa61405 mono`mono_handle_native_crash(signal="SIGABRT", mctx=0x000070001034f2f8, info=0x000070001034f8d8) at mini-exceptions.c:3324:2 [opt]
    frame #4: 0x000000010aac74b1 mono`sigabrt_signal_handler(_dummy=6, _info=0x000070001034f8d8, context=0x000070001034f940) at mini-posix.c:234:3 [opt]
    frame #5: 0x00007fff5fbceb5d libsystem_platform.dylib`_sigtramp + 29
    frame #6: 0x00007fff5fb242c7 libsystem_kernel.dylib`__pthread_kill + 11
    frame #7: 0x00007fff5fbd9bf1 libsystem_pthread.dylib`pthread_kill + 284
    frame #8: 0x00007fff5fa8e6a6 libsystem_c.dylib`abort + 127
    frame #9: 0x00007fff5fb9caef libsystem_malloc.dylib`malloc_vreport + 545
    frame #10: 0x00007fff5fb9c8b0 libsystem_malloc.dylib`malloc_report + 151
    frame #11: 0x000000010abafd79 mono`mono_metadata_free_type at metadata.c:3231:3 [opt]
    frame #12: 0x000000010abafd60 mono`mono_metadata_free_type(type=0x00007f8f60ef3e60) at metadata.c:4057 [opt]
    frame #13: 0x000000010abb84f3 mono`free_inflated_method [inlined] mono_metadata_free_inflated_signature(sig=0x00007f8f60eeecd0) at metadata.c:2369:3 [opt]
    frame #14: 0x000000010abb84e6 mono`free_inflated_method(imethod=0x00007f8f6163a240) at metadata.c:3194 [opt]
    frame #15: 0x000000010acc9f15 mono`monoeg_g_hash_table_foreach_remove(hash=, func=(mono`inflated_method_in_image at metadata.c:3059), user_data=0x00007f8f5e093c00) at ghashtable.c:450:6 [opt]
    frame #16: 0x000000010abb05b1 mono`mono_metadata_clean_for_image(image=) at metadata.c:3156:3 [opt]
    frame #17: 0x000000010ab935ed mono`mono_image_close_except_pools(image=0x00007f8f5e093c00) at image.c:2115:2 [opt]
    frame #18: 0x000000010ab45e68 mono`mono_assembly_close_except_image_pools(assembly=0x00007f8f611a8750) at assembly.c:4538:7 [opt]
    frame #19: 0x000000010ab3d90d mono`mono_domain_free(domain=0x00007f8f609f8460, force=) at domain.c:1161:8 [opt]
    frame #20: 0x000000010ab3b2e4 mono`unload_thread_main(arg=) at appdomain.c:2988:2 [opt]
    frame #21: 0x000000010abfdb23 mono`start_wrapper [inlined] start_wrapper_internal at threads.c:1220:3 [opt]
    frame #22: 0x000000010abfda93 mono`start_wrapper(data=0x00007f8f61d5ed80) at threads.c:1293 [opt]
    frame #23: 0x00007fff5fbd72eb libsystem_pthread.dylib`_pthread_body + 126
    frame #24: 0x00007fff5fbda249 libsystem_pthread.dylib`_pthread_start + 66
    frame #25: 0x00007fff5fbd640d libsystem_pthread.dylib`thread_start + 13
@lambdageek

This comment has been minimized.

Copy link
Member

@lambdageek lambdageek commented Apr 2, 2019

It's a double free when we're cleaning up the aggregate custom modifiers. #13320
Not sure why it only started failing recently - it should've been failing from day one.

lambdageek added a commit to lambdageek/mono that referenced this issue Apr 2, 2019
The aggregate modifiers container is owned by a MonoImageSet.  The aggregate
modifiers may be shared by more than one MonoType that happen to need the same
modifiers. (Similar to how generic instantiations are shared)

When the image set is cleaned up the aggregate modifier cleanup will free the
types in the aggregate modifiers container (the call to
free_aggregate_modifiers in mono_metadata_clean_for_image).

With this extra call to free_aggregate_modifiers in mono_metadata_free_type may
cause the same memory to be freed twice.

Fixes mono#13775
monojenkins added a commit to monojenkins/mono that referenced this issue Apr 2, 2019
The aggregate modifiers container is owned by a MonoImageSet.  The aggregate
modifiers may be shared by more than one MonoType that happen to need the same
modifiers. (Similar to how generic instantiations are shared)

When the image set is cleaned up the aggregate modifier cleanup will free the
types in the aggregate modifiers container (the call to
free_aggregate_modifiers in mono_metadata_clean_for_image).

With this extra call to free_aggregate_modifiers in mono_metadata_free_type may
cause the same memory to be freed twice.

Fixes mono#13775
lambdageek added a commit that referenced this issue Apr 5, 2019
…13785)

The aggregate modifiers container is owned by a MonoImageSet.  The aggregate
modifiers may be shared by more than one MonoType that happen to need the same
modifiers. (Similar to how generic instantiations are shared)

When the image set is cleaned up the aggregate modifier cleanup will free the
types in the aggregate modifiers container (the call to
free_aggregate_modifiers in mono_metadata_clean_for_image).

With this extra call to free_aggregate_modifiers in mono_metadata_free_type may
cause the same memory to be freed twice.

Fixes #13775
lambdageek added a commit that referenced this issue Apr 9, 2019
…13796)

The aggregate modifiers container is owned by a MonoImageSet.  The aggregate
modifiers may be shared by more than one MonoType that happen to need the same
modifiers. (Similar to how generic instantiations are shared)

When the image set is cleaned up the aggregate modifier cleanup will free the
types in the aggregate modifiers container (the call to
free_aggregate_modifiers in mono_metadata_clean_for_image).

With this extra call to free_aggregate_modifiers in mono_metadata_free_type may
cause the same memory to be freed twice.

Fixes #13775
alexanderkyte added a commit to alexanderkyte/mono that referenced this issue May 6, 2019
…ono#13785)

The aggregate modifiers container is owned by a MonoImageSet.  The aggregate
modifiers may be shared by more than one MonoType that happen to need the same
modifiers. (Similar to how generic instantiations are shared)

When the image set is cleaned up the aggregate modifier cleanup will free the
types in the aggregate modifiers container (the call to
free_aggregate_modifiers in mono_metadata_clean_for_image).

With this extra call to free_aggregate_modifiers in mono_metadata_free_type may
cause the same memory to be freed twice.

Fixes mono#13775
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.