Skip to content

Commit

Permalink
[runtime] Add write barrier to end of JIT info table changes
Browse files Browse the repository at this point in the history
The MERP stress test often catches a crash like so:

```
 * frame #0: 0x0000000103409827 mono-sgen`mono_get_hazardous_pointer(pp=0x00007fc559a1a1f8, hp=0x00000001037e9600, hazard_index=1) at hazard-pointer.c:208
    frame mono#1: 0x000000010320eb8e mono-sgen`jit_info_table_chunk_index(chunk=0x00007fc4186060a0, hp=0x00000001037e9600, addr=0x00007fff73bffb66) at jit-info.c:200
    frame mono#2: 0x000000010320d7fa mono-sgen`jit_info_table_find(table=0x00007fc4186057c0, hp=0x00000001037e9600, addr=0x00007fff73bffb66) at jit-info.c:222
    frame mono#3: 0x000000010320d5e6 mono-sgen`mono_jit_info_table_find_internal(domain=0x00007fc418512bc0, addr=0x00007fff73bffb66, try_aot=1, allow_trampolines=1) at jit-info.c:293
    frame mono#4: 0x00000001030d67af mono-sgen`sigabrt_signal_handler(_dummy=6, _info=0x00007ffeecd4ea28, context=0x00007ffeecd4ea90) at mini-posix.c:219
    frame mono#5: 0x00007fff73dbdf5a libsystem_platform.dylib`_sigtramp + 26
    frame mono#6: 0x00007fff73bffb67 libsystem_kernel.dylib`__pthread_kill + 11
    frame mono#7: 0x00007fff73dca080 libsystem_pthread.dylib`pthread_kill + 333
    frame mono#8: 0x00007fff73b5b1ae libsystem_c.dylib`abort + 127
    frame mono#9: 0x00007fff73c59822 libsystem_malloc.dylib`free + 521
    frame mono#10: 0x0000000103c7d9d4 libtest.0.dylib`monoeg_g_free(ptr=0x00007ffeecd4ec34) at gmem.c:86
    frame mono#11: 0x0000000103c7733b libtest.0.dylib`mono_test_MerpCrashMalloc at libtest.c:7710
    frame mono#12: 0x0000000103c5c288
    frame mono#13: 0x000000010385dfd1
    frame mono#14: 0x0000000102ec85c3 mono-sgen`mono_jit_runtime_invoke(method=0x00007fc418514288, obj=0x0000000000000000, params=0x00007ffeecd4f180, exc=0x00007ffeecd4ef08, error=0x00007ffeecd4f250) at mini-runtime.c:3215
    frame mono#15: 0x000000010326f39d mono-sgen`do_runtime_invoke(method=0x00007fc418514288, obj=0x0000000000000000, params=0x00007ffeecd4f180, exc=0x0000000000000000, error=0x00007ffeecd4f250) at object.c:2977
    frame mono#16: 0x0000000103267c61 mono-sgen`mono_runtime_invoke_checked(method=0x00007fc418514288, obj=0x0000000000000000, params=0x00007ffeecd4f180, error=0x00007ffeecd4f250) at object.c:3145
    frame mono#17: 0x0000000103274d58 mono-sgen`do_exec_main_checked(method=0x00007fc418514288, args=0x00000001040003e8, error=0x00007ffeecd4f250) at object.c:5042
    frame mono#18: 0x0000000103272b03 mono-sgen`mono_runtime_exec_main_checked(method=0x00007fc418514288, args=0x00000001040003e8, error=0x00007ffeecd4f250) at object.c:5138
    frame mono#19: 0x0000000103272b56 mono-sgen`mono_runtime_run_main_checked(method=0x00007fc418514288, argc=2, argv=0x00007ffeecd4f760, error=0x00007ffeecd4f250) at object.c:4599
    frame mono#20: 0x0000000102f78bff mono-sgen`mono_jit_exec_internal(domain=0x00007fc418512bc0, assembly=0x00007fc4185445f0, argc=2, argv=0x00007ffeecd4f760) at driver.c:1298
    frame mono#21: 0x0000000102f78a2d mono-sgen`mono_jit_exec(domain=0x00007fc418512bc0, assembly=0x00007fc4185445f0, argc=2, argv=0x00007ffeecd4f760) at driver.c:1257
    frame mono#22: 0x0000000102f7d60f mono-sgen`main_thread_handler(user_data=0x00007ffeecd4f6a0) at driver.c:1375
    frame mono#23: 0x0000000102f7b8dd mono-sgen`mono_main(argc=3, argv=0x00007ffeecd4f758) at driver.c:2551
    frame mono#24: 0x0000000102eb1e4e mono-sgen`mono_main_with_options(argc=3, argv=0x00007ffeecd4f758) at main.c:50
    frame mono#25: 0x0000000102eb145d mono-sgen`main(argc=3, argv=0x00007ffeecd4f758) at main.c:406
    frame mono#26: 0x00007fff73aaf015 libdyld.dylib`start + 1
    frame mono#27: 0x00007fff73aaf015 libdyld.dylib`start + 1
```

because of

```
(lldb) p table->num_chunks
(int) $24 = 6
(lldb) p table->chunks [0]->num_elements
(volatile int) $18 = 50
(lldb) p table->chunks [1]->num_elements
(volatile int) $19 = 48
(lldb) p table->chunks [2]->num_elements
(volatile int) $20 = 48
(lldb) p table->chunks [3]->num_elements
(volatile int) $21 = 32
(lldb) p table->chunks [4]->num_elements
(volatile int) $23 = 32
(lldb) p table->chunks [5]->num_elements
(volatile int) $22 = 1347440720
```
.

It reproduces very often. Adding assertions / memory fences at the end
of the add/remove functions seems to remove this crash.

Looking at the code in the add function, I can picture how a missing
fence right at the end could lead to what we were seeing.

Not 100% happy with this fix, as it's going to have a perf cost. We have
an earlier fence that might be able to be coalesced into this one, but
the logic as-is makes that look unsafe.
  • Loading branch information
alexanderkyte committed Dec 27, 2018
1 parent 4f9af7a commit 78c1dde
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions mono/metadata/jit-info.c
Expand Up @@ -660,6 +660,7 @@ jit_info_table_add (MonoDomain *domain, MonoJitInfoTable *volatile *table_ptr, M

/* Debugging code, should be removed. */
//jit_info_table_check (table);
mono_memory_write_barrier ();
}

void
Expand Down Expand Up @@ -750,6 +751,7 @@ jit_info_table_remove (MonoJitInfoTable *table, MonoJitInfo *ji)

/* Debugging code, should be removed. */
//jit_info_table_check (table);
mono_memory_write_barrier ();
}

void
Expand Down

0 comments on commit 78c1dde

Please sign in to comment.