Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Linux] Mono (5.18.0.268) crashes in String:IndexOf(char value, int startIndex) #13452

Closed
VirusFree opened this issue Mar 14, 2019 · 9 comments
Assignees

Comments

@VirusFree
Copy link

@VirusFree VirusFree commented Mar 14, 2019

Steps to Reproduce

  1. Run sample app (provided in attachment) -> mono ConsoleApp1.exe

Current Behavior

mono crashes and produces a mono_crash.json file (included in attachment)

Expected Behavior

not crash mono

On which platforms did you notice this

[ ] macOS
[X] Linux
[ ] Windows

Version Used:
Mono JIT compiler version 5.18.0.268 (tarball Tue Mar 12 14:15:15 UTC 2019)

Stacktrace

      at System.SpanHelpers:IndexOf <0x000fc>
      at System.String:IndexOf <0x00072>
      at System.String:IndexOf <0x0003a>
      at ConsoleApp1.Program:DoTheThing <0x00052>
      at ConsoleApp1.Program:Main <0x0002a>
      at <Module>:runtime_invoke_void_object <0x000d8>
  Aborted (core dumped)

Attachments

BugReport.zip

@VirusFree

This comment has been minimized.

Copy link
Author

@VirusFree VirusFree commented Mar 14, 2019

It took some time, but I was able to narrow down the problem using a small and simple sample code.
The source and binary are also in the zip file, but basically this is it :

    static void Main(string[] args)
    {
        var str = "this is a string";
        Console.WriteLine(DoTheThing(str, 's'));
    }
    public static string DoTheThing(string source, char c, int n = 1, int StartIndex = 0)
    {
        int idx = StartIndex - 1;
        while (n != 0)
        {
            idx = source.IndexOf(c, idx + 1);
            --n;
        }
        return null;
    }
@lewurm

This comment has been minimized.

Copy link
Member

@lewurm lewurm commented Mar 14, 2019

(1) What CPU are you running on?
(2) Could you post the full crash log?
(3) Did you install mono from our Debian reprository?
(4) Could you try to run your sample with MONO_ENV_OPTIONS=-O=-aot?

@VirusFree

This comment has been minimized.

Copy link
Author

@VirusFree VirusFree commented Mar 14, 2019

$ cat /etc/apt/sources.list.d/mono-official-stable.list
deb https://download.mono-project.com/repo/ubuntu stable-bionic main

CPU info

processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 63
model name : Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz
stepping : 2
microcode : 0x41
cpu MHz : 2400.099
cache size : 30720 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm cpuid_fault invpcid_single pti fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf
bogomips : 4800.13
clflush size : 64
cache_alignment : 64
address sizes : 46 bits physical, 48 bits virtual
power management:

Crash log


=================================================================
	Native Crash Reporting
=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

=================================================================
	Basic Fault Adddress Reporting
=================================================================
Memory around native instruction pointer (0x40041f0c):0x40041efc  23 f9 e9 62 00 00 00 48 8d 64 24 00 41 83 ef 04  #..b...H.d$.A...
0x40041f0c  0f b7 45 00 41 3b c6 0f 84 fc 02 00 00 0f b7 45  ..E.A;.........E
0x40041f1c  02 41 3b c6 0f 84 eb 02 00 00 b9 02 00 00 00 48  .A;............H
0x40041f2c  d1 e1 48 8b c5 48 03 c1 0f b7 00 41 3b c6 0f 84  ..H..H.....A;...

=================================================================
	Native stacktrace:
=================================================================
	0x557fa6be29a8 - mono : (null)
	0x557fa6b78284 - mono : (null)
	0x557fa6af97c8 - mono : (null)
	0x7f8aeab2b890 - /lib/x86_64-linux-gnu/libpthread.so.0 : (null)
	0x40041f0c - Unknown

=================================================================
	Telemetry Dumper:
=================================================================
Pkilling 0x7f8aea2bb700 from 0x7f8aeb6f9780
Entering thread summarizer pause from 0x7f8aeb6f9780
Finished thread summarizer pause from 0x7f8aeb6f9780.

Waiting for dumping threads to resume

=================================================================
	Managed Stacktrace:
=================================================================
	  at System.SpanHelpers:IndexOf <0x000fc>
	  at System.String:IndexOf <0x00072>
	  at System.String:IndexOf <0x0003a>
	  at ConsoleApp1.Program:DoTheThing <0x00052>
	  at ConsoleApp1.Program:Main <0x0002a>
	  at <Module>:runtime_invoke_void_object <0x000d8>
=================================================================

@lewurm

This comment has been minimized.

Copy link
Member

@lewurm lewurm commented Mar 14, 2019

I poked around with a similar CPU:

$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 85
model name      : Intel(R) Xeon(R) W-2140B CPU @ 3.20GHz
stepping        : 4
microcode       : 0x200004d
cpu MHz         : 3191.615
cache size      : 11264 KB
physical id     : 0
siblings        : 3
core id         : 0
cpu cores       : 3
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xsaves arat flush_l1d arch_capabilities
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf
bogomips        : 6383.23
clflush size    : 64
cache_alignment : 64
address sizes   : 43 bits physical, 48 bits virtual

but I couldn't reproduce your issue 😕 do you have a gdb or lldb installed? if not, please install one of it, then our crash reporting subsystem should give a more detailed description where the crash happens.

Also could you try to run your repro with MONO_ENV_OPTIONS=-O=-aot?

@VirusFree

This comment has been minimized.

Copy link
Author

@VirusFree VirusFree commented Mar 14, 2019

if it helps, this is a t2.micro instance in AWS using ubuntu-bionic-18.04-amd64-server-20190212.1 (ami-005bdb005fb00e791), apt-get upgrade everything and installed mono-complete

@VirusFree

This comment has been minimized.

Copy link
Author

@VirusFree VirusFree commented Mar 14, 2019

with installed gdb (also attached mono_crash.json)

=================================================================
        Native Crash Reporting
=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================
/proc/self/maps:
400a9000-400b9000 rwxp 00000000 00:00 0
41a34000-41a44000 rwxp 00000000 00:00 0
556490e91000-5564912dd000 r-xp 00000000 ca:01 55495                      /usr/bin/mono-sgen
5564914dc000-5564914e3000 r--p 0044b000 ca:01 55495                      /usr/bin/mono-sgen
5564914e3000-5564914e8000 rw-p 00452000 ca:01 55495                      /usr/bin/mono-sgen
5564914e8000-5564914ff000 rw-p 00000000 00:00 0
556491d33000-556491e3c000 rw-p 00000000 00:00 0                          [heap]
7f3de8000000-7f3de8021000 rw-p 00000000 00:00 0
7f3de8021000-7f3dec000000 ---p 00000000 00:00 0
7f3decda7000-7f3decda8000 ---p 00000000 00:00 0
7f3decda8000-7f3decda9000 rw-p 00000000 00:00 0
7f3decda9000-7f3decdb1000 ---p 00000000 00:00 0
7f3decdb1000-7f3decfa8000 rw-p 00000000 00:00 0
7f3decfa8000-7f3ded3ff000 r--p 00000000 ca:01 274338                     /usr/lib/mono/4.5/mscorlib.dll
7f3ded3ff000-7f3dee3ff000 rw-p 00000000 00:00 0
7f3dee3ff000-7f3dee400000 ---p 00000000 00:00 0
7f3dee400000-7f3def000000 rw-p 00000000 00:00 0
7f3def168000-7f3def1e8000 rw-p 00000000 00:00 0
7f3def1ec000-7f3def1ed000 rw-p 00000000 00:00 0
7f3def1ed000-7f3def24c000 ---p 00000000 00:00 0
7f3def24c000-7f3def3bf000 r--p 00000000 ca:01 7807                       /usr/lib/locale/C.UTF-8/LC_COLLATE
7f3def3bf000-7f3def5a6000 r-xp 00000000 ca:01 2079                       /lib/x86_64-linux-gnu/libc-2.27.so
7f3def5a6000-7f3def7a6000 ---p 001e7000 ca:01 2079                       /lib/x86_64-linux-gnu/libc-2.27.so
7f3def7a6000-7f3def7aa000 r--p 001e7000 ca:01 2079                       /lib/x86_64-linux-gnu/libc-2.27.so
7f3def7aa000-7f3def7ac000 rw-p 001eb000 ca:01 2079                       /lib/x86_64-linux-gnu/libc-2.27.so

=================================================================
        Basic Fault Adddress Reporting
=================================================================
Memory around native instruction pointer (0x400aaf0c):0x400aaefc  23 f9 e9 62 00 00 00 48 8d 64 24 00 41 83 ef 04  #..b...H.d$.A...
0x400aaf0c  0f b7 45 00 41 3b c6 0f 84 fc 02 00 00 0f b7 45  ..E.A;.........E
0x400aaf1c  02 41 3b c6 0f 84 eb 02 00 00 b9 02 00 00 00 48  .A;............H
0x400aaf2c  d1 e1 48 8b c5 48 03 c1 0f b7 00 41 3b c6 0f 84  ..H..H.....A;...

=================================================================
        Native stacktrace:
=================================================================
        0x556490fbb9a8 - mono : (null)
        0x556490f51284 - mono : (null)
        0x556490ed27c8 - mono : (null)
        0x7f3def9da890 - /lib/x86_64-linux-gnu/libpthread.so.0 : (null)
        0x400aaf0c - Unknown

=================================================================
        Telemetry Dumper:
=================================================================
Pkilling 0x7f3decfa7700 from 0x7f3df05a8780
Entering thread summarizer pause from 0x7f3df05a8780
Finished thread summarizer pause from 0x7f3df05a8780.

Waiting for dumping threads to resume

Debug info from gdb:


=================================================================
        External Debugger Dump:
=================================================================
[New LWP 6648]
[New LWP 6649]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f3def9da23a in __waitpid (pid=6651, stat_loc=0x7f3df03c3d04, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
30      ../sysdeps/unix/sysv/linux/waitpid.c: No such file or directory.
  Id   Target Id         Frame
* 1    Thread 0x7f3df05a8780 (LWP 6647) "mono" 0x00007f3def9da23a in __waitpid (pid=6651, stat_loc=0x7f3df03c3d04, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
  2    Thread 0x7f3deebff700 (LWP 6648) "SGen worker" 0x00007f3def9d59f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x5564914fcbc8) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
  3    Thread 0x7f3decfa7700 (LWP 6649) "Finalizer" 0x00007f3def9d86d6 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x5564914edd80) at ../sysdeps/unix/sysv/linux/futex-internal.h:205

Thread 3 (Thread 0x7f3decfa7700 (LWP 6649)):
#0  0x00007f3def9d86d6 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x5564914edd80) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  do_futex_wait (sem=sem@entry=0x5564914edd80, abstime=0x0) at sem_waitcommon.c:111
#2  0x00007f3def9d87c8 in __new_sem_wait_slow (sem=0x5564914edd80, abstime=0x0) at sem_waitcommon.c:181
#3  0x0000556491122758 in ?? ()
#4  0x00005564910d858b in ?? ()
#5  0x00007f3def9cf6db in start_thread (arg=0x7f3decfa7700) at pthread_create.c:463
#6  0x00007f3def4e088f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 2 (Thread 0x7f3deebff700 (LWP 6648)):
#0  0x00007f3def9d59f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x5564914fcbc8) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x5564914fcbe0, cond=0x5564914fcba0) at pthread_cond_wait.c:502
#2  __pthread_cond_wait (cond=0x5564914fcba0, mutex=0x5564914fcbe0) at pthread_cond_wait.c:655
#3  0x00005564911838fa in ?? ()
#4  0x00007f3def9cf6db in start_thread (arg=0x7f3deebff700) at pthread_create.c:463
#5  0x00007f3def4e088f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7f3df05a8780 (LWP 6647)):
#0  0x00007f3def9da23a in __waitpid (pid=6651, stat_loc=0x7f3df03c3d04, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
#1  0x0000556490fbbbf8 in ?? ()
#2  0x0000556490f51284 in ?? ()
#3  0x0000556490ed27c8 in ?? ()
#4  <signal handler called>
#5  0x00000000400aaf0c in ?? ()
#6  0x0000000000000001 in ?? ()
#7  0x00007ffc76e65910 in ?? ()
#8  0x00007f3def1c4130 in ?? ()
#9  0x0000000100000000 in ?? ()
#10 0x0000000000000010 in ?? ()
#11 0xff413e8b4900eb00 in ?? ()
#12 0x60ec8348ec8b4855 in ?? ()
#13 0xf075894ce86d894c in ?? ()
#14 0xbb490024648d48fd in ?? ()
#15 0x0000000000000000 in ?? ()

=================================================================
        Managed Stacktrace:
=================================================================
          at System.SpanHelpers:IndexOf <0x000fc>
          at System.String:IndexOf <0x00072>
          at System.String:IndexOf <0x0003a>
          at ConsoleApp1.Program:DoTheThing <0x00052>
          at ConsoleApp1.Program:Main <0x0002a>
          at <Module>:runtime_invoke_void_object <0x000d8>
=================================================================
Aborted (core dumped)

mono_crash.4f9d86d8a.0.txt

@VirusFree

This comment has been minimized.

Copy link
Author

@VirusFree VirusFree commented Mar 14, 2019

$ MONO_ENV_OPTIONS=-O=-aot
$ echo $MONO_ENV_OPTIONS
-O=-aot
$ mono ConsoleApp1.exe

=================================================================
        Native Crash Reporting
=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================
/proc/self/maps:
402a3000-402b3000 rwxp 00000000 00:00 0
402da000-402ea000 rwxp 00000000 00:00 0
55cd4b5e1000-55cd4ba2d000 r-xp 00000000 ca:01 55495                      /usr/bin/mono-sgen
55cd4bc2c000-55cd4bc33000 r--p 0044b000 ca:01 55495                      /usr/bin/mono-sgen
55cd4bc33000-55cd4bc38000 rw-p 00452000 ca:01 55495                      /usr/bin/mono-sgen
55cd4bc38000-55cd4bc4f000 rw-p 00000000 00:00 0
55cd4d9e5000-55cd4daee000 rw-p 00000000 00:00 0                          [heap]
7f2a98000000-7f2a98021000 rw-p 00000000 00:00 0
7f2a98021000-7f2a9c000000 ---p 00000000 00:00 0
7f2a9d3a8000-7f2a9d7ff000 r--p 00000000 ca:01 274338                     /usr/lib/mono/4.5/mscorlib.dll
7f2a9d7ff000-7f2a9e7ff000 rw-p 00000000 00:00 0
7f2a9e7ff000-7f2a9e800000 ---p 00000000 00:00 0
7f2a9e800000-7f2a9f400000 rw-p 00000000 00:00 0
7f2a9f42b000-7f2a9f42c000 ---p 00000000 00:00 0
7f2a9f42c000-7f2a9f42d000 rw-p 00000000 00:00 0
7f2a9f42d000-7f2a9f435000 ---p 00000000 00:00 0
7f2a9f435000-7f2a9f6ac000 rw-p 00000000 00:00 0
7f2a9f6b0000-7f2a9f6b1000 rw-p 00000000 00:00 0
7f2a9f6b1000-7f2a9f710000 ---p 00000000 00:00 0
7f2a9f710000-7f2a9f883000 r--p 00000000 ca:01 7807                       /usr/lib/locale/C.UTF-8/LC_COLLATE
7f2a9f883000-7f2a9fa6a000 r-xp 00000000 ca:01 2079                       /lib/x86_64-linux-gnu/libc-2.27.so
7f2a9fa6a000-7f2a9fc6a000 ---p 001e7000 ca:01 2079                       /lib/x86_64-linux-gnu/libc-2.27.so
7f2a9fc6a000-7f2a9fc6e000 r--p 001e7000 ca:01 2079                       /lib/x86_64-linux-gnu/libc-2.27.so
7f2a9fc6e000-7f2a9fc70000 rw-p 001eb000 ca:01 2079                       /lib/x86_64-linux-gnu/libc-2.27.so
7f2a9fc70000-7f2a9fc74000 rw-p 00000000 00:00 0

=================================================================
        Basic Fault Adddress Reporting
=================================================================
Memory around native instruction pointer (0x402dbf0c):0x402dbefc  23 f9 e9 62 00 00 00 48 8d 64 24 00 41 83 ef 04  #..b...H.d$.A...
0x402dbf0c  0f b7 45 00 41 3b c6 0f 84 fc 02 00 00 0f b7 45  ..E.A;.........E
0x402dbf1c  02 41 3b c6 0f 84 eb 02 00 00 b9 02 00 00 00 48  .A;............H
0x402dbf2c  d1 e1 48 8b c5 48 03 c1 0f b7 00 41 3b c6 0f 84  ..H..H.....A;...

=================================================================
        Native stacktrace:
=================================================================
        0x55cd4b70b9a8 - mono : (null)
        0x55cd4b6a1284 - mono : (null)
        0x55cd4b6227c8 - mono : (null)
        0x7f2a9fe9e890 - /lib/x86_64-linux-gnu/libpthread.so.0 : (null)
        0x402dbf0c - Unknown

=================================================================
        Telemetry Dumper:
=================================================================
Pkilling 0x7f2a9f62b700 from 0x7f2aa0a6c780
Entering thread summarizer pause from 0x7f2aa0a6c780
Finished thread summarizer pause from 0x7f2aa0a6c780.

Waiting for dumping threads to resume

Debug info from gdb:


=================================================================
        External Debugger Dump:
=================================================================
[New LWP 6661]
[New LWP 6662]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f2a9fe9e23a in __waitpid (pid=6664, stat_loc=0x7f2aa0887d04, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
30      ../sysdeps/unix/sysv/linux/waitpid.c: No such file or directory.
  Id   Target Id         Frame
* 1    Thread 0x7f2aa0a6c780 (LWP 6660) "mono" 0x00007f2a9fe9e23a in __waitpid (pid=6664, stat_loc=0x7f2aa0887d04, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
  2    Thread 0x7f2a9efff700 (LWP 6661) "SGen worker" 0x00007f2a9fe999f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55cd4bc4cbc8) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
  3    Thread 0x7f2a9f62b700 (LWP 6662) "Finalizer" 0x00007f2a9fe9c6d6 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x55cd4bc3dd80) at ../sysdeps/unix/sysv/linux/futex-internal.h:205

Thread 3 (Thread 0x7f2a9f62b700 (LWP 6662)):
#0  0x00007f2a9fe9c6d6 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x55cd4bc3dd80) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  do_futex_wait (sem=sem@entry=0x55cd4bc3dd80, abstime=0x0) at sem_waitcommon.c:111
#2  0x00007f2a9fe9c7c8 in __new_sem_wait_slow (sem=0x55cd4bc3dd80, abstime=0x0) at sem_waitcommon.c:181
#3  0x000055cd4b872758 in ?? ()
#4  0x000055cd4b82858b in ?? ()
#5  0x00007f2a9fe936db in start_thread (arg=0x7f2a9f62b700) at pthread_create.c:463
#6  0x00007f2a9f9a488f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 2 (Thread 0x7f2a9efff700 (LWP 6661)):
#0  0x00007f2a9fe999f3 in futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x55cd4bc4cbc8) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55cd4bc4cbe0, cond=0x55cd4bc4cba0) at pthread_cond_wait.c:502
#2  __pthread_cond_wait (cond=0x55cd4bc4cba0, mutex=0x55cd4bc4cbe0) at pthread_cond_wait.c:655
#3  0x000055cd4b8d38fa in ?? ()
#4  0x00007f2a9fe936db in start_thread (arg=0x7f2a9efff700) at pthread_create.c:463
#5  0x00007f2a9f9a488f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7f2aa0a6c780 (LWP 6660)):
#0  0x00007f2a9fe9e23a in __waitpid (pid=6664, stat_loc=0x7f2aa0887d04, options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
#1  0x000055cd4b70bbf8 in ?? ()
#2  0x000055cd4b6a1284 in ?? ()
#3  0x000055cd4b6227c8 in ?? ()
#4  <signal handler called>
#5  0x00000000402dbf0c in ?? ()
#6  0x0000000000000001 in ?? ()
#7  0x00007fff36df5fc0 in ?? ()
#8  0x00007f2a9f688130 in ?? ()
#9  0x0000000100000000 in ?? ()
#10 0x0000000000000010 in ?? ()
#11 0xff413e8b4900eb00 in ?? ()
#12 0x60ec8348ec8b4855 in ?? ()
#13 0xf075894ce86d894c in ?? ()
#14 0xbb490024648d48fd in ?? ()
#15 0x0000000000000000 in ?? ()

=================================================================
        Managed Stacktrace:
=================================================================
          at System.SpanHelpers:IndexOf <0x000fc>
          at System.String:IndexOf <0x00072>
          at System.String:IndexOf <0x0003a>
          at ConsoleApp1.Program:DoTheThing <0x00052>
          at ConsoleApp1.Program:Main <0x0002a>
          at <Module>:runtime_invoke_void_object <0x000d8>
=================================================================
Aborted (core dumped)
@lewurm

This comment has been minimized.

Copy link
Member

@lewurm lewurm commented Mar 22, 2019

I can reproduce on a t2.micro instance on AWS, investigating now.

lewurm added a commit to lewurm/mono that referenced this issue Mar 22, 2019
We do 64bit operations, but operate on 32bit values. We need to sanitize those values.

Consider:
```
(lldb) x/8i 0x40011357
    0x40011357: 0f 84 c7 00 00 00  je     0x40011424
    0x4001135d: 49 8d 7c 24 14     leaq   0x14(%r12), %rdi
->  0x40011362: 49 8b c6           movq   %r14, %rax
    0x40011365: 48 d1 e0           shlq   %rax
    0x40011368: 48 03 f8           addq   %rax, %rdi
    0x4001136b: 0f b7 74 24 20     movzwl 0x20(%rsp), %esi
    0x40011370: 49 8b d7           movq   %r15, %rdx
    0x40011373: e8 bc 00 00 00     callq  0x40011434
(lldb) register read r14 rax rdi
     r14 = 0x0000000100000000
     rax = 0x0000000000000010
     rdi = 0x00007ffff7f48144
(lldb) si
(lldb) si
(lldb) si
(lldb) register read r14 rax rdi
     r14 = 0x0000000100000000
     rax = 0x0000000200000000
     rdi = 0x00008001f7f48144
```
In this snippet `r14` has some garbage in the 32bit upper half, which unfortunately is taken into account for the multiplication (`shl`) and addition in order to compute the address (`rdi`).

Several factors why we didn't see it earlier:
1. It doesn't trigger on AOT, because when compiling mscorlib.dll, `Unsafe.dll` isn't available to the AOT compiler and thus inlining won't happen. We use AOT where possible in our CI. When doing a call the upper halfs are properly cleared.
2. It doesn't trigger on macOS due to different memory organization.  We don't end up with gargabe in the registers.
3. Recently we added intrinsics for Unsafe.* https://github.com/mono/mono/pull/12605/files#diff-89fe30e1e87c5db21be052ddb0c8e28dR329 It masks this specific problem. Funnily enough, the assumed IL for the `Add ()` intrinsics didn't match what we had before. Now it does :-)

Fixes mono#13452 and mono#13597
monojenkins added a commit to monojenkins/mono that referenced this issue Mar 22, 2019
We do 64bit operations, but operate on 32bit values. We need to sanitize those values.

Consider:
```
(lldb) x/8i 0x40011357
    0x40011357: 0f 84 c7 00 00 00  je     0x40011424
    0x4001135d: 49 8d 7c 24 14     leaq   0x14(%r12), %rdi
->  0x40011362: 49 8b c6           movq   %r14, %rax
    0x40011365: 48 d1 e0           shlq   %rax
    0x40011368: 48 03 f8           addq   %rax, %rdi
    0x4001136b: 0f b7 74 24 20     movzwl 0x20(%rsp), %esi
    0x40011370: 49 8b d7           movq   %r15, %rdx
    0x40011373: e8 bc 00 00 00     callq  0x40011434
(lldb) register read r14 rax rdi
     r14 = 0x0000000100000000
     rax = 0x0000000000000010
     rdi = 0x00007ffff7f48144
(lldb) si
(lldb) si
(lldb) si
(lldb) register read r14 rax rdi
     r14 = 0x0000000100000000
     rax = 0x0000000200000000
     rdi = 0x00008001f7f48144
```
In this snippet `r14` has some garbage in the 32bit upper half, which unfortunately is taken into account for the multiplication (`shl`) and addition in order to compute the address (`rdi`).

Several factors why we didn't see it earlier:
1. It doesn't trigger on AOT, because when compiling mscorlib.dll, `Unsafe.dll` isn't available to the AOT compiler and thus inlining won't happen. We use AOT where possible in our CI. When doing a call the upper halfs are properly cleared.
2. It doesn't trigger on macOS due to different memory organization.  We don't end up with gargabe in the registers.
3. Recently we added intrinsics for Unsafe.* https://github.com/mono/mono/pull/12605/files#diff-89fe30e1e87c5db21be052ddb0c8e28dR329 It masks this specific problem. Funnily enough, the assumed IL for the `Add ()` intrinsics didn't match what we had before. Now it does :-)

Fixes mono#13452 and mono#13597
monojenkins added a commit to monojenkins/mono that referenced this issue Mar 22, 2019
We do 64bit operations, but operate on 32bit values. We need to sanitize those values.

Consider:
```
(lldb) x/8i 0x40011357
    0x40011357: 0f 84 c7 00 00 00  je     0x40011424
    0x4001135d: 49 8d 7c 24 14     leaq   0x14(%r12), %rdi
->  0x40011362: 49 8b c6           movq   %r14, %rax
    0x40011365: 48 d1 e0           shlq   %rax
    0x40011368: 48 03 f8           addq   %rax, %rdi
    0x4001136b: 0f b7 74 24 20     movzwl 0x20(%rsp), %esi
    0x40011370: 49 8b d7           movq   %r15, %rdx
    0x40011373: e8 bc 00 00 00     callq  0x40011434
(lldb) register read r14 rax rdi
     r14 = 0x0000000100000000
     rax = 0x0000000000000010
     rdi = 0x00007ffff7f48144
(lldb) si
(lldb) si
(lldb) si
(lldb) register read r14 rax rdi
     r14 = 0x0000000100000000
     rax = 0x0000000200000000
     rdi = 0x00008001f7f48144
```
In this snippet `r14` has some garbage in the 32bit upper half, which unfortunately is taken into account for the multiplication (`shl`) and addition in order to compute the address (`rdi`).

Several factors why we didn't see it earlier:
1. It doesn't trigger on AOT, because when compiling mscorlib.dll, `Unsafe.dll` isn't available to the AOT compiler and thus inlining won't happen. We use AOT where possible in our CI. When doing a call the upper halfs are properly cleared.
2. It doesn't trigger on macOS due to different memory organization.  We don't end up with gargabe in the registers.
3. Recently we added intrinsics for Unsafe.* https://github.com/mono/mono/pull/12605/files#diff-89fe30e1e87c5db21be052ddb0c8e28dR329 It masks this specific problem. Funnily enough, the assumed IL for the `Add ()` intrinsics didn't match what we had before. Now it does :-)

Fixes mono#13452 and mono#13597
@lewurm lewurm self-assigned this Mar 22, 2019
monojenkins added a commit to monojenkins/mono that referenced this issue Mar 22, 2019
We do 64bit operations, but operate on 32bit values. We need to sanitize those values.

Consider:
```
(lldb) x/8i 0x40011357
    0x40011357: 0f 84 c7 00 00 00  je     0x40011424
    0x4001135d: 49 8d 7c 24 14     leaq   0x14(%r12), %rdi
->  0x40011362: 49 8b c6           movq   %r14, %rax
    0x40011365: 48 d1 e0           shlq   %rax
    0x40011368: 48 03 f8           addq   %rax, %rdi
    0x4001136b: 0f b7 74 24 20     movzwl 0x20(%rsp), %esi
    0x40011370: 49 8b d7           movq   %r15, %rdx
    0x40011373: e8 bc 00 00 00     callq  0x40011434
(lldb) register read r14 rax rdi
     r14 = 0x0000000100000000
     rax = 0x0000000000000010
     rdi = 0x00007ffff7f48144
(lldb) si
(lldb) si
(lldb) si
(lldb) register read r14 rax rdi
     r14 = 0x0000000100000000
     rax = 0x0000000200000000
     rdi = 0x00008001f7f48144
```
In this snippet `r14` has some garbage in the 32bit upper half, which unfortunately is taken into account for the multiplication (`shl`) and addition in order to compute the address (`rdi`).

Several factors why we didn't see it earlier:
1. It doesn't trigger on AOT, because when compiling mscorlib.dll, `Unsafe.dll` isn't available to the AOT compiler and thus inlining won't happen. We use AOT where possible in our CI. When doing a call the upper halfs are properly cleared.
2. It doesn't trigger on macOS due to different memory organization.  We don't end up with gargabe in the registers.
3. Recently we added intrinsics for Unsafe.* https://github.com/mono/mono/pull/12605/files#diff-89fe30e1e87c5db21be052ddb0c8e28dR329 It masks this specific problem. Funnily enough, the assumed IL for the `Add ()` intrinsics didn't match what we had before. Now it does :-)

Fixes mono#13452 and mono#13597
monojenkins added a commit that referenced this issue Mar 25, 2019
[amd64] use 32bit variant of lea_membase for 32bit operations

Consider
```
 xor %r15, %r15       // %r15 = 0x0
 dec %r15d            // %r15 = 0xffff_ffff
 lea 0x1(%r15), %rdx  // %rdx = 0x1_0000_0000 but should be 0x0
```

instead `lea 0x1(%r15), %edx` should be generated.


Fixes #13452
Fixes #13597
@marek-safar

This comment has been minimized.

Copy link
Member

@marek-safar marek-safar commented Apr 2, 2019

Duplicate of #13597

@marek-safar marek-safar marked this as a duplicate of #13597 Apr 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.