Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dmd -O optimization breaks app #18983

Open
dlangBugzillaToGithub opened this issue Apr 25, 2015 · 9 comments
Open

dmd -O optimization breaks app #18983

dlangBugzillaToGithub opened this issue Apr 25, 2015 · 9 comments

Comments

@dlangBugzillaToGithub
Copy link

Tomáš Chaloupka (@tchaloupka) reported this on 2015-04-25T21:28:01Z

Transferred from https://issues.dlang.org/show_bug.cgi?id=14502

CC List

Description

I've come across a problem that when -O is used, app which normally works end up with segfault.

It's this benchmark: https://github.com/kostya/benchmarks/blob/master/havlak/havlak.d

When compiled with: dmd -ofhavlak_d -O -release -inline havlak.d
It segfaults at least for me (it seams to work for repository author according to test results)

DMD64 D Compiler v2.067.0
Linux 3.19.3-gentoo #1 SMP Fri Mar 27 10:41:00 CET 2015 x86_64 Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz GenuineIntel GNU/Linux

GDC, LDC and DMD without -O are working ok.

I'm trying to dustmite the problem and I'll post the results.

!!!There are attachements in the bugzilla issue that have not been copied over!!!

@dlangBugzillaToGithub
Copy link
Author

chalucha commented on 2015-04-26T18:48:06Z

Its really hard for me (as a D noob) to dustmite something usable from this :(

The output from gdb is:
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7aa5407 in object.TypeInfo_Class.getHash(const(void*)) () from /opt/dmd-2.067/lib64/libphobos2.so.0.67
#0  0x00007ffff7aa5407 in object.TypeInfo_Class.getHash(const(void*)) () from /opt/dmd-2.067/lib64/libphobos2.so.0.67
#1  0x00007ffff7ac371a in _aaInX () from /opt/dmd-2.067/lib64/libphobos2.so.0.67
#2  0x00007ffff7ac36d0 in _aaGetRvalueX () from /opt/dmd-2.067/lib64/libphobos2.so.0.67
#3  0x0000000000407bf8 in test.HavlakLoopFinder.DSF(test.BasicBlock, test.UnionFindNode[], int[test.BasicBlock], int[], int) ()
#4  0x0000000000407c48 in test.HavlakLoopFinder.DSF(test.BasicBlock, test.UnionFindNode[], int[test.BasicBlock], int[], int) ()
#5  0x0000000000407c48 in test.HavlakLoopFinder.DSF(test.BasicBlock, test.UnionFindNode[], int[test.BasicBlock], int[], int) ()

It has deterministic behaviour, always segfaults on the 12th iteration.

I've made a PR https://github.com/kostya/benchmarks/pull/23
which ads final to classes to speed it up a bit and this problem no longer occur with these changes - I have no idea why

@dlangBugzillaToGithub
Copy link
Author

chalucha commented on 2015-04-27T07:39:59Z

I can confirm, that the problem still exists for 2.067.1
Only it sometimes behaves differently.

On the first try it worked, but after that no..

Once it ended up with:
*** Error in `./test': double free or corruption (out): 0x00007f7b0cd1a010 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x72f5f)[0x7f7b1450bf5f]
/lib64/libc.so.6(+0x7839e)[0x7f7b1451139e]
/lib64/libc.so.6(+0x790e6)[0x7f7b145120e6]
/opt/dmd-2.067/lib64/libphobos2.so.0.67(_D2gc2gc3Gcx4DtorMFZv+0x401)[0x7f7b152fbac9]
======= Memory map: ========
00400000-00407000 r-xp 00000000 08:04 5904474                            /home/tomas/workspace/benchmarks/havlak/bug/clean/test
00607000-00608000 r--p 00007000 08:04 5904474                            /home/tomas/workspace/benchmarks/havlak/bug/clean/test
00608000-00609000 rw-p 00008000 08:04 5904474                            /home/tomas/workspace/benchmarks/havlak/bug/clean/test
01b83000-0218a000 rw-p 00000000 00:00 0                                  [heap]
7f7b035f9000-7f7b07ff9000 rw-p 00000000 00:00 0
7f7b0861a000-7f7b0cd65000 rw-p 00000000 00:00 0
7f7b0e6f3000-7f7b0e709000 r-xp 00000000 00:10 147065185                  /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.2/libgcc_s.so.1
7f7b0e709000-7f7b0e908000 ---p 00016000 00:10 147065185                  /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.2/libgcc_s.so.1
7f7b0e908000-7f7b0e909000 r--p 00015000 00:10 147065185                  /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.2/libgcc_s.so.1
7f7b0e909000-7f7b0e90a000 rw-p 00016000 00:10 147065185                  /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.2/libgcc_s.so.1
7f7b0e965000-7f7b12ee5000 rw-p 00000000 00:00 0
7f7b12ee5000-7f7b12ef8000 r-xp 00000000 00:10 138567531                  /lib64/libresolv-2.20.so
7f7b12ef8000-7f7b130f8000 ---p 00013000 00:10 138567531                  /lib64/libresolv-2.20.so
7f7b130f8000-7f7b130f9000 r--p 00013000 00:10 138567531                  /lib64/libresolv-2.20.so
7f7b130f9000-7f7b130fa000 rw-p 00014000 00:10 138567531                  /lib64/libresolv-2.20.so
7f7b130fa000-7f7b130fc000 rw-p 00000000 00:00 0
7f7b130fc000-7f7b13111000 r-xp 00000000 00:10 39352853                   /lib64/libz.so.1.2.8
7f7b13111000-7f7b13310000 ---p 00015000 00:10 39352853                   /lib64/libz.so.1.2.8
7f7b13310000-7f7b13311000 r--p 00014000 00:10 39352853                   /lib64/libz.so.1.2.8
7f7b13311000-7f7b13312000 rw-p 00015000 00:10 39352853                   /lib64/libz.so.1.2.8
7f7b13312000-7f7b13359000 r-xp 00000000 00:10 136807347                  /usr/lib64/libldap-2.4.so.2.10.3
7f7b13359000-7f7b13559000 ---p 00047000 00:10 136807347                  /usr/lib64/libldap-2.4.so.2.10.3
7f7b13559000-7f7b1355a000 r--p 00047000 00:10 136807347                  /usr/lib64/libldap-2.4.so.2.10.3
7f7b1355a000-7f7b1355c000 rw-p 00048000 00:10 136807347                  /usr/lib64/libldap-2.4.so.2.10.3
7f7b1355c000-7f7b1356a000 r-xp 00000000 00:10 136807348                  /usr/lib64/liblber-2.4.so.2.10.3
7f7b1356a000-7f7b13769000 ---p 0000e000 00:10 136807348                  /usr/lib64/liblber-2.4.so.2.10.3
7f7b13769000-7f7b1376a000 r--p 0000d000 00:10 136807348                  /usr/lib64/liblber-2.4.so.2.10.3
7f7b1376a000-7f7b1376b000 rw-p 0000e000 00:10 136807348                  /usr/lib64/liblber-2.4.so.2.10.3
7f7b1376b000-7f7b13976000 r-xp 00000000 00:10 142872611                  /usr/lib64/libcrypto.so.1.0.0
7f7b13976000-7f7b13b75000 ---p 0020b000 00:10 142872611                  /usr/lib64/libcrypto.so.1.0.0
7f7b13b75000-7f7b13b91000 r--p 0020a000 00:10 142872611                  /usr/lib64/libcrypto.so.1.0.0
7f7b13b91000-7f7b13b9d000 rw-p 00226000 00:10 142872611                  /usr/lib64/libcrypto.so.1.0.0
7f7b13b9d000-7f7b13ba0000 rw-p 00000000 00:00 0
7f7b13ba0000-7f7b13c08000 r-xp 00000000 00:10 142872612                  /usr/lib64/libssl.so.1.0.0
7f7b13c08000-7f7b13e08000 ---p 00068000 00:10 142872612                  /usr/lib64/libssl.so.1.0.0
7f7b13e08000-7f7b13e0d000 r--p 00068000 00:10 142872612                  /usr/lib64/libssl.so.1.0.0
7f7b13e0d000-7f7b13e14000 rw-p 0006d000 00:10 142872612                  /usr/lib64/libssl.so.1.0.0
7f7b13e14000-7f7b13e2f000 r-xp 00000000 00:10 91821790                   /usr/lib64/librtmp.so.1
7f7b13e2f000-7f7b1402e000 ---p 0001b000 00:10 91821790                   /usr/lib64/librtmp.so.1
7f7b1402e000-7f7b1402f000 r--p 0001a000 00:10 91821790                   /usr/lib64/librtmp.so.1
7f7b1402f000-7f7b14030000 rw-p 0001b000 00:10 91821790                   /usr/lib64/librtmp.so.1
7f7b14030000-7f7b14093000 r-xp 00000000 00:10 149049390                  /usr/lib64/libcurl.so.4.3.0
7f7b14093000-7f7b14292000 ---p 00063000 00:10 149049390                  /usr/lib64/libcurl.so.4.3.0
7f7b14292000-7f7b14294000 r--p 00062000 00:10 149049390                  /usr/lib64/libcurl.so.4.3.0
7f7b14294000-7f7b14295000 rw-p 00064000 00:10 149049390                  /usr/lib64/libcurl.so.4.3.0
7f7b14295000-7f7b14297000 r-xp 00000000 00:10 138566303                  /lib64/libdl-2.20.so
7f7b14297000-7f7b14497000 ---p 00002000 00:10 138566303                  /lib64/libdl-2.20.so
7f7b14497000-7f7b14498000 r--p 00002000 00:10 138566303                  /lib64/libdl-2.20.so
7f7b14498000-7f7b14499000 rw-p 00003000 00:10 138566303                  /lib64/libdl-2.20.so
7f7b14499000-7f7b14627000 r-xp 00000000 00:10 138568423                  /lib64/libc-2.20.so
7f7b14627000-7f7b14827000 ---p 0018e000 00:10 138568423                  /lib64/libc-2.20.so
7f7b14827000-7f7b1482b000 r--p 0018e000 00:10 138568423                  /lib64/libc-2.20.so
7f7b1482b000-7f7b1482d000 rw-p 00192000 00:10 138568423                  /lib64/libc-2.20.so
7f7b1482d000-7f7b14831000 rw-p 00000000 00:00 0
7f7b14831000-7f7b14838000 r-xp 00000000 00:10 138567074                  /lib64/librt-2.20.so
7f7b14838000-7f7b14a37000 ---p 00007000 00:10 138567074                  /lib64/librt-2.20.so
7f7b14a37000-7f7b14a38000 r--p 00006000 00:10 138567074                  /lib64/librt-2.20.so
7f7b14a38000-7f7b14a39000 rw-p 00007000 00:10 138567074                  /lib64/librt-2.20.so
7f7b14a39000-7f7b14b31000 r-xp 00000000 00:10 138566264                  /lib64/libm-2.20.so
7f7b14b31000-7f7b14d30000 ---p 000f8000 00:10 138566264                  /lib64/libm-2.20.so
7f7b14d30000-7f7b14d31000 r--p 000f7000 00:10 138566264                  /lib64/libm-2.20.so
7f7b14d31000-7f7b14d32000 rw-p 000f8000 00:10 138566264                  /lib64/libm-2.20.so
7f7b14d32000-7f7b14d48000 r-xp 00000000 00:10 138568474                  /lib64/libpthread-2.20.so
7f7b14d48000-7f7b14f47000 ---p 00016000 00:10 138568474                  /lib64/libpthread-2.20.so
7f7b14f47000-7f7b14f48000 r--p 00015000 00:10 138568474                  /lib64/libpthread-2.20.so
7f7b14f48000-7f7b14f49000 rw-p 00016000 00:10 138568474                  /lib64/libpthread-2.20.so
7f7b14f49000-7f7b14f4d000 rw-p 00000000 00:00 0
7f7b14f4d000-7f7b15351000 r-xp 00000000 00:10 149775571                  /opt/dmd-2.067/lib64/libphobos2.so.0.67.1
7f7b15351000-7f7b15550000 ---p 00404000 00:10 149775571                  /opt/dmd-2.067/lib64/libphobos2.so.0.67.1
7f7b15550000-7f7b15573000 r--p 00403000 00:10 149775571                  /opt/dmd-2.067/lib64/libphobos2.so.0.67.1
7f7b15573000-7f7b15619000 rw-p 00426000 00:10 149775571                  /opt/dmd-2.067/lib64/libphobos2.so.0.67.1
7f7b15619000-7f7b1561b000 rw-p 00000000 00:00 0
7f7b1561b000-7f7b1563c000 r-xp 00000000 00:10 138568416                  /lib64/ld-2.20.so
7f7b15688000-7f7b157df000 rw-p 00000000 00:00 0
7f7b157ea000-7f7b1583b000 rw-p 00000000 00:00 0
7f7b1583b000-7f7b1583c000 r--p 00020000 00:10 138568416                  /lib64/ld-2.20.so
7f7b1583c000-7f7b1583d000 rw-p 00021000 00:10 138568416                  /lib64/ld-2.20.so
7f7b1583d000-7f7b1583e000 rw-p 00000000 00:00 0
7ffed5cae000-7ffed5fc6000 rw-p 00000000 00:00 0                          [stack]
7ffed5fee000-7ffed5ff0000 r--p 00000000 00:00 0                          [vvar]
7ffed5ff0000-7ffed5ff2000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Neúspěšně ukončen (SIGABRT)

@dlangBugzillaToGithub
Copy link
Author

dlang-bugzilla (@CyberShadow) commented on 2015-04-27T07:45:04Z

(In reply to Tomáš Chaloupka from comment #1)
> Its really hard for me (as a D noob) to dustmite something usable from this
> :(

See here:

https://github.com/CyberShadow/DustMite/wiki/Reducing-a-regression-between-two-D-versions

Except that instead of compiling with two versions of D, compile once without -O and once with -O.

@dlangBugzillaToGithub
Copy link
Author

chalucha commented on 2015-04-27T18:32:00Z

Created attachment 1516
version which fails

@dlangBugzillaToGithub
Copy link
Author

chalucha commented on 2015-04-27T18:32:29Z

Created attachment 1517
patch with which its ok

@dlangBugzillaToGithub
Copy link
Author

chalucha commented on 2015-04-27T18:42:08Z

dustmite is problematic because it can lead to versions which segfaults too, but with other reason or which eats all mem, etc. And is slow as hell to test outputs from all variants (-O -release, -release, -debug) - I let it run for more than 24hours actually with just slightly shorter version..

So I tried manually compare version which works and version which don't and made an attachment with failing version and with simplified patch to working version.

Patch consists of 3 changes.
I would guess something with:
new BasicBlockEdge(... instead of just BasicBlockEdge(...

BasicBlockEdge is a struct

But don't understand internals of what new means for structs?

@dlangBugzillaToGithub
Copy link
Author

dlang-bugzilla (@CyberShadow) commented on 2015-04-27T21:28:30Z

(In reply to Tomáš Chaloupka from comment #6)
> dustmite is problematic because it can lead to versions which segfaults too,
> but with other reason or which eats all mem, etc. And is slow as hell to
> test outputs from all variants (-O -release, -release, -debug) - I let it
> run for more than 24hours actually with just slightly shorter version..

Um, did you read the linked page?

Your test script should do the following:

1. Compile the program without -O
2. Run the program and check that it exits successfully. You can use "timeout" and "ulimit" to limit how much time and memory the program can consume.
3. If the above program works fine, compile it again with -O
4. Run the program and checks that it crashes (or produces output different from the first run)
5. If the first run was OK and the second run crashed, return 0 (you have reproduced the bug). Otherwise, return 1

@dlangBugzillaToGithub
Copy link
Author

chalucha commented on 2015-04-28T21:05:51Z

Actually read it before. Tried to use prepared script with specific backtrace, but could not make it work with timeout (wrong output from gdb then).

So now trying with just simple:

#!/bin/bash
TIME=12
timeout $TIME rdmd --force -O -release test.d 2>&1 > /dev/null
if [ $? -eq 139 ]; then
    timeout $TIME rdmd --force -release test.d 2>&1 > /dev/null
    if [ $? -eq 124 ]; then
        timeout $TIME rdmd --force test.d 2>&1 > /dev/null
        if [ $? -eq 124 ]; then
            exit 0
        else
            exit 1
        fi
    else
        exit 1
    fi
fi
exit 1

which is at least faster thanks to timeout usage.

@dlangBugzillaToGithub
Copy link
Author

chalucha commented on 2015-04-29T04:32:31Z

So it ended up with an indeterministic version which sometimes pass, sometimes fails in all switches :(
Is it actually failing for anyone else?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants