rubiks fails doctest with gcc 4.6.0 and -O2 optimisation.

[sagetrac-drkirkby]

On a Sun Ultra 27 running OpenSolaris 06/2009, the rubiks-20070912.p12 package builds, but fails a doctest

drkirkby@hawk:~/sage-4.7.alpha3$ ./sage -t  -long -force_lib devel/sage/sage/interfaces/rubik.py
sage -t -long -force_lib "devel/sage/sage/interfaces/rubik.py"
**********************************************************************
File "/export/home/drkirkby/sage-4.7.alpha3/devel/sage/sage/interfaces/rubik.py", line 132:
    sage: solver = OptimalSolver() # long time
Exception raised:
    Traceback (most recent call last):
      File "/export/home/drkirkby/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/export/home/drkirkby/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/export/home/drkirkby/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_3[4]>", line 1, in <module>
        solver = OptimalSolver() # long time###line 132:
    sage: solver = OptimalSolver() # long time
      File "/export/home/drkirkby/sage-4.7.alpha3/local/lib/python/site-packages/sage/interfaces/rubik.py", line 98, in __init__
        self.ready()
      File "/export/home/drkirkby/sage-4.7.alpha3/local/lib/python/site-packages/sage/interfaces/rubik.py", line 117, in ready
        self.child.expect('enter cube')
      File "/export/home/drkirkby/sage-4.7.alpha3/local/lib/python/site-packages/pexpect.py", line 912, in expect
        return self.expect_list(compiled_pattern_list, timeout, searchwindowsize)
      File "/export/home/drkirkby/sage-4.7.alpha3/local/lib/python/site-packages/pexpect.py", line 978, in expect_list
        raise EOF (str(e) + '\n' + str(self))
    EOF: End Of File (EOF) in read_nonblocking(). Empty string style platform.
    <pexpect.spawn instance at 0xc5367ac>
    version: 2.0 ($Revision: 1.151 $)
    command: /export/home/drkirkby/sage-4.7.alpha3/local/bin/optimal
    args: ['/export/home/drkirkby/sage-4.7.alpha3/local/bin/optimal']
    patterns:
        enter cube
    buffer (last 100 chars): 
    before (last 100 chars): olutions
    
    initializing transformation tables
    
    init_fulledge_to_edgequot : too many  edgequot's

    after: <class 'pexpect.EOF'>
    match: None
    match_index: None
    exitstatus: None
    flag_eof: 1
    pid: 10994
    child_fd: 3
    timeout: None
    delimiter: <class 'pexpect.EOF'>
    logfile: None
    maxread: 2000
    searchwindowsize: None
    delaybeforesend: 0.1
**********************************************************************
1 items had failures:
   1 of  12 in __main__.example_3
***Test Failed*** 1 failures.
For whitespace errors, see the file /export/home/drkirkby/.sage//tmp/.doctest_rubik.py
         [9.4 s]
 
----------------------------------------------------------------------
The following tests failed:


        sage -t -long -force_lib "devel/sage/sage/interfaces/rubik.py"
Total time for all tests: 9.4 seconds

However, if the optimisation level is changed from -O2 to -O1, the program behaves properly.

drkirkby@hawk:~/sage-4.7.alpha3$ ./sage -t  -long -force_lib devel/sage/sage/interfaces/rubik.py
sage -t -long -force_lib "devel/sage/sage/interfaces/rubik.py"
         [26.7 s]
 
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 26.7 seconds

Similar problems have been observed on other platforms too. A revised package, which works around the gcc 4.6.0 bug by adding the flag -fno-ivopts:

http://boxen.math.washington.edu/home/jdemeyer/spkg/rubiks-20070912.p16.spkg

Reported upstream to gcc: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48702

For other gcc 4.6.0 specific problems see #11216

Upstream: Fixed upstream, but not in a stable release.

CC: @jhpalmieri

Component: porting: Solaris

Author: David Kirkby, Jeroen Demeyer

Reviewer: John Palmieri, David Kirkby

Merged: sage-4.7.rc2

Issue created by migration from https://trac.sagemath.org/ticket/11168

[sagetrac-drkirkby]

Author: David Kirkby

[jhpalmieri]

comment:3

The patch is very simple. The new spkg builds on both hawk (OpenSolaris on x86, with gcc 4.5.0 and gcc 4.6.0) and t2 (Solaris on sparc, with gcc 4.5.1 and 4.6.0), and tests pass in all cases. With the old version, tests failed on both machines using gcc 4.6.0 (but everything worked fine with the earlier version of gcc).

Note: to build with gcc 4.6.0 on t2, I had to set LD_LIBRARY_PATH and unset LD_OPTIONS; I couldn't get LD_OPTIONS to work. But I don't really know how to properly use these variables anyway...

[jhpalmieri]

Reviewer: John Palmieri

[jdemeyer]

Merged: sage-4.7.alpha5

[jdemeyer]

Changed merged from sage-4.7.alpha5 to none

[jdemeyer]

comment:6

Marking this as needs_work because there are more systems with the same symptom. In particular, on the buildbot Fedora 14-32 (cicero).

[jdemeyer]

comment:7

Also, I really dislike simply lowering the optimization level. That is wiping the problem under the carpet, not solving it. Also should be reported upstream.

[sagetrac-drkirkby]

comment:8

Replying to @jdemeyer:

Also, I really dislike simply lowering the optimization level. That is wiping the problem under the carpet, not solving it. Also should be reported upstream.

In the short term, I don't see we can do anything other than just lower the optimisation.

It should be noted that there are quite a few Sage patches applied - some directly to the source code, which should not have been done, but has been done. This package has been patched 13 times already!

Dave

[sagetrac-drkirkby]

comment:9

Here's a revised package, which reduces the optimisation level to -O1 (as my previous patch), but this time only on all platforms, and not just Solaris.

Dave

[sagetrac-drkirkby]

comment:10

Replying to @sagetrac-drkirkby:

Here's a revised package, which reduces the optimisation level to -O1 (as my previous patch), but this time only on all platforms, and not just Solaris.

Dave

The grammar was not very good there! The point being, this patch changes the optimisation to -O1 on all platforms.

Dave

[sagetrac-mariah]

comment:11

I believe only the file src/reid/optimal.c needs to be reduced
to -O1 due to an optimization regression in gcc-4.6.0. See gcc bugzilla # 48702 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48702

[sagetrac-mariah]

Changed upstream from Not yet reported upstream; Will do shortly. to Reported upstream. Little or no feedback.

[sagetrac-drkirkby]

comment:27

Replying to @jdemeyer:

David, I agree with everything you say, but I hope you can agree that the 2>/dev/null doesn't hurt anyway. Can it get positive_review now?

Sure.

[jdemeyer]

Merged: sage-4.7.rc1

[jdemeyer]

Changed merged from sage-4.7.rc1 to none

[jdemeyer]

Changed reviewer from John Palmieri to John Palmieri, David Kirkby

[jdemeyer]

Attachment: rubiks-20070912.p15-p16.diff.gz

Diff for the rubiks spkg, for reviewing only

[sagetrac-mariah]

comment:32

The patch will be applied for later versions than just gcc-4.6.0.
Recommend that the patch only be applied for gcc-4.6.0.

[jdemeyer]

comment:33

Replying to @sagetrac-mariah:

The patch will be applied for later versions than just gcc-4.6.0.
Recommend that the patch only be applied for gcc-4.6.0.

Why?

There already exist pre-release versions of gcc 4.6.1 manifesting the same problem.

I think that having the workaround for all versions 4.6.x doesn't really hurt. It is certainly a safer alternative than applying the workaround only for gcc 4.6.0. If at some point, this bug is fixed in a later gcc 4.6.x, then we can still change the spkg accordingly.

[jdemeyer]

comment:35

Can somebody please review? Thanks.

[sagetrac-drkirkby]

comment:36

Replying to @jdemeyer:

Can somebody please review? Thanks.

Sure:

drkirkby@hawk:~/sage-4.7.rc0$  ./sage -t  -long -force_lib devel/sage/sage/interfaces/rubik.py
sage -t -long -force_lib "devel/sage/sage/interfaces/rubik.py"
	 [25.5 s]
 
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 25.6 seconds

so that option is working to resolve the bug on OpenSolaris too. This looks good to me.

Dave

[jdemeyer]

Merged: sage-4.7.rc2

[jdemeyer]

Changed upstream from Reported upstream. Developers acknowledge bug. to Fixed upstream, but not in a stable release.

[jdemeyer]

comment:39

See #11437 for a follow-up.