optional package cbc-2.3.p2.spkg does not build on some platforms on sage-4.7.1.alpha1

[sagetrac-mariah]

The optional package cbc-2.3.p2.spkg does not build on skynet/eno (x86_64-Linux-core2-fc) on sage-4.7.1.alpha1:

...
 /usr/local/gcc-4.6.0/x86_64-Linux-core2-fc/bin/g++ -DHAVE_CONFIG_H -I. -I. -I../inc -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Clp/src -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Osi/src/OsiClp -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cbc/../Clp/inc -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglClique -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglDuplicateRow -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglFlowCover -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglGomory -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglKnapsackCover -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglMixedIntegerRounding -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglMixedIntegerRounding2 -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglOddHole -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglPreProcess -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglProbing -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglRedSplit -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglTwomir -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglLandP -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglResidualCapacity -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cgl/src/CglZeroHalf -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/CoinUtils/src -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cbc/../CoinUtils/inc -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Osi/src -I/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cbc/../Osi/inc -DCOIN_NO_CLP_MESSAGE -DUSE_CBCCONFIG -O3 -fomit-frame-pointer -pipe -DNDEBUG -pedantic-errors -Wimplicit -Wparentheses -Wreturn-type -Wcast-qual -Wall -Wpointer-arith -Wwrite-strings -Wconversion -Wno-unknown-pragmas -MT CbcEventHandler.lo -MD -MP -MF .deps/CbcEventHandler.Tpo -c CbcEventHandler.cpp  -fPIC -DPIC -o .libs/CbcEventHandler.o
cc1plus: warning: command line option '-Wimplicit' is valid for C/ObjC but not for C++ [enabled by default]
In file included from CbcEventHandler.cpp:8:0:
CbcEventHandler.hpp:137:43: error: 'NULL' was not declared in this scope
make[2]: *** [CbcEventHandler.lo] Error 1
make[2]: Leaving directory `/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cbc/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/mariah/sage/sage-4.7.1.alpha1-x86_64-Linux-core2-fc/spkg/build/cbc-2.3.p2/src/Cbc'
make: *** [all-recursive] Error 1

real    5m40.762s
user    4m8.662s
sys     0m49.002s
sage: An error occurred while installing cbc-2.3.p2

CC: @nathanncohen @kcrisman @dandrake

Component: packages: optional

Keywords: sd32 sd40.5

Reviewer: Karl-Dieter Crisman, Dan Drake, John Perry

Issue created by migration from https://trac.sagemath.org/ticket/11444

[kcrisman]

comment:1

This did install normally on 4.7.1.alpha1 on Mac OS X 10.6. So perhaps it's just certain Linuces that don't work.

---

On a related note, there seems to be exactly one set of doctests which even require this package optionally. I'm surprised that numerical/backends/ and the graph stuff doesn't. I'm cc:ing Nathann about this. After all, hopefully optional tests are at least sometimes run for major releases...

And when I run them, I get

sage -t -optional "devel/sage/sage/combinat/integer_vector.py"
**********************************************************************
File "/Users/karl-dietercrisman/Downloads/sage-4.7.1.alpha1/devel/sage/sage/combinat/integer_vector.py", line 279:
    sage: print gale_ryser_theorem(p1, p2, algorithm="gale") # Optional - requires GLPK or CBC
Expected:
    [1 0 1 1 0]
    [1 0 1 0 1]
    [1 1 0 0 0]
    [0 1 0 0 0]
Got:
    [0 0 1 1 1]
    [1 1 1 0 0]
    [1 1 0 0 0]
    [1 0 0 0 0]

And all the ones with 'generic backend' fail as well... Nathann?

[nathanncohen]

comment:2

Hellooooooo !!!

Well, this doctest has been written before GLPK got installed as a standard spkg. Since then, a function using LP needs not be optional anymore. On the other hand, if you have CBC or CPLEX installed, the LP will be solved with them instead of GLPK. The truth is this #optional keyword could be removed. The only occurrences of "#optional CBC" should appear in the file numerical/backends/coin_backend.pyx

Nathann

[kcrisman]

comment:3

Replying to @nathanncohen:

Hellooooooo !!!

Well, this doctest has been written before GLPK got installed as a standard spkg. Since then, a function using LP needs not be optional anymore. On the other hand, if you have CBC or CPLEX installed, the LP will be solved with them instead of GLPK. The truth is this #optional keyword could be removed. The only occurrences of "#optional CBC" should appear in the file numerical/backends/coin_backend.pyx

Interesting! So maybe we should open another ticket to fix that.

That doesn't explain this doctest, though I have to admit I rarely run optional doctests because they usually involve programs I don't have.

[nathanncohen]

comment:4

Then maybe this doctest should be removed. We've been adding a very hard one in #11324 anyway, so this small example could disappear without much to worry about. This is a problem that appears very often with LP solvers : they all give good answers, but it's a mess to write doctests when the answers have no reason to be unique :-)

If it's fine by you I can modify my patch at #11324 to fix it there !

Nathann

[johnperry-math]

comment:5

Replying to @nathanncohen:

Then maybe this doctest should be removed. We've been adding a very hard one in #11324 anyway, so this small example could disappear without much to worry about. This is a problem that appears very often with LP solvers : they all give good answers, but it's a mess to write doctests when the answers have no reason to be unique :-)

If it's fine by you I can modify my patch at #11324 to fix it there !

Is there no way to generate a "canonical" answer from both packages? (Does that even make sense mathematically? I'm not familiar with the gale ryser theorem.) If not, I agree with Nathann.

[williamstein]

Changed keywords from none to sd32

[nathanncohen]

comment:7

Is there no way to generate a "canonical" answer from both packages? (Does that even make sense mathematically? I'm not familiar with the gale ryser theorem.) If not, I agree with Nathann.

Not that I know. When I write doctests using LP, I try to find examples for which the solution is unique, but there are many matrices solving the gale-ryser proble (ryser's algorithm even lets one enumerate them), and the solvers have no reason to return one rather than the other. Adding a fake objective functions to make the answer unique on the same problem would touch the running time just to make the doctests easier to write, which clearly isn't a good idea :-D

Nathann

[johnperry-math]

comment:8

This is listed as a "high priority" bug for sage days 40.5, but sage-4.7.1 is way out of date, and the cbc package has since been updated to a newer version. Should the status of this ticket be changed to invalid or obsolete, if possible, or can someone test sage-5.0 on the systems reported as defective, or... something?

[kcrisman]

comment:9

Dan Drake is just now trying sage -i cbc-2.7.5 (the current version according to http://sagemath.org/packages/optional/) with Sage 5.0 and the result is... it builds! He's running tests now.

[kcrisman]

Changed keywords from sd32 to sd32 sd40.5

[kcrisman]

Reviewer: Karl-Dieter Crisman, Dan Drake, John Perry

[johnperry-math]

comment:10

Replying to @kcrisman:

Dan Drake is just now trying sage -i cbc-2.7.5 (the current version according to http://sagemath.org/packages/optional/) with Sage 5.0 and the result is... it builds! He's running tests now.

Does he have the same hardware? I know it builds on x86_64 with Phenom, and x86_32 Pentium something-or-other (I have an OLD computer at home :-)) and some other machine at work. i think the problem is whether it builds on the specific hardware listed.

[dandrake]

comment:11

sage -i cbc-2.7.5 works on skynet/eno (a Fedora machine, 64 bit, Linux 3.2.3, gcc 4.7.0) and almost all doctests pass -- there's one failure in cplex_backend.pyx which I think is a typo.

[drake@eno sage-5.0]$ ./sage -tp 8 -only-optional=Coin devel/sage/sage/numerical/backends/cplex_backend.pyx
Global iterations: 1
File iterations: 1
Using cached timings to run longest doctests first.
Doctesting 1 file using 1 thread
sage -t -only-optional=Coin devel/sage/sage/numerical/backends/cplex_backend.pyx
**********************************************************************
File "/home/drake/sage-5.0/devel/sage-main/sage/numerical/backends/cplex_backend.pyx", line 424:
    sage: p.add_linear_constraints(2, None, 2, names=['foo','bar']) # optional - Coin
Exception raised:
    Traceback (most recent call last):
      File "/home/drake/sage-5.0/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/home/drake/sage-5.0/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/home/drake/sage-5.0/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_0[3]>", line 1, in <module>
        p.add_linear_constraints(Integer(2), None, Integer(2), names=['foo','bar']) # optional - Coin###line 424:
    sage: p.add_linear_constraints(2, None, 2, names=['foo','bar']) # optional - Coin
    NameError: name 'p' is not defined
**********************************************************************

Should that test be "optional - CPLEX"?

If all we care about is building on eno with gcc 4.7, this works fine. Here's info on eno: http://wiki.sagemath.org/skynet#Eno

[kcrisman]

comment:12

Should that test be "optional - CPLEX"?

As discussed and relevant to comment:2, this definitely was a typo.

If all we care about is building on eno with gcc 4.7, this works fine. Here's info on eno: http://wiki.sagemath.org/skynet#Eno

As regards John's comment, this is exactly the place where the problem is reported.

I've reported the typo to the appropriate person, so this is good to go.