WIP: More tests #103

odow · 2017-08-01T21:47:31Z

Port some of the MPB tests from linprog (1 c,d,e from #23) and add tests for the result count attribute #99 (comment)

Closes #105

joaquimg · 2017-08-01T23:16:06Z

test/contlinear.jl

-        @test MOI.cangetattribute(m, MOI.PrimalStatus())
-        @test MOI.getattribute(m, MOI.PrimalStatus()) == MOI.FeasiblePoint
+        @test MOI.cangetattribute(m, MOI.ResultCount())
+        @test MOI.getattribute(m, MOI.ResultCount()) == 1


I am not sure, but maybe we could test for MOI.getattribute(m, MOI.ResultCount()) >= 1 instead of == 1.

joaquimg · 2017-08-01T23:41:11Z

test/intlinear.jl

+        @test MOI.getattribute(m, MOI.ObjectiveValue()) ≈ 19 atol=ε
+
+        @test MOI.cangetattribute(m, MOI.VariablePrimal(), v)
+        @test MOI.getattribute(m, MOI.VariablePrimal(), v) ≈ [4,5,1] atol=ε


The problem seems to be degenerate, x = 0, y = 7, z = 1, maybe we could exchange the bounds between x and y

Yeah. I just copied the CPLEX problem

odow · 2017-08-02T01:36:48Z

~~@joaquimg do you have SOS working in Xpress? I can't get CPLEX to pass the SOSII test with MOI...~~

~~It's like I'm adding in the wrong constraint somewhere...~~ Indeed I was...

odow · 2017-08-02T05:02:29Z

This should probably get merged soon as it contains a lot of new tests and various fixes to current ones that are necessary to get passing.

mlubin · 2017-08-02T06:13:35Z

test/contlinear.jl

+            @test MOI.getattribute(m, MOI.TerminationStatus()) == MOI.Success
+            @test MOI.getattribute(m, MOI.DualStatus()) == MOI.InfeasibilityCertificate
+            @test MOI.cangetattribute(m, MOI.ConstraintDual(), c)
+            @test MOI.getattribute(m, MOI.ConstraintDual(), c) ≈ -1 atol=ε


What should the dual values be on the bound constraints? These should be tested also.

mlubin · 2017-08-02T06:15:48Z

test/contlinear.jl

+        if MOI.getattribute(m, MOI.ResultCount()) == 1
+            # solver returned an unbounded ray
+            @test MOI.getattribute(m, MOI.TerminationStatus()) == MOI.Success
+            @test MOI.getattribute(m, MOI.PrimalStatus()) == MOI.ReductionCertificate


ReductionCertificate is incorrect here. Looks like we forgot to include a ray status.

What exactly is a reduction certificate then? I just assumed it was some fancy conic word for an unbounded ray

Actually the ray status is there and is called InfeasibilityCertificate because a primal unbounded ray is an infeasibility certificate of the dual. Facial reduction certificates are mentioned briefly in the manual; they're for cases of duality failures or unattained solutions which never happen for LP.

joaquimg · 2017-08-04T13:41:20Z

@odow I am planning to add SOS support this week

joaquimg · 2017-08-04T13:44:10Z

test/intlinear.jl

@@ -26,6 +298,10 @@ function knapsacktest(solver::MOI.AbstractSolver, eps=Base.rtoldefault(Float64))

        MOI.setobjective!(m, MOI.MaxSense, MOI.ScalarAffineFunction(v, [5.0, 3.0, 2.0, 7.0, 4.0], 0.0))

+        if MOI.cansetattribute(m, MOI.VariablePrimalStart(), v)
+            MOI.setattribute!(m, MOI.VariablePrimalStart(), v, [0.0, 0.0, 0.0, 0.0, 0.0])


Any ideas on how do we check this method is working?

joaquimg · 2017-08-04T13:45:33Z

I remember a bug when modifying quadratic objectives: the quadratic terms were not being re-set to zero. Are we testing that?

odow · 2017-08-07T23:13:19Z

Okay. Before I make this PR any longer, lets get this merged. @blegat or @joaquimg looks like you have write access?

joaquimg · 2017-08-08T12:10:39Z

Great job! Contlinear and intlinear are all passing in CPLEX? I agree with merging today. @blegat do you have any problem with this?

blegat · 2017-08-08T13:30:24Z

test/contlinear.jl

+
+        @test MOI.getattribute(m, MOI.ObjectiveValue()) ≈ 79e4/11 atol=ε
+        @test MOI.getattribute(m, MOI.VariablePrimal(), x) ≈ 650/11 atol=ε
+        @test MOI.getattribute(m, MOI.VariablePrimal(), y) ≈ 400/11 atol=ε


I would use rtol instead of atol here. isapprox is implemented as

abs(x-y) <= atol + rtol*max(abs(x), abs(y)))

where rtol is 1.4901161193847656e-8 for Float64 by default.
For the objective rtol * max(abs(x), abs(y)) is approximately 1e-3 so if atol is like 1e-5 it will be neglibeable.
CSDP fails the test for VariablePrimal/x because epsilon is 1e-7 which means that atol is 1e-7butrtol * max(abs(x), abs(y))is about1e-7too. CSDP has a relative tolerance of1e-7` so it should pass the test but here it is tested for an absolute tolerance of 1e-7.

In fact, it makes more sense to only use atol when one of the two numbers being compared is 0 or very close to 0. Why we use atol everywhere in MOI's tests ? Usually it does not matter much but here since the numbers are large, it does matter.

Solvers are able to specify different tolerances... for example https://github.com/JuliaOpt/CPLEX.jl/pull/139/files#diff-fce720c43af3c52c862fd7451c7374b8R30

Yes but the tolerance should be relative, not absolute right ?

How about setting atol and rtol?

It might be cumbersome since usually the solver just tells its accuracy and we should expect to pass the tests with epsilon = this accuracy.
I would suggest using atol when what we compare to zero and rtol when we compare far from zero. rtol is pretty bad close to zero:

isapprox(1e-9, 1e-10, rtol=1e-2)

So we do

@test x ≈ 0 atol=ε @test x ≈ 1 rtol=ε @test x ≈ y atol=ε rtol=ε # x and y might be close to zero or far from zero, we don't know. I don't know if this case happens in the tests though

What do you think ? Setting atol and rtol might also be a flexible solution.
@joaquimg @joehuchette @chriscoey Any preference ?

Have you seen 83f919e?

Yes, we can use this solution if everyone agree on it.

odow · 2017-08-08T21:20:04Z

Contlinear and intlinear are all passing in CPLEX?

Contlinear, intlinear, contquadratic, and most of contconic are passing.

odow changed the title ~~More contlinear tests~~ More tests Aug 1, 2017

odow changed the title ~~More tests~~ WIP: More tests Aug 1, 2017

joaquimg reviewed Aug 1, 2017

View reviewed changes

odow mentioned this pull request Aug 2, 2017

Solver wrappers tracker #70

Closed

13 tasks

mlubin reviewed Aug 2, 2017

View reviewed changes

odow mentioned this pull request Aug 2, 2017

Tests to be ported from MPB #23

Closed

51 tasks

odow added 20 commits August 3, 2017 14:04

status tests

1b6192a

Add come integer tests

b38a87a

typo

5866a89

another typo

1439f95

More typos

c059186

allow multiple solns

55dfbdb

Add SOS tests

d800bb7

add variables bounds

88de901

tidy SOS1 test

2e6eb1b

more tests for sos constraint sets and functions

d56ee9d

typo

d9d8ef2

put certificates in dual status

ee2f9b4

Typo fixes to contquadratic

2fbe2ef

Add new QP test from CPLEX QP01

a205102

addlinearconstraints!

4d2ff8b

Add tests for solver attributes

7d17edc

add tests for support attributes

327caca

remove show

208a0ad

change to objective sense

556524e

fix primal start

f8b827d

joaquimg reviewed Aug 4, 2017

View reviewed changes

odow added 8 commits August 7, 2017 10:28

Test quadratic->linear objective

e7d3a4f

tidy tests

984a780

Reenable broken tests

46d2118

tidy conic tests

bd1d5ab

Add ranged constraints

cba6e3c

Add interval duals

f2e0be7

Add test for int with ranged constraint

d9d32c8

test modifying interval constraints

37eabac

blegat reviewed Aug 8, 2017

View reviewed changes

odow added 3 commits August 9, 2017 12:24

Explicity atol and rtol for tests

83f919e

fix infeas test

a901d5a

Can return infeasible points

a25598f

joaquimg approved these changes Aug 9, 2017

View reviewed changes

blegat merged commit 24f924b into jump-dev:master Aug 9, 2017

odow deleted the newtests branch October 18, 2017 22:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: More tests #103

WIP: More tests #103

odow commented Aug 1, 2017 •

edited

Loading

joaquimg Aug 1, 2017

joaquimg Aug 1, 2017

odow Aug 1, 2017

odow commented Aug 2, 2017 •

edited

Loading

odow commented Aug 2, 2017

mlubin Aug 2, 2017

mlubin Aug 2, 2017

odow Aug 2, 2017

mlubin Aug 2, 2017

joaquimg commented Aug 4, 2017

joaquimg Aug 4, 2017

odow Aug 7, 2017

joaquimg commented Aug 4, 2017

odow commented Aug 7, 2017

joaquimg commented Aug 8, 2017

blegat Aug 8, 2017

odow Aug 8, 2017

blegat Aug 8, 2017

odow Aug 9, 2017

blegat Aug 9, 2017 •

edited

Loading

odow Aug 9, 2017

blegat Aug 9, 2017

odow commented Aug 8, 2017

WIP: More tests #103

WIP: More tests #103

Conversation

odow commented Aug 1, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

odow commented Aug 2, 2017 • edited Loading

odow commented Aug 2, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joaquimg commented Aug 4, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joaquimg commented Aug 4, 2017

odow commented Aug 7, 2017

joaquimg commented Aug 8, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

blegat Aug 9, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

odow commented Aug 8, 2017

odow commented Aug 1, 2017 •

edited

Loading

odow commented Aug 2, 2017 •

edited

Loading

blegat Aug 9, 2017 •

edited

Loading