Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate testing of examples #1712

Merged
merged 19 commits into from Jan 2, 2019

Conversation

@odow
Copy link
Member

commented Dec 29, 2018

This PR adds a build stage to Linux/Julia 1.0 on Travis that runs the examples in /examples. This requires the installation of some solvers. So far, I have added GLPK (for LP/MIP), Ipopt (for NLP), and SCS (for SDP).

List of examples and whether they have been updated:

  • basic
  • cannery
  • cluster
  • corr_sdp
  • diet
  • knapsack
  • maxcut_sdp
  • mindistortion
  • minellipse
  • mle
  • multi
  • optcontrol
  • prod
  • qcp
  • robust_uncertainty
  • rosenbrock
  • steelT3
  • sudoku
  • transp
  • urbanplan

Closes #699
Closes #1175

  • PR currently includes Manifest.toml because latest GLPK tag isn't in METADATA. Once it is, we can remove Manifest.tom.
@codecov

This comment has been minimized.

Copy link

commented Dec 29, 2018

Codecov Report

Merging #1712 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1712   +/-   ##
=======================================
  Coverage   68.69%   68.69%           
=======================================
  Files          30       30           
  Lines        3891     3891           
=======================================
  Hits         2673     2673           
  Misses       1218     1218

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd8dbe5...5b167b3. Read the comment docs.

@codecov

This comment has been minimized.

Copy link

commented Dec 29, 2018

Codecov Report

Merging #1712 into master will increase coverage by 0.02%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1712      +/-   ##
==========================================
+ Coverage   68.69%   68.72%   +0.02%     
==========================================
  Files          30       30              
  Lines        3891     3904      +13     
==========================================
+ Hits         2673     2683      +10     
- Misses       1218     1221       +3
Impacted Files Coverage Δ
src/copy.jl 76.31% <0%> (-1.47%) ⬇️
src/JuMP.jl 71.28% <0%> (+0.58%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd8dbe5...8997c86. Read the comment docs.

@mlubin
Copy link
Member

left a comment

This is great. I like how we can see if JuMP is passing its tests (without solvers) separately from whether the examples pass.

A lot of the CI time is spent in Ipopt compilation. Hopefully we can switch to BinaryBuilder on linux soon...

examples/basic.jl Show resolved Hide resolved

odow added some commits Dec 30, 2018

examples/mindistortion.jl Outdated Show resolved Hide resolved
It is known that the 4-point metric space defined by the star graph:
x
\\

This comment has been minimized.

Copy link
@mlubin

mlubin Dec 30, 2018

Member

Extra "\"

This comment has been minimized.

Copy link
@odow

odow Dec 30, 2018

Author Member

Because it is in the docstring it needed to escape the \.

examples/minellipse.jl Outdated Show resolved Hide resolved
examples/minellipse.jl Outdated Show resolved Hide resolved
examples/run_examples.jl Outdated Show resolved Hide resolved
examples/urbanplan.jl Outdated Show resolved Hide resolved

odow added some commits Dec 30, 2018

@chriscoey

This comment has been minimized.

Copy link
Member

commented Dec 30, 2018

Perhaps for a separate PR: can I suggest a renaming of some examples to further improve consistency (eg underscores) and understandability. I think we can be verbose with names here.

Eg steelT3 - no idea what the T3 means, off the bat

@hochB

This comment has been minimized.

Copy link

commented Dec 30, 2018

I want to say that for newcomers examples are more helpful than anything else.
So a big thank you for working on this.

examples/sudoku.jl Outdated Show resolved Hide resolved
@odow

This comment has been minimized.

Copy link
Member Author

commented Dec 30, 2018

Perhaps for a separate PR: can I suggest a renaming of some examples to further improve consistency (eg underscores) and understandability. I think we can be verbose with names here.

Totally agree. I'm almost verging on these examples should be included in the documentation and structured as tutorial/notebooky type things. But let's get them tidied up and passing tests first.

odow added some commits Dec 30, 2018

@odow

This comment has been minimized.

Copy link
Member Author

commented Dec 30, 2018

Okay, all the examples have been updated to the latest JuMP master and are tested in a separate build stage on travis.

While I have tidied up many of the examples for style and more JuMPish conventions (see diet.jl whic uses DenseAxisArray to the data :)), I didn't fix all of them (e.g. prod.jl).

Once this PR is in, I might move a few of the examples into the docs with doctests. Maybe one MIP, one NLP, and one conic.

@blegat

This comment has been minimized.

Copy link
Member

commented Dec 31, 2018

Once this PR is in, I might move a few of the examples into the docs with doctests. Maybe one MIP, one NLP, and one conic.

With MockOptimizer ? So that the doctests keep their independence with solvers.

examples/run_examples.jl Outdated Show resolved Hide resolved
examples/sudoku.jl Outdated Show resolved Hide resolved
@mlubin

mlubin approved these changes Jan 1, 2019

@@ -4,7 +4,7 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
#############################################################################
# JuMP
# An algebraic modeling langauge for Julia
# An algebraic modeling language for Julia

This comment has been minimized.

Copy link
@mlubin

mlubin Jan 1, 2019

Member

I can't believe this typo was still here.

This comment has been minimized.

Copy link
@odow

odow Jan 1, 2019

Author Member

See #1720

@odow odow referenced this pull request Jan 1, 2019

@odow odow merged commit 9483613 into master Jan 2, 2019

5 checks passed

codecov/patch Coverage not affected when comparing cd8dbe5...8997c86
Details
codecov/project 68.72% (+0.02%) compared to cd8dbe5
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@odow odow deleted the od/test_examples branch Jan 2, 2019

@barpit20 barpit20 referenced this pull request Mar 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.