Add package for linear programming solving and Simplex implementation

[btracey]

No description provided.

[btracey]

PTAL @vladimir-ch

There are lots and lots of minor cleanups that can be done. I'm afraid that if too many are done, it'll be hard to understand the algorithm at the first pass. I figure like this you have a hope of doing a look-through and afterwards cleanup can happen.

It's been a while since I was working with this code, but at the time I had done extensive testing on it. There are still cases that fail, though some of them also cause the matlab LP solver to crash. You can see that the tests are generating a 5 digit number of tests, so I'm pretty sure the algorithm is correct (if not perfectly robust to numeric error).

[sbinet]

license header?

[btracey]

Done.

[sbinet]

I would perhaps add an example:

// Copyright ©2016 The gonum Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package lp_test

import (
    "fmt"
    "log"

    "github.com/gonum/matrix/mat64"
    "github.com/gonum/optimize/convex/lp"
)

func Example_Simplex() {
    c := []float64{-1, -2, 0, 0}
    A := mat64.NewDense(2, 4, []float64{-1, 2, 1, 0, 3, 1, 0, 1})
    b := []float64{4, 9}

    opt, x, err := lp.Simplex(c, A, b, 0, nil)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("opt: %v\n", opt)
    fmt.Printf("x: %v\n", x)
    // Output:
    // opt: -8
    // x: [2 3 0 0]
}

tested with scipy.optimize.linprog:
http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linprog.html#scipy.optimize.linprog

[sbinet]

perhaps also add an article/book reference?

[btracey]

Do you have one in mind? In the documentation, I like to link references that are actually useful, and I learned the simplex from those lectures.

[sbinet]

the canonical one seems to be:

J.A. Nelder and R. Mead. A simplex method for function minimization. Comput. J., 7:308, 1965.

in my field, we usually just refer to MINUIT:

• https://en.wikipedia.org/wiki/MINUIT
• http://seal.web.cern.ch/seal/documents/minuit/mntutorial.pdf

[btracey]

Wrong simplex method :). This is Danzig's Simplex, not Nelder-Mead

[btracey]

Done.

[sbinet]

ha! :)

[kortschak]

In the opening sentence of the doc comment it looks like you are aligning with tabs. Please do that with spaces; tabs are not always rendered the same width on all platforms.

[sbinet]

-Standard Frm
+Standard Form

[btracey]

Done.

[vladimir-ch]

I quickly skimmed through the code and pointed out a couple of things that I saw. I think that the algorithm is quite clear from the code although I didn't go into the details. The algorithm can be surely made more efficient by updating the factorization but given our limited manpower that's a future work. There are many possible cleanups, especially in the docs, too many to point out individually. So I think it is best to merge this and fix things in another PR. So LGTM.

[btracey]

I added your example @sbinet