Adding explicit MINLP support #115

darioizzo · 2017-05-28T12:07:58Z

In reply to #112, this PR modifies the problem interface adding explicit support for MINLP type of problems. In order to do so the implemented logic is based on the following design drivers

All integers are represented as floats.
pagmo only check is that the bounds of a problem are integers for the int part.
pagmo guarantees to generate integers when constructing a population, but does not prevent floats to be used in the decision vector where integers are expected.
Hessians and gradients are untouched. So the user can define them for the integer part and find
the solution of the relaxed problem calling some NLP solver.
It is up to the algorithm to decide what to do with the integer part. By default, algorithms that are designed for continuous optimization will just treat it as a continuous variable, thus solving the relaxed problem.

bluescarni · 2017-06-03T19:00:50Z

doc/sphinx/docs/algorithm_list.rst

@@ -20,12 +21,12 @@ Differential Evolution (DE)                                :cpp:class:`pagmo::de
 Self-adaptive DE (jDE and iDE)                             :cpp:class:`pagmo::sade`                  :class:`pygmo.sade`                       S-U            
 Self-adaptive DE (de_1220 aka pDE)                         :cpp:class:`pagmo::de1220`                :class:`pygmo.de1220`                     S-U             
 Particle Swarm Optimization (PSO)                          :cpp:class:`pagmo::pso`                   :class:`pygmo.pso`                        S-U             
-(N+1)-ES Simple Evolutionary Algorithm                     :cpp:class:`pagmo::sea`                   :class:`pygmo.sea`                        S-U (sto)       
-Simple Genetic Algorithm                                   :cpp:class:`pagmo::sga`                   :class:`pygmo.sga`                        S-U (sto)       
+(N+1)-ES Simple Evolutionary Algorithm                     :cpp:class:`pagmo::sea`                   :class:`pygmo.sea`                        S-U   (sto)       


Seems like an extra space slipped in here.

bluescarni · 2017-06-03T19:03:41Z

doc/sphinx/docs/python/tutorials/coding_udp_minlp.rst

+We found a feasible solution!
+
+.. note::
+   The solution strategy above is, in general, flawed in assuming the best solution of the relaxed problem is colse to the 


"colse" -> "close"

bluescarni · 2017-06-03T19:03:56Z

doc/sphinx/docs/python/tutorials/coding_udp_minlp.rst

+
+.. note::
+   The solution strategy above is, in general, flawed in assuming the best solution of the relaxed problem is colse to the 
+   the full MINLP problem solution. More sophisticated techniques would instead search the combinatorial part more exhaustvely.


"exhaustively"

bluescarni · 2017-06-03T19:04:13Z

doc/sphinx/docs/python/tutorials/coding_udp_minlp.rst

+.. note::
+   The solution strategy above is, in general, flawed in assuming the best solution of the relaxed problem is colse to the 
+   the full MINLP problem solution. More sophisticated techniques would instead search the combinatorial part more exhaustvely.
+   We used here this approach only to show how simple is, in pygmo, to define and solve the relaxed problem and


"How simple it is"

bluescarni · 2017-06-03T19:04:50Z

doc/sphinx/docs/python/tutorials/using_archipelago.rst

@@ -109,7 +109,7 @@ After inspection, let us now run the evolution.
    array([  4.07802374, -23.82020921,  -4.07802374]),
    array([  1.71396489, -25.90794514,  -1.71396489]),
    ...]
-    >>> archi.evolve()
+    >>> archi.evolve() #doctest: +SKIP


Why do this (and in a bunch of places below) need to be skipped as no output is being produced?

It does not need to be skipped, I choose to skip it as it makes the test much longer and this call syntax is not likely to change ever.

bluescarni · 2017-06-03T19:33:59Z

include/pagmo/utils/generic.hpp

        retval[i] = uniform_real_from_range(bounds.first[i], bounds.second[i], r_engine);
    }
+    if (nix > 0u) {


I don't think this is the right way to go.

First, we are hard-coding the integer type to int via std::uniform_int_distribution<>. This effectively means restricting the range of the integer variables to 32 bits on modern platforms. This seems a rather important (and undocumented) implementation characteristic which may be limiting (?).

Secondly, the static casts are going to result in undefined behaviour if the floating point values are outside the int range.

What about instead generating numbers with a uniform real distribution in the specified range and calling something like std::round on them? I'd like to avoid going through a conversion to a C++ integer type, as that it's a shitcan to deal with and in the end we are always dealing with floating point values in the bounds, dvs etc. anyway.

Ok I then have to use floor and add 1 to the upper bound as to ensure the obtained distribution from int is uniform in the bounds.

bluescarni · 2017-06-03T19:35:07Z

include/pagmo/utils/generic.hpp

 *
 * @throws std::invalid_argument if:
- * - the bounds are not of equal length, they contain NaNs or infs, or \f$ \mathbf{ub} < \mathbf {lb}\f$,
+ * - the bounds are not of equal length, they have zero size, they contain NaNs or infs,


Missing failure modes.

added (except infinites)

bluescarni · 2017-06-03T19:37:11Z

pygmo/docstrings.cpp

+
+Raises:
+    ValueError: if *nix* is negative or greater than an implementation-defined value
+    ValueError: if *lb* and *ub* are malformed (unequal lenght, zero size, *nix* larger than len(lb) or bounds are not integers in their last *nix* components)


lenght-> length, missing failure mode, "len(lb)" should probably be in backticks as it's a fragment of code.

bluescarni · 2017-06-03T19:38:45Z

pygmo/test.py

+        self.assertTrue(int(x[1]) != x[1])
+        set_global_rng_seed(42)
+        y = random_decision_vector(lb = [1.1,2.1,-3], ub = [2.1, 3.4,5], nix = 1)
+        self.assertTrue((x == y).all())


Would be good to have a couple of tests for failure modes as well.

those are tested in the C++ part. I usually only test the boost interface in test.py

I would consider exception translation part of the boost interface.

bluescarni · 2017-06-03T19:39:05Z

tests/generic.cpp

@@ -71,16 +71,33 @@ BOOST_AUTO_TEST_CASE(random_decision_vector_test)
    BOOST_CHECK_THROW(random_decision_vector({{1, -inf}, {0, 32}}, r_engine), std::invalid_argument);
    BOOST_CHECK_THROW(random_decision_vector({{1, 2, 3}, {0, 3}}, r_engine), std::invalid_argument);
    BOOST_CHECK_THROW(random_decision_vector({{0, 2, 3}, {1, 4, nan}}, r_engine), std::invalid_argument);
+    BOOST_CHECK_THROW(random_decision_vector({{0, 2, nan}, {1, 4, 4}}, r_engine), std::invalid_argument);


Maybe add tests for the infinity checks?

So I think there is some confusion on how we deal with infinites.

Status quo:

It is allowed to have infinite bounds on the problem.

It is not allowed to draw at random a number if the bounds are infinite.

These cases are already tested on the c++ side.

With the addition of the integer part I would not change this behaviour and keep the possibility to have infinite bounds on the problem (also for integers), while preventing to draw random decision vectors from infinite bounds.

The reason for this seemingly asymmetric behaviour is that

It makes sense to signal to the optimizer algo.evolve() that the problem is unbounded.

It makes little sense, if the bounds are infinite, to construct a decision vector drawing randomly from it, better to have the user explicitly init the pop in that case.

Right, I forgot that the random vector stuff errors out with infinities.

bluescarni · 2017-06-03T19:40:14Z

@darioizzo looks good. The only real issue I have found is the one about generating random decision vectors (see comments).

bluescarni · 2017-06-04T10:14:00Z

include/pagmo/utils/generic.hpp

        retval[i] = uniform_real_from_range(bounds.first[i], bounds.second[i], r_engine);
    }
+    if (nix > 0u) {


Two notes:

I think you can skip the if (nix > 0u) because you have already the for cycle checking for the same condition right?

How about using std::nextafter for having a closed interval. See: http://en.cppreference.com/w/cpp/numeric/random/uniform_real_distribution/uniform_real_distribution

To create a distribution over the closed interval [a,b], std::nextafter(b, std::numeric_limits::max()) may be used as the second parameter.

bluescarni · 2017-06-04T10:14:32Z

include/pagmo/utils/generic.hpp

        retval[i] = uniform_real_from_range(bounds.first[i], bounds.second[i], r_engine);
    }
+    if (nix > 0u) {
+        for (decltype(nx) i = ncx; i < nx; ++i) {


auto i = ncx

bluescarni · 2017-06-04T10:15:22Z

tests/generic.cpp

@@ -71,16 +71,33 @@ BOOST_AUTO_TEST_CASE(random_decision_vector_test)
    BOOST_CHECK_THROW(random_decision_vector({{1, -inf}, {0, 32}}, r_engine), std::invalid_argument);
    BOOST_CHECK_THROW(random_decision_vector({{1, 2, 3}, {0, 3}}, r_engine), std::invalid_argument);
    BOOST_CHECK_THROW(random_decision_vector({{0, 2, 3}, {1, 4, nan}}, r_engine), std::invalid_argument);
+    BOOST_CHECK_THROW(random_decision_vector({{0, 2, nan}, {1, 4, 4}}, r_engine), std::invalid_argument);


Right, I forgot that the random vector stuff errors out with infinities.

bluescarni · 2017-06-04T10:16:22Z

pygmo/test.py

+        self.assertTrue(int(x[1]) != x[1])
+        set_global_rng_seed(42)
+        y = random_decision_vector(lb = [1.1,2.1,-3], ub = [2.1, 3.4,5], nix = 1)
+        self.assertTrue((x == y).all())


I would consider exception translation part of the boost interface.

darioizzo added 19 commits May 27, 2017 19:41

null problem now with integer part

fe09708

test null_problem c++ updated

85357b7

type trait has_integer_part added

60e4c2f

method get_nix added to the problem API

c82391e

C++ tests added fro the new problem class

33fefbb

random_vector now generates also the integer part

6393e4a

check bounds now checks the integer part

11e9f32

c++ test added for random_decision_vector (int part)

386e4a8

C++ tests finalized

92b3fd0

python exposition step

bdcbc22

python docs update

542a9a5

doctest fixes

937e7f5

int - double conversion fix

48581f6

small CI fix

096e229

WIP minlp problem

089eaf2

test no compiles

a39e624

rastrigin minlp added

02d2e0f

zdt5 updated

d4b3121

fix for test failiure

c9cb383

darioizzo mentioned this pull request May 30, 2017

estimate_sparsity #116

Open

darioizzo added 2 commits May 30, 2017 16:06

radnom decision vector exposition and test

10173a8

radnom decision vector exposition and test

bd38a57

darioizzo mentioned this pull request May 30, 2017

Improve support to integers #112

Closed

4 tasks

darioizzo changed the title ~~Adding explicit integer programming support~~ Adding explicit MINLP support May 30, 2017

darioizzo added 6 commits May 30, 2017 17:07

nsga2 simplified to the new integer support mechanism

42ce8b4

sga now uses the new integer interface

53c7f45

sga now uses the new integer interface

145aac5

fixes for failing CI

2c31b66

new tutorial on minlp (WIP)

85571a0

Merge remote-tracking branch 'upstream/master' into integer

fb41a8f

darioizzo added 3 commits May 31, 2017 22:28

End of tutorial

270798f

Alg list update

d286472

[skip CI] Alg list update

1277ee1

darioizzo requested a review from bluescarni June 1, 2017 06:50

[ci skip] tutorial and pythin docs fixes

b9da121

bluescarni requested changes Jun 3, 2017

View reviewed changes

darioizzo added 5 commits June 4, 2017 12:40

Merge remote-tracking branch 'upstream/master' into integer

73876b4

[ci skip] fixing merge

5ea3f61

[ci skip] fixing merge

70cc4ca

[ci skip] review part 1

c794955

review end

4f0e547

bluescarni requested changes Jun 4, 2017

View reviewed changes

darioizzo added 3 commits June 4, 2017 22:07

review 2

3abf043

review3 - still not clear what to do on the python assert

e22bc0c

review3 - python assert fixed

8b62e15

bluescarni approved these changes Jun 4, 2017

View reviewed changes

bluescarni merged commit 729e4c6 into esa:master Jun 4, 2017

darioizzo deleted the integer branch June 5, 2017 04:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding explicit MINLP support #115

Adding explicit MINLP support #115

darioizzo commented May 28, 2017

bluescarni Jun 3, 2017

bluescarni Jun 3, 2017

darioizzo Jun 4, 2017

bluescarni Jun 3, 2017

darioizzo Jun 4, 2017

bluescarni Jun 3, 2017

darioizzo Jun 4, 2017

bluescarni Jun 3, 2017

darioizzo Jun 4, 2017 •

edited

Loading

bluescarni Jun 3, 2017

darioizzo Jun 4, 2017

darioizzo Jun 4, 2017

bluescarni Jun 3, 2017

darioizzo Jun 4, 2017

darioizzo Jun 4, 2017

bluescarni Jun 3, 2017

bluescarni Jun 3, 2017

darioizzo Jun 4, 2017

bluescarni Jun 4, 2017

bluescarni Jun 3, 2017

darioizzo Jun 4, 2017

bluescarni Jun 4, 2017

bluescarni commented Jun 3, 2017

bluescarni Jun 4, 2017

bluescarni Jun 4, 2017

bluescarni Jun 4, 2017

bluescarni Jun 4, 2017

Adding explicit MINLP support #115

Adding explicit MINLP support #115

Conversation

darioizzo commented May 28, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

darioizzo Jun 4, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bluescarni commented Jun 3, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

darioizzo Jun 4, 2017 •

edited

Loading