CMAES inconsistency fix #88

favre49 · 2019-03-01T08:12:31Z

This is a fix for issue #70 . The boundary handler has been adapted from the python code for CMA-ES
I had to change the bounds in the test functions since they did not align with the solution.
I will update the documentation for the code in a couple hours if there are no problems

zoq · 2019-03-05T23:40:57Z

You can put the update right after the Fixes for SPSA (#87). line.

zoq · 2019-03-05T23:41:57Z

Ah wrong PR.

favre49 · 2019-03-08T08:30:43Z

weird error here, the sudo apt-get update failed?

rcurtin

Hey @favre49, sorry the response took so long here. It's looking good to me and it'll be nice to fix these issues, but I have a few questions still. Let me know what you think. 👍

rcurtin · 2019-03-24T21:34:00Z

include/ensmallen_bits/cmaes/cmaes_impl.hpp

@@ -129,7 +129,7 @@ double CMAES<SelectionPolicyType>::Optimize(
    const size_t idx1 = i % 2;

    // Perform Cholesky decomposition. If the matrix is not positive definite,
-    // add a small value and try again
+    // add a small value and try again.


Thanks for adding that. :)

rcurtin · 2019-03-24T21:34:34Z

include/ensmallen_bits/cmaes/cmaes.hpp

@@ -146,6 +166,16 @@ class CMAES

  //! The selection policy used to calculate the objective.
  SelectionPolicyType selectionPolicy;
+
+  //! Step size damping.
+  double ds;


Just for consistency, it might be beter to name this stepSizeDamping since the method to access it is StepSizeDamping(). (Similar for CumulationConstant().) If we are exposing these parameters, should we also add constructor parameters for these also? I think maybe I don't fully follow what these parameters are for (or how a user would set them).

I'll change the name for sure. I included those two based on Marcus' suggestion in the issue discussion. These parameters are used in the update of the variables. It provides some extra flexibility for the user, but I wonder if including them in the constructor would become more confusing to the user, or how useful they would be (I use CMA-ES a lot for my work, and have never really toyed with these values). Maybe @zoq can shed some light on this?

Ah, that's fair. It doesn't matter to me if the variables are included or not, but if we do include them then I think we should make them accessible from the constructor. 👍

rcurtin · 2019-03-24T21:34:46Z

include/ensmallen_bits/cmaes/cmaes.hpp

  size_t PopulationSize() const { return lambda; }
-  //! Modify the step size.
+  //! Modify the population size.


Nice catch, thanks! 👍

rcurtin · 2019-03-24T21:44:48Z

include/ensmallen_bits/cmaes/cmaes.hpp

@@ -70,6 +71,7 @@ class CMAES
   *     objective.
   */
  CMAES(const size_t lambda = 0,
+        const double initialSigma = 0.6,


I'm concerned that this would break reverse compatibility and we'd have to bump the major version (I don't think that's justified by a relatively simple change like this). According to our versioning policy,

* A MAJOR version bump implies a backwards-incompatible change to the user-facing optimizers. Code using one major version of ensmallen may not work with a newer major version of ensmallen.

We could instead add initialSigma as the last parameter with a default value, and then we could optionally mark that constructor deprecated ("will be removed in 2.10.0") and add another constructor just like this one here. What do you think?

Yup, didn't think of that. I'll change it :)

rcurtin · 2019-03-24T21:47:53Z

include/ensmallen_bits/cmaes/cmaes_impl.hpp

  const double cs = (muEffective + 2) / (iterate.n_elem + muEffective + 5);
-  const double ds = 1 + cs + 2 * std::max(std::sqrt((muEffective - 1) /
+  ds = 1 + cs + 2 * std::max(std::sqrt((muEffective - 1) /
      (iterate.n_elem + 1)) - 1, 0.0);


I guess, then, that if the user has set ds or cc, they get overwritten here when Optimize() is called?

rcurtin · 2019-03-24T21:49:05Z

include/ensmallen_bits/cmaes/cmaes_impl.hpp

+      double x = iterate(it,ii);
+
+      // Shift x into [lowerBound-al, upperBound+au].
+      if (x < lowerBound - 2 * al - (upperBound - lowerBound) / 2 || x > upperBound + 2 * au + (upperBound - lowerBound) / 2)


Style issue---this line is over 80 characters long. Can you wrap it please?

rcurtin · 2019-03-24T21:49:15Z

include/ensmallen_bits/cmaes/cmaes_impl.hpp

+  {
+    for (size_t it = 0; it < iterate.n_rows; it++)
+    {
+      double x = iterate(it,ii);


Tiny style issue, this should be iterate(it, ii).

rcurtin · 2019-03-24T21:50:20Z

include/ensmallen_bits/cmaes/cmaes_impl.hpp

@@ -283,6 +287,79 @@ double CMAES<SelectionPolicyType>::Optimize(
  return overallObjective;
 }

+// Transforms the candidate into the given bounds.
+template<typename SelectionPolicyType>
+void CMAES<SelectionPolicyType>::BoundaryTransform(arma::mat& iterate)


So, if I'm understanding that this function, given iterate, will transform each element of iterate such that it falls between lowerBound and upperBound, you could just use arma::clamp(), I think. Correct me if I'm wrong and there's a bit that I missed. (It looks like maybe there is some handling for when upperBound and lowerBound are too close together? I have not figured out exactly what that does.)

rcurtin · 2019-03-24T21:51:08Z

include/ensmallen_bits/cmaes/cmaes_impl.hpp

      pObjective(idx(j)) = selectionPolicy.Select(function, batchSize,
          pPosition.slice(idx(j)));
+      BoundaryTransformInverse(pPosition.slice(idx(j)));


Hmm, I guess I'm a little confused here. I would imagine it would be easier to take pPosition.slice(idx(j)) and simply clamp it to be within the desired range, but I'm not sure what the reverse transformation is. Can you clarify the functionality here a bit? Maybe I misunderstood.

mlpack-bot · 2019-07-19T01:53:36Z

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions! 👍

gaurav-singh1998 · 2020-03-24T07:45:51Z

Hi, @favre49, @zoq, @rcurtin since @favre49 has implemented and addressed the majority part regarding the issue #70 I want to take up this and implement the necessary changes that @rcurtin suggested above. Can I move forward with it?

gaurav-singh1998 · 2020-03-24T07:51:58Z

Also, @favre49 can you provide the link of the source code or the paper that you referred for applying box constraints for each parameter here?

favre49 · 2020-03-24T10:38:52Z

@gaurav-singh1998 That would be great. I'll try to find the sources I used and get back to you

favre49 · 2020-03-24T10:40:46Z

If I'm not wrong, I used pycma

favre49 · 2020-03-24T10:45:41Z

Also, what I failed to address in Ryan's comment pertained to how Nikolas Hansen, the author of the paper on CMA-ES advised handling bound constraints. I am unable to find the resources that I read this from, but the gist of it was that at every step, the result obtained would be "projected" into another space, where it would exist in the bounds (if I remember correctly). The way this has been done is in my code (though probably flawed) as well as in the pycma code. If I'm able to find it again, I'll let you know.

gaurav-singh1998 · 2020-04-02T07:46:04Z

Hi, @favre49 sorry for responding so late on this. After going through this implementation of boundary handling and an example which shows how to use it in the final CMAES implementation I found that they are not applying the inverse function after transforming them into the given box constraints like you have implemented here and here. I may be wrong somewhere or missing something but kindly help me with this before moving forward. Thanks.

favre49 · 2020-04-02T09:25:57Z

I think instead of calling inverse here they write the same thing as in the function here. Not sure why, though

gaurav-singh1998 · 2020-04-02T11:30:33Z

Hi, @favre49 in the libcmaes library, C++ version of CMAES by N Hansen, box constraints are handles in the same way i.e. they just provided a function for the inverse here. So, before moving further with this wanted to clarify should I also apply this concept for handling the box constraints? Thanks.

favre49 · 2020-04-03T15:56:39Z

Yeah, I think that would make sense

gaurav-singh1998 · 2020-04-03T18:53:34Z

Hi, @favre49 sorry to disturb you again and again but wanted to know can I open a new PR for this?

favre49 · 2020-04-04T04:45:16Z

No issues, I'll close this when you open your PR

favre49 added 4 commits February 25, 2019 21:59

documentation fix

ca03d06

fixes for the issue besides transformation

6e58ad6

Boundary handler

c3845cb

changes test params

442d589

zoq mentioned this pull request Mar 5, 2019

CMA-ES inconsistencies #70

Open

favre49 added 2 commits March 6, 2019 20:04

Added parameter for initial step size

9f4a65a

Added some full stops.

67936de

favre49 added 4 commits March 8, 2019 23:05

More full stops.

2312805

Fixed examples

14f9c79

Style fixes

d9062c8

Removed redundant newline

75955ea

rcurtin added c: optimizers s: needs review t: bugfix labels Mar 10, 2019

rcurtin reviewed Mar 24, 2019

View reviewed changes

rcurtin removed the s: needs review label Mar 25, 2019

mlpack-bot bot added the s: stale label Jul 19, 2019

zoq added s: keep open and removed s: stale labels Jul 19, 2019

gaurav-singh1998 mentioned this pull request Apr 8, 2020

Fix for CMA-ES inconsistencies. #193

Closed

conradsnicta added s: stale and removed s: keep open labels Jul 12, 2021

mlpack-bot bot closed this Jul 19, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CMAES inconsistency fix #88

CMAES inconsistency fix #88

favre49 commented Mar 1, 2019 •

edited

Loading

zoq commented Mar 5, 2019

zoq commented Mar 5, 2019

favre49 commented Mar 8, 2019

rcurtin left a comment

rcurtin Mar 24, 2019

rcurtin Mar 24, 2019

favre49 Mar 25, 2019

rcurtin Apr 2, 2019

rcurtin Mar 24, 2019

rcurtin Mar 24, 2019

favre49 Mar 25, 2019

rcurtin Mar 24, 2019

rcurtin Mar 24, 2019

rcurtin Mar 24, 2019

rcurtin Mar 24, 2019

rcurtin Mar 24, 2019

mlpack-bot bot commented Jul 19, 2019

gaurav-singh1998 commented Mar 24, 2020

gaurav-singh1998 commented Mar 24, 2020

favre49 commented Mar 24, 2020

favre49 commented Mar 24, 2020

favre49 commented Mar 24, 2020

gaurav-singh1998 commented Apr 2, 2020 •

edited

Loading

favre49 commented Apr 2, 2020

gaurav-singh1998 commented Apr 2, 2020 •

edited

Loading

favre49 commented Apr 3, 2020

gaurav-singh1998 commented Apr 3, 2020

favre49 commented Apr 4, 2020

CMAES inconsistency fix #88

CMAES inconsistency fix #88

Conversation

favre49 commented Mar 1, 2019 • edited Loading

zoq commented Mar 5, 2019

zoq commented Mar 5, 2019

favre49 commented Mar 8, 2019

rcurtin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mlpack-bot bot commented Jul 19, 2019

gaurav-singh1998 commented Mar 24, 2020

gaurav-singh1998 commented Mar 24, 2020

favre49 commented Mar 24, 2020

favre49 commented Mar 24, 2020

favre49 commented Mar 24, 2020

gaurav-singh1998 commented Apr 2, 2020 • edited Loading

favre49 commented Apr 2, 2020

gaurav-singh1998 commented Apr 2, 2020 • edited Loading

favre49 commented Apr 3, 2020

gaurav-singh1998 commented Apr 3, 2020

favre49 commented Apr 4, 2020

favre49 commented Mar 1, 2019 •

edited

Loading

gaurav-singh1998 commented Apr 2, 2020 •

edited

Loading

gaurav-singh1998 commented Apr 2, 2020 •

edited

Loading