ElectricalMix Simulator using nevergrad

[Foloso]

pull request to apply for the Nevergrad competition

Types of changes

• Docs change / refactoring / dependency upgrade
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Motivation and Context / Related issue

How Has This Been Tested (if it applies)

Checklist

• The documentation is up-to-date with the changes I made.
• I have read the CONTRIBUTING document and completed the CLA (see CLA).
• All tests passed, and additional code has been covered with new tests.

[facebook-github-bot]

Hi @Foloso!

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file.

In order for us to review and merge your code, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

[teytaud]

You have to check the CLA, this should be easy :-)
Very interesting work, I'll comment in your github repository, this is definitely one of the priorities for Nevergrad!

[facebook-github-bot]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

[Foloso]

You have to check the CLA, this should be easy :-)
Very interesting work, I'll comment in your github repository, this is definitely one of the priorities for Nevergrad!

thank you

[jrapin]

Thanks for adding this ;)
@teytaud let's merge it, but soon we'll need to have a dedicated page for this, it will be easier to find than in the machine learning part of the doc

[TokyAxel]

Hi! :-)
These plots are the results of a simulation based on data from the Toamasina Madagascar Inter-connected Mix. We can distinguish the current and "optimal" use coefficients as well as performance indices (price of electricity production, environmental impact, satisfaction of demand), In the case of satisfaction of demand, a negative number means an overproduction (production> demand) and a positive number means a production deficit (production <demand).

Note: The current values ​​of the usage coefficients are entirely arbitrary (we have not yet received the exact values) and some informations from the dataset are estimated.

We have two graphs here: the first is based on the OnePlusOne optimizer from nevergrad and the second is based on the sequence of OnePlusOne and DE.

[Foloso]

Hello, we've just integrated more optimizers from Nevergrad.
Here is the list of available optimizers:
['cGA', 'SplitOptimizer', 'RecombiningPortfolioOptimisticNoisyDiscreteOnePlusOne', 'RecES', 'RealSpacePSO', 'RandomSearchPlusMiddlePoint', 'QrDE', 'QORandomSearch', 'OptimisticNoisyOnePlusOne', 'OptimisticDiscreteOnePlusOne', 'ORandomSearch', 'NoisyOnePlusOne', 'NoisyDiscreteOnePlusOne', 'NoisyDE', 'NoisyBandit', 'NelderMead', 'NaiveTBPSA', 'NaiveIsoEMNA', 'LhsDE', 'FCMA', 'ES', 'DoubleFastGADiscreteOnePlusOne', 'DiscreteOnePlusOne', 'CauchyOnePlusOne', 'CM', 'AlmostRotationInvariantDE', 'TwoPointsDE', 'RandomSearch', 'OnePlusOne', 'DE', 'CMA', 'PSO', 'TBPSA']

[Foloso]

hey :-) we've just added a method to compare results from differents optimizers :-)

[teytaud]

hey :-) we've just added a method to compare results from differents optimizers :-)

Just a remark: I think you plot the objective values of points obtained through "ask".
This is cool, but this includes points with bad values, just tested for exploration. You might
either use the value obtained by optimizer.recommend() (but this require re-running your experiments), or
simply plot for x-axis t the best value met up to time t, rather than the last visited point.

That's a simple thing and your results will be much cleaner :-)

[teytaud]

Hello, we've just integrated more optimizers from Nevergrad.
Here is the list of available optimizers:
['cGA', 'SplitOptimizer', 'RecombiningPortfolioOptimisticNoisyDiscreteOnePlusOne', 'RecES', 'RealSpacePSO', 'RandomSearchPlusMiddlePoint', 'QrDE', 'QORandomSearch', 'OptimisticNoisyOnePlusOne', 'OptimisticDiscreteOnePlusOne', 'ORandomSearch', 'NoisyOnePlusOne', 'NoisyDiscreteOnePlusOne', 'NoisyDE', 'NoisyBandit', 'NelderMead', 'NaiveTBPSA', 'NaiveIsoEMNA', 'LhsDE', 'FCMA', 'ES', 'DoubleFastGADiscreteOnePlusOne', 'DiscreteOnePlusOne', 'CauchyOnePlusOne', 'CM', 'AlmostRotationInvariantDE', 'TwoPointsDE', 'RandomSearch', 'OnePlusOne', 'DE', 'CMA', 'PSO', 'TBPSA']

Interesting :-) you might remove "OptimisticDiscreteOnePlusOne" and "Noisy*" and "TBPSA" because I guess your objective function is deterministic and these algorithms are for the deterministic case.
"NGOpt" could be added as it's the pointer to the best algorithm so far overall (this does not mean that it will be the best here, but it's often a strong competitor!).

[teytaud]

hey :-) we've just added a method to compare results from differents optimizers :-)

Just a remark: I think you plot the objective values of points obtained through "ask".
This is cool, but this includes points with bad values, just tested for exploration. You might
either use the value obtained by optimizer.recommend() (but this require re-running your experiments), or
simply plot for x-axis t the best value met up to time t, rather than the last visited point.

That's a simple thing and your results will be much cleaner :-)

Sorry! I was wrong, you are actually correctly using everything.
The result is a bit noisy but this is probably just because there is a single run per x-axis value, without averaging.

[Foloso]

hey :-) we've just added a method to compare results from differents optimizers :-)

Just a remark: I think you plot the objective values of points obtained through "ask".
This is cool, but this includes points with bad values, just tested for exploration. You might
either use the value obtained by optimizer.recommend() (but this require re-running your experiments), or
simply plot for x-axis t the best value met up to time t, rather than the last visited point.
That's a simple thing and your results will be much cleaner :-)

Sorry! I was wrong, you are actually correctly using everything.
The result is a bit noisy but this is probably just because there is a single run per x-axis value, without averaging.

Hello :-)
We tried to make it less noisy

[teytaud]

The cost you are optimizing is the sum of these 3 costs, right ? maybe a curve with this sum would be cool as it would be a fair comparison between methods.

[teytaud]

If the computational cost is an issue maybe there is a solution: if you package your code so that we can pip install it and pip import it, then we can include it inside nevergrad, and it will be run automatically each time someone runs the dashboard (plenty of algorithms, sufficiently replicas for significance). Up to you :-)

[Foloso]

The cost you are optimizing is the sum of these 3 costs, right ? maybe a curve with this sum would be cool as it would be a fair comparison between methods.

Actually we are only optimizing the production_cost. But we are using nevergrad register_cheap_constraint() method for the carbon_impact constraint and the demand_satisfaction_constraint.

[Foloso]

Hello :-) , here is the new list of supported Optimizers from now.

['ASCMA2PDEthird', 'ASCMADEQRthird', 'ASCMADEthird', 'AdaptiveDiscreteOnePlusOne', 'AlmostRotationInvariantDE', 'CM', 'CMA', 'CMandAS', 'CMandAS2', 'CMandAS3', 'CauchyLHSSearch', 'CauchyOnePlusOne', 'CauchyScrHammersleySearch', 'Cobyla', 'DE', 'DiagonalCMA', 'DiscreteBSOOnePlusOne', 'DiscreteOnePlusOne', 'DoubleFastGADiscreteOnePlusOne', 'EDA', 'ES', 'FCMA', 'LHSSearch', 'LhsDE', 'MEDA', 'MPCEDA', 'MetaModel', 'MixES', 'MultiCMA', 'NGOpt', 'NaiveIsoEMNA', 'NaiveTBPSA', 'NelderMead', 'NoisyBandit', 'NoisyDE', 'NoisyDiscreteOnePlusOne', 'NoisyOnePlusOne', 'ORandomSearch', 'OnePlusOne', 'OptimisticDiscreteOnePlusOne', 'OptimisticNoisyOnePlusOne', 'PCEDA', 'PSO', 'PolyCMA', 'QORandomSearch', 'QrDE', 'RandomSearch', 'RandomSearchPlusMiddlePoint', 'RealSpacePSO', 'RecES', 'RecMixES', 'RecombiningPortfolioOptimisticNoisyDiscreteOnePlusOne', 'RotationInvariantDE', 'SplitOptimizer', 'TBPSA', 'TripleCMA', 'TwoPointsDE', 'cGA']

We are still working on the others

[Foloso]

If the computational cost is an issue maybe there is a solution: if you package your code so that we can pip install it and pip import it, then we can include it inside nevergrad, and it will be run automatically each time someone runs the dashboard (plenty of algorithms, sufficiently replicas for significance). Up to you :-)

We are currently working on it :-)

[Foloso]

If the computational cost is an issue maybe there is a solution: if you package your code so that we can pip install it and pip import it, then we can include it inside nevergrad, and it will be run automatically each time someone runs the dashboard (plenty of algorithms, sufficiently replicas for significance). Up to you :-)

Hello, you can now a pip import it :-) the package name is0 "mixsimulator"