DEV: Add "Particle Swarm Optimization" algorithm to SciPy.optimization #12821
Conversation
Added a new optimization algorithm and some tests to ensure the correct behaviour of the code. Also the __init__.py file of SciPy.optimization has been modified to include this new feature.
|
@damarro3 thanks for your interest in contributing. I might not be able to review this, but one thing that would help us decide whether to include it would be to compare this against existing global optimizers. Can you add it to the benchmarks, run them, and share the results? Please add the function name to the list of global optimizers in You can check to see that it renders properly by clicking "Details" of the ci/circleci: build_docs_artifact link.
|
|
Hi @damarro3, thanks for opening this PR! Adding a particle swarm optimization was discussed a few months back on the mailing list, with mildly positive feedback: https://mail.python.org/pipermail/scipy-dev/2020-May/024173.html. Note that there's a detailed reply from @andyfaff to that first message with steps to take for adding a new optimization method. In particular, adding it to the benchmarks and showing that the performance (in terms of number of problems it solves and amount of function evaluations) is competitive. There was also an unanswered question from me about a comparison to PyGMO. |
|
Thank you @mdhaber and @rgommers, I didn't know about those specific benchmarks. I will implement them and get the results to check if the algorithm is competitive. Regarding to the PyGMO comparison, I thought about adding this method because is quite used in literature and not everybody install new packages in order to use a general "black box" optimization method. |
|
@damarro3, the email topic pointed to by Ralf, is the sequence of steps that we should follow before anything is added. In future, it's best to bring up proposed changes on the scipy-dev mailing list first, before any significant amount of code is written. |
|
From my experience adding the |
There was a problem hiding this comment.
I just glanced at a few things, so it's not a comprehensive review. Before you go any further can you set it up to run against the global optimization benchmarks? New global optimizers need to be run against the benchmark suite to show that they offer a performance advantage to the existing minimizer functions in order for them to be merged.
i.e. a greater rate of success, or smaller amount of function evaluations (for a reasonable success), etc.
| Initial bounds for variables, inside the global search bounds. If it is | ||
| not specified, the initial bounds are settle as the global bounds. | ||
| Initial bounds for variables, inside the global search bounds. If it is | ||
| not specified, the initial bounds are settle as the global bounds. | ||
| There are two ways to specify the initial bounds: | ||
| 1. Instance of `Bounds` class. | ||
| 2. ``(min, max)`` pairs for each element in ``x``, defining the finite |
There was a problem hiding this comment.
I would only accept the Bounds class for this, it's a more modern way of doing things.
| self.bestof = bestof | ||
| self.func = func | ||
|
|
||
| def updatevel(self, w, c1, c2, bestgpos, vmin, vmax): |
| self.velocity = np.maximum(self.velocity, vmin) | ||
| self.velocity = np.minimum(self.velocity, vmax) | ||
|
|
||
| def updatepos(self, pmin, pmax): |
|
|
||
| pos_value = self.func(self.position) | ||
| if pos_value <= self.bestof: | ||
| self.bestof = pos_value |
| pos_value = self.func(self.position) | ||
| if pos_value <= self.bestof: | ||
| self.bestof = pos_value | ||
| self.bestpos = self.position |
There was a problem hiding this comment.
best_pos.
Although if best_pos is actually x, then why not use best_x?
|
|
||
| # test w | ||
| if self.w <= 0: | ||
| if (((self.velocity_strategy == 'constant') or (self.velocity_strategy == 'random')) |
There was a problem hiding this comment.
self.velocity_strategy in ['constant', 'random']:
| bestof=self.func(init_position), | ||
| func=self.func) | ||
|
|
||
| def _update(self): |
There was a problem hiding this comment.
It'd be preferable if the object was iterable, implementing __iter__ and __next__.
| """ Update of the swarm particles """ | ||
| for i in range(self.n_particles): | ||
| self._particle_update(i) | ||
| if self.particles[i].bestof <= self.bestgof: |
There was a problem hiding this comment.
Is this parallelisable? If so, then we should use a workers keyword to enable parallelisation (see differential_evolution).
| self.bestgpos = self.particles[i].bestpos | ||
| self.bestgof = self.particles[i].bestof | ||
|
|
||
| self.fevals += self.n_particles |
| self.bestgof = self.particles[i].bestof | ||
|
|
||
| self.fevals += self.n_particles | ||
| self.it += 1 |
|
I have executed the benchmarks. As soon as they finish, I will post the results. |
What does this implement/fix?
Added a new optimization algorithm and some tests to ensurethe correct behaviour of the code.
Additional information
Also the init.py file of
SciPy.optimization has been modified to include this new
feature.