Skip to content

Commit

Permalink
DQD Emitter batch_size description (#336)
Browse files Browse the repository at this point in the history
## Description

<!-- Provide a brief description of the PR's purpose here. -->
We originally had batch_size for DQD emitter include the solution
returned by ask_dqd, i.e., if you want ask to return 36 solutions you
need to put batch_size=37. We reverted this decision and now assume that
ask_dqd only returns one solution, i.e., you will put batch_size=36 for
the above example. This make the batch_size description between DQD and
non-DQD emitters consistent.

However, at some point we must have forgot to change the docstrings
according. This PR address that.

**Furthermore, I find that some configuration in sphere.py is not
consistent with the original paper.** For example, the [CMA-MAE
paper](https://arxiv.org/pdf/2205.10752.pdf) "select[s] a batch size
$\lambda = 36$ following prior work" (Appendix A), while [this
line](https://github.com/icaros-usc/pyribs/blob/7099f1ab7679d557f41a9608b9cdaaf075d30981/examples/sphere.py#L490C1-L490C26)
clearly uses $\lambda=37$. Similarly, our sphere experiment with
CMA-MAEGA also uses $\lambda=37$.

I could be missing something here, so I figured I should try to clarify
in this PR in case other users have same confusion as me.

## TODO

<!-- Notable points that this PR has either accomplished or will
accomplish. -->

## Questions

<!-- Any concerns or points of confusion? -->

## Status

- [x] I have read the guidelines in
[CONTRIBUTING.md](https://github.com/icaros-usc/pyribs/blob/master/CONTRIBUTING.md)
- [x] I have formatted my code using `yapf`
- [x] I have tested my code by running `pytest`
- [x] I have linted my code with `pylint`
- [ ] ~I have added a one-line description of my change to the changelog
in `HISTORY.md`~
- [x] This PR is ready to go
  • Loading branch information
itsdawei committed Jul 7, 2023
1 parent 4385f02 commit b1fe409
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 24 deletions.
22 changes: 7 additions & 15 deletions examples/sphere.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,17 +43,9 @@
- `cma_maega`: GridArchive (learning_rate = 0.01) with
GradientArborescenceEmitter using ImprovementRanker.
All algorithms use 15 emitters, each with a batch size of 37. Each one runs for
4500 iterations for a total of 15 * 37 * 4500 ~= 2.5M evaluations.
Notes:
- `cma_mega` and `cma_mega_adam` use only one emitter and run for 10,000
iterations. This is to be consistent with the paper (`Fontaine 2021
<https://arxiv.org/abs/2106.03894>`_) in which these algorithms were proposed.
- `cma_mae` and `cma_maega` run for 10,000 iterations as well.
- CVTArchive in this example uses 10,000 cells, as opposed to the 250,000
(500x500) in the GridArchive, so it is not fair to directly compare
`cvt_map_elites` and `line_cvt_map_elites` to the other algorithms.
The parameters for each algorithm are stored in CONFIG. The parameters
reproduce the experiments presented in the paper in which each algorithm is
introduced.
Outputs are saved in the `sphere_output/` directory by default. The archive is
saved as a CSV named `{algorithm}_{dim}_archive.csv`, while snapshots of the
Expand Down Expand Up @@ -431,7 +423,7 @@
"archive_dims": (100, 100),
"use_result_archive": False,
"is_dqd": True,
"batch_size": 36,
"batch_size": 35,
"archive": {
"class": GridArchive,
"kwargs": {
Expand Down Expand Up @@ -459,7 +451,7 @@
"archive_dims": (100, 100),
"use_result_archive": False,
"is_dqd": True,
"batch_size": 36,
"batch_size": 35,
"archive": {
"class": GridArchive,
"kwargs": {
Expand Down Expand Up @@ -487,7 +479,7 @@
"archive_dims": (100, 100),
"use_result_archive": True,
"is_dqd": False,
"batch_size": 37,
"batch_size": 36,
"archive": {
"class": GridArchive,
"kwargs": {
Expand Down Expand Up @@ -516,7 +508,7 @@
"archive_dims": (100, 100),
"use_result_archive": True,
"is_dqd": True,
"batch_size": 37,
"batch_size": 35,
"archive": {
"class": GridArchive,
"kwargs": {
Expand Down
17 changes: 8 additions & 9 deletions ribs/emitters/_gradient_arborescence_emitter.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

class GradientArborescenceEmitter(EmitterBase):
"""Generates solutions with a gradient arborescence, with coefficients
parameterized by an ES.
parameterized by an evolution strategy.
This emitter originates in `Fontaine 2021
<https://arxiv.org/abs/2106.03894>`_. It leverages the gradient information
Expand Down Expand Up @@ -100,10 +100,9 @@ class GradientArborescenceEmitter(EmitterBase):
an open problem. Hence, this argument must be set to None.
batch_size (int): Number of solutions to return in :meth:`ask`. If not
passed in, a batch size will be automatically calculated using the
default CMA-ES rules. Note that `batch_size` **does not** include
the number of solutions returned by :meth:`ask_dqd`, but also note
that :meth:`ask_dqd` always returns one solution, i.e. the solution
point.
default CMA-ES rules. This **does not** account for the **one**
solution returned by :meth:`ask_dqd`, which is the solution point
maintained by the gradient optimizer.
epsilon (float): For numerical stability, we add a small epsilon when
normalizing gradients in :meth:`tell_dqd` -- refer to the
implementation `here
Expand Down Expand Up @@ -252,12 +251,12 @@ def ask(self):
The multivariate Gaussian is parameterized by the evolution strategy
optimizer ``self._opt``.
Note that this method returns `batch_size - 1` solution as one solution
is returned via ask_dqd.
This method returns ``batch_size`` solutions, even though one solution
is returned via ``ask_dqd``.
Returns:
(batch_size, :attr:`solution_dim`) array -- a batch of new solutions
to evaluate.
(:attr:`batch_size`, :attr:`solution_dim`) array -- a batch of new
solutions to evaluate.
"""
coeff_lower_bounds = np.full(
self._num_coefficients,
Expand Down

0 comments on commit b1fe409

Please sign in to comment.