Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement batch addition in archives #221

Merged
merged 55 commits into from
Jul 24, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
2826da9
Prototype (does not work)
btjanaka Jun 27, 2022
83770ea
Bug fixes and tests
btjanaka Jun 27, 2022
e121192
Bugfix
btjanaka Jun 28, 2022
c399ac6
Add numpy_groupies to setup.py
btjanaka Jun 28, 2022
9dac9f5
Merge branch 'master' into add-batch
btjanaka Jun 29, 2022
e598450
Merge branch 'master' into add-batch
btjanaka Jul 2, 2022
23ac54e
Merge branch 'master' into add-batch
btjanaka Jul 3, 2022
fa2d68e
Update docstring
btjanaka Jul 6, 2022
7439ca1
Docstring
btjanaka Jul 6, 2022
3f0000a
Merge branch 'master' into add-batch
btjanaka Jul 6, 2022
4943420
Docstrings
btjanaka Jul 6, 2022
b5df701
Refactor shape checking functions
btjanaka Jul 6, 2022
766b73b
Tidy up add_single
btjanaka Jul 6, 2022
e198d2a
Fix some tests
btjanaka Jul 6, 2022
ad23c49
Grid archive test
btjanaka Jul 6, 2022
1e4f1dc
CVT archive tests
btjanaka Jul 6, 2022
cc50c8e
visualize tests
btjanaka Jul 6, 2022
8d49143
sliding boundaries tests
btjanaka Jul 6, 2022
0af3402
Fix grid archive benchmark
btjanaka Jul 6, 2022
c438146
Allow extra msg in error funcs
btjanaka Jul 7, 2022
28325cf
Remvoe branch limit
btjanaka Jul 7, 2022
e6996ed
Formatting
btjanaka Jul 7, 2022
c298f90
Merge branch 'master' into add-batch
itsdawei Jul 12, 2022
d583f47
Merge branch 'master' into add-batch
btjanaka Jul 13, 2022
e6c7133
Merge branch 'add-batch' of github.com:icaros-usc/pyribs into add-batch
btjanaka Jul 13, 2022
cc8f14e
Merge branch 'master' into add-batch
btjanaka Jul 13, 2022
7bd7a60
Fix add_batch bug?
btjanaka Jul 22, 2022
ef177f2
Clean up add code
btjanaka Jul 22, 2022
349c955
Use add in optimizer
btjanaka Jul 22, 2022
f03ff54
Add test for add
btjanaka Jul 22, 2022
2b69856
Fix CVTArchive benchmarks
btjanaka Jul 22, 2022
bc29569
Fix some emitter tests
btjanaka Jul 22, 2022
6f4515e
Clarify
btjanaka Jul 22, 2022
122a361
Fix emitter base test
btjanaka Jul 22, 2022
5270d84
Fix ranker test
btjanaka Jul 22, 2022
c296cf7
Fix 1D grid vis test
btjanaka Jul 22, 2022
729c324
Proper checking of elite batch
btjanaka Jul 22, 2022
0b71213
Fix optimizer tests
btjanaka Jul 22, 2022
4d6d63e
Increase max locals
btjanaka Jul 22, 2022
4f28541
Shuffle solutions in add
btjanaka Jul 22, 2022
73ecd3e
Metadata warning
btjanaka Jul 22, 2022
8d9688d
Revert "Shuffle solutions in add"
btjanaka Jul 22, 2022
8caedc5
Clarify about ties
btjanaka Jul 22, 2022
a327d59
Check shapes in add()
btjanaka Jul 22, 2022
21507da
Tweak err messages
btjanaka Jul 22, 2022
75888be
history
btjanaka Jul 22, 2022
c08ef8e
Handle ties
btjanaka Jul 22, 2022
ff3e318
Rearrange docstring
btjanaka Jul 22, 2022
29e4ff2
Docstring tweaks
btjanaka Jul 22, 2022
3c13e86
Start slidingboundariesarchive
btjanaka Jul 22, 2022
1d9d178
Skip sliding boundaries tests for now
btjanaka Jul 22, 2022
953bd84
Remove unused stats_update
btjanaka Jul 22, 2022
933b89d
Simplify extra messages for errors
btjanaka Jul 24, 2022
4402a77
Clarify batch size check
btjanaka Jul 24, 2022
070874a
Merge branch 'master' into add-batch
btjanaka Jul 24, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions .pylintrc
Original file line number Diff line number Diff line change
Expand Up @@ -344,13 +344,13 @@ max-args=9999
ignored-argument-names=_.*

# Maximum number of locals for function / method body
max-locals=30
max-locals=100

# Maximum number of return / yield for function / method body
max-returns=6

# Maximum number of branch for function / method body
max-branches=12
max-branches=100

# Maximum number of statements in function / method body
max-statements=100
Expand Down
3 changes: 3 additions & 0 deletions HISTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@

#### API

- **Backwards-incompatible:** Implement batch addition in archives (#221)
- `add` now adds a batch of solutions to the archive
- `add_single` adds a single solution
- `emitter.tell` now takes in `status_batch` and `value_batch` (#227)
- Make epsilon configurable in archives (#226)
- **Backwards-incompatible:** Remove ribs.factory (#225,#228)
Expand Down
57 changes: 39 additions & 18 deletions ribs/_utils.py
Original file line number Diff line number Diff line change
@@ -1,27 +1,48 @@
"""Miscellaneous internal utilities."""


def check_measures_batch_shape(measures_batch,
measure_dim,
name="measures_batch"):
"""Checks the shape of a batch of measures.
def check_batch_shape(array, array_name, dim, dim_name, extra_msg=""):
"""Checks that the array has shape (batch_size, dim).

`measures_batch` must be a numpy array.
`batch_size` can be any value.

`array` must be a numpy array, and `dim` must be an int.
"""
shape = measures_batch.shape
if len(shape) != 2 or shape[1] != measure_dim:
raise ValueError(f"Expected {name} to be a 2D array with shape "
f"(batch_size, {measure_dim}) (i.e. shape "
f"(batch_size, measure_dim)) but it had shape {shape}")
if array.ndim != 2 or array.shape[1] != dim:
btjanaka marked this conversation as resolved.
Show resolved Hide resolved
raise ValueError(f"Expected {array_name} to be a 2D array with shape "
f"(batch_size, {dim}) (i.e. shape "
f"(batch_size, {dim_name})) but it had shape "
f"{array.shape}.{extra_msg}")


def check_measures_shape(measures, measure_dim, name="measures"):
"""Checks the shape of a 1D vector of measures.
def check_1d_shape(array, array_name, dim, dim_name, extra_msg=""):
"""Checks that the array has shape (dim,).

`measures` must be a numpy array.
`array` must be a numpy array, and `dim` must be an int.
"""
shape = measures.shape
if len(shape) != 1 or shape[0] != measure_dim:
raise ValueError(f"Expected {name} to be a 1D array with shape "
f"({measure_dim},) (i.e. shape (measure_dim,)) "
f"but it had shape {shape}")
if array.ndim != 1 or array.shape[0] != dim:
raise ValueError(
f"Expected {array_name} to be a 1D array with shape "
f"({dim},) (i.e. shape ({dim_name},)) but it had shape "
f"{array.shape}.{extra_msg}")


def check_is_1d(array, array_name, extra_msg=""):
"""Checks that an array is 1D."""
if array.ndim != 1:
raise ValueError(f"Expected {array_name} to be a 1D array but it had "
f"shape {array.shape}.{extra_msg}")


def check_solution_batch_dim(array,
btjanaka marked this conversation as resolved.
Show resolved Hide resolved
array_name,
batch_size,
is_1d=False,
extra_msg=""):
"""Checks the batch dimension of an array with respect to solution_batch."""
if array.shape[0] != batch_size:
raise ValueError(f"{array_name} does not match the batch dimension of "
"solution_batch -- since solution_batch has shape "
f"({batch_size}, ..), {array_name} should have shape "
f"({batch_size},{'' if is_1d else ' ..'}), but it has "
f"shape {array.shape}.{extra_msg}")