Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement batch addition in archives #221

Merged
merged 55 commits into from Jul 24, 2022
Merged

Implement batch addition in archives #221

merged 55 commits into from Jul 24, 2022

Conversation

btjanaka
Copy link
Member

@btjanaka btjanaka commented Jun 27, 2022

Description

The add() method will add solutions to the archive as a batch, which should greatly increase the archive speed.

add_single() is now the corresponding method which adds just one solution to the archive.

We obtained roughly 2x speedup on the sphere example -- python sphere.py cma_me_imp_mu and python sphere.py cma_me_imp took 60-65s (compare with 140s in #220 (comment)). Note that this is all without numba, i.e. batched operations without numba are faster than single operations with numba.

TODO

  • Implement add_batch()
  • Update docstrings
  • Rename add() and add_batch() to insert() and insert_single() for consistency with the rest of the API, e.g. index_of() and index_of_single() No point since we would have to change AddStatus, and we will check array shapes anyway
  • Rename add() and add_batch() to add_single() and add()
  • Write tests
  • Fix existing tests
  • Fix add() in SlidingBoundariesArchive -> followup PR
  • Check for regression on sphere example

Questions

Status

  • I have read the guidelines in CONTRIBUTING.md.
  • I have formatted my code using yapf
  • I have tested my code by running pytest
  • I have linted my code with pylint
  • I have added a one-line description of my change to the changelog in HISTORY.md.
  • This PR is ready to go

@btjanaka btjanaka changed the title Implement add_batch method for archives Implement batch insertion in archives Jul 6, 2022
@btjanaka btjanaka changed the title Implement batch insertion in archives Implement batch addition in archives Jul 6, 2022
Copy link
Member Author

@btjanaka btjanaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self-review

Copy link
Collaborator

@itsdawei itsdawei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work!

ribs/_utils.py Show resolved Hide resolved
ribs/_utils.py Outdated Show resolved Hide resolved
ribs/_utils.py Show resolved Hide resolved
tests/archives/grid_archive_benchmark.py Show resolved Hide resolved
@btjanaka btjanaka merged commit 3fb80ba into master Jul 24, 2022
@btjanaka btjanaka deleted the add-batch branch July 24, 2022 09:57
@btjanaka btjanaka restored the add-batch branch July 24, 2022 10:00
@btjanaka btjanaka deleted the add-batch branch July 24, 2022 20:30
@itsdawei itsdawei mentioned this pull request Aug 24, 2022
17 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants