-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Doc: extension of contribution documentation #445
Merged
Merged
Changes from 57 commits
Commits
Show all changes
58 commits
Select commit
Hold shift + click to select a range
14a7faf
fixed some typos
musslick 8d4feec
added additional info
musslick 64849db
fixed typos
musslick 7943462
added hint and pointer to forum
musslick 2faf248
added pointer
musslick 331b94c
some minor fixes
musslick 278afc6
added some more details on how to make a module contribution
musslick c116c57
fixed theorist contribution (and cross-referenced to module contribut…
musslick b479be7
increased figure resolution
musslick 69cde78
added experimentalist module doc
musslick bd2364b
fixed broken link
musslick ec70d6f
renaming title
musslick b118866
added experiment runner image
musslick b055166
replaced fig with higher resolution
musslick c719b83
Update docs/contribute/core.md
musslick b0fe554
Update docs/contribute/core.md
musslick 8fae546
fixed synthetic link
musslick 05d78e6
Update docs/contribute/experimentalist-module.md
musslick 7cd2ec3
Update docs/contribute/experimentalist-module.md
musslick 6ef3c80
Update docs/contribute/module.md
musslick 7b6a3ba
Update docs/contribute/module.md
musslick d98e786
Update docs/contribute/module.md
musslick 1242b1b
Update docs/contribute/module.md
musslick 3ca5e10
included conda
musslick 946e8ab
Merge branch 'main' into doc/exp-module
benwandrew 54b9ebb
addressed younes' suggestions
musslick 3af9643
Update docs/contribute/experimentalist-module.md
musslick 735ec46
Update docs/contribute/experimentalist-module.md
musslick 8032c33
Update docs/contribute/experimentalist-module.md
musslick ae7fe20
Update docs/contribute/experimentalist-module.md
musslick 02b5f39
Update docs/contribute/experimentalist-module.md
musslick 5a76b3c
Update docs/contribute/experimentalist-module.md
musslick 83a9dfb
Update docs/contribute/experimentalist-module.md
musslick 674ca8e
Update docs/contribute/index.md
musslick f5b868f
Update docs/contribute/index.md
musslick ac00888
Update docs/contribute/index.md
musslick efa7ae2
Update docs/contribute/index.md
musslick 67b3934
Update docs/contribute/module.md
musslick c08f709
Update docs/contribute/module.md
musslick c741b6c
Update docs/contribute/module.md
musslick 4a2e8d1
Update docs/contribute/experimentalist-module.md
musslick b55927a
Update docs/contribute/experimentalist-module.md
musslick fe86e85
Update docs/contribute/experimentalist-module.md
musslick 83a8d0d
Update docs/contribute/experimentalist-module.md
musslick b91e51d
Update docs/contribute/module.md
musslick a09699e
Update docs/contribute/module.md
musslick b59cad7
Update docs/contribute/module.md
musslick 235bfb3
Update docs/contribute/module.md
musslick 6977843
Update docs/contribute/module.md
musslick 133a72d
Update docs/contribute/module.md
musslick 55aa4bb
Update docs/contribute/module.md
musslick a4e730c
Update docs/contribute/module.md
musslick 0fa4744
Update docs/contribute/module.md
musslick fae97e6
Update docs/contribute/module.md
musslick d28ff97
Update docs/contribute/module.md
musslick 486340e
docs: add experimentalist-module.md to nav in mkdocs.yml
benwandrew efaf4fb
docs: sentence case for all headings in experimentalist-module.md
benwandrew ca638aa
docs: sentence case for headings throughout contributor docs
benwandrew File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,127 @@ | ||
# Contribute an experimentalist | ||
|
||
AutoRA experimentalists are meant to return novel experimental conditions based on prior experimental conditions, prior | ||
observations, and/or prior models. Such conditions may serve as a basis for new, informative experiments conducted | ||
by an experiment runner. Experimentalists are generally implemented as functions that can be integrated into an | ||
[Experimentalist Pipeline](https://autoresearch.github.io/autora/core/docs/pipeline/Experimentalist%20Pipeline%20Examples/). | ||
|
||
![Experimentalist Module](../img/experimentalist.png) | ||
|
||
Experimentalists can be implemented as *poolers* or as *samplers*. | ||
- **Poolers** return a pool of candidate experimental conditions, which can be passed to a sampler that selects | ||
a subset of conditions from the pool to be used in the next experiment. | ||
- **Samplers** directly return a subset of experimental conditions from a pool of candidate experimental conditions that already exist. | ||
|
||
## Repository setup | ||
|
||
We recommend using the [cookiecutter template](https://github.com/AutoResearch/autora-template-cookiecutter) to set up | ||
a repository for your experimentalist. Alternatively, you can use the | ||
[unguided template](https://github.com/AutoResearch/autora-template). If you choose the cookiecutter template, you can set up your repository using | ||
|
||
```shell | ||
cookiecutter https://github.com/AutoResearch/autora-template-cookiecutter | ||
``` | ||
|
||
Make sure to select the `experimentalist` option when prompted. You may also select whether you want to implement an experimentalist as a sampler, pooler, or custom function. You can skip all other prompts pertaining to other modules | ||
(e.g., experiment runners) by pressing enter. | ||
|
||
## Implementation | ||
|
||
Irrespective of whether you are implementing a pooler or a sampler, | ||
you should implement a function that returns a set of experimental conditions. This set may be | ||
a numpy array, iterator variable or other data format. | ||
|
||
!!! hint | ||
We generally **recommend using 2-dimensional numpy arrays as outputs** in which | ||
each row represents a set of experimental conditions. The columns of the array correspond to the independent variables. | ||
|
||
### Implementing poolers | ||
|
||
Once you've created your repository, you can implement your experimentalist pooler by editing the `init.py` file in | ||
``src/autora/experimentalist/pooler/name_of_your_experimentalist/``. | ||
You may also add additional files to this directory if needed. | ||
It is important that the `init.py` file contains a function called `name_of_your_experimentalist` | ||
which returns a pool of experimental conditions (e.g., as an iterator object or numpy array). | ||
|
||
The following example ``init.py`` illustrates the implementation of a simple experimentalist pooler | ||
that generates a grid of samples within the specified bounds of each independent variable (IV): | ||
|
||
```python | ||
|
||
""" | ||
Example Experimentalist Pooler | ||
""" | ||
|
||
from itertools import product | ||
from typing import List | ||
from autora.variable import IV | ||
|
||
|
||
def grid_pool(ivs: List[IV]): | ||
""" | ||
Creates exhaustive pool from discrete values using a Cartesian product of sets | ||
|
||
Arguments: | ||
ivs {List[IV]}: List of independent variables | ||
|
||
Returns: | ||
pool: An iterator over all possible combinations of IV values | ||
""" | ||
|
||
l_iv_values = [] | ||
for iv in ivs: | ||
assert iv.allowed_values is not None, ( | ||
f"gridsearch_pool only supports independent variables with discrete allowed values, " | ||
f"but allowed_values is None on {iv=} " | ||
) | ||
l_iv_values.append(iv.allowed_values) | ||
|
||
# Return Cartesian product of all IV values | ||
return product(*l_iv_values) | ||
|
||
|
||
``` | ||
|
||
### Implementing samplers | ||
|
||
Once you've created your repository, you can implement your experimentalist sampler by editing the `init.py` file in | ||
``src/autora/experimentalist/sampler/name_of_your_experimentalist/``. | ||
You may also add additional files to this directory if needed. | ||
It is important that the `init.py` file contains a function called `name_of_your_experimentalist` | ||
which returns a set of experimental conditions (e.g., as a numpy array) given a pool of candidate experimental conditions. | ||
|
||
The following example ``init.py`` illustrates the implementation of a simple experimentalist sampler | ||
that uniformly samples without replacement from a pool of candidate conditions. | ||
|
||
```python | ||
""" | ||
Example Experimentalist Sampler | ||
""" | ||
|
||
import random | ||
from typing import Iterable, Sequence, Union | ||
|
||
random_sample(conditions: Union[Iterable, Sequence], n: int = 1): | ||
""" | ||
Uniform random sampling without replacement from a pool of conditions. | ||
Args: | ||
conditions: Pool of conditions | ||
n: number of samples to collect | ||
|
||
Returns: Sampled pool | ||
|
||
""" | ||
|
||
if isinstance(conditions, Iterable): | ||
conditions = list(conditions) | ||
random.shuffle(conditions) | ||
samples = conditions[0:n] | ||
|
||
return samples | ||
``` | ||
|
||
|
||
## Next steps: testing, documentation, publishing | ||
|
||
For more information on how to test, document, and publish your experimentalist, please refer to the | ||
[general guideline for module contributions](module.md) . |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is super nitpicky, but is there logic to when we use title case vs sentence case in headings?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. I think we should generally stick with sentence case. What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sounds good to me. addressed in this file for this PR, and will address globally in a separate PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
actually, decided to just implement in this PR...