Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 22 additions & 65 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,15 @@ package alongside direct interfaces to Optuna and scikit-learn optimizers, suppo
<h3>
<a href="https://github.com/SimonBlanke/Hyperactive#overview">Overview</a> •
<a href="https://github.com/SimonBlanke/Hyperactive#installation">Installation</a> •
<a href=https://nbviewer.org/github/SimonBlanke/hyperactive-tutorial/blob/main/notebooks/hyperactive_tutorial.ipynb>Tutorial</a> •
<a href="https://nbviewer.org/github/SimonBlanke/hyperactive-tutorial/blob/main/notebooks/hyperactive_tutorial.ipynb">Tutorial</a> •
<a href="https://github.com/SimonBlanke/Hyperactive#citing-hyperactive">Citation</a> •
<a href="https://github.com/SimonBlanke/Hyperactive#license">License</a>
</h3>
</div>

| **Open&#160;Source** | [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![GC.OS Sponsored](https://img.shields.io/badge/GC.OS-Sponsored%20Project-orange.svg?style=flat&colorA=0eac92&colorB=2077b4)](https://gc-os-ai.github.io/) |
|---|---|
| **Community** | [![!discord](https://img.shields.io/static/v1?logo=discord&label=discord&message=chat&color=lightgreen)](https://discord.com/invite/54ACzaFsn7) [![!slack](https://img.shields.io/static/v1?logo=linkedin&label=LinkedIn&message=news&color=lightblue)](https://www.linkedin.com/company/german-center-for-open-source-ai) |
| **Community** | [![Discord](https://img.shields.io/static/v1?logo=discord&label=Discord&message=chat&color=lightgreen)](https://discord.gg/7uKdHfdcJG) [![LinkedIn](https://img.shields.io/static/v1?logo=linkedin&label=LinkedIn&message=news&color=lightblue)](https://www.linkedin.com/company/german-center-for-open-source-ai) |
| **CI/CD** | [![github-actions](https://img.shields.io/github/actions/workflow/status/SimonBlanke/hyperactive/test.yml?logo=github)](https://github.com/SimonBlanke/hyperactive/actions/workflows/test.yml) [![readthedocs](https://img.shields.io/readthedocs/hyperactive?logo=readthedocs)](https://www.hyperactive.net/en/latest/?badge=latest)
| **Code** | [![!pypi](https://img.shields.io/pypi/v/hyperactive?color=orange)](https://pypi.org/project/hyperactive/) [![!python-versions](https://img.shields.io/pypi/pyversions/hyperactive)](https://www.python.org/) [![!black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) |

Expand All @@ -50,7 +50,7 @@ def problem(params):
x = params["x"]
y = params["y"]

return - x ** 2 + - y ** 2
return -(x**2 + y**2)

# discrete search space: dict of iterable, scikit-learn like grid space
# (valid search space types depends on optimizer)
Expand Down Expand Up @@ -105,7 +105,7 @@ score, add_info = sklearn_exp.score(params)
from hyperactive.opt.gfo import HillClimbing

search_space = {
"C": np.logspace(0.01, 100, num=10),
"C": np.logspace(-2, 2, num=10),
"kernel": ["linear", "rbf"],
}

Expand Down Expand Up @@ -218,7 +218,7 @@ Hyperactive features a collection of optimization algorithms that can be used fo
<a><b>Population Methods:</b></a>
<ul>
<li><a href="./examples/gfo/parallel_tempering_example.py">Parallel Tempering</a></li>
<li><a href="./examples/gfo/particle_swarm_optimization_example.py">Particle Swarm Optimizer</a></li>
<li><a href="./examples/gfo/particle_swarm_example.py">Particle Swarm Optimizer</a></li>
<li><a href="./examples/gfo/spiral_optimization_example.py">Spiral Optimization</a></li>
<li><a href="./examples/gfo/genetic_algorithm_example.py">Genetic Algorithm</a></li>
<li><a href="./examples/gfo/evolution_strategy_example.py">Evolution Strategy</a></li>
Expand All @@ -227,11 +227,11 @@ Hyperactive features a collection of optimization algorithms that can be used fo
<a><b>Sequential Methods:</b></a>
<ul>
<li><a href="./examples/gfo/bayesian_optimization_example.py">Bayesian Optimization</a></li>
<li><a href="./examples/gfo/lipschitz_optimization_example.py">Lipschitz Optimization</a></li>
<li><a href="./examples/gfo/lipschitz_optimizer_example.py">Lipschitz Optimization</a></li>
<li><a href="./examples/gfo/direct_algorithm_example.py">Direct Algorithm</a></li>
<li><a href="./examples/gfo/tree_structured_parzen_estimators_example.py">Tree of Parzen Estimators</a></li>
<li><a href="./examples/gfo/forest_optimization_example.py">Forest Optimizer</a>
[<a href="#/./overview#references">dto</a>] </li>
<li><a href="./examples/gfo/forest_optimizer_example.py">Forest Optimizer</a>
[<a href="#references">ref</a>] </li>
</ul><br>
<a><b>Optuna Backend:</b></a>
<ul>
Expand All @@ -248,60 +248,11 @@ Hyperactive features a collection of optimization algorithms that can be used fo
<td>
<a><b>Machine Learning:</b></a>
<ul>
<li><a href="./examples/sklearn/grid_search_example.py">Scikit-learn</a></li>
<li><a href="./examples/tested_and_supported_packages/xgboost_example.py">XGBoost</a></li>
<li><a href="./examples/tested_and_supported_packages/lightgbm_example.py">LightGBM</a></li>
<li><a href="./examples/tested_and_supported_packages/catboost_example.py">CatBoost</a></li>
<li><a href="./examples/tested_and_supported_packages/rgf_example.py">RGF</a></li>
<li><a href="./examples/tested_and_supported_packages/mlxtend_example.py">Mlxtend</a></li>
</ul><br>
<a><b>Deep Learning:</b></a>
<ul>
<li><a href="./examples/tested_and_supported_packages/tensorflow_example.py">Tensorflow</a></li>
<li><a href="./examples/tested_and_supported_packages/keras_example.py">Keras</a></li>
<li><a href="./examples/tested_and_supported_packages/pytorch_example.py">Pytorch</a></li>
</ul><br>
<a><b>Parallel Computing:</b></a>
<ul>
<li><a href="./examples/tested_and_supported_packages/multiprocessing_example.py">Multiprocessing</a></li>
<li><a href="./examples/tested_and_supported_packages/joblib_example.py">Joblib</a></li>
<li>Pathos</li>
<li><a href="./examples/sklearn/README.md">Scikit-learn</a></li>
</ul>
</td>
<td>
<a><b>Feature Engineering:</b></a>
<ul>
<li><a href="./examples/optimization_applications/feature_transformation.py">Feature Transformation</a></li>
<li><a href="./examples/optimization_applications/feature_selection.py">Feature Selection</a></li>
</ul>
<a><b>Machine Learning:</b></a>
<ul>
<li><a href="./examples/optimization_applications/hyperpara_optimize.py">Hyperparameter Tuning</a></li>
<li><a href="./examples/optimization_applications/model_selection.py">Model Selection</a></li>
<li><a href="./examples/optimization_applications/sklearn_pipeline_example.py">Sklearn Pipelines</a></li>
<li><a href="./examples/optimization_applications/ensemble_learning_example.py">Ensemble Learning</a></li>
</ul>
<a><b>Deep Learning:</b></a>
<ul>
<li><a href="./examples/optimization_applications/neural_architecture_search.py">Neural Architecture Search</a></li>
<li><a href="./examples/optimization_applications/pretrained_nas.py">Pretrained Neural Architecture Search</a></li>
<li><a href="./examples/optimization_applications/transfer_learning.py">Transfer Learning</a></li>
</ul>
<a><b>Data Collection:</b></a>
<ul>
<li><a href="./examples/optimization_applications/meta_data_collection.py">Search Data Collection</a></li>
<li><a href="./examples/optimization_applications/meta_optimization.py">Meta Optimization</a></li>
<li><a href="./examples/optimization_applications/meta_learning.py">Meta Learning</a></li>
</ul>
<a><b>Miscellaneous:</b></a>
<ul>
<li><a href="./examples/optimization_applications/test_function.py">Test Functions</a></li>
<li>Fit Gaussian Curves</li>
<li><a href="./examples/optimization_applications/multiple_scores.py">Managing multiple objectives</a></li>
<li><a href="./examples/optimization_applications/search_space_example.py">Managing objects in search space</a></li>
<li><a href="./examples/optimization_applications/constrained_optimization.py">Constrained Optimization</a></li>
<li><a href="./examples/optimization_applications/memory.py">Memorize evaluations</a></li>
</ul>

</td>
</tr>
</tbody>
Expand Down Expand Up @@ -391,10 +342,15 @@ Reduce the search space size to resolve this error.

<br>

This is because you have classes and/or non-top-level objects in the search space. Pickle (used by multiprocessing) cannot serialize them. Setting distribution to "joblib" or "pathos" may fix this problem:
```python
hyper = Hyperactive(distribution="joblib")
```
This typically means your search space or parameter suggestions include non-serializable
objects (e.g., classes, bound methods, lambdas, local functions, locks). Ensure that all
values in `search_space`/`param_space` are plain Python/scientific types such as ints,
floats, strings, lists/tuples, or numpy arrays. Avoid closures and non-top-level callables
in parameter values.

Hyperactive v5 does not expose a global “distribution” switch. If you parallelize outside
Hyperactive (e.g., with joblib/dask/ray), choose an appropriate backend and make sure the
objective and arguments are picklable for process-based backends.

</details>

Expand Down Expand Up @@ -425,8 +381,9 @@ warnings.warn = warn

<br>

This warning occurs because Hyperactive needs more initial positions to choose from to generate a population for the optimization algorithm:
The number of initial positions is determined by the `initialize`-parameter in the `add_search`-method.
This warning occurs because the optimizer needs more initial positions to generate a
population for the search. In v5, initial positions are controlled via the optimizer’s
`initialize` parameter.
```python
# This is how it looks per default
initialize = {"grid": 4, "random": 2, "vertices": 4}
Expand Down