Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Dask integration #2023

Merged
merged 55 commits into from
Nov 15, 2022
+830 −0
Merged

Add Dask integration #2023

Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
a20c9a7
Initial content
jrbourbeau Nov 14, 2020
b9083fd
Linting related changes
jrbourbeau Nov 14, 2020
fa02ff5
Add distributed to integration requirements
jrbourbeau Nov 14, 2020
032d837
Update .gitignore
jrbourbeau Nov 14, 2020
339ceea
Update integration import tests
jrbourbeau Nov 14, 2020
726eac0
Add distributed to testing dependencies
jrbourbeau Nov 14, 2020
b0bbf7b
Remove OptunaSchedulerExtension from public API
jrbourbeau Nov 16, 2020
5dbcfe9
Add dask_simple.py example script
jrbourbeau Nov 16, 2020
568dfb7
Add DaskStorage to API docs
jrbourbeau Nov 16, 2020
81b838e
Add more underscores to function names
jrbourbeau Nov 16, 2020
1a9c789
Merge branch 'master' of https://github.com/optuna/optuna into dask-s…
jrbourbeau Feb 6, 2021
30a9d17
Incorporate recent multi objective updates
jrbourbeau Feb 6, 2021
928e02c
Add DaskStorage.get_trial_id_from_study_id_trial_number
jrbourbeau Feb 6, 2021
6292bac
Initial DaskStudy implementation
jrbourbeau Feb 7, 2021
40b626c
Update documentation dependencies
jrbourbeau Feb 7, 2021
8ace4b1
Merge branch 'master' of https://github.com/optuna/optuna into dask-s…
jrbourbeau Jul 22, 2021
c344594
Changes due to recent updates
jrbourbeau Jul 22, 2021
39f8475
Linting
jrbourbeau Jul 22, 2021
1b379b8
Update examples/dask_simple.py
jrbourbeau Jul 22, 2021
f23b7e3
Use in-memory sql in tests
jrbourbeau Jul 22, 2021
147ad20
Use larger dataset in example
jrbourbeau Jul 22, 2021
c51a1fc
Improve doc links
jrbourbeau Jul 22, 2021
cbfe742
Lint
jrbourbeau Jul 22, 2021
1c3a249
Update type annotations
jrbourbeau Jul 22, 2021
992f54d
Revert in-memory sqlite changes to avoid threading issue
jrbourbeau Jul 22, 2021
30d1068
Improved comments / docstrings
jrbourbeau Jul 23, 2021
0ebb6b6
Remove Dask example
jrbourbeau Aug 11, 2021
20a9bf8
Merge branch 'master' of https://github.com/optuna/optuna into dask-s…
jrbourbeau Aug 11, 2021
1576d38
Merge branch 'master' of https://github.com/optuna/optuna into dask-s…
jrbourbeau Sep 21, 2022
012bde1
Revive
jrbourbeau Sep 21, 2022
6eec2c0
Unused comm argument no longer needed
jrbourbeau Sep 21, 2022
50e37c6
Roll back get_study_class changes
jrbourbeau Sep 21, 2022
29dfdb8
Add optuna.integration.dask.create_study(...) method
jrbourbeau Sep 22, 2022
d5064a1
Temporarily disable fast fail for integration tests
jrbourbeau Sep 22, 2022
755c0f7
Typo
jrbourbeau Sep 22, 2022
76be9ec
Support older versions of distributed and make flake8 happy
jrbourbeau Sep 22, 2022
8d5e71f
mypy
jrbourbeau Sep 22, 2022
9e0d40b
Flake8
jrbourbeau Sep 22, 2022
3ccd7f6
Lingering code review comment
jrbourbeau Sep 22, 2022
b936592
Reseeding sampler suggestion
jrbourbeau Sep 22, 2022
6f232da
Remove temporary GHA workflow changes
jrbourbeau Sep 22, 2022
5113fc1
_optimize_sequential uses progress_bar=None to disable the progress bar
jrbourbeau Oct 4, 2022
12237c5
Merge branch 'master' of https://github.com/optuna/optuna into dask-s…
jrbourbeau Oct 17, 2022
3c004c9
Add more study method test coverage
jrbourbeau Oct 17, 2022
09c0e2c
Add support for DaskStudy.stop()
jrbourbeau Oct 17, 2022
fc0f7ca
__study attribute is no longer needed
jrbourbeau Oct 17, 2022
b2a1ac0
Merge branch 'master' of https://github.com/optuna/optuna into dask-s…
jrbourbeau Nov 7, 2022
2d92abb
Remove DaskStudy-related code
jrbourbeau Nov 8, 2022
1e00967
Update dependencies
jrbourbeau Nov 9, 2022
2a3dedc
Address code review suggestions
jrbourbeau Nov 9, 2022
05518e4
Merge branch 'master' of https://github.com/optuna/optuna into dask-s…
jrbourbeau Nov 10, 2022
d8f8593
Use datetime.fromisoformat
jrbourbeau Nov 10, 2022
c88fa2a
Apply suggestions from code review
jrbourbeau Nov 10, 2022
048774f
Merge branch 'master' of https://github.com/optuna/optuna into dask-s…
jrbourbeau Nov 11, 2022
269098c
Fix mypy
jrbourbeau Nov 11, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,3 +120,6 @@ docs/source/tutorial/

# asv
.asv

# Dask
dask-worker-space/
9 changes: 9 additions & 0 deletions docs/source/reference/integration.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,15 @@ Chainer
optuna.integration.ChainerPruningExtension
optuna.integration.ChainerMNStudy

Dask
----

.. autosummary::
:toctree: generated/
:nosignatures:

optuna.integration.DaskStorage

fast.ai
-------

Expand Down
2 changes: 2 additions & 0 deletions optuna/integration/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
"chainer": ["ChainerPruningExtension"],
"chainermn": ["ChainerMNStudy"],
"cma": ["CmaEsSampler", "PyCmaSampler"],
"dask": ["DaskStorage"],
"mlflow": ["MLflowCallback"],
"wandb": ["WeightsAndBiasesCallback"],
"keras": ["KerasPruningCallback"],
Expand Down Expand Up @@ -47,6 +48,7 @@
from optuna.integration.chainermn import ChainerMNStudy # NOQA
from optuna.integration.cma import CmaEsSampler # NOQA
from optuna.integration.cma import PyCmaSampler # NOQA
from optuna.integration.dask import DaskStorage # NOQA
from optuna.integration.fastaiv1 import FastAIV1PruningCallback # NOQA
from optuna.integration.fastaiv2 import FastAIPruningCallback # NOQA
from optuna.integration.fastaiv2 import FastAIV2PruningCallback # NOQA
Expand Down