Skip to content

Commit

Permalink
Ravin Kohli: [ADD] Missing Batchnorm (#317)
Browse files Browse the repository at this point in the history
  • Loading branch information
Github Actions committed Nov 11, 2021
1 parent a4a7a02 commit 8f0af69
Show file tree
Hide file tree
Showing 24 changed files with 439 additions and 506 deletions.
Binary file modified development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip
View file
Binary file not shown.
Binary file modified development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip
View file
Binary file not shown.
Binary file modified development/_images/sphx_glr_example_visualization_001.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified development/_images/sphx_glr_example_visualization_002.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified development/_images/sphx_glr_example_visualization_thumb.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
17 changes: 7 additions & 10 deletions development/_sources/examples/20_basics/example_image_classification.rst.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -86,16 +86,13 @@ Image Classification
________________________________________
Configuration:
image_augmenter:GaussianBlur:use_augmenter, Value: False
image_augmenter:GaussianNoise:use_augmenter, Value: False
image_augmenter:RandomAffine:rotate, Value: 328
image_augmenter:RandomAffine:scale_offset, Value: 0.26538653822859176
image_augmenter:RandomAffine:shear, Value: 43
image_augmenter:RandomAffine:translate_percent_offset, Value: 0.0013477722050754704
image_augmenter:RandomAffine:use_augmenter, Value: True
image_augmenter:RandomCutout:p, Value: 0.9271524955899881
image_augmenter:GaussianNoise:sigma_offset, Value: 0.6648162951470324
image_augmenter:GaussianNoise:use_augmenter, Value: True
image_augmenter:RandomAffine:use_augmenter, Value: False
image_augmenter:RandomCutout:p, Value: 0.41272250662075116
image_augmenter:RandomCutout:use_augmenter, Value: True
image_augmenter:Resize:use_augmenter, Value: True
image_augmenter:ZeroPadAndCrop:percent, Value: 0.04276538172142241
image_augmenter:Resize:use_augmenter, Value: False
image_augmenter:ZeroPadAndCrop:percent, Value: 0.04095527521678438
normalizer:__choice__, Value: 'NoNormalizer'

Fitting the pipeline...
Expand Down Expand Up @@ -175,7 +172,7 @@ Image Classification
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 0 minutes 8.275 seconds)
**Total running time of the script:** ( 0 minutes 8.334 seconds)


.. _sphx_glr_download_examples_20_basics_example_image_classification.py:
Expand Down
78 changes: 60 additions & 18 deletions development/_sources/examples/20_basics/example_tabular_classification.rst.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ Search for an ensemble of machine learning algorithms
.. code-block:: none
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f91159b1e80>
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f3b2933a070>
Expand Down Expand Up @@ -162,7 +162,7 @@ Print the final ensemble performance

.. code-block:: none
<smac.runhistory.runhistory.RunHistory object at 0x7f91159b14f0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
<smac.runhistory.runhistory.RunHistory object at 0x7f3b295019d0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
data_loader:batch_size, Value: 64
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
Expand Down Expand Up @@ -194,7 +194,7 @@ Print the final ensemble performance
scaler:__choice__, Value: 'StandardScaler'
trainer:StandardTrainer:weighted_loss, Value: True
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0014147758483886719, budget=0), TrajEntry(train_perf=0.14619883040935677, incumbent_id=1, incumbent=Configuration:
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0014591217041015625, budget=0), TrajEntry(train_perf=0.16374269005847952, incumbent_id=1, incumbent=Configuration:
data_loader:batch_size, Value: 64
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
Expand Down Expand Up @@ -226,28 +226,70 @@ Print the final ensemble performance
scaler:__choice__, Value: 'StandardScaler'
trainer:StandardTrainer:weighted_loss, Value: True
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=1, ta_time_used=4.973806142807007, wallclock_time=6.012700080871582, budget=5.555555555555555)]
{'accuracy': 0.8670520231213873}
| | Preprocessing | Estimator | Weight |
|---:|:------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
| 0 | None | KNNLearner | 0.18 |
| 1 | None | RFLearner | 0.14 |
| 2 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.14 |
| 3 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.14 |
| 4 | None | CBLearner | 0.1 |
| 5 | None | ETLearner | 0.1 |
| 6 | None | LGBMLearner | 0.08 |
| 7 | SimpleImputer,OneHotEncoder,MinMaxScaler,PowerTransformer | embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
| 8 | None | SVMLearner | 0.04 |
| 9 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
, ta_runs=1, ta_time_used=4.630058765411377, wallclock_time=5.667309522628784, budget=5.555555555555555), TrajEntry(train_perf=0.14619883040935677, incumbent_id=2, incumbent=Configuration:
data_loader:batch_size, Value: 154
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:KernelPCA:coef0, Value: 0.27733876378393374
feature_preprocessor:KernelPCA:kernel, Value: 'sigmoid'
feature_preprocessor:KernelPCA:n_components, Value: 3
feature_preprocessor:__choice__, Value: 'KernelPCA'
imputer:categorical_strategy, Value: 'most_frequent'
imputer:numerical_strategy, Value: 'mean'
lr_scheduler:StepLR:gamma, Value: 0.5658285105415104
lr_scheduler:StepLR:step_size, Value: 10
lr_scheduler:__choice__, Value: 'StepLR'
network_backbone:ResNetBackbone:activation, Value: 'tanh'
network_backbone:ResNetBackbone:blocks_per_group_0, Value: 2
network_backbone:ResNetBackbone:blocks_per_group_1, Value: 2
network_backbone:ResNetBackbone:max_shake_drop_probability, Value: 0.4280891218905112
network_backbone:ResNetBackbone:num_groups, Value: 1
network_backbone:ResNetBackbone:num_units_0, Value: 623
network_backbone:ResNetBackbone:num_units_1, Value: 42
network_backbone:ResNetBackbone:use_dropout, Value: False
network_backbone:ResNetBackbone:use_shake_drop, Value: True
network_backbone:ResNetBackbone:use_shake_shake, Value: True
network_backbone:__choice__, Value: 'ResNetBackbone'
network_embedding:LearnedEntityEmbedding:dimension_reduction_0, Value: 0.7061800992159439
network_embedding:LearnedEntityEmbedding:dimension_reduction_1, Value: 0.40404533505032336
network_embedding:LearnedEntityEmbedding:dimension_reduction_2, Value: 0.14124612419045746
network_embedding:LearnedEntityEmbedding:dimension_reduction_3, Value: 0.24304972767199295
network_embedding:LearnedEntityEmbedding:dimension_reduction_4, Value: 0.8403938666630251
network_embedding:LearnedEntityEmbedding:dimension_reduction_5, Value: 0.11081539209354929
network_embedding:LearnedEntityEmbedding:dimension_reduction_6, Value: 0.5150164644256714
network_embedding:LearnedEntityEmbedding:dimension_reduction_7, Value: 0.6185258490472787
network_embedding:LearnedEntityEmbedding:min_unique_values_for_embedding, Value: 3
network_embedding:__choice__, Value: 'LearnedEntityEmbedding'
network_head:__choice__, Value: 'fully_connected'
network_head:fully_connected:activation, Value: 'relu'
network_head:fully_connected:num_layers, Value: 2
network_head:fully_connected:units_layer_1, Value: 506
network_init:NoInit:bias_strategy, Value: 'Zero'
network_init:__choice__, Value: 'NoInit'
optimizer:AdamWOptimizer:beta1, Value: 0.9639206805787317
optimizer:AdamWOptimizer:beta2, Value: 0.9439342949959634
optimizer:AdamWOptimizer:lr, Value: 0.05110804312778185
optimizer:AdamWOptimizer:weight_decay, Value: 0.026136253949706992
optimizer:__choice__, Value: 'AdamWOptimizer'
scaler:__choice__, Value: 'NoScaler'
trainer:StandardTrainer:weighted_loss, Value: True
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=17, ta_time_used=214.60139346122742, wallclock_time=276.6278324127197, budget=50.0)]
{'accuracy': 0.8728323699421965}
| | Preprocessing | Estimator | Weight |
|---:|:-----------------------------------------------|:----------------------------------------------------------|---------:|
| 0 | None | CBLearner | 0.92 |
| 1 | SimpleImputer,OneHotEncoder,NoScaler,KernelPCA | embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
| 2 | None | RFLearner | 0.02 |
| 3 | None | ETLearner | 0.02 |
| 4 | None | SVMLearner | 0.02 |
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 5 minutes 35.756 seconds)
**Total running time of the script:** ( 5 minutes 28.652 seconds)


.. _sphx_glr_download_examples_20_basics_example_tabular_classification.py:
Expand Down
20 changes: 11 additions & 9 deletions development/_sources/examples/20_basics/example_tabular_regression.rst.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ Search for an ensemble of machine learning algorithms
.. code-block:: none
<autoPyTorch.api.tabular_regression.TabularRegressionTask object at 0x7f91ae0714f0>
<autoPyTorch.api.tabular_regression.TabularRegressionTask object at 0x7f3bcbf0aa90>
Expand Down Expand Up @@ -157,7 +157,7 @@ Print the final ensemble performance

.. code-block:: none
<smac.runhistory.runhistory.RunHistory object at 0x7f91b817eac0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
<smac.runhistory.runhistory.RunHistory object at 0x7f3bcbf0ae80> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
data_loader:batch_size, Value: 64
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
Expand Down Expand Up @@ -188,7 +188,7 @@ Print the final ensemble performance
optimizer:__choice__, Value: 'AdamOptimizer'
scaler:__choice__, Value: 'StandardScaler'
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0013966560363769531, budget=0), TrajEntry(train_perf=0.4945322850469632, incumbent_id=1, incumbent=Configuration:
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0013344287872314453, budget=0), TrajEntry(train_perf=0.30855378234329356, incumbent_id=1, incumbent=Configuration:
data_loader:batch_size, Value: 64
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
Expand Down Expand Up @@ -219,21 +219,23 @@ Print the final ensemble performance
optimizer:__choice__, Value: 'AdamOptimizer'
scaler:__choice__, Value: 'StandardScaler'
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=1, ta_time_used=3.1266889572143555, wallclock_time=4.161885023117065, budget=5.555555555555555)]
{'r2': 0.9246659176848098}
, ta_runs=1, ta_time_used=2.8718996047973633, wallclock_time=3.902437925338745, budget=5.555555555555555)]
{'r2': 0.9433209451809809}
| | Preprocessing | Estimator | Weight |
|---:|:------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
| 0 | None | CBLearner | 0.58 |
| 1 | None | LGBMLearner | 0.24 |
| 2 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.18 |
| 0 | None | CBLearner | 0.44 |
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.32 |
| 2 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.1 |
| 3 | None | LGBMLearner | 0.08 |
| 4 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 5 minutes 38.186 seconds)
**Total running time of the script:** ( 5 minutes 38.626 seconds)


.. _sphx_glr_download_examples_20_basics_example_tabular_regression.py:
Expand Down
8 changes: 4 additions & 4 deletions development/_sources/examples/20_basics/sg_execution_times.rst.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@

Computation times
=================
**11:22.217** total execution time for **examples_20_basics** files:
**11:15.612** total execution time for **examples_20_basics** files:

+--------------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_examples_20_basics_example_tabular_regression.py` (``example_tabular_regression.py``) | 05:38.186 | 0.0 MB |
| :ref:`sphx_glr_examples_20_basics_example_tabular_regression.py` (``example_tabular_regression.py``) | 05:38.626 | 0.0 MB |
+--------------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_examples_20_basics_example_tabular_classification.py` (``example_tabular_classification.py``) | 05:35.756 | 0.0 MB |
| :ref:`sphx_glr_examples_20_basics_example_tabular_classification.py` (``example_tabular_classification.py``) | 05:28.652 | 0.0 MB |
+--------------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_examples_20_basics_example_image_classification.py` (``example_image_classification.py``) | 00:08.275 | 0.0 MB |
| :ref:`sphx_glr_examples_20_basics_example_image_classification.py` (``example_image_classification.py``) | 00:08.334 | 0.0 MB |
+--------------------------------------------------------------------------------------------------------------+-----------+--------+

0 comments on commit 8f0af69

Please sign in to comment.