Skip to content

Commit

Permalink
Ravin Kohli: [ADD] Pytest schedule (#234)
Browse files Browse the repository at this point in the history
  • Loading branch information
Github Actions committed Jun 10, 2021
1 parent fdbdaf1 commit 8a61289
Show file tree
Hide file tree
Showing 24 changed files with 401 additions and 329 deletions.
Binary file modified development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip
View file
Binary file not shown.
Binary file modified development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip
View file
Binary file not shown.
Binary file modified development/_images/sphx_glr_example_visualization_001.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified development/_images/sphx_glr_example_visualization_002.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified development/_images/sphx_glr_example_visualization_thumb.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
23 changes: 11 additions & 12 deletions development/_sources/examples/20_basics/example_image_classification.rst.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,26 +87,25 @@ Image Classification
Pipeline Random Config:
________________________________________
Configuration:
image_augmenter:GaussianBlur:sigma_min, Value: 2.7815796172433904
image_augmenter:GaussianBlur:sigma_offset, Value: 2.2162192719004334
image_augmenter:GaussianBlur:use_augmenter, Value: True
image_augmenter:GaussianBlur:use_augmenter, Value: False
image_augmenter:GaussianNoise:use_augmenter, Value: False
image_augmenter:RandomAffine:rotate, Value: 211
image_augmenter:RandomAffine:scale_offset, Value: 0.1303552201986558
image_augmenter:RandomAffine:shear, Value: 24
image_augmenter:RandomAffine:translate_percent_offset, Value: 0.1470709970650069
image_augmenter:RandomAffine:rotate, Value: 139
image_augmenter:RandomAffine:scale_offset, Value: 0.23068913599566782
image_augmenter:RandomAffine:shear, Value: 15
image_augmenter:RandomAffine:translate_percent_offset, Value: 0.3649730895991128
image_augmenter:RandomAffine:use_augmenter, Value: True
image_augmenter:RandomCutout:use_augmenter, Value: False
image_augmenter:RandomCutout:p, Value: 0.8026417457168646
image_augmenter:RandomCutout:use_augmenter, Value: True
image_augmenter:Resize:use_augmenter, Value: True
image_augmenter:ZeroPadAndCrop:percent, Value: 0.29994917149110223
normalizer:__choice__, Value: 'ImageNormalizer'
image_augmenter:ZeroPadAndCrop:percent, Value: 0.26059070078518315
normalizer:__choice__, Value: 'NoNormalizer'

Fitting the pipeline...
________________________________________
ImageClassificationPipeline
________________________________________
0-) normalizer:
ImageNormalizer
NoNormalizer

1-) preprocessing:
EarlyPreprocessing
Expand Down Expand Up @@ -178,7 +177,7 @@ Image Classification
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 0 minutes 8.362 seconds)
**Total running time of the script:** ( 0 minutes 6.439 seconds)


.. _sphx_glr_download_examples_20_basics_example_image_classification.py:
Expand Down
95 changes: 47 additions & 48 deletions development/_sources/examples/20_basics/example_tabular_classification.rst.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ Search for an ensemble of machine learning algorithms
.. code-block:: none
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f7f96017d90>
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7fe257cbbe50>
Expand Down Expand Up @@ -162,7 +162,7 @@ Print the final ensemble performance

.. code-block:: none
<smac.runhistory.runhistory.RunHistory object at 0x7f7f96017130> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
<smac.runhistory.runhistory.RunHistory object at 0x7fe257b3a0d0> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
data_loader:batch_size, Value: 64
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
Expand Down Expand Up @@ -194,7 +194,7 @@ Print the final ensemble performance
scaler:__choice__, Value: 'StandardScaler'
trainer:StandardTrainer:weighted_loss, Value: True
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0012085437774658203, budget=0), TrajEntry(train_perf=0.17543859649122806, incumbent_id=1, incumbent=Configuration:
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0010366439819335938, budget=0), TrajEntry(train_perf=0.17543859649122806, incumbent_id=1, incumbent=Configuration:
data_loader:batch_size, Value: 64
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
Expand Down Expand Up @@ -226,63 +226,62 @@ Print the final ensemble performance
scaler:__choice__, Value: 'StandardScaler'
trainer:StandardTrainer:weighted_loss, Value: True
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=1, ta_time_used=2.6043033599853516, wallclock_time=3.6323750019073486, budget=5.555555555555555), TrajEntry(train_perf=0.1578947368421053, incumbent_id=2, incumbent=Configuration:
data_loader:batch_size, Value: 131
encoder:__choice__, Value: 'NoEncoder'
feature_preprocessor:KernelPCA:coef0, Value: -0.2027355777455664
feature_preprocessor:KernelPCA:degree, Value: 2
feature_preprocessor:KernelPCA:gamma, Value: 0.0029756156161293078
feature_preprocessor:KernelPCA:kernel, Value: 'poly'
feature_preprocessor:KernelPCA:n_components, Value: 4
feature_preprocessor:__choice__, Value: 'KernelPCA'
, ta_runs=1, ta_time_used=1.9255990982055664, wallclock_time=2.953387975692749, budget=5.555555555555555), TrajEntry(train_perf=0.14619883040935677, incumbent_id=2, incumbent=Configuration:
data_loader:batch_size, Value: 54
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
imputer:categorical_strategy, Value: 'constant_!missing!'
imputer:numerical_strategy, Value: 'mean'
lr_scheduler:CosineAnnealingWarmRestarts:T_0, Value: 20
lr_scheduler:CosineAnnealingWarmRestarts:T_mult, Value: 1.2502829975237466
lr_scheduler:__choice__, Value: 'CosineAnnealingWarmRestarts'
network_backbone:ShapedResNetBackbone:activation, Value: 'sigmoid'
network_backbone:ShapedResNetBackbone:blocks_per_group, Value: 2
network_backbone:ShapedResNetBackbone:max_units, Value: 21
network_backbone:ShapedResNetBackbone:num_groups, Value: 11
network_backbone:ShapedResNetBackbone:output_dim, Value: 128
network_backbone:ShapedResNetBackbone:resnet_shape, Value: 'stairs'
network_backbone:ShapedResNetBackbone:use_dropout, Value: False
network_backbone:ShapedResNetBackbone:use_shake_drop, Value: False
network_backbone:ShapedResNetBackbone:use_shake_shake, Value: False
network_backbone:__choice__, Value: 'ShapedResNetBackbone'
lr_scheduler:CosineAnnealingLR:T_max, Value: 307
lr_scheduler:__choice__, Value: 'CosineAnnealingLR'
network_backbone:ShapedMLPBackbone:activation, Value: 'relu'
network_backbone:ShapedMLPBackbone:max_dropout, Value: 0.543030049110043
network_backbone:ShapedMLPBackbone:max_units, Value: 35
network_backbone:ShapedMLPBackbone:mlp_shape, Value: 'hexagon'
network_backbone:ShapedMLPBackbone:num_groups, Value: 3
network_backbone:ShapedMLPBackbone:output_dim, Value: 18
network_backbone:ShapedMLPBackbone:use_dropout, Value: True
network_backbone:__choice__, Value: 'ShapedMLPBackbone'
network_embedding:__choice__, Value: 'NoEmbedding'
network_head:__choice__, Value: 'fully_connected'
network_head:fully_connected:activation, Value: 'tanh'
network_head:fully_connected:num_layers, Value: 4
network_head:fully_connected:units_layer_1, Value: 415
network_head:fully_connected:units_layer_2, Value: 290
network_head:fully_connected:units_layer_3, Value: 313
network_init:KaimingInit:bias_strategy, Value: 'Normal'
network_init:__choice__, Value: 'KaimingInit'
optimizer:AdamOptimizer:beta1, Value: 0.9981587455677909
optimizer:AdamOptimizer:beta2, Value: 0.9934737249657393
optimizer:AdamOptimizer:lr, Value: 0.0015351906927605823
optimizer:AdamOptimizer:weight_decay, Value: 0.06126849297256112
optimizer:__choice__, Value: 'AdamOptimizer'
network_head:fully_connected:activation, Value: 'relu'
network_head:fully_connected:num_layers, Value: 3
network_head:fully_connected:units_layer_1, Value: 316
network_head:fully_connected:units_layer_2, Value: 503
network_init:SparseInit:bias_strategy, Value: 'Normal'
network_init:__choice__, Value: 'SparseInit'
optimizer:AdamWOptimizer:beta1, Value: 0.9489565046389004
optimizer:AdamWOptimizer:beta2, Value: 0.9647522172509646
optimizer:AdamWOptimizer:lr, Value: 0.0030477242366055836
optimizer:AdamWOptimizer:weight_decay, Value: 0.061913730296919815
optimizer:__choice__, Value: 'AdamWOptimizer'
scaler:__choice__, Value: 'MinMaxScaler'
trainer:StandardTrainer:weighted_loss, Value: False
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=20, ta_time_used=161.57836866378784, wallclock_time=232.07247638702393, budget=50.0)]
{'accuracy': 0.8728323699421965}
| | Preprocessing | Estimator | Weight |
|---:|:------------------------------------------------------------------|:-------------------------------------------------------------------|---------:|
| 0 | None | CatBoostClassifier | 0.92 |
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
| 2 | SimpleImputer,NoEncoder,MinMaxScaler,KernelPCA | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
| 3 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
trainer:MixUpTrainer:alpha, Value: 0.8559230573827334
trainer:MixUpTrainer:weighted_loss, Value: True
trainer:__choice__, Value: 'MixUpTrainer'
, ta_runs=18, ta_time_used=170.50614953041077, wallclock_time=220.6971218585968, budget=50.0)]
{'accuracy': 0.861271676300578}
| | Preprocessing | Estimator | Weight |
|---:|:------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
| 0 | None | CatBoostClassifier | 0.28 |
| 1 | SimpleImputer,NoEncoder,Normalizer,Nystroem | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.2 |
| 2 | None | KNNClassifier | 0.1 |
| 3 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.1 |
| 4 | SimpleImputer,OneHotEncoder,StandardScaler,Nystroem | no embedding,ResNetBackbone,FullyConnectedHead,nn.Sequential | 0.06 |
| 5 | None | LGBMClassifier | 0.06 |
| 6 | None | RFClassifier | 0.06 |
| 7 | SimpleImputer,OneHotEncoder,MinMaxScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
| 8 | SimpleImputer,NoEncoder,NoScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
| 9 | None | SVC | 0.04 |
| 10 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 5 minutes 28.336 seconds)
**Total running time of the script:** ( 5 minutes 24.537 seconds)


.. _sphx_glr_download_examples_20_basics_example_tabular_classification.py:
Expand Down
30 changes: 15 additions & 15 deletions development/_sources/examples/20_basics/example_tabular_regression.rst.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ Search for an ensemble of machine learning algorithms
.. code-block:: none
<autoPyTorch.api.tabular_regression.TabularRegressionTask object at 0x7f803aa3a6a0>
<autoPyTorch.api.tabular_regression.TabularRegressionTask object at 0x7fe2fda056a0>
Expand Down Expand Up @@ -158,7 +158,7 @@ Print the final ensemble performance

.. code-block:: none
<smac.runhistory.runhistory.RunHistory object at 0x7f802dfa0040> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
<smac.runhistory.runhistory.RunHistory object at 0x7fe2f0fa5040> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
data_loader:batch_size, Value: 64
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
Expand Down Expand Up @@ -189,7 +189,7 @@ Print the final ensemble performance
optimizer:__choice__, Value: 'AdamOptimizer'
scaler:__choice__, Value: 'StandardScaler'
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0011713504791259766, budget=0), TrajEntry(train_perf=5.132313191053143, incumbent_id=1, incumbent=Configuration:
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0010111331939697266, budget=0), TrajEntry(train_perf=4.001648104799329, incumbent_id=1, incumbent=Configuration:
data_loader:batch_size, Value: 64
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
Expand Down Expand Up @@ -220,7 +220,7 @@ Print the final ensemble performance
optimizer:__choice__, Value: 'AdamOptimizer'
scaler:__choice__, Value: 'StandardScaler'
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=1, ta_time_used=1.7346484661102295, wallclock_time=2.760028600692749, budget=5.555555555555555), TrajEntry(train_perf=3.390282144755361, incumbent_id=2, incumbent=Configuration:
, ta_runs=1, ta_time_used=1.2097136974334717, wallclock_time=2.233612060546875, budget=5.555555555555555), TrajEntry(train_perf=3.1930668561547075, incumbent_id=2, incumbent=Configuration:
data_loader:batch_size, Value: 67
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:TruncatedSVD:target_dim, Value: 6
Expand Down Expand Up @@ -256,7 +256,7 @@ Print the final ensemble performance
optimizer:__choice__, Value: 'RMSpropOptimizer'
scaler:__choice__, Value: 'NoScaler'
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=3, ta_time_used=7.148277759552002, wallclock_time=18.647205114364624, budget=5.555555555555555), TrajEntry(train_perf=1.7652469933219221, incumbent_id=3, incumbent=Configuration:
, ta_runs=3, ta_time_used=5.242362022399902, wallclock_time=14.604888677597046, budget=5.555555555555555), TrajEntry(train_perf=1.63612106672499, incumbent_id=3, incumbent=Configuration:
data_loader:batch_size, Value: 150
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:TruncatedSVD:target_dim, Value: 9
Expand Down Expand Up @@ -297,7 +297,7 @@ Print the final ensemble performance
scaler:Normalizer:norm, Value: 'max'
scaler:__choice__, Value: 'Normalizer'
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=4, ta_time_used=10.926101922988892, wallclock_time=23.508082389831543, budget=5.555555555555555), TrajEntry(train_perf=1.3579328988923458, incumbent_id=4, incumbent=Configuration:
, ta_runs=4, ta_time_used=7.889399766921997, wallclock_time=18.329752445220947, budget=5.555555555555555), TrajEntry(train_perf=1.303718113076385, incumbent_id=4, incumbent=Configuration:
data_loader:batch_size, Value: 82
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:TruncatedSVD:target_dim, Value: 8
Expand Down Expand Up @@ -332,7 +332,7 @@ Print the final ensemble performance
scaler:Normalizer:norm, Value: 'mean_abs'
scaler:__choice__, Value: 'Normalizer'
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=6, ta_time_used=16.41826319694519, wallclock_time=36.227760314941406, budget=5.555555555555555), TrajEntry(train_perf=1.0, incumbent_id=5, incumbent=Configuration:
, ta_runs=6, ta_time_used=11.728835821151733, wallclock_time=27.961014986038208, budget=5.555555555555555), TrajEntry(train_perf=1.0, incumbent_id=5, incumbent=Configuration:
data_loader:batch_size, Value: 64
encoder:__choice__, Value: 'NoEncoder'
feature_preprocessor:TruncatedSVD:target_dim, Value: 8
Expand Down Expand Up @@ -374,7 +374,7 @@ Print the final ensemble performance
scaler:__choice__, Value: 'Normalizer'
trainer:MixUpTrainer:alpha, Value: 0.15674505157760243
trainer:__choice__, Value: 'MixUpTrainer'
, ta_runs=9, ta_time_used=51.96426844596863, wallclock_time=80.42096281051636, budget=5.555555555555555), TrajEntry(train_perf=1.0008126311940073, incumbent_id=6, incumbent=Configuration:
, ta_runs=9, ta_time_used=38.322489976882935, wallclock_time=61.81372833251953, budget=5.555555555555555), TrajEntry(train_perf=1.0090091679942723, incumbent_id=6, incumbent=Configuration:
data_loader:batch_size, Value: 82
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:TruncatedSVD:target_dim, Value: 8
Expand Down Expand Up @@ -409,7 +409,7 @@ Print the final ensemble performance
scaler:Normalizer:norm, Value: 'mean_abs'
scaler:__choice__, Value: 'Normalizer'
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=10, ta_time_used=54.072187185287476, wallclock_time=83.59581923484802, budget=16.666666666666664), TrajEntry(train_perf=1.0, incumbent_id=7, incumbent=Configuration:
, ta_runs=10, ta_time_used=39.92764472961426, wallclock_time=64.51126146316528, budget=16.666666666666664), TrajEntry(train_perf=1.0, incumbent_id=7, incumbent=Configuration:
data_loader:batch_size, Value: 130
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:TruncatedSVD:target_dim, Value: 6
Expand Down Expand Up @@ -467,19 +467,19 @@ Print the final ensemble performance
optimizer:__choice__, Value: 'AdamOptimizer'
scaler:__choice__, Value: 'StandardScaler'
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=17, ta_time_used=165.81303334236145, wallclock_time=210.14278411865234, budget=50.0)]
{'r2': -0.002203283150869373}
| | Preprocessing | Estimator | Weight |
|---:|:------------------------------------------------|:-------------------------------------------------------------|---------:|
| 0 | SimpleImputer,OneHotEncoder,Normalizer,TruncSVD | embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 1 |
, ta_runs=17, ta_time_used=133.23015427589417, wallclock_time=170.4870789051056, budget=50.0)]
{'r2': 0.13995994112861065}
| | Preprocessing | Estimator | Weight |
|---:|:--------------------------------------------------------------|:-------------------------------------------------------------------|---------:|
| 0 | SimpleImputer,OneHotEncoder,StandardScaler,PolynomialFeatures | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 1 |
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 5 minutes 17.148 seconds)
**Total running time of the script:** ( 5 minutes 9.389 seconds)


.. _sphx_glr_download_examples_20_basics_example_tabular_regression.py:
Expand Down

0 comments on commit 8a61289

Please sign in to comment.