Skip to content

Commit

Permalink
Francisco Rivera Valverde: [ADD] Coverage calculation (#224)
Browse files Browse the repository at this point in the history
  • Loading branch information
Github Actions committed Jun 2, 2021
1 parent 842537e commit f7c29d1
Show file tree
Hide file tree
Showing 24 changed files with 591 additions and 631 deletions.
Binary file modified development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip
View file
Binary file not shown.
Binary file modified development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip
View file
Binary file not shown.
Binary file modified development/_images/sphx_glr_example_visualization_001.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified development/_images/sphx_glr_example_visualization_002.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified development/_images/sphx_glr_example_visualization_thumb.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
28 changes: 14 additions & 14 deletions development/_sources/examples/20_basics/example_image_classification.rst.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,26 +87,26 @@ Image Classification
Pipeline Random Config:
________________________________________
Configuration:
image_augmenter:GaussianBlur:use_augmenter, Value: False
image_augmenter:GaussianNoise:sigma_offset, Value: 2.6558905363999727
image_augmenter:GaussianNoise:use_augmenter, Value: True
image_augmenter:RandomAffine:rotate, Value: 297
image_augmenter:RandomAffine:scale_offset, Value: 0.10167631197409942
image_augmenter:RandomAffine:shear, Value: 26
image_augmenter:RandomAffine:translate_percent_offset, Value: 0.018225027710844
image_augmenter:GaussianBlur:sigma_min, Value: 2.7815796172433904
image_augmenter:GaussianBlur:sigma_offset, Value: 2.2162192719004334
image_augmenter:GaussianBlur:use_augmenter, Value: True
image_augmenter:GaussianNoise:use_augmenter, Value: False
image_augmenter:RandomAffine:rotate, Value: 211
image_augmenter:RandomAffine:scale_offset, Value: 0.1303552201986558
image_augmenter:RandomAffine:shear, Value: 24
image_augmenter:RandomAffine:translate_percent_offset, Value: 0.1470709970650069
image_augmenter:RandomAffine:use_augmenter, Value: True
image_augmenter:RandomCutout:p, Value: 0.9504486965250005
image_augmenter:RandomCutout:use_augmenter, Value: True
image_augmenter:Resize:use_augmenter, Value: False
image_augmenter:ZeroPadAndCrop:percent, Value: 0.48642929026494963
normalizer:__choice__, Value: 'NoNormalizer'
image_augmenter:RandomCutout:use_augmenter, Value: False
image_augmenter:Resize:use_augmenter, Value: True
image_augmenter:ZeroPadAndCrop:percent, Value: 0.29994917149110223
normalizer:__choice__, Value: 'ImageNormalizer'

Fitting the pipeline...
________________________________________
ImageClassificationPipeline
________________________________________
0-) normalizer:
NoNormalizer
ImageNormalizer

1-) preprocessing:
EarlyPreprocessing
Expand Down Expand Up @@ -178,7 +178,7 @@ Image Classification
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 0 minutes 5.588 seconds)
**Total running time of the script:** ( 0 minutes 8.362 seconds)


.. _sphx_glr_download_examples_20_basics_example_image_classification.py:
Expand Down
92 changes: 48 additions & 44 deletions development/_sources/examples/20_basics/example_tabular_classification.rst.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ Search for an ensemble of machine learning algorithms
.. code-block:: none
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f0eb3815eb0>
<autoPyTorch.api.tabular_classification.TabularClassificationTask object at 0x7f7f96017d90>
Expand Down Expand Up @@ -162,7 +162,7 @@ Print the final ensemble performance

.. code-block:: none
<smac.runhistory.runhistory.RunHistory object at 0x7f0eb35b7790> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
<smac.runhistory.runhistory.RunHistory object at 0x7f7f96017130> [TrajEntry(train_perf=2147483648, incumbent_id=1, incumbent=Configuration:
data_loader:batch_size, Value: 64
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
Expand Down Expand Up @@ -194,7 +194,7 @@ Print the final ensemble performance
scaler:__choice__, Value: 'StandardScaler'
trainer:StandardTrainer:weighted_loss, Value: True
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0013079643249511719, budget=0), TrajEntry(train_perf=0.17543859649122806, incumbent_id=1, incumbent=Configuration:
, ta_runs=0, ta_time_used=0.0, wallclock_time=0.0012085437774658203, budget=0), TrajEntry(train_perf=0.17543859649122806, incumbent_id=1, incumbent=Configuration:
data_loader:batch_size, Value: 64
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
Expand Down Expand Up @@ -226,59 +226,63 @@ Print the final ensemble performance
scaler:__choice__, Value: 'StandardScaler'
trainer:StandardTrainer:weighted_loss, Value: True
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=1, ta_time_used=2.003859281539917, wallclock_time=3.0306620597839355, budget=5.555555555555555), TrajEntry(train_perf=0.14619883040935677, incumbent_id=2, incumbent=Configuration:
data_loader:batch_size, Value: 54
encoder:__choice__, Value: 'OneHotEncoder'
feature_preprocessor:__choice__, Value: 'NoFeaturePreprocessor'
, ta_runs=1, ta_time_used=2.6043033599853516, wallclock_time=3.6323750019073486, budget=5.555555555555555), TrajEntry(train_perf=0.1578947368421053, incumbent_id=2, incumbent=Configuration:
data_loader:batch_size, Value: 131
encoder:__choice__, Value: 'NoEncoder'
feature_preprocessor:KernelPCA:coef0, Value: -0.2027355777455664
feature_preprocessor:KernelPCA:degree, Value: 2
feature_preprocessor:KernelPCA:gamma, Value: 0.0029756156161293078
feature_preprocessor:KernelPCA:kernel, Value: 'poly'
feature_preprocessor:KernelPCA:n_components, Value: 4
feature_preprocessor:__choice__, Value: 'KernelPCA'
imputer:categorical_strategy, Value: 'constant_!missing!'
imputer:numerical_strategy, Value: 'mean'
lr_scheduler:CosineAnnealingLR:T_max, Value: 307
lr_scheduler:__choice__, Value: 'CosineAnnealingLR'
network_backbone:ShapedMLPBackbone:activation, Value: 'relu'
network_backbone:ShapedMLPBackbone:max_dropout, Value: 0.543030049110043
network_backbone:ShapedMLPBackbone:max_units, Value: 35
network_backbone:ShapedMLPBackbone:mlp_shape, Value: 'hexagon'
network_backbone:ShapedMLPBackbone:num_groups, Value: 3
network_backbone:ShapedMLPBackbone:output_dim, Value: 18
network_backbone:ShapedMLPBackbone:use_dropout, Value: True
network_backbone:__choice__, Value: 'ShapedMLPBackbone'
lr_scheduler:CosineAnnealingWarmRestarts:T_0, Value: 20
lr_scheduler:CosineAnnealingWarmRestarts:T_mult, Value: 1.2502829975237466
lr_scheduler:__choice__, Value: 'CosineAnnealingWarmRestarts'
network_backbone:ShapedResNetBackbone:activation, Value: 'sigmoid'
network_backbone:ShapedResNetBackbone:blocks_per_group, Value: 2
network_backbone:ShapedResNetBackbone:max_units, Value: 21
network_backbone:ShapedResNetBackbone:num_groups, Value: 11
network_backbone:ShapedResNetBackbone:output_dim, Value: 128
network_backbone:ShapedResNetBackbone:resnet_shape, Value: 'stairs'
network_backbone:ShapedResNetBackbone:use_dropout, Value: False
network_backbone:ShapedResNetBackbone:use_shake_drop, Value: False
network_backbone:ShapedResNetBackbone:use_shake_shake, Value: False
network_backbone:__choice__, Value: 'ShapedResNetBackbone'
network_embedding:__choice__, Value: 'NoEmbedding'
network_head:__choice__, Value: 'fully_connected'
network_head:fully_connected:activation, Value: 'relu'
network_head:fully_connected:num_layers, Value: 3
network_head:fully_connected:units_layer_1, Value: 316
network_head:fully_connected:units_layer_2, Value: 503
network_init:SparseInit:bias_strategy, Value: 'Normal'
network_init:__choice__, Value: 'SparseInit'
optimizer:AdamWOptimizer:beta1, Value: 0.9489565046389004
optimizer:AdamWOptimizer:beta2, Value: 0.9647522172509646
optimizer:AdamWOptimizer:lr, Value: 0.0030477242366055836
optimizer:AdamWOptimizer:weight_decay, Value: 0.061913730296919815
optimizer:__choice__, Value: 'AdamWOptimizer'
network_head:fully_connected:activation, Value: 'tanh'
network_head:fully_connected:num_layers, Value: 4
network_head:fully_connected:units_layer_1, Value: 415
network_head:fully_connected:units_layer_2, Value: 290
network_head:fully_connected:units_layer_3, Value: 313
network_init:KaimingInit:bias_strategy, Value: 'Normal'
network_init:__choice__, Value: 'KaimingInit'
optimizer:AdamOptimizer:beta1, Value: 0.9981587455677909
optimizer:AdamOptimizer:beta2, Value: 0.9934737249657393
optimizer:AdamOptimizer:lr, Value: 0.0015351906927605823
optimizer:AdamOptimizer:weight_decay, Value: 0.06126849297256112
optimizer:__choice__, Value: 'AdamOptimizer'
scaler:__choice__, Value: 'MinMaxScaler'
trainer:MixUpTrainer:alpha, Value: 0.8559230573827334
trainer:MixUpTrainer:weighted_loss, Value: True
trainer:__choice__, Value: 'MixUpTrainer'
, ta_runs=18, ta_time_used=191.332772731781, wallclock_time=243.9774169921875, budget=50.0)]
{'accuracy': 0.861271676300578}
| | Preprocessing | Estimator | Weight |
|---:|:------------------------------------------------------------------|:----------------------------------------------------------------|---------:|
| 0 | SimpleImputer,NoEncoder,Normalizer,Nystroem | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.2 |
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.2 |
| 2 | None | CatBoostClassifier | 0.18 |
| 3 | None | SVC | 0.18 |
| 4 | None | RFClassifier | 0.08 |
| 5 | None | ExtraTreesClassifier | 0.06 |
| 6 | None | KNNClassifier | 0.06 |
| 7 | SimpleImputer,OneHotEncoder,MinMaxScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
trainer:StandardTrainer:weighted_loss, Value: False
trainer:__choice__, Value: 'StandardTrainer'
, ta_runs=20, ta_time_used=161.57836866378784, wallclock_time=232.07247638702393, budget=50.0)]
{'accuracy': 0.8728323699421965}
| | Preprocessing | Estimator | Weight |
|---:|:------------------------------------------------------------------|:-------------------------------------------------------------------|---------:|
| 0 | None | CatBoostClassifier | 0.92 |
| 1 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.04 |
| 2 | SimpleImputer,NoEncoder,MinMaxScaler,KernelPCA | no embedding,ShapedResNetBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
| 3 | SimpleImputer,OneHotEncoder,StandardScaler,NoFeaturePreprocessing | no embedding,ShapedMLPBackbone,FullyConnectedHead,nn.Sequential | 0.02 |
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 5 minutes 19.640 seconds)
**Total running time of the script:** ( 5 minutes 28.336 seconds)


.. _sphx_glr_download_examples_20_basics_example_tabular_classification.py:
Expand Down

0 comments on commit f7c29d1

Please sign in to comment.