Skip to content

Latest commit

 

History

History
323 lines (202 loc) · 7 KB

algorithms.rst

File metadata and controls

323 lines (202 loc) · 7 KB
Raw

composer.algorithms

composer.algorithms

We describe programmatic modifications to the model or training process as "algorithms." Examples include :pysmoothing the labels <composer.algorithms.label_smoothing.LabelSmoothing> and adding :pySqueeze-and-Excitation <composer.algorithms.squeeze_excite.SqueezeExcite> blocks, among many others.

Algorithms can be used in two ways:

  • Using :py~composer.algorithm.Algorithm objects. These objects provide callbacks to be run in the training loop.
  • Using algorithm-specific functions and classes, such as :pysmooth_labels <composer.algorithms.label_smoothing.smooth_labels> or :py~composer.algorithms.squeeze_excite.SqueezeExcite2d.

The former are the easier to compose together, since they all have the same public interface and work automatically with the Composer :py~composer.trainer.Trainer. The latter are easier to integrate piecemeal into an existing codebase.

See :py~composer.algorithms.Algorithm for more information.

The following algorithms are available in Composer:

~alibi.Alibi ~augmix.AugMix ~blurpool.BlurPool ~channels_last.ChannelsLast ~colout.ColOut ~cutout.CutOut ~ghost_batchnorm.GhostBatchNorm ~label_smoothing.LabelSmoothing ~layer_freezing.LayerFreezing ~mixup.MixUp ~progressive_resizing.ProgressiveResizing ~randaugment.RandAugment ~sam.SAM ~scale_schedule.ScaleSchedule ~selective_backprop.SelectiveBackprop ~squeeze_excite.SqueezeExcite ~stochastic_depth.StochasticDepth ~swa.SWA

Alibi

Algorithm

composer.algorithms.alibi.Alibi

composer.algorithms.alibi.AlibiHparams

Standalone

composer.algorithms.alibi.apply_alibi

Augmix

Algorithm

composer.algorithms.augmix.AugMix

composer.algorithms.augmix.AugMixHparams

Standalone

composer.algorithms.augmix.augment_and_mix

BlurPool

Algorithm

composer.algorithms.blurpool.BlurPool

composer.algorithms.blurpool.BlurPoolHparams

Standalone

composer.algorithms.blurpool.BlurConv2d

composer.algorithms.blurpool.BlurMaxPool2d

composer.algorithms.blurpool.BlurPool2d

composer.algorithms.blurpool.blur_2d

composer.algorithms.blurpool.apply_blurpool

Channels Last

Algorithm

composer.algorithms.channels_last.ChannelsLast

composer.algorithms.channels_last.ChannelsLastHparams

ColOut

Algorithm

composer.algorithms.colout.ColOut

composer.algorithms.colout.ColOutHparams

Standalone

composer.algorithms.colout.colout

CutOut

Algorithm

composer.algorithms.cutout.CutOut

composer.algorithms.cutout.CutOutHparams

Standalone

composer.algorithms.cutout.cutout

Ghost Batch Normalization

Algorithm

composer.algorithms.ghost_batchnorm.GhostBatchNorm

composer.algorithms.ghost_batchnorm.GhostBatchNormHparams

Standalone

composer.algorithms.ghost_batchnorm.apply_ghost_batchnorm

Label Smoothing

Algorithm

composer.algorithms.label_smoothing.LabelSmoothing

composer.algorithms.label_smoothing.LabelSmoothingHparams

Standalone

composer.algorithms.label_smoothing.smooth_labels

Layer Freezing

Algorithm

composer.algorithms.layer_freezing.LayerFreezing

composer.algorithms.layer_freezing.LayerFreezingHparams

Standalone

composer.algorithms.layer_freezing.freeze_layers

MixUp

Algorithm

composer.algorithms.mixup.MixUp

composer.algorithms.mixup.MixUpHparams

Standalone

composer.algorithms.mixup.mixup_batch

Progressive Resizing

Algorithm

composer.algorithms.progressive_resizing.ProgressiveResizing

composer.algorithms.progressive_resizing.ProgressiveResizingHparams

Standalone

composer.algorithms.progressive_resizing.resize_inputs

RandAugment

Algorithm

composer.algorithms.randaugment.RandAugment

composer.algorithms.randaugment.RandAugmentHparams

Standalone

composer.algorithms.randaugment.randaugment

Sequence Length Warmup

Algorithm

composer.algorithms.seq_length_warmup

composer.algorithms.seq_length_warmup.SeqLengthWarmup

composer.algorithms.seq_length_warmup.SeqLengthWarmupHparams

Standalone

composer.algorithms.seq_length_warmup

composer.algorithms.seq_length_warmup.apply_seq_length_warmup

Sharpness-Aware Minimization

Algorithm

composer.algorithms.sam.SAM

composer.algorithms.sam.SAMHparams

Scaling the Learning Rate Schedule

Algorithm

composer.algorithms.scale_schedule.ScaleSchedule

composer.algorithms.scale_schedule.ScaleScheduleHparams

Standalone

composer.algorithms.scale_schedule.scale_scheduler

Selective Backpropagation

Algorithm

composer.algorithms.selective_backprop.SelectiveBackprop

composer.algorithms.selective_backprop.SelectiveBackpropHparams

Squeeze-and-Excitation

Algorithm

composer.algorithms.squeeze_excite.SqueezeExcite

composer.algorithms.squeeze_excite.SqueezeExciteHparams

Standalone

composer.algorithms.squeeze_excite.SqueezeExcite2d

composer.algorithms.squeeze_excite.SqueezeExciteConv2d

composer.algorithms.squeeze_excite.apply_se

Stochastic Depth

Algorithm

composer.algorithms.stochastic_depth.StochasticDepth

composer.algorithms.stochastic_depth.StochasticDepthHparams

Standalone

composer.algorithms.stochastic_depth.StochasticBottleneck

composer.algorithms.stochastic_depth.apply_stochastic_depth

Stochastic Weight Averaging

Algorithm

composer.algorithms.swa.SWA

composer.algorithms.swa.SWAHparams