Skip to content

AutoML for Model Compression (AMC): Trials and Tribulations

Neta Zmora edited this page Jan 24, 2019 · 15 revisions

Work on AMC currently takes place in branch 'amc'.

Replacing DDPG with Clipped PPO creates better results as presented below. I created these diagram using this Distiller Jupyter notebook. For details about our attempts on using DDPG for AMC see this notebook.

Clipped PPO is more stable and reliable but converges slower compared to DDPG (although I haven't enabled PPO's full parallelization capabilities yet, so I'm optimistic). Its stability is quite promising, as seen below.

Research questions:

  • How does this fare on larger models (e.g. ResNet50)?
  • Can we improve the final Top1 score if we use the model-structure (per-layer density) as input to AGP-for-Structure with a short pruning schedule?

MACs-Constrained Compression

python3 --arch=plain20_cifar ../../../data.cifar --amc --resume=checkpoint.plain20_cifar.pth.tar --lr=0.05 --amc-action-range 0.0 0.80 --vs=0.8

Accuracy-Guaranteed Compression

python3 --arch=plain20_cifar ../../../data.cifar --amc --resume=checkpoint.plain20_cifar.pth.tar --lr=0.05 --amc-action-range 0.2 0.80 --vs=0.8 --amc-protocol=accuracy-guaranteed

You can’t perform that action at this time.