How to compress model in Multi-machine

[as754770178]

I want to compress the model in How to compress model in Multi-machine

[jiaxiang-wu]

You mean how to train a compressed model with multiple nodes/machines?

[xieydd]

When using the horovod, i think it`s feasible.

[as754770178]

Yes, but I mean compress the model by Prunning/Quantization, then train the compress model with multiple nodes/machines.

I have other questions:

1. Can PocketFlow compress object detection network， such as Yolo, fasterCNN?
2. I how to verify the inference speed of compressed model is faster than the full-precision uncompressed network?

thanks

[as754770178]

When using the horovod, i think it`s feasible.

What's it?

[xieydd]

Horovod is Distributed training framework for TensorFlow, Keras, and PyTorch.

1. AFAIK，it not yet support the state-of-art models lke the Yolo or fasterCNN
2. you can test in TensorRT
   @as754770178

[as754770178]

I exec command ./scripts/run_local.sh nets/resnet_at_cifar10_run.py --learner dis-chn-pruned.

I read the checkpoint file of compressed model (saved in models_dcp_eval file) and pre-trained model, but I find the structure of net is same. Compressed model only delete the variable w.r.t Momentum and BatchNorm.
So I want to test the inference speed of compressed model and full-precision uncompressed network

[jiaxiang-wu]

@as754770178

1. PocketFlow does not support object detection models (YOLO / Faster R-CNN) for the moment. We are working on this right now.
2. For multi-node training, please use horovod, as suggested by @xieydd .
3. To verify the inference speed, use TF-Lite models generated by tools/conversion/export_pb_tflite_models.py.

[as754770178]

I mean use tf-serving to test the speed? Can horovod support compress model? I want compress model and train it together, like PocketFlow.

[jiaxiang-wu]

1. Horovod is a distributed training framework. It does not care what code is running within.
2. The speed improvement on GPU will be less significant than on ARM-based mobile devices, according to our experience.

[as754770178]

Can PocketFlow compress multi-label Classification network?

[jiaxiang-wu]

Yes, you can define a new ModelHelper class, including the network architecture and the computation of loss function's value, similar to other ModelHelper classes defined under the nets directory.

[as754770178]

OK, thanks.