Knowledge Distillation

[HKLee2040]

I had read "Knowledge Distillation" https://nervanasystems.github.io/distiller/schedule/index.html#knowledge-distillation

Would you please help to give me a simple example about knowledge distillation?

[guyjacob]

If you're looking for examples of how to use knowledge distillation in Distiller, I updated a couple of the sample schedulers in the quantization samples dir with sample command lines and results - FP32 sample, quantized sample.
If you're looking for an example of where knowledge distillation might be used in general, we have a nice introduction to the subject in our docs, with some links to papers using knowledge distillation with other compression methods.

[HKLee2040]

@guyjacob The quantization samples are what I need. Thanks!