Quantization with 1D convolutions

[dorian95]

Hi, I have a question, does Distiller support 1D convolutions? I'm trying to compress a CNN with 1D convolutions for binary classification of strings using quantization.

[guyjacob]

Indeed at the moment only Conv2d is explicitly supported in the code, as we haven't had a chance to experiment with quantization of models that have 1D convolutions. But I don't see anything that should prevent Conv1d from working, seeing as Conv1d and Conv2d are both sub-classes of the same parent (_ConvNd).

To test, please make the following changes:

• https://github.com/NervanaSystems/distiller/blob/c247b57f6f14e609b39cb3e8c1fbae29ca0fe049/distiller/quantization/quantizer.py#L141-L142
  Add nn.Conv1d to the list in the if condition.

• Depending on which quantization method you are using, you might need to map Conv1d to a "replacement factory" function. At the moment only SymmetricLinearQuantizer replaces convolution modules. If this is the one you use, add a line here:
  https://github.com/NervanaSystems/distiller/blob/master/distiller/quantization/range_linear.py#L195-L196

If and when you find this works, feel free to make a PR with the changes. Let me know if you have additional questions.

[levzlotnik]

Hi,

Closing issue since it seems resolved and stale. Feel free to reopen.