-
Notifications
You must be signed in to change notification settings - Fork 799
Quantize my model question? #360
Comments
@nzmora Thanks for your quick reply! But I am not understand the example of resnet20-cifar can output 1/2 model(8 bits and 16 bits are same?). |
Hi @Linxxxx, |
@nzmora My question is "Are all quantized model will not be compressed through quantization on computer?" |
@nzmora Hi, I had another question about quantize my own model. When I random a dummy_input to prepare_model, I found this in your code, as show: self.model.quantizer_metadata["dummy_input"] = dummy_input But 'self.adjacency_map' was not used in your project, what's the function of this step? |
Hi @Linxxxx Question 1: correct, as explained in the documentation, to benefit from quantization you need the framework and HW to support execution of quantized models. Pytorch 1.2 supports quantization, so hopefully this will change soon. Question 2: class Cheers |
@nzmora Thanks! For question 1: When I try your example of resnet20-cifar, the quantized model is compressed from 2.2M(original) to 1.1M(INT8). According to you, the quantized model should not be compressed? I did not understand. |
@nzmora For question 2: When I quantize my own model, in
I debug the error then I found |
Hi @Linxxxx , Please provide your PyTorch model. |
@nzmora Hello, This is my model: `import copy num_classes = opt.num_classes # change this depend on your dataset class MGN(nn.Module):
` |
Hi @Linxxxx Please try adding a try-except block in |
@nzmora Hi~ It does work, but is there any influence to the quantization result? This is my revised code: |
Add try/except block around code accessing missing convolution shape information.
Please re-open if you need further information or assistance. |
Add try/except block around code accessing missing convolution shape information.
Hi~, I implemented my model quantization like this:
`import distiller
from distiller.quantization import PostTrainLinearQuantizer
quantizer = PostTrainLinearQuantizer(model)
quantizer.prepare_model(torch.rand(*your_input_shape))
apputils.save_checkpoint(0, 'mymodel', model, optimizer=None, name='model', dir='quantization')`
But output model did not resize to 1/4,just not changed, would you please to help me? Thanks!
The text was updated successfully, but these errors were encountered: