Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How to save quantized model in PyTorch1.3 with quantization information #28331

Closed
vippeterhou opened this issue Oct 19, 2019 · 4 comments
Closed

How to save quantized model in PyTorch1.3 with quantization information #28331

vippeterhou opened this issue Oct 19, 2019 · 4 comments
Labels
oncall: quantization Quantization support in PyTorch triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@vippeterhou
Copy link

vippeterhou commented Oct 19, 2019 •
edited by pytorch-probot bot
Loading

❓ How to save the quantized model in PyTorch1.3 with quantization information

Is there any way to save the quantized model in PyTorch1.3, which keeps the original information remaining?

I have known that I can save it after tracing it by:

# Save
torch.jit.save(torch.jit.script(self.model_q), "quant_model.pth")
# Load
mq = torch.jit.load("quant_model.pth")

Although mq has the right result, it, however, losts the quantized information, such as module(layer) name, zero point, scale, etc.

cc @jerryzh168 @jianyuh @dzhulgakov @raghuramank100
@vippeterhou vippeterhou changed the title How to save quantized model in PyTorch1.3 with qutization infomation How to save quantized model in PyTorch1.3 with quantization infomation Oct 19, 2019
@vippeterhou vippeterhou changed the title How to save quantized model in PyTorch1.3 with quantization infomation How to save quantized model in PyTorch1.3 with quantization information Oct 19, 2019
@lly-zero-one lly-zero-one added the oncall: quantization Quantization support in PyTorch label Oct 21, 2019
@lly-zero-one
Copy link
Contributor

I think the zero_point and scale should be kept in the model. Can you let us know how to reproduce the issue?

@lly-zero-one lly-zero-one added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Oct 21, 2019
@vippeterhou
Copy link
Author

@lly-zero-one

# load float model -> add observer -> calibration -> convert -> print
print(self.model_q) # original quantized model

# jit script save and load
torch.jit.save(torch.jit.script(self.model_q), "quant_model.pth")
mq = torch.jit.load("quant_model.pth")
print(mq) # script model

The output as follows:

MobileNetV2(
  (features): Sequential(
    (0): ConvBNReLU(
      (0): QuantizedConvReLU2d(3, 32, kernel_size=(3, 3), stride=(2, 2), scale=0.02576013095676899, zero_point=0, padding=(1, 1))
      (1): Identity()
      (2): Identity()
    )
    (1): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(32, 32, kernel_size=(3, 3), stride=(1, 1), scale=0.1074332669377327, zero_point=0, padding=(1, 1), groups=32)
          (1): Identity()
          (2): Identity()
        )
        (1): QuantizedConv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), scale=0.13626477122306824, zero_point=66)
        (2): Identity()
      )
      (skip_add): QFunctional()
    )
    (2): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(16, 96, kernel_size=(1, 1), stride=(1, 1), scale=0.06834518909454346, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(96, 96, kernel_size=(3, 3), stride=(2, 2), scale=0.06564690172672272, zero_point=0, padding=(1, 1), groups=96)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(96, 24, kernel_size=(1, 1), stride=(1, 1), scale=0.07728171348571777, zero_point=61)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (3): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(24, 144, kernel_size=(1, 1), stride=(1, 1), scale=0.01681913249194622, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(144, 144, kernel_size=(3, 3), stride=(1, 1), scale=0.026830462738871574, zero_point=0, padding=(1, 1), groups=144)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(144, 24, kernel_size=(1, 1), stride=(1, 1), scale=0.09138959646224976, zero_point=58)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (4): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(24, 144, kernel_size=(1, 1), stride=(1, 1), scale=0.021155843511223793, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(144, 144, kernel_size=(3, 3), stride=(2, 2), scale=0.024789324030280113, zero_point=0, padding=(1, 1), groups=144)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(144, 32, kernel_size=(1, 1), stride=(1, 1), scale=0.05252280458807945, zero_point=69)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (5): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(32, 192, kernel_size=(1, 1), stride=(1, 1), scale=0.011180322617292404, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(192, 192, kernel_size=(3, 3), stride=(1, 1), scale=0.014178555458784103, zero_point=0, padding=(1, 1), groups=192)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(192, 32, kernel_size=(1, 1), stride=(1, 1), scale=0.04321109130978584, zero_point=65)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (6): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(32, 192, kernel_size=(1, 1), stride=(1, 1), scale=0.012100644409656525, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(192, 192, kernel_size=(3, 3), stride=(1, 1), scale=0.018376799300312996, zero_point=0, padding=(1, 1), groups=192)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(192, 32, kernel_size=(1, 1), stride=(1, 1), scale=0.05106690526008606, zero_point=59)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (7): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(32, 192, kernel_size=(1, 1), stride=(1, 1), scale=0.01497643068432808, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(192, 192, kernel_size=(3, 3), stride=(2, 2), scale=0.021964795887470245, zero_point=0, padding=(1, 1), groups=192)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(192, 64, kernel_size=(1, 1), stride=(1, 1), scale=0.04775148257613182, zero_point=72)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (8): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(64, 384, kernel_size=(1, 1), stride=(1, 1), scale=0.008965414948761463, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(384, 384, kernel_size=(3, 3), stride=(1, 1), scale=0.012472052127122879, zero_point=0, padding=(1, 1), groups=384)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), scale=0.03361879661679268, zero_point=71)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (9): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(64, 384, kernel_size=(1, 1), stride=(1, 1), scale=0.006726221181452274, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(384, 384, kernel_size=(3, 3), stride=(1, 1), scale=0.011396719142794609, zero_point=0, padding=(1, 1), groups=384)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), scale=0.02584526129066944, zero_point=67)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (10): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(64, 384, kernel_size=(1, 1), stride=(1, 1), scale=0.007458422798663378, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(384, 384, kernel_size=(3, 3), stride=(1, 1), scale=0.020480144768953323, zero_point=0, padding=(1, 1), groups=384)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), scale=0.036506373435258865, zero_point=78)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (11): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(64, 384, kernel_size=(1, 1), stride=(1, 1), scale=0.00952215027064085, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(384, 384, kernel_size=(3, 3), stride=(1, 1), scale=0.019453968852758408, zero_point=0, padding=(1, 1), groups=384)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(384, 96, kernel_size=(1, 1), stride=(1, 1), scale=0.042090073227882385, zero_point=58)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (12): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(96, 576, kernel_size=(1, 1), stride=(1, 1), scale=0.010614714585244656, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(576, 576, kernel_size=(3, 3), stride=(1, 1), scale=0.0174615029245615, zero_point=0, padding=(1, 1), groups=576)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), scale=0.027665885165333748, zero_point=66)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (13): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(96, 576, kernel_size=(1, 1), stride=(1, 1), scale=0.013487894088029861, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(576, 576, kernel_size=(3, 3), stride=(1, 1), scale=0.024134499952197075, zero_point=0, padding=(1, 1), groups=576)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(576, 96, kernel_size=(1, 1), stride=(1, 1), scale=0.036522701382637024, zero_point=60)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (14): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(96, 576, kernel_size=(1, 1), stride=(1, 1), scale=0.012434634380042553, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(576, 576, kernel_size=(3, 3), stride=(2, 2), scale=0.036095958203077316, zero_point=0, padding=(1, 1), groups=576)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(576, 160, kernel_size=(1, 1), stride=(1, 1), scale=0.027086306363344193, zero_point=61)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (15): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(160, 960, kernel_size=(1, 1), stride=(1, 1), scale=0.012404436245560646, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(960, 960, kernel_size=(3, 3), stride=(1, 1), scale=0.022417496889829636, zero_point=0, padding=(1, 1), groups=960)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(960, 160, kernel_size=(1, 1), stride=(1, 1), scale=0.02385452575981617, zero_point=61)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (16): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(160, 960, kernel_size=(1, 1), stride=(1, 1), scale=0.008413408882915974, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(960, 960, kernel_size=(3, 3), stride=(1, 1), scale=0.019144069403409958, zero_point=0, padding=(1, 1), groups=960)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(960, 160, kernel_size=(1, 1), stride=(1, 1), scale=0.03730720281600952, zero_point=67)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (17): InvertedResidual(
      (conv): Sequential(
        (0): ConvBNReLU(
          (0): QuantizedConvReLU2d(160, 960, kernel_size=(1, 1), stride=(1, 1), scale=0.007915135473012924, zero_point=0)
          (1): Identity()
          (2): Identity()
        )
        (1): ConvBNReLU(
          (0): QuantizedConvReLU2d(960, 960, kernel_size=(3, 3), stride=(1, 1), scale=0.011366610415279865, zero_point=0, padding=(1, 1), groups=960)
          (1): Identity()
          (2): Identity()
        )
        (2): QuantizedConv2d(960, 320, kernel_size=(1, 1), stride=(1, 1), scale=0.026501065120100975, zero_point=63)
        (3): Identity()
      )
      (skip_add): QFunctional()
    )
    (18): ConvBNReLU(
      (0): QuantizedConvReLU2d(320, 1280, kernel_size=(1, 1), stride=(1, 1), scale=0.09207352995872498, zero_point=0)
      (1): Identity()
      (2): Identity()
    )
  )
  (classifier): Sequential(
    (0): Dropout(p=0.2, inplace=False)
    (1): QuantizedLinear(in_features=1280, out_features=1000, scale=0.33595794439315796, zero_point=41)
  )
  (quant): Quantize(scale=tensor([0.0375]), zero_point=tensor([57]), dtype=torch.quint8)
  (dequant): DeQuantize()
)
ScriptModule(
  original_name=MobileNetV2
  (quant): ScriptModule(original_name=Quantize)
  (features): ScriptModule(
    original_name=_ConstSequential
    (0): ScriptModule(
      original_name=_ConstSequential
      (0): ScriptModule(original_name=ConvReLU2d)
      (1): ScriptModule(original_name=Identity)
      (2): ScriptModule(original_name=Identity)
    )
    (1): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(original_name=Conv2d)
        (2): ScriptModule(original_name=Identity)
      )
    )
    (2): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (3): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (4): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (5): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (6): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (7): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (8): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (9): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (10): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (11): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (12): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (13): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (14): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (15): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (16): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (17): ScriptModule(
      original_name=InvertedResidual
      (skip_add): ScriptModule(original_name=QFunctional)
      (conv): ScriptModule(
        original_name=_ConstSequential
        (0): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (1): ScriptModule(
          original_name=_ConstSequential
          (0): ScriptModule(original_name=ConvReLU2d)
          (1): ScriptModule(original_name=Identity)
          (2): ScriptModule(original_name=Identity)
        )
        (2): ScriptModule(original_name=Conv2d)
        (3): ScriptModule(original_name=Identity)
      )
    )
    (18): ScriptModule(
      original_name=_ConstSequential
      (0): ScriptModule(original_name=ConvReLU2d)
      (1): ScriptModule(original_name=Identity)
      (2): ScriptModule(original_name=Identity)
    )
  )
  (classifier): ScriptModule(
    original_name=_ConstSequential
    (0): ScriptModule(original_name=Dropout)
    (1): ScriptModule(original_name=Linear)
  )
  (dequant): ScriptModule(original_name=DeQuantize)
)

It's obvious that the latter has not quantization information, such as zero point, scale, etc.

Thanks.

@jamesr66a
Copy link
Collaborator

jamesr66a commented Oct 23, 2019 •
edited
Loading

It’s more likely that the special __repr__ functions are not saved, resulting in a different print out. Please check if scale and zero_point exist as attributes on the modules rather than relying on print

@jamesr66a jamesr66a mentioned this issue Oct 23, 2019
Open
@jamesr66a
Copy link
Collaborator

Closed in favor of #28517

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
oncall: quantization Quantization support in PyTorch triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jamesr66a @vippeterhou @lly-zero-one