Train with PACT but the value for cliping weights and activations which denoted as alpha seems not change.

[jianyin2016]

the value for cliping weights and activations which denoted as alpha is initialized to 6.0, In my opinion, this value should be updated during training, but I found it not, I am training with the imagenet_example just adding such following configs to make PACT working.

if args.quant:
        extra_params = {
            'extra_qconfig_dict': {
                'w_observer': "MinMaxObserver",
                'a_observer': "EMAMinMaxObserver",
                'w_fakequantize': "PACTFakeQuantize",
                'a_fakequantize': "PACTFakeQuantize",
                'a_fakeq_params': {},
                'w_qscheme': {
                    'bit': 8,
                    'symmetry': True,
                    'per_channel': False,
                    'pot_scale': False
                },
                'a_qscheme': {
                    'bit': 8,
                    'symmetry': True,
                    'per_channel': False,
                    'pot_scale': False
                }
            },
            'extra_quantizer_dict': {},
            'preserve_attr': {},
            'concrete_args': {},
            'extra_fuse_dict': {}
        }
        print("==> config with extra params", extra_params)
        model = prepare_by_platform(model, args.backend, extra_params)

[zhiwei-dong]

Can you give us a minimal reproducible code tar file?

[jianyin2016]

Can you give us a minimal reproducible code tar file?

I am afraid not due to the restriction of information security policy of our company.
but the minimal reproduction is not hard to reproduce following below steps.

• (necessary) in application/imagenet_example/main.py ,around line 167, before calling of prepare_by_platform, add the PACT configs which has been presented.
• (optinal) for more obvious watch of alpha, you could print the alpha in PACTFakeQuantize.forward.
• (optinal and maybe wrong) in mqbench/deploy/deploy_linear.py change line 99 from if scale.shape[0] > 1: to if len(scale.shape) != 0 and scale.shape[0] > 1:
• (necessary) python setup.py install

[zhiwei-dong]

OK，I get.

[Tracin]

@jianyin2016 Problem fixed? Anything wrong in code or not?

[jianyin2016]

I think there is something wrong with the code for PACT, but I have to validate the correction first.

[jianyin2016]

Closed for finally found the reason why the performance was not as expected is that I wrongly use of some configs.

[feizi]

I got the same error.
@jianyin2016 , How did you fix it.