Hi, I can't import tinynet

[RichardMrLu]

Could you updata the tinynet's file and the URL of tinynet.pth.tar? Thanks for your excellent work.

[jack-willturner]

Hi there,

Sorry, that whole quantization folder needs to be redone since it's mostly just pseudocode! If you don't need the quantization step then take a look at the stuff in the sparsity folder - the iPython notebook is what I have put most of my time into.

If you want to use the quantization code as a skeleton for your own implementation then you can just put any of these files in the quantization folder and rename it to tinynet.py (or import it by its actual name).

I'll get round to fixing all the quantization code soon I hope.

Thanks,
Jack

[RichardMrLu]

Thanks, I have run your sparsity code, but after prune, the net's params didn't decreased...maybe after I change the code to .py file caused some error.
Could you help me? I find some quantization code on Github, but after quantized, the model's memory are not changed, while it should be smaller? How to save the model after quantized?

[jack-willturner]

Any chance you can put your version of this in a public repository? That way I can look through your code and try to help.

Usually in pytorch to save a model you would use something along the lines of:

        state = {
            'model': model.state_dict(),
            'acc': acc,
        }
        filename = './ckpt_' + model_name + '.t7'
        if not os.path.isdir('checkpoint'):
            os.mkdir('checkpoint')
        torch.save(state, filename)

[RichardMrLu]

https://github.com/aaron-xichen/pytorch-playground

python quantize.py --type alexnet --quant_method minmax --param_bits 8 --fwd_bits 8 --bn_bits 8 --ngpu 1

type=alexnet, quant_method=minmax, param_bits=8, bn_bits=8, fwd_bits=8, overflow_rate=0.0, acc1=0.5514, acc5=0.7819

print('save model...')
torch.save(model_raw.state_dict(), 'model_quantized.pth')
I am puzzled, after quantized, the model's memory are still 234M, how to save the quantied model？
Thanks for your help.

"This is expected. Because it is a simulation tool. The quantized parameters are represented in float format, not integers. You need to do some post precess work yourself before deployment."

[RichardMrLu]

I have run another quantizition method:
https://github.com/TropComplique/trained-ternary-quantization

his model are also unchanged...
"
Simple example how to save quantized values as int8:

import numpy as np

full precision weights

x = np.random.randn(5000, 5000).astype('float32')
np.save('x.npy', x) # 96 MB

quantization

threshold = 0.7
x[x > threshold] = 1.0
x[np.logical_and(x <= threshold, x >= -threshold)] = 0.0
x[x < -threshold] = -1.0
x = x.astype('int8')
np.save('quantized_x.npy', x) # 24 MB
In case of CNN weights, you have to find an appropriate
data structure to store 2-bit values.
Maybe try numpy.packbits.
"