A bug in bitsback coding

[bfs18]

sandbox/bitsback/bitsback.py

Line 29 in 3dac794

samples = norm.ppf(ppf_range, loc=mu, scale=np.exp(log_var))

scale should be standard deviation, not variance. changing scale to scale=np.exp(0.5. * log_var) improves the result a lot.

[bjlkeng]

Nice find, thank you!

I'll double check once I get a few minutes and fix the notebook and blog post up.

[bfs18]

@bjlkeng Thank you for the detailed introduction and concise code. It helped me understand bitsback encoding very well

[bjlkeng]

Thanks for finding the bug @bfs18 ! I fixed the bug you mentioned and then realized I had some more issues with the encoding/decoding of the latent variables. So I fixed those too, and now I can achieve 1.53 bits/pixel! Much better (although not as good as the original paper). See: https://github.com/bjlkeng/sandbox/tree/master/bitsback

I also updated the blog post too for those who come across it in the future: https://bjlkeng.io/posts/lossless-compression-with-latent-variable-models-using-bits-back-coding/

Thanks again for finding that bug!