Slow and memory-costly compilation

[ilyaraz]

Currently, compiling anything that includes FALCONN takes quite a bit of time and RAM (on towhee running make python_package_install eats more than 2.4 Gb of RAM).

It's a bit painful: for instance, one can't run make python_package_install on the machine, where the website is hosted, which is unfortunate, since it'd be way more convenient to generate docs on it, instead of pulling them from somewhere.

[ludwigschmidt]

I also noticed that compiling the C++ wrapper test takes more time than before. I think this is because the wrapper has more template options now: 4 different table types and 2 different hash type sizes (32 bit and 64 bit integers). So overall there are 8x more template classes to generate.

I think currently the situation is inconvenient but still acceptable. In the future, we can write the wrapper so that we only instantiate the LSHTable template with one hash table wrapper class that does virtual function calls to the actual hash table internally. This will lead to slightly more runtime overhead, but should not be an issue overall. And we don't need to change the core classes, so the full template solution would still be available if necessary.

[Leonidas-from-XIV]

I had the problem as well, where compiling FALCONN took forever on Docker on Mac and then failed with what I assume was the OOM killing GCC. I ended up creating a wheel on a different machine and forcing pip to use that wheel instead of the tarball.

I think it would be nice if FALCONN would submit some pre-built wheels to PyPI, so I wouldn't need to depend on my binary.