[FEA] Optimize use of main memory for ANN indexes

[cjnolet]

We need to find a good approach to efficiently performing queries on architectures like G&H. Ideally, we would be able to train and keep very large indices in memory but have smart ways of caching only what we need on the GPU such that we can increase cached hits as much as possible and reduce the number of transfers to/from the GPU.

I'm creating this issue so we can track discussions and progress on this feature.