made various changes to cleanup retrieval

[epinzur]

made a bunch of changes that generally make sure our ColBERT retrieval code interacts nicely with Cassandra (doesn't overload the database when making queries).

There is little change in retrieval latency with these changes running on CPU.

• wrapped async calls to cassandra inside a ThreadPoolExecutor.

  • We should experiment to see what value of max_casandra_workers works the best.

• automatically dropping down to half-precision when calculating chunk scores, if running on GPU and the GPU supports it.

  • We should test to see if is_fp16 flag is getting set to True on boxes with CUDA support.
  • I "think" that half-precision won't hurt our results, and chatGPT claims this can "significantly speed up our calculations & reduce memory usage on the GPU".

[nicoloboschi]

LGTM in general

I think we should improve the error handling