Reduce Atlas memory footprint

[zabeen]

Even when Atlas is idle, the max/avg memory usage hovers around 30-40%. This is likely due to the number of items we have in app cache, such as HMD, scoring, and H.F. sets. Search, particularly match prediction, then adds to this footprint when calculating match probabilities.

Issues that could reduce this footprint:

• Matching Optimisations #1278
• Memory profile Atlas search locally #1281
• Use shared distributed cache across all apps #730
• Caching match prediction calculations #1131
• Investigate memory usage of data refresh HLA processing step #1288
  • This is not as urgent a task as the job only runs once every 3 months and typically we try to run over the weekend.
  • However, fixing it would mean data refresh job could be run on live as early as possible, even during business days, which reduces risk of donor updates/search requests being rejected due to HMD errors.