[hail][performance] Fix literals lazy decoding#6605
Conversation
Cloud providers don't want you to know this one weird trick to save thousands on your compute bill
|
This change fixes a huge problem caused by these lines of code and context: https://github.com/hail-is/hail/pull/6605/files#diff-1278c1788239002cc63ccb82cbef8d76L190 The problem is that in our generated code, every literal is decoded each time any literal is referenced. This is extremely expensive! In this change, we instead decode the literals once with the function is constructed from the partition index (used with randomness), by adding a new region argument which the literals are decoded into. This region must live as long as the RegionValues returned by any invocation of the function. The primary error mode I might expect is that we use the wrong region to generate the function, causing use-after-free errors. These are well-covered by tests, since I had a few of these bugs and fixed them due to test failures. The region we shouldn't be using is
|
Cloud providers don't want you to know this one weird trick to save
thousands on your compute bill
This PR:
Master:
Lower bound: 900x faster