Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
[hail][performance] Fix literals lazy decoding #6605
Cloud providers don't want you to know this one weird trick to save
Lower bound: 900x faster
This change fixes a huge problem caused by these lines of code and context:
The problem is that in our generated code, every literal is decoded each time any literal is referenced. This is extremely expensive!
In this change, we instead decode the literals once with the function is constructed from the partition index (used with randomness), by adding a new region argument which the literals are decoded into. This region must live as long as the RegionValues returned by any invocation of the function.
The primary error mode I might expect is that we use the wrong region to generate the function, causing use-after-free errors. These are well-covered by tests, since I had a few of these bugs and fixed them due to test failures. The region we shouldn't be using is