Join GitHub today
High memory usage compiling `keccak` benchmark #54208
The spike is due to a single allocation of 500,363,244 bytes here:
Each vector element is a
I have one idea to improve this:
Alternatively, if we packed the
@nikomatsakis: do you have any ideas for improving this on the algorithmic side? Is this dense
Now for NLL. According to perf.rust-lang.org, an "Nll" build of
The three allocation sites are here:
In each case
I tried changing
One trivial idea: it looks like
@nikomatsakis: any other thoughts here from the algorithmic side?
I have implemented this in #54211.
I have implemented this in #54213.
@nikomatsakis: I have run out of ideas on this one.
If it helps, here is what the
In other words, it is 25994 x 94976 bits (308.6MB), and the rows start off almost entirely set, and by the end drop down to about half set. About 75% of the bits are set.
And here's what
It is 25994 x 83328 bits (270.8MB). Apart from the second row, the rows start of almost empty and get fuller until they are 77% full by the end. About 38% of the bits are set.
I didn't look at
I can't see how to represent this data more compactly, and I don't understand the algorithm in enough detail to know if less data could be stored. I also looked into separating the lifetimes of the two structures but they are used in tandem, as far as I can tell.
Discussed with @nikomatsakis during triage of NLL issues.
We decided that the memory usage on this case should not block NLL's inclusion in RC2.
In terms of whether to put this on the Release milestone or not, we decided that it would be a better idea, at least in the short-to-middle term, to focus effort more on Polonius, since that component might end up replacing the dataflow entirely, and thus the pay-off from optimizing
So, tagging as NLL-deferred, with the intention of revisiting after we've learned more about what we plan to do with Polonius, if anything.