Implement sharing for hls-graph Keys

[wz1000]

On large projects, duplicate Keys start to comprise a significant (~5-10%) fraction of the heap.
This becomes better if we use Ints as keys and de-duplicate the actual values by storing them
in maps. Keys are mostly just compared and hash during usual operation, printing them out is
rare so a lookup shouldn't pessimize it much.

Also use HashSets to keep track of dependencies.
However, this means that the underlying value for Keys will never be garbage collected once allocated.

[pepeiborra]

Approved. Please share some benchmarks showing the memory savings when you have a chance!

[pepeiborra]

Can we drop the Int proxy (Key) and the IntMap?
newKey will lookup in the KeyMap and return the existing value if any, recovering sharing.

What's the benefit of keeping an extra mapping?

[wz1000]

Can we drop the Int proxy (Key) and the IntMap? newKey will lookup in the KeyMap and return the existing value if any, recovering sharing.

What's the benefit of keeping an extra mapping?

I tried this first, but GHC unpacks the Key and we lose sharing at that point.

[pepeiborra]

I tried this first, but GHC unpacks the Key and we lose sharing at that point.

Worth adding a comment about this, I believe my previous attempt to share keys must have failed for this reason

[michaelpj]

I tried this first, but GHC unpacks the Key and we lose sharing at that point.

Maybe it's a bit too sensitive, but can we do something to prevent GHC from unpacking the key? e.g. not import the constructor in the module in question?

[wz1000]

Maybe it's a bit too sensitive, but can we do something to prevent GHC from unpacking the key? e.g. not import the constructor in the module in question?

I don't think this is possible currently. Would be a good feature request though.

[wz1000]

I've also added sharing of the rendered Key as Text to avoid duplicating the rendered Key in the diagnostic store, else we get a duplicate of each key as a rendered Text value for each file in the project.

[pepeiborra]

Can we have those benchmark results?

[wz1000]

Here are some profiles from before and after

before (heap size):

before (keys in heap):

In the second image you can see that just the top allocation site of duplicate keys is responsible for ~200MB of live bytes. These are all eliminated by this patch.

after (heap size):