Use process dictionary for hash like aggregation

[pichi]

This version uses process dictionary for hash like K/V storage. It is traditionally fastest K/V storage available in Erlang. There are also maps available in new R17 Erlang/OTP version, which is cleaner but a little bit slower alternative. Process dictionary works even in older releases. Using process dictionary is perceived as dirty unless you use it in your own process so it is used in this patch.

This version is about 50 times faster on mine computer and natively compiled version subtract another 20% from that result.

[andreaferretti]

So, is this process dictionary the erlang equivalent of a hasmap? I am not sure which one of the two erlang pull requests I got is actually comparable to the implementations in other languages.

I understand that this one works more or less the same as the others, so if you confirm this I will merge it

[pichi]

Yes, process dictionary is hash map implementation. It is mutable and uses hash internally which makes it closest to hash map implementation in common imperative languages. Only drawback is there is only one per process and is mutable which makes it bad practice in Erlang because it is not clear how data are transferred between two lines of groupBy/2 function. I don't know if you should merge it because it is not Erlang best practice use this trick. BTW sofs is not good either in mine eyes, because it requires higher knowledge of the set theory.

[andreaferretti]

Er... I was about to merge but you closed your PR. So, if process dictionary is not idiomatic and sofs is not good, what would be a natural way to write a groupBy in Erlang?

[andreaTP]

👎
pretty sure that process dictionary are not intended to be used as computational hash map into functions.
Not reliable, and "tricky" implementation.

[pichi]

I agree. Process dictionary is a good way how to write really fast code, but it is not good practice. But I found why the original code was so slow. The issue is dict:append/3 which appends at the end of the list. I had run into this before and I would like dict:prepend/3 in stdlib instead. You have to use dict:update/4 instead and id works nice even not fast as process dictionary.