Replace std::map with std::unordered_map.

[denravonska]

This pulls the std::map -> std::unordered_map change from Bitcoin. The benefit is that unordered map is almost always faster in all aspects, plus it makes the code a bit more readable.

CPU usage

The CPU usage is drastically reduced. I did some profiling when syncing to see how much time was spent comparing uint256 when locating blocks in the maps while syncing on a Raspberry Pi:

  %   cumulative   self              self     total           
 time   seconds   seconds    calls  Ks/call  Ks/call
 46.28    749.82   749.82 702389359     0.00     0.00  map
 13.43    332.04   332.04 2917441538    0.00     0.00  unordered_map

As we can see, even though the std::map did fewer iterations due to shorter sync time and thus having its results skewed by scrypt operations, it still consumed far more CPU cycles than the unordered_map version.

Memory usage

There is a slight memory usage increase with unordered_map:

                    VIRT   RES   SHR
std::map            2076M  974M  398M
std::unordered_map: 1978M  990M  421M

I think this can be optimized away by improving CBlockIndex.

Remaining

• Sync from 0 (ongoing)
• Ensure that there are no collisions due to only using 64bit keys
• Check increase memory consumption
• Run CPU usage tests once more. unordered_map might have used asm optimized scrypt

[tomasbrod]

That 64 bit hash requires a test or research. Is it just a hash for hash table implementation and then the hash table handles collisions by comparing full key or is the 64 bit key directly used as key to the map?

[denravonska]

Seems to be the former:

// Key
Type of the key values. Each element in an unordered_map is uniquely identified by its key value.
Aliased as member type unordered_map::key_type.

// Hash
The unordered_map object uses the hash values returned by this function to organize its elements internally, speeding up the process of locating individual elements.

http://www.cplusplus.com/reference/unordered_map/unordered_map/

I will verify this just to be sure.

[iFoggz]

look forward to testing this one. good work.

[denravonska]

Test program with same hash for different keys:

#include <iostream>
#include <unordered_map>
#include <string>

struct hasher {
   size_t operator()(const std::string& key) const { return 0; }
};

int main(int argc, char** argv) {
   std::unordered_map<std::string, int, hasher> map;
   map["first"]  = 1;
   map["second"] = 2;
   std::cout << "First="    << map["first"] <<
                ", second=" << map["second"] <<
                ", size=" << map.size() << std::endl;
                
   return 0;
}

$ ./apa 
First=1, second=2, size=2

[denravonska]

Updated with some benchmarks. If everyone is ok with a little more RAM usage I think we should include this in the version after the current mandatory.