new hash table #24

Open
rurban opened this Issue Sep 11, 2015 · 4 comments

Projects

None yet

1 participant

@rurban
Member
rurban commented Sep 11, 2015 edited

The old hash table uses:

  • linked lists for its collisions, with slow out-of-cache pointer chasing and data overhead.
  • unsorted flags at the end, while some flags are needed for compare.
  • has questionable security measures to slow down all cases. seed ok, randomize iter maybe, but randomize the collisions and slow hash funcs is stupid. the security should be fixed with proper implementations, not by pseudo-security theatre.

either use:

  • rurban/hash-sortbuckets: a sorted static list of collisions (up to 8, maybe start with 3, then 8) as in knuth sorted hash table,
  • khash: use open addressing as everyone else. faster, less space.
  • PERTURB_KEYS_TOP move-to-front with a linked list is the only sane strategy for simple chained bucket lists.
  • HV_FILL_RATE: try lower fill rates than 100%. 100% is pretty insane, esp. with our bad hash funcs. make it fast with builtin_ctz.
  • ✓ use __builtin_ctz for faster division in DO_HSPLIT. Allow -DHV_FILL_RATE=90 definition. (Tested to be the best with FNV1A)
  • HE_ARRAY: According to http://goanna.cs.rmit.edu.au/~jz/fulltext/spire05.pdf
    the best for chained hashing is currently a cache-friendly array:
    cache-friendly continuous buffer of HE's w/ inlined HEK (char_) + SV_ val, but no hash, no next ptr. Also for shared he's: PL_strtab.
  • array_he: inline parts of the HE into the array. array_he vs ll_he. (linked list, see also the he-array branch). array_he (HvARRAY = AHE[]) can contain { hent_he, hent_hash }. this way the hash catches 99% of all comparisons already, and we don't have to chase the external hek ptr, when the hash check fails. every HE entry will then be 2 words (128), instead of one word (64), called AHE. The linked list still contains the old HE*, with { hent_next, hent_hek, hent_val }.
  • one-word-AHE: As possible improvement on that on 64bit use 48bits for the HE ptr, and 16bits of the hash to be compared first. See https://www.semanticscholar.org/paper/Fast-Dynamically-Sized-Concurrent-Hash-Table-Barnat-Rockai/ab7dd007587f411cf99bfe056639e055eff22e0c/pdf
  • use robin-hood as this is currently the best worse-case strategy (being super defensive, but not so stupid to use SipHash, which adds no benefit). with better threading support (shared hashes) eventually use leapfrog.
  • compact ordered hash. use an array of short indices into a compacted array of hash/key/val entries as in PyPy and now python: "compact ordered dict". This saves a lot of space and only add's one indirect lookup into cache-friendly array. See methane/cpython#1 https://mail.python.org/pipermail/python-dev/2016-September/146327.html
    This also iterates over the hash in insertion order, which effectively hides any attempt to get the seed over the iterators. For attacks you need to get collision and robin-hood reordering timings.

maybe use the glibc htable (hsearch, hcreate), just add random seeds and flood detection (collision counts). (or khash, which is basically the same). coucal looks good also. https://github.com/xroche/coucal
but best is probably preshing's http://preshing.com/20160314/leapfrog-probing/, which even can be made concurrent easily.

open addressing is cache friendly and much faster, but table growth is slow.
potion uses khash, first double open, then quadratic open.

either way, first I need to abstract away the collision search in MACROS
and reduce the needed comparisons from 4 to 1. (edit: after my critic, they improved that part to one ptr cmp)
maybe we can normalize all keys to utf8 for faster checks. (probably not)
utf8 is a HEK flag. See http://cpansearch.perl.org/src/RURBAN/illguts-0.49/index.html#hek
The current implementation is just insanity and it only got worse over the years.

The last attempt was here: https://github.com/rurban/perl/commits/rurban/hash-sortbuckets
See the featurex/gh24-*hash* branches.
See also GH #102


plan

  • ✓ -Dhash_func, add and test many more funcs
  • ✓ PERTURB_KEYS_TOP
  • ✓ HV_FILL_RATE
  • ✓ d_builtin_ctz
  • ✓ seperate hv_common_magic
  • ✓ security: warn and protect against hash flood
  • ✓ fix NODEFAULT_SHAREKEYS (#201)
  • ✓ hash_loop: abstract loops and array_he
  • ✓ array_he: seperate HvARRAY from collisions entries, add hent_hash
  • 80% new_hash_table: merge HE fields for faster loop, usehekbitfield, d_little_endian.
  • one-word-AHE (hash+he on 64bit)
  • 40% open_hash, compact_ordered: various strategies with the new abstractions
@rurban rurban added the enhancement label Sep 11, 2015
@rurban rurban self-assigned this Sep 11, 2015
@rurban rurban added this to the v5.24.0 milestone Sep 13, 2015
@rurban rurban added a commit that referenced this issue Mar 1, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash (WIP)
calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24
5a3d225
@rurban rurban added a commit that referenced this issue Mar 1, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash (WIP)
calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
2a513be
@rurban rurban added a commit that referenced this issue Mar 1, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
6df5f0e
@rurban rurban added a commit that referenced this issue Mar 1, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
1a73b31
@rurban rurban added a commit that referenced this issue Mar 2, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
2fdc180
@rurban rurban added a commit that referenced this issue Mar 8, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
456d64f
@rurban rurban added a commit that referenced this issue Mar 12, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
15c4251
@rurban rurban added a commit that referenced this issue Mar 19, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
890cc1e
@rurban rurban added a commit that referenced this issue Mar 21, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
31c8d10
@rurban rurban added a commit that referenced this issue Mar 23, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
bd65fb9
@rurban rurban added a commit that referenced this issue Mar 23, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
0f82e9a
@rurban rurban added a commit that referenced this issue Mar 24, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
244095c
@rurban
Member
rurban commented Mar 24, 2016

Merged now very simple, traditional improvements:

  • default HV_FILL_RATE 90%,
  • PERTURB_KEYS_TOP move-to-front

no hashtable rewrite yet.

@rurban rurban added a commit that referenced this issue Mar 24, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
e5cd682
@rurban rurban added a commit that referenced this issue Mar 26, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
e1c695c
@rurban rurban added a commit that referenced this issue Mar 26, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
4438133
@rurban rurban added a commit that referenced this issue Mar 27, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
9e30efc
@rurban rurban added a commit that referenced this issue Mar 28, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
0f8dd28
@rurban rurban added a commit that referenced this issue Mar 28, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
5e12ee2
@rurban rurban added a commit that referenced this issue Mar 30, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
ad7f351
@rurban rurban added a commit that referenced this issue Apr 3, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
470e1a2
@rurban rurban added a commit that referenced this issue Apr 5, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
bc2eed8
@rurban rurban added a commit that referenced this issue Apr 6, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
1f5f5c3
@rurban rurban added a commit that referenced this issue Apr 8, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
45bdf48
@rurban rurban added a commit that referenced this issue Apr 8, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
b7f8111
@rurban rurban added a commit that referenced this issue Apr 9, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
2dcc4cd
@rurban rurban added a commit that referenced this issue Apr 12, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
36dea52
@rurban rurban added a commit that referenced this issue Apr 13, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
1a00907
@rurban rurban added a commit that referenced this issue Apr 13, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
9b783ec
@rurban rurban added a commit that referenced this issue Apr 17, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
7b99185
@rurban rurban added a commit that referenced this issue Apr 17, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
76a289f
@rurban rurban added a commit that referenced this issue Apr 17, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
409c677
@rurban rurban added a commit that referenced this issue Apr 18, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
2925776
@rurban rurban added a commit that referenced this issue Apr 19, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
60875fd
@rurban rurban added a commit that referenced this issue Apr 21, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
58477bf
@rurban rurban added a commit that referenced this issue Apr 25, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
fba6572
@rurban rurban added a commit that referenced this issue May 2, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
afad2ce
@rurban rurban added a commit that referenced this issue May 3, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
fc804b0
@rurban rurban added a commit that referenced this issue May 5, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
f7d8611
@rurban rurban added a commit that referenced this issue May 9, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
4c1e52a
@rurban rurban modified the milestone: v5.26.0, v5.24.0 May 10, 2016
@rurban rurban added a commit that referenced this issue May 22, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
958f72a
@rurban rurban added a commit that referenced this issue May 28, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
3ba3919
@rurban rurban added a commit that referenced this issue May 30, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
ed096bf
@rurban rurban added a commit that referenced this issue May 30, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
3fa24ba
@rurban rurban added a commit that referenced this issue Jun 4, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
a6cd53c
@rurban rurban added a commit that referenced this issue Jun 6, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
c112b00
@rurban rurban added a commit that referenced this issue Jun 17, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
df401b6
@rurban rurban added a commit that referenced this issue Jun 18, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
9367f4e
@rurban rurban added a commit that referenced this issue Jun 20, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
4fdb824
@rurban rurban added a commit that referenced this issue Jun 20, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
e86b607
@rurban rurban added a commit that referenced this issue Jun 20, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
2f8ee26
@rurban rurban added a commit that referenced this issue Jun 23, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
51acd4f
@rurban rurban added a commit that referenced this issue Jun 23, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
a60b94c
@rurban rurban added a commit that referenced this issue Jun 24, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
71fb914
@rurban rurban added a commit that referenced this issue Jun 24, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
a7636b3
@rurban rurban added a commit that referenced this issue Jun 25, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
a0b5dcb
@rurban rurban added a commit that referenced this issue Jun 27, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
b2ab9ff
@rurban rurban added a commit that referenced this issue Jul 4, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
c6af1b7
@rurban rurban added a commit that referenced this issue Jul 4, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
dbb8d77
@rurban rurban added a commit that referenced this issue Jul 6, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
33422fd
@rurban rurban added a commit that referenced this issue Jul 11, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
39b22eb
@rurban rurban added a commit that referenced this issue Jul 11, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
66ca01b
@rurban rurban added a commit that referenced this issue Jul 11, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
3580222
@rurban rurban added a commit that referenced this issue Jul 13, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
509f264
@rurban rurban added a commit that referenced this issue Jul 14, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
5ed84bc
@rurban rurban added a commit that referenced this issue Jul 17, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
ba19908
@rurban rurban added a commit that referenced this issue Jul 19, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
ac25d18
@rurban rurban added a commit that referenced this issue Jul 25, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
6133f0f
@rurban rurban added a commit that referenced this issue Jul 26, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
6d3569e
@rurban rurban added a commit that referenced this issue Jul 26, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
2b0e425
@rurban rurban added a commit that referenced this issue Jul 27, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
d38b8ea
@rurban rurban added a commit that referenced this issue Jul 27, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
36530aa
@rurban rurban added a commit that referenced this issue Jul 27, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
ccb6eab
@rurban rurban added a commit that referenced this issue Jul 28, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
8855570
@rurban rurban added a commit that referenced this issue Jul 28, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
5cb559d
@rurban rurban added a commit that referenced this issue Jul 28, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
ace022b
@rurban rurban added a commit that referenced this issue Jul 28, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
0d36158
@rurban rurban added a commit that referenced this issue Jul 31, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
fae500b
@rurban rurban added a commit that referenced this issue Jul 31, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
30c1a2d
@rurban rurban added a commit that referenced this issue Aug 2, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
c828cd3
@rurban rurban added a commit that referenced this issue Aug 2, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
568b564
@rurban rurban added a commit that referenced this issue Aug 7, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
8790770
@rurban rurban added a commit that referenced this issue Aug 7, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
005e320
@rurban rurban added a commit that referenced this issue Aug 8, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
e2bffbc
@rurban rurban added a commit that referenced this issue Aug 8, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
e126510
@rurban rurban added a commit that referenced this issue Aug 8, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
6a430d3
@rurban rurban added a commit that referenced this issue Aug 8, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
4675136
@rurban rurban added a commit that referenced this issue Aug 10, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
da41cd6
@rurban rurban added a commit that referenced this issue Aug 10, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
9f6be14
@rurban rurban added a commit that referenced this issue Aug 11, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
688d7c6
@rurban rurban added a commit that referenced this issue Aug 11, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
343df74
@rurban rurban added a commit that referenced this issue Aug 14, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
a4051a1
@rurban rurban added a commit that referenced this issue Aug 14, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
2813837
@rurban rurban added a commit that referenced this issue Aug 14, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
d521046
@rurban rurban added a commit that referenced this issue Aug 14, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1% slower,
but we hope to get to speed with the HeARRAY patch. See below.

But the endgoal is to get rid of linked lists and store the collisions
in consecutive memory, in a HeARRAY. Measurements in
"Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
713eec2
@rurban rurban added a commit that referenced this issue Nov 7, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
e2f54c8
@rurban rurban added a commit that referenced this issue Nov 11, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
d827480
@rurban rurban added a commit that referenced this issue Nov 11, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
69c6c3a
@rurban rurban added a commit that referenced this issue Nov 15, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
4c5a4f6
@rurban rurban added a commit that referenced this issue Nov 15, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
2e30767
@rurban rurban added a commit that referenced this issue Nov 15, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
3d87878
@rurban rurban added a commit that referenced this issue Nov 15, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
918c110
@rurban rurban added a commit that referenced this issue Nov 18, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
213291e
@rurban rurban added a commit that referenced this issue Nov 18, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
08ff088
@rurban rurban added a commit that referenced this issue Nov 20, 2016
@rurban rurban AHe: seperate HvARRAY[] (AHE), PERL_INLINE_HASH
struct array_he and struct he can now be seperate, to allow open
addressing or inline hent_hash into the array, for faster hash checks.

Prepare #define PERL_INLINE_HASH for hash inlined into AHE.
Avoids HEK pointer chasing, when the hash lookup failed.
Much earlier exit before the loop.

Re-arrange hsplit a bit.

Add seperate mro.h

See GH #24
a85e70c
@rurban rurban added a commit that referenced this issue Nov 20, 2016
@rurban rurban AHe: seperate HvARRAY[] (AHE), PERL_INLINE_HASH
struct array_he and struct he can now be seperate, to allow open
addressing or inline hent_hash into the array, for faster hash checks.

Prepare #define PERL_INLINE_HASH for hash inlined into AHE.
Avoids HEK pointer chasing, when the hash lookup failed.
Much earlier exit before the loop.

Re-arrange hsplit a bit.

Add seperate mro.h

See GH #24
5a8bc2d
@rurban rurban added a commit that referenced this issue Nov 20, 2016
@rurban rurban AHe: seperate HvARRAY[] (AHE), PERL_INLINE_HASH
struct array_he and struct he can now be seperate, to allow open
addressing or inline hent_hash into the array, for faster hash checks.

Prepare #define PERL_INLINE_HASH for hash inlined into AHE.
Avoids HEK pointer chasing, when the hash lookup failed.
Much earlier exit before the loop.

Re-arrange hsplit a bit.

Add seperate mro.h

See GH #24
48fdbfa
@rurban rurban added a commit that referenced this issue Nov 20, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
97ac59b
@rurban rurban added a commit that referenced this issue Nov 20, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
defcbf5
@rurban rurban added a commit that referenced this issue Nov 20, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
b35ece9
@rurban rurban added a commit that referenced this issue Nov 20, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
9daf4d5
@rurban rurban added a commit that referenced this issue Nov 22, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
b8a3020
@rurban rurban added a commit that referenced this issue Nov 22, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
1636c2f
@rurban rurban added a commit that referenced this issue Nov 22, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
d2e70f9
@rurban rurban added a commit that referenced this issue Nov 22, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
af9a1fc
@rurban rurban added a commit that referenced this issue Nov 23, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
c60edc7
@rurban rurban added a commit that referenced this issue Nov 23, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
a12098a
@rurban rurban added a commit that referenced this issue Nov 24, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
8348465
@rurban rurban added a commit that referenced this issue Nov 27, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
a44fa29
@rurban rurban added a commit that referenced this issue Nov 27, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
0e03d47
@rurban rurban added a commit that referenced this issue Nov 27, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
f94fbe9
@rurban rurban added a commit that referenced this issue Nov 27, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
abd2027
@rurban rurban added a commit that referenced this issue Nov 27, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
c8b00c4
@rurban rurban added a commit that referenced this issue Nov 27, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
f49687b
@rurban rurban added a commit that referenced this issue Nov 30, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
f661c44
@rurban rurban added a commit that referenced this issue Nov 30, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
31defea
@rurban rurban added the in progress label Dec 1, 2016
@rurban rurban added a commit that referenced this issue Dec 1, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
7218eeb
@rurban rurban added a commit that referenced this issue Dec 1, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
a3b4c57
@rurban rurban added a commit that referenced this issue Dec 4, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
95bced3
@rurban rurban added a commit that referenced this issue Dec 4, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
cd25966
@rurban rurban added a commit that referenced this issue Dec 4, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
45c9ee3
@rurban rurban added a commit that referenced this issue Dec 4, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
419b9d1
@rurban rurban added a commit that referenced this issue Dec 10, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
b92d12c
@rurban rurban added a commit that referenced this issue Dec 10, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
8a81fcd
@rurban rurban added a commit that referenced this issue Dec 11, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
2897e29
@rurban rurban added a commit that referenced this issue Dec 11, 2016
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
0aa3e42
@rurban rurban added a commit that referenced this issue Dec 31, 2016
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
8b3bab3
@rurban rurban added a commit that referenced this issue Jan 1, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
c62d919
@rurban rurban added a commit that referenced this issue Jan 1, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
fb68e4e
@rurban rurban added a commit that referenced this issue Jan 1, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
9d6f3ee
@rurban rurban added a commit that referenced this issue Jan 2, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
ad38224
@rurban rurban added a commit that referenced this issue Jan 2, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
9759238
@rurban rurban added a commit that referenced this issue Jan 2, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
4ded4cf
@rurban rurban added a commit that referenced this issue Jan 2, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
4290876
@rurban rurban added a commit that referenced this issue Jan 4, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
dc7850f
@rurban rurban added a commit that referenced this issue Jan 4, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
ddbbe14
@rurban rurban added a commit that referenced this issue Jan 4, 2017
@rurban rurban AHe: seperate HvARRAY[] (AHE), PERL_INLINE_HASH
struct array_he and struct he can now be seperate, to allow open
addressing or inline hent_hash into the array, for faster hash checks.

Prepare #define PERL_INLINE_HASH for hash inlined into AHE.
Avoids HEK pointer chasing, when the hash lookup failed.
Much earlier exit before the loop.

Re-arrange hsplit a bit.

Add seperate mro.h

See GH #24
2243bb8
@rurban rurban added a commit that referenced this issue Jan 4, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
5e49e63
@rurban rurban added a commit that referenced this issue Jan 4, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
a5111af
@rurban rurban added a commit that referenced this issue Jan 4, 2017
@rurban rurban AHe: seperate HvARRAY[] (AHE), PERL_INLINE_HASH
struct array_he and struct he can now be seperate, to allow open
addressing or inline hent_hash into the array, for faster hash checks.

Prepare #define PERL_INLINE_HASH for hash inlined into AHE.
Avoids HEK pointer chasing, when the hash lookup failed.
Much earlier exit before the loop.

Re-arrange hsplit a bit.

Add seperate mro.h

See GH #24
8543286
@rurban rurban added a commit that referenced this issue Jan 4, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
351053a
@rurban rurban added a commit that referenced this issue Jan 4, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
7e9c64e
@rurban rurban added a commit that referenced this issue Jan 7, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
6c023d2
@rurban rurban added a commit that referenced this issue Jan 7, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
c177a7c
@rurban rurban added a commit that referenced this issue Jan 9, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
7b08554
@rurban rurban added a commit that referenced this issue Jan 9, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
7386e14
@rurban rurban added a commit that referenced this issue Jan 13, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
26287a2
@rurban rurban added a commit that referenced this issue Jan 13, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
0bda5ec
@rurban rurban added a commit that referenced this issue Jan 14, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
b0bffea
@rurban rurban added a commit that referenced this issue Jan 14, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
4cbd931
@rurban rurban added a commit that referenced this issue Jan 15, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
55fbf48
@rurban rurban added a commit that referenced this issue Jan 15, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
d8b8e0a
@rurban rurban added a commit that referenced this issue Jan 15, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
c648aeb
@rurban rurban added a commit that referenced this issue Jan 15, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
713bbb8
@rurban rurban added a commit that referenced this issue Jan 17, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
0a2df9f
@rurban rurban added a commit that referenced this issue Jan 17, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
430bf0b
@rurban rurban added a commit that referenced this issue Jan 24, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
ee067b1
@rurban rurban added a commit that referenced this issue Jan 24, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
6d48ff3
@rurban rurban added a commit that referenced this issue Jan 25, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
1498e2c
@rurban rurban added a commit that referenced this issue Jan 25, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
5a5983f
@rurban rurban added a commit that referenced this issue Jan 27, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
d15044f
@rurban rurban added a commit that referenced this issue Jan 27, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
7e042df
@rurban rurban added a commit that referenced this issue Jan 30, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
1b91d7f
@rurban rurban added a commit that referenced this issue Jan 30, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
75cda2a
@rurban rurban added a commit that referenced this issue Jan 31, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
3ed9519
@rurban rurban added a commit that referenced this issue Jan 31, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
84b7665
@rurban rurban added a commit that referenced this issue Feb 4, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
8a6ac96
@rurban rurban added a commit that referenced this issue Feb 4, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
2b8686c
@rurban rurban added a commit that referenced this issue Feb 8, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
87670f7
@rurban rurban added a commit that referenced this issue Feb 8, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
05e2a44
@rurban rurban added a commit that referenced this issue Feb 8, 2017
@rurban rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
c190456
@rurban rurban added a commit that referenced this issue Feb 8, 2017
@rurban rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
4a4d419
@rurban rurban added a commit that referenced this issue Feb 12, 2017
@rurban @rurban rurban + rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
e64cced
@rurban rurban added a commit that referenced this issue Feb 12, 2017
@rurban @rurban rurban + rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
4801ae8
@rurban rurban added a commit that referenced this issue Feb 16, 2017
@rurban @rurban rurban + rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
bb2d92e
@rurban rurban added a commit that referenced this issue Feb 16, 2017
@rurban @rurban rurban + rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
72ce9aa
@rurban rurban added a commit that referenced this issue Feb 16, 2017
@rurban @rurban rurban + rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.
176cb01
@rurban rurban added a commit that referenced this issue Feb 16, 2017
@rurban @rurban rurban + rurban open hash: use open addressing with linear probing (WIP)
and a layout similar to array_he, inlined 1-2 word hash array.

khash tried double hashing and found it slower. robin-hood with
backward deletion propagation would be an alternative, but first
we need to re-architecture and abstract the various probing implementations
and search loops scattered all over.

See GH #24
1a2baf4
@rurban rurban added a commit that referenced this issue Feb 18, 2017
@rurban @rurban rurban + rurban HeArray: remove hek_hash and refcounted_he_hash
Calculate hashes on demand, but not store it in a HEK
to make HEK shorter to fill more entries into a cache line.
HEK_HASH(hek) is now invalid and gone.
Use the new HeHASH_calc(he), HEK_HASH_calc(hek), SvSHARED_HASH_calc(sv)
instead.
See http://www.ilikebigbits.com/blog/2016/8/28/designing-a-fast-hash-table
for benchmarks (HashCache).

And using 4 tests in the hot hash loop also makes not much sense,
when checking the length and the string is enough to weed out
collisions.
This strategy, recomputing the hash wehen needed, is so far 1-7% slower,
but we hope to get to speed with the HeARRAY patch. See below.

The endgoal is to get rid of linked lists and store the collisions
inlined in consecutive memory, in a HekARRAY. (len,cmp-flags,char*,other-flags,val)
Measurements in "Cache-Conscious Collision Resolution in String Hash Tables"
by Nikolas Askitis and Justin Zobel, Melbourne 2005 show that this is the
fastest strategy for Open Hashing (chained) tables.
See GH #24 and GH #102

The next idea is to use MSB varint encoding of the str length in a HEK,
because our strings are usually short, len < 63, fits into one byte.
We can then merge it with the cmp-flags, the flags only needed for comparison.
See https://techoverflow.net/blog/2013/01/25/efficiently-encoding-variable-length-integers-in-cc/
or just <63 one byte, >63 MSB: I32 len.
Note that the 1st MSB bit is already taken for UTF8.