BITSET index: salloc_index_to_ptr is broken in 1.5 and 1.6

[rtsisyk]

salloc_index_to_ptr is broken because tuple pointer is shifted by tuple_alloc

1.5 and 1.6 will likely require different fixes

[rtsisyk]

I promised to put some light on this bug.
tuple_alloc shifts a tuple pointer and uses this extra space to store field_map:

    char *ptr = (char *) salloc(total, "tuple");
    struct tuple *tuple = (struct tuple *)(ptr + format->field_map_size);

salloc_index_to_ptr always returns the original pointer.
There is no way to get shifted pointer from original because the size of field_map cannot be determined.

[kostja]

Roman, why do tests pass in 1.5? What's wrong with tests that they don't verify this? 1.5 needs a test case to reproduce the problem. Then, for now, simply disable salloc_ptr_to_index in 1.5. In 1.6 we will solve this after adding msgpack.

[rtsisyk]

test/box/bitset.lua works with (num, num) tuples. Such kind of tuples do not need the offset table, because offsets can be precalculated and saved in the tuple format definition.

BITSET need a (tuple <-> some_number_id) mapping where the right space is very dense.
ptr_to_index is not so good approximation to the ideal mapping, but it was best that I could implement in our architecture. Replacing ptr_to_index with tuple pointer make BITSET very sparse and therefore ineffective.

Possible ways to fix the bug:

1. Remove the ugly hack which shifting pointers and move all offsets to the end of tuple
2. Try to find a better way to enumerate tuples in the space

[rtsisyk]

I added a workaround for 1.5 to fix bitset.
Anyway, this bug is not fixed.

[kostja]

I think we need to abandon allocator-based pointer translation, and store a translation hash in the bitset itself. The id in the bitset would be simply the offset in the hash with linear collision resolution.
More memory used, but significantly less headache.
Thoughts?

[rtsisyk]

I'm not sure about offset in mhash, but I totally agree with the idea.
This ticket should be re-scheduled to 1.5.x.