Text index: add internal bloom filter layer

[ahmadov]

Similar to #83485 but better. Reverts #85271 which reverts #85054.

Adds an additional bloom filter layer on top of text index segments. This prevents loading the segment dictionary into memory when a term does not exist in the dictionary.

The bloom filter is constructed from the segment data on writeSegment call.

Introduce a new text index parameter bloom_filter_false_positive_rate to control false-positive rate while building the bloom filter. By default it's set to 0.1%.

Changelog category (leave one):

• Not for changelog (changelog entry is not required)

[clickhouse-gh]

Workflow [PR], commit [afcf9a0]

Summary: ⏳

job_name	test_name	status	info	comment
Stateless tests (amd_debug, distributed plan, s3 storage, parallel)		failure
	02443_detach_attach_partition	FAIL
Integration tests (asan, old analyzer, 3/6)		failure
	test_ttl_replicated/test.py::test_ttl_compatibility[node_left2-node_right2-2]	FAIL

[rschu1ze]

We should use std::lock_guard here (SO).

More importantly, the caller readSegmentedPostingsLists accesses fst as well:

if (seg_dict.second->fst == nullptr)

TSAN will sooner or later complain. It technically comes down to the question if the pointer itself needs synchronization or only the pointed-to object. I would argue for the former - the pointer gets set once the FST is loaded.

Therefore, can we omit locking here and rewrite readSegmentedPostingsLists to something like this:

FST::FiniteStateTransducer::Output fst_output;

{
    std::lock_guard lock(segment_dictionary.second->fst_mutex);

    if (segment_dictionary.second->fst == nullptr)
    {
        /// Segment dictionary is not loaded. First check the bloom filter if we can avoid the load.
        if (segment_dictionary.second->bloom_filter && !segment_dictionary.second->bloom_filter->contains(term))
            continue;

        /// Term might be in segment dictionary
        readSegmentFST(segment_dictionary.second);
    }

    fst_output = segment_dictionary.second->fst->getOutput(term);
    if (!fst_output.found)
        continue;
}

(I also changed the return type of FiniteStateTransducer::getOutput a little bit:

struct Output {
    UInt64 offset;
    bool found;
};
Output getOutput(std::string_view term);

)

[rschu1ze]

02443_detach_attach_partition: #54748

test_ttl_replicated/test.py::test_ttl_compatibility[node_left2-node_right2-2]: #83789 (comment)