Bloom filter indices

[nikvas0]

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Category (leave one):

• New Feature

Short description (up to few sentences):
A new type of data skipping indices based on bloom filters (can be used for equal, in and like functions).

• ngrambf(n, bloom_filter_size_in_bytes, hash_functions_count, random_seed)
• tokenbf(bloom_filter_size_in_bytes, hash_functions_count, random_seed) (saves all tokens, so query WHERE s LIKE '%cats%' will use fullscan, but query WHERE s LIKE %.cats.% will use 'cat' token)

[nikvas0]

Some comparison (ngrambf(3, 512, 2, 0), tokenbf(512, 2, 0) and no index)
https://gist.github.com/nikvas0/a6d66e833c0adaa42762fdf77a026e2b

[nikvas0]

Can be also used in KeyCondition (RPNBuilder is simply a copy-paste of some of its functions), but it fails performance tests. https://clickhouse-test-reports.s3.yandex.net/4499/fcb82ba901651b73229a0be1bbd71fba308a4d57/performance_test.html
(For BloomFilterIndex I have not noticed any difference in performance between copy-pasted functions and RPNBuilder)

[nikvas0]

Comparision between no index and ngram index (50 runs for each)
https://github.com/nikvas0/CHDataSkippingTest/blob/master/indices_test.ipynb

[nikvas0]

Comparison for insert

# insert into ... select * from datasets.hits_v1

no index: 0 rows in set. Elapsed: 20.463 sec. Processed 8.87 million rows, 8.46 GB (433.65 thousand rows/s., 413.45 MB/s.)

3 x ngrambf(3, 512, 2, 0) (URLDomain, SearchPhrase, Title): 0 rows in set. Elapsed: 42.752 sec. Processed 8.87 million rows, 8.46 GB (207.57 thousand rows/s., 197.90 MB/s.)
3 x ngrambf(4, 512, 1, 0) (URLDomain, SearchPhrase, Title): 0 rows in set. Elapsed: 39.391 sec. Processed 8.87 million rows, 8.46 GB (225.28 thousand rows/s., 214.78 MB/s.)

1 x ngrambf(4, 512, 1, 0) (Title): 0 rows in set. Elapsed: 35.820 sec. Processed 8.87 million rows, 8.46 GB (247.74 thousand rows/s., 236.19 MB/s.)
1 x ngrambf(4, 512, 1, 0) (URLDomain): 0 rows in set. Elapsed: 25.047 sec. Processed 8.87 million rows, 8.46 GB (354.29 thousand rows/s., 337.79 MB/s.)

1 x tokenbf(512, 1, 0) (Title): 0 rows in set. Elapsed: 27.339 sec. Processed 8.87 million rows, 8.46 GB (324.59 thousand rows/s., 309.47 MB/s.)
1 x tokenbf(512, 1, 0) (URLDomain): 0 rows in set. Elapsed: 22.537 sec. Processed 8.87 million rows, 8.46 GB (393.74 thousand rows/s., 375.40 MB/s.)

https://gist.github.com/nikvas0/c60ecb9c37d4a61b4cd924090ff6a806