Hashing query performance improvements (1.5-2x faster on benchmarks) #114

alexklibisz · 2020-07-23T00:59:42Z

Rewrote custom hashing query (MatchHashesAndScoreQuery) in Java.
This doesn't necessarily make it faster, rather less likely that you introduce an expensive scala abstraction.
Also easier to get help from Lucene users.

Made match counting faster by using an array instead of a map.
This works because each counter only deals with the consecutive doc ids in a single segment.
So instead of a Map from doc id to count, you have an array where the index is the doc id and value is the count.

Made candidate identification faster using a similar construct.
Since you know the highest possible count is the number of terms, you can us an array to build a histogram of the counts,
then traverse from the end of the array to find the kth largest count.

Still need to understand how the PrefixCodedTerms work and if there's any possible optimization.

Specific timing improvements (p90 benchmark times):

Angular LSH: 121ms -> 50ms
L2 LSH: 18ms -> 11ms
Jaccard LSH: 58ms -> 36ms

github-actions · 2020-07-23T01:02:02Z

Benchmark Results

Commit: b0012f1300d32ba268d99f63dcfc8becbd24a88f
Github Workflow: 179199627

dataset	similarity	algorithm	k	recallP10	durationP10	recallP50	durationP50	recallP90	durationP90	mapping	query
Random1000d50K1K	Angular	Exact	100	1.0	136.013	1.0	199.005	1.0	205.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"angular","vec":{},"model":"exact"}
Random1000d50K1K	Angular	LSH	100	0.15	41.001	0.16005	46.005	0.18	49.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"angular","dims":1000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"angular","model":"lsh"}
Random1000d50K1K	L2	Exact	100	1.0	136.006	1.0	146.01	1.0	151.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"l2","vec":{},"model":"exact"}
Random1000d50K1K	L2	LSH	100	1.0E-5	9.001	0.02	10.0	0.02	11.0	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"l2","dims":1000,"L":400,"k":1,"r":3}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"l2","model":"lsh"}
Random3000d50K1K	Jaccard	Exact	100	1.0	396.034	1.0	507.015	1.0	529.009	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"dims":3000}}	{"field":"vec","similarity":"jaccard","vec":{},"model":"exact"}
Random3000d50K1K	Jaccard	LSH	100	0.28	26.004	0.29005	34.0	0.30009	36.0	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"model":"lsh","similarity":"jaccard","dims":3000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"jaccard","model":"lsh"}

github-actions · 2020-07-23T02:16:43Z

Benchmark Results

Commit: 006d78596d78c36620af500371536185887d594f
Github Workflow: 179259570

dataset	similarity	algorithm	k	recallP10	durationP10	recallP50	durationP50	recallP90	durationP90	mapping	query
Random1000d50K1K	Angular	Exact	100	1.0	130.048	1.0	187.02	1.0	197.0	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"angular","vec":{},"model":"exact"}
Random1000d50K1K	Angular	LSH	100	0.15	37.001	0.16005	39.005	0.18	40.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"angular","dims":1000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"angular","model":"lsh"}
Random1000d50K1K	L2	Exact	100	1.0	113.01	1.0	131.005	1.0	136.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"l2","vec":{},"model":"exact"}
Random1000d50K1K	L2	LSH	100	1.0E-5	9.0	0.02	10.0	0.02	10.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"l2","dims":1000,"L":400,"k":1,"r":3}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"l2","model":"lsh"}
Random3000d50K1K	Jaccard	Exact	100	1.0	318.13	1.0	479.01	1.0	492.009	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"dims":3000}}	{"field":"vec","similarity":"jaccard","vec":{},"model":"exact"}
Random3000d50K1K	Jaccard	LSH	100	0.28	28.001	0.29005	30.0	0.30009	30.009	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"model":"lsh","similarity":"jaccard","dims":3000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"jaccard","model":"lsh"}

github-actions · 2020-07-23T02:47:14Z

Benchmark Results

Commit: 1c71b39548a66b6536235bc02ff304ac89695abc
Github Workflow: 179284149

dataset	similarity	algorithm	k	recallP10	durationP10	recallP50	durationP50	recallP90	durationP90	mapping	query
Random1000d50K1K	Angular	Exact	100	1.0	184.01	1.0	209.02	1.0	218.063	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"angular","vec":{},"model":"exact"}
Random1000d50K1K	Angular	LSH	100	0.15	40.005	0.16005	48.0	0.18	49.0	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"angular","dims":1000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"angular","model":"lsh"}
Random1000d50K1K	L2	Exact	100	1.0	113.009	1.0	136.005	1.0	140.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"l2","vec":{},"model":"exact"}
Random1000d50K1K	L2	LSH	100	1.0E-5	9.0	0.02	10.0	0.02	10.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"l2","dims":1000,"L":400,"k":1,"r":3}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"l2","model":"lsh"}
Random3000d50K1K	Jaccard	Exact	100	1.0	429.023	1.0	502.04	1.0	514.0	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"dims":3000}}	{"field":"vec","similarity":"jaccard","vec":{},"model":"exact"}
Random3000d50K1K	Jaccard	LSH	100	0.28	29.0	0.29005	31.005	0.30009	33.009	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"model":"lsh","similarity":"jaccard","dims":3000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"jaccard","model":"lsh"}

github-actions · 2020-07-24T00:27:41Z

Benchmark Results

Commit: d6ec464bd25b39184792a225c91ed02084f8f0aa
Github Workflow: 180508704

dataset	similarity	algorithm	k	recallP10	durationP10	recallP50	durationP50	recallP90	durationP90	mapping	query
Random1000d50K1K	Angular	Exact	100	1.0	177.01	1.0	197.005	1.0	203.018	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"angular","vec":{},"model":"exact"}
Random1000d50K1K	Angular	LSH	100	0.15	40.004	0.16005	47.005	0.18	49.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"angular","dims":1000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"angular","model":"lsh"}
Random1000d50K1K	L2	Exact	100	1.0	135.006	1.0	144.0	1.0	146.027	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"l2","vec":{},"model":"exact"}
Random1000d50K1K	L2	LSH	100	1.0E-5	9.001	0.02	10.0	0.02	11.0	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"l2","dims":1000,"L":400,"k":1,"r":3}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"l2","model":"lsh"}
Random3000d50K1K	Jaccard	Exact	100	1.0	413.033	1.0	484.01	1.0	492.027	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"dims":3000}}	{"field":"vec","similarity":"jaccard","vec":{},"model":"exact"}
Random3000d50K1K	Jaccard	LSH	100	0.28	25.002	0.29005	32.005	0.30009	34.0	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"model":"lsh","similarity":"jaccard","dims":3000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"jaccard","model":"lsh"}

github-actions · 2020-07-24T00:37:58Z

Benchmark Results

Commit: cd1e79d441a418b0d152a6c504b24af121249982
Github Workflow: 180520350

dataset	similarity	algorithm	k	recallP10	durationP10	recallP50	durationP50	recallP90	durationP90	mapping	query
Random1000d50K1K	Angular	Exact	100	1.0	189.034	1.0	227.015	1.0	237.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"angular","vec":{},"model":"exact"}
Random1000d50K1K	Angular	LSH	100	0.15	43.002	0.16005	50.0	0.18	52.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"angular","dims":1000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"angular","model":"lsh"}
Random1000d50K1K	L2	Exact	100	1.0	130.001	1.0	146.005	1.0	149.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"l2","vec":{},"model":"exact"}
Random1000d50K1K	L2	LSH	100	1.0E-5	10.0	0.02	11.0	0.02	12.0	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"l2","dims":1000,"L":400,"k":1,"r":3}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"l2","model":"lsh"}
Random3000d50K1K	Jaccard	Exact	100	1.0	447.024	1.0	489.015	1.0	504.0	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"dims":3000}}	{"field":"vec","similarity":"jaccard","vec":{},"model":"exact"}
Random3000d50K1K	Jaccard	LSH	100	0.28	32.0	0.29005	34.005	0.30009	35.0	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"model":"lsh","similarity":"jaccard","dims":3000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"jaccard","model":"lsh"}

github-actions · 2020-07-24T00:53:50Z

Benchmark Results

Commit: b62db48f7c94b1e3bf40c533731207156f0603e8
Github Workflow: 180538935

dataset	similarity	algorithm	k	recallP10	durationP10	recallP50	durationP50	recallP90	durationP90	mapping	query
Random1000d50K1K	Angular	Exact	100	1.0	211.001	1.0	217.01	1.0	228.09	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"angular","vec":{},"model":"exact"}
Random1000d50K1K	Angular	LSH	100	0.15	50.005	0.16005	60.0	0.18	61.0	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"angular","dims":1000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"angular","model":"lsh"}
Random1000d50K1K	L2	Exact	100	1.0	124.005	1.0	144.025	1.0	151.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"l2","vec":{},"model":"exact"}
Random1000d50K1K	L2	LSH	100	1.0E-5	10.0	0.02	11.0	0.02	12.0	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"l2","dims":1000,"L":400,"k":1,"r":3}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"l2","model":"lsh"}
Random3000d50K1K	Jaccard	Exact	100	1.0	396.1	1.0	504.025	1.0	527.036	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"dims":3000}}	{"field":"vec","similarity":"jaccard","vec":{},"model":"exact"}
Random3000d50K1K	Jaccard	LSH	100	0.28	35.002	0.29005	38.005	0.30009	40.0	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"model":"lsh","similarity":"jaccard","dims":3000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"jaccard","model":"lsh"}

github-actions · 2020-07-24T01:29:35Z

Benchmark Results

Commit: 64d6a692dd0bd03a616eba59566f6e9efe2c7434
Github Workflow: 180575379

dataset	similarity	algorithm	k	recallP10	durationP10	recallP50	durationP50	recallP90	durationP90	mapping	query
Random1000d50K1K	Angular	Exact	100	1.0	140.027	1.0	172.015	1.0	177.0	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"angular","vec":{},"model":"exact"}
Random1000d50K1K	Angular	LSH	100	0.15	32.0	0.16005	35.0	0.18	36.009	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"angular","dims":1000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"angular","model":"lsh"}
Random1000d50K1K	L2	Exact	100	1.0	123.001	1.0	128.005	1.0	134.0	{"type":"elastiknn_dense_float_vector","elastiknn":{"dims":1000}}	{"field":"vec","similarity":"l2","vec":{},"model":"exact"}
Random1000d50K1K	L2	LSH	100	1.0E-5	9.0	0.02	9.0	0.02	10.0	{"type":"elastiknn_dense_float_vector","elastiknn":{"model":"lsh","similarity":"l2","dims":1000,"L":400,"k":1,"r":3}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"l2","model":"lsh"}
Random3000d50K1K	Jaccard	Exact	100	1.0	379.008	1.0	401.015	1.0	415.054	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"dims":3000}}	{"field":"vec","similarity":"jaccard","vec":{},"model":"exact"}
Random3000d50K1K	Jaccard	LSH	100	0.28	21.001	0.29005	24.005	0.30009	27.0	{"type":"elastiknn_sparse_bool_vector","elastiknn":{"model":"lsh","similarity":"jaccard","dims":3000,"L":400,"k":1}}	{"field":"vec","candidates":1000,"vec":{},"similarity":"jaccard","model":"lsh"}

alexklibisz added 8 commits July 18, 2020 16:27

Decent first pass at Lucene testing harness for HashingQuery

ec08707

Re-implemented hashing query in java and tests pass

69b948a

Separate test for benchmarking hash queries

a9e72fe

~4x speedup by using array instead of map

1f837a1

Slight cleanup and running 10x more searches

6769a1c

Tinkering with quickselect. No big wins

4f5eb92

klargestIndices method cuts benchmark query time from 120s to 97s

4a7d48c

Some internal cleanup.

b0012f1

alexklibisz changed the title ~~Hashing query performance improvements~~ Hashing query performance improvements (1.5-2x faster on benchmarks) Jul 23, 2020

alexklibisz added 6 commits July 22, 2020 21:14

Version and changelog

ae08864

revert version will change later

ef00614

Remove infinite loop so tests actually run

45ee7b5

Doing less

d0bd994

Explanation

006d785

possibly fixed index out of bounds

bc6e1c9

Fixed advance

1c71b39

alexklibisz added 3 commits July 23, 2020 20:01

Show docker-compose logs if any previous step fails

d6ec464

Additional tests for kthGreatest

cd1e79d

more specific exception message

b62db48

Fixing edge case where the number of candidates is >= the number of docs

64d6a69

alexklibisz merged commit c75b23f into master Jul 24, 2020

alexklibisz deleted the perf-issue-58 branch July 24, 2020 01:29

alexklibisz mentioned this pull request Jul 29, 2020

Revisit MatchTermsAndScoreQuery performance #112

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hashing query performance improvements (1.5-2x faster on benchmarks) #114

Hashing query performance improvements (1.5-2x faster on benchmarks) #114

alexklibisz commented Jul 23, 2020 •

edited

Loading

github-actions bot commented Jul 23, 2020

github-actions bot commented Jul 23, 2020

github-actions bot commented Jul 23, 2020

github-actions bot commented Jul 24, 2020

github-actions bot commented Jul 24, 2020

github-actions bot commented Jul 24, 2020

github-actions bot commented Jul 24, 2020

Hashing query performance improvements (1.5-2x faster on benchmarks) #114

Hashing query performance improvements (1.5-2x faster on benchmarks) #114

Conversation

alexklibisz commented Jul 23, 2020 • edited Loading

github-actions bot commented Jul 23, 2020

Benchmark Results

github-actions bot commented Jul 23, 2020

Benchmark Results

github-actions bot commented Jul 23, 2020

Benchmark Results

github-actions bot commented Jul 24, 2020

Benchmark Results

github-actions bot commented Jul 24, 2020

Benchmark Results

github-actions bot commented Jul 24, 2020

Benchmark Results

github-actions bot commented Jul 24, 2020

Benchmark Results

alexklibisz commented Jul 23, 2020 •

edited

Loading