Reported by rcurtin on 25 Oct 42625579 22:12 UTC
Using current sources (r13421), CoverTree<>::DualTreeTraverser performs particularly slow searches. For example, running (and then profiling) allknn with query and reference sets equal to 50k points out of the covtype dataset (covtype_r-50k.csv), here are the number of base case computations:
So the problem is fairly clear. When getting numbers for this, one must be careful to compile disabling inlining if using profiles to get the numbers (-fno-inline).
Commented by rcurtin on 15 Jun 42899215 14:50 UTC
For a simpler test case (reference = test_data_3_1000.csv, query = test_data_3_1000.csv), with k = 5, here are the number of base case computations:
Also some other information:
Those numbers are the distance evaluations for both tree constructions (so, half that for the per-dataset cover tree construction evaluations).
Commented by rcurtin on 29 Jun 42899840 07:39 UTC
Using a jl-code constructed cover tree on test_data_3_1000.csv and k = 5, as before, we get these numbers for the number of distance evaluations in the dual tree traversal:
Therefore the issue is not with the actual way we are creating cover trees, but instead we know that the bug must be in the actual dual-tree traversal. The jl-to-mlpack cover tree converter is in contrib/rcurtin/jl-to-mlpack_cover_tree/ for posterity.
Commented by rcurtin on 14 Jan 42915951 10:48 UTC
With some changes to how the bound is propagated in r13955 through r13959, the following numbers are produced:
We have some improvement, but I believe jl's implementation makes a few more opportunistic prunes than our implementation currently does. So this isn't quite done yet.
Commented by rcurtin on 14 Jul 42925871 04:14 UTC
This is the prune which is not being done by our tree:
inline bool shell(float parent_query_dist, float child_parent_dist, float upper_bound)
return parent_query_dist - child_parent_dist <= upper_bound;
// && child_parent_dist - parent_query_dist <= upper_bound;
It is called when descending the reference cover set. The bound can prune without evaluating the base case of a child.
Commented by rcurtin on 4 Mar 42929546 20:06 UTC
Things are getting better with r13976:
The jl-constructed cover tree must have some interesting properties then. I believe there is still one prune with shell() that we are not getting.
Commented by rcurtin on 19 Nov 42931369 14:32 UTC
Some more improvements:
So we are now outperforming jl's implementation with his tree, but not with the mlpack-constructed tree. Fortunately I still have one more trick up my sleeve. The latest (somewhat hackish) improvements are committed in r13979.
Commented by rcurtin on 25 Oct 43028289 09:30 UTC
There were some errors I introduced with my modification (#264), so now with those fixed we are at:
This is not doing parent-child prunes, so I need to reimplement that now (safely).
Commented by rcurtin on 24 Oct 43044072 17:43 UTC
r14120 makes the bound a little tighter, giving
The parent-child prune has not been implemented safely yet.
Commented by rcurtin on 11 Jul 43761046 14:04 UTC
Realistically at this point, the CalculateBound() function probably needs to be rethought, with some assumptions on how cover trees and kd-trees are traversed. Moving to 1.0.8...
Commented by rcurtin on 12 Jan 44029419 04:13 UTC
After some very time-consuming debugging, I discovered an invalid prune in the jl dual-tree traversal. Thus, we now have
Note that this is done with a base of 2.0.
Commented by rcurtin on 15 Aug 44113956 00:40 UTC
Using the new TraversalInfo concept committed in r16217 through r16230, the child-parent prunes are now possible and the results are now (on test_data_3_1000.csv):
And runtime results for the same dataset:
Then, on a larger dataset, the corel dataset (32x37748) (one duplicate point had to be removed):
I am not sure the runtime numbers for the jl-constructed cover tree and mlpack dual-tree traversal are reasonable, because the tree needs to be reformatted and this is done inefficiently. Additional slowdown is probably present because the bound calculation (NeighborSearchRules::CalculateBound()) is slow to calculate and should be improved. But that is outside the scope of this ticket because this ticket is only about the traversal, so I can (finally) mark this resolved.