Repository navigation
Proposal: Sparse CSR fast path for large Numba NMS and GreedyNMM workloads #1413
Closed
ZephyrKeXiner
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hi, @onuralpszr.
I would like to get feedback on a possible optimization for the Numba postprocessing backend before preparing a PR.
Context
#1395 introduced the sparse CSR path to avoid materializing an
N x Nmetric matrix for large inputs.In the current Numba backend,
nmm_numba()delegates large inputs to the sparse implementation because it otherwise builds a full metric matrix. However,nms_numba()andgreedy_nmm_numba()still use their fully JIT-compiled exhaustive pairwise loops.These loops are memory-safe because they do not materialize an
N x Nmatrix, but their runtime is still approximatelyO(N²). For small inputs this is very efficient, while for large and spatially sparse sliced-inference outputs, most box pairs being checked cannot intersect.Proposal
Keep the existing dense JIT path for small inputs, but add a Numba-specific sparse path for large inputs:
SPARSE_MIN_BOXESunconditionally.Conceptually:
The same approach can be used for GreedyNMM, with the JIT function returning keep_order and keeper_of, followed by the existing Python dict conversion.
This would keep the current low-overhead behavior for small inputs while avoiding comparisons between boxes that cannot spatially overlap on large inputs.
All reactions