Ideas

[ankane]

Please create a new issue to discuss any ideas or share your own.

HNSW

• Add support for streaming I/O - hnsw-read-stream3 branch
• Use TidStore for vacuuming for Postgres 17+ - hnsw-tidstore2 branch
• Improve colocation of neighbors on disk for index builds - hnsw-build-order branch
• Add support for inline filtering
  • HQANN - hqann3 branch (WIP)
  • ACORN
• Add time budget option - time-budget branch
• Use scan->kill_prior_tuple - kill-prior-tuple branch (WIP)
• Add support for index-only scans - hnsw-index-only-v2 branch
• Add support for INCLUDE clause with CREATE INDEX (not more performant since still accesses heap)
• Improve performance of on-disk index builds by keeping neighbors and closer in memory

IVFFlat

• Add support for inline filtering - ivfflat-filtering branch (needs different design for better performance)
• Add support for product quantization
• Add support for another coarse quantizer
• Add support for parallel k-means
• Update parallel workers calculation - ivfflat-parallel-workers branch
• Increase default value of ivfflat.probes
• Add support for setting default probes on index level
• Scan additional lists if number of entries is low after scanning probes
• Use pairing heap for index scan for performance - stages and pairingheap branches
• Use tuplesort_set_bound for performance - bound branch (not needed w/ pairing heap)
• Add support for parallel index scans (planner gets cost estimate but doesn't use) - parallel-index-scan4 branch
• Use Lloyd's or MO-Elkan's k-means when not enough memory for Elkan's
• Use FB-Elkan's k-means for faster index creation
• Use mini-batch k-means for index creation for reduced memory - minibatch branch
• Add support for index-only scans - ivfflat-index-only branch
• Add support for INCLUDE clause with CREATE INDEX
• Prevent compression for index tuples - ivfflat-storage branch

Indexes

• Include vectors with zero norm in indexes for cosine distance - index-zero-norm branch

Types

• Add support for fp8 vectors (e4m3) - minivec branch
• Add support for int8 vectors - intvec branch

Functions

• Add subscript function - subscript branch (requires Postgres 14+)
• Add angular_distance function - angular_distance branch - Adding angular distance for nn search #234
• Add random_vector function - random_vector branch
• Add functions to view lists and/or pages like pageinspect (require superuser)
• Add function to estimate index balance (require superuser?) - Estimating ivfflat balance / need to recompute index #105

Installation

• Create installer for Windows - Offer precompiled OS packages (feature request) #108