v0.6.0

v0.6.0: Config Refactor, PQ, and Performance Boost

This release introduces a significant refactoring of the Config struct, adds Product Quantization (PQ) for memory efficiency, and includes several performance optimizations that lay the groundwork for world-class speed.

🚀 Features

• Generic Config Struct: The Config struct is now generic, allowing for greater flexibility and type safety when working with different floating-point types (float32, float64).

• Simplified Configuration: We've introduced DefaultConfig() to provide a sane starting point for your HNSW graph. The Config fields are now un-exported, and tuning is done via the SetParams method on the Graph instance.

  Old Way (before v0.6.0):

  cfg := needle.Config[float32]{
      Threshold: 0.5,
      MaxEdges:  32,
  }
  g, _ := needle.NewGraphFromConfig(cfg)

  New Way (v0.6.0 and later):

  // Start with a default configuration
  cfg := needle.DefaultConfig()
  
  // Create the graph
  g := needle.NewGraphFromConfig(cfg)
  
  // Tune parameters as needed
  g.SetParams(32, 64, 128) // m, efSearch, efConstruction

✨ Optimizations

• Product Quantization (PQ): This release adds an implementation of Product Quantization, which dramatically reduces the memory footprint of the graph, especially for large datasets. The graph will automatically train a PQ codec and use it for distance calculations once a certain threshold of vectors has been added.

• Heap-based Search: The core search algorithm now utilizes minHeap and maxHeap data structures (based on container/heap) to efficiently manage candidate nodes during graph traversal. This is a fundamental component for high-performance HNSW implementations.

🐛 Fixes

• Benchmark Suite (bench/bench_test.go): The benchmarking test has been updated to use the new configuration API, ensuring that our performance measurements are accurate and reflect the latest changes. The benchmark now correctly initializes the graph and runs without errors.