Skip to content

v0.3.0

Choose a tag to compare

@flori flori released this 18 May 08:10
  • Source Tracking & Versioning:
    • Introduced Documentrix::Utils::Digests for SHA256 hashing of strings
      and files, featuring an mtime-based cache.
    • Implemented source-based document management including
      normalize_source, source_exist?, source_modified?, source_update,
      and source_remove.
    • Added support for digest in Documentrix::Documents#add and
      source_update to enable version tracking.
  • Text Splitting:
    • Added Documentrix::Documents::Splitters::Common to implement
      force_split behavior.
    • Integrated force splitting into Character, RecursiveCharacter, and
      Semantic splitters to ensure chunks strictly respect chunk_size.
  • Cache Enhancements:
    • Added each_source to Documentrix::Documents::Cache::Common and an
      optimized SELECT DISTINCT source implementation in SQLiteCache.
    • Implemented a SQLite trigger delete_embedding_after_record to
      automatically clean the embeddings table when records are deleted.
    • Made object_class a required keyword argument in
      Documentrix::Documents::RedisCache#initialize.
  • Search & Retrieval:
    • Added min_similarity parameter to Documentrix::Documents#find and
      associated cache retrieval methods.
    • Optimized SQLiteCache#find_records by moving similarity calculations
      and filtering into the SQL query using 1 - vec_distance_cosine.
    • Streamlined Documentrix::Documents#find_where logic and improved
      utilization of max_records.
  • Bug Fixes:
    • Fixed an issue in Documentrix::Documents#find where max_records was
      hardcoded to nil when calling the cache.