v1.17 - Replication, Hybrid Search, and more
Breaking Changes
none
New Features
Leaderless Replication
Weaviate v1.17 introduces leaderless replication with tunable consistency. With replication, Weaviate becomes highly available and can be scaled to increase throughput. Imports can be automatically replicated across the cluster. Existing datasets can be scaled up to allow for higher read-throughput and fail-safety.
To learn more about the architecture behind Weaviate's leaderless replication, how to use replication, what limitations there are, and what the future roadmap looks like, please see the documentation.
Contributed by @parkerduckworth in #2344, by @parkerduckworth in #2361, by @redouan-rhazouani in #2362, by @parkerduckworth in #2365, by @redouan-rhazouani in #2371, by @redouan-rhazouani in #2373, by @redouan-rhazouani in #2377, by @redouan-rhazouani in #2380, by @parkerduckworth in #2382, by @redouan-rhazouani in #2384, by @redouan-rhazouani in #2389, by @parkerduckworth in #2395, by @parkerduckworth in #2420, by @redouan-rhazouani in #2418, by @redouan-rhazouani in #2446, by @parkerduckworth in #2447, by @etiennedi in #2463, by @etiennedi in #2461, by @redouan-rhazouani in #2464, by @redouan-rhazouani in #2399, by @redouan-rhazouani in #2415, by @redouan-rhazouani in #2402, by @redouan-rhazouani in #2417, by @redouan-rhazouani in #2437, by @redouan-rhazouani in #2426
Hybrid Search (Dense Vector & BM25F)
Weaviate v1.17 introduces the ability to perform a hybrid search consisting of a BM25(F) and dense vector search. The results are combined using rank fusion. In addition to Hybrid Search, Weaviate now also supports pure BM25 and BM25F search.
To learn more about Hybrid Search and BM25 Search, see the documentation.
Contributed by @donomii in #2319, by @donomii in #2381, by @donomii in #2450, by @donomii in #2467, by @aliszka in #2469, by @aliszka in #2468, by @donomii in #2466, by @donomii in #2391
Other Features
- Dynamically Add Nodes To Running Cluster after data has been imported by @etiennedi in #2350
- Make startup cluster schema sync backward-compatible by @etiennedi in #2413
- Add TTLs to cluster-wide transactions (Schema, Classifications) by @etiennedi in #2364
- Adjust Memtable Size Dynamically by @etiennedi in #2425
Fixes
Performance
- Improve Startup Time by persisting more LSM Segment Information by @etiennedi in #2385
- Improve p9999 Batch Import Latency by precomputing segment metadata for newly compacted segments by @etiennedi in #2421
- Improve HNSW startup time by improving allocation efficiency in Deserializer by @etiennedi in #2392
UX
- UX: No error message on empty schema by @trengrj in #2345
- Updated contributor guide by @databyjp in #2353
- Update README.md by @erika-cardenas in #2354
- Fix "schema out of sync" error when upgrading to v1.17 by @etiennedi in #2419
- nit: Improve GitHub Community Standards Score by @MarcusSorealheis in #2409
Other / Internal / Security
- Bugfix: created and last updated timestamps missing from reference additional fields by @parkerduckworth in #2342
- Fix typos by @trengrj in #2374
- Add Go-specific Vulnerability Scanner by @etiennedi in #2379
- misc: fix typo in indexTimestamps prop name by @dandv in #2396
- Fix broken logger in tx manager by @etiennedi in #2411
- update go/x/net version by @etiennedi in #2427
- Change references from Travis to GitHub actions in PR summary by @dirkkul in #2422
- Add Codeowners file by @dirkkul in #2404
- Run docker push step on larger machine by @etiennedi in #2435
- Fix file descriptor leak in compaction pre-compute by @etiennedi in #2440
New Contributors
- @databyjp made their first contribution in #2353
- @erika-cardenas made their first contribution in #2354
- @dandv made their first contribution in #2396
- @MarcusSorealheis made their first contribution in #2409
Full Changelog: v1.16.9...v1.17.0
