Skip to content

Vectorwave v0.2.05 released

Choose a tag to compare

View all tags
@junyeong0619 junyeong0619 released this 21 Dec 03:16
· 84 commits to main since this release
v0.2.05
e75d928

🚀 VectorWave v0.2.05 Release Notes

"Experience Unmatched Performance with the New Rust Core"

In version 0.2.05, we have completely re-engineered the core data ingestion and logging module by porting it to Rust. By breaking free from the Global Interpreter Lock (GIL) of Python, VectorWave can now process massive amounts of log data in real-time without compromising the performance of your main application.

🔥 Highlights

1. 🦀 Introducing Rust Core Engine (vectorwave_core)

  • The WeaviateBatchManager, previously reliant on Python's threading and queue, has been rewritten in Rust.
  • True Parallelism: Leveraging Rust's native threads, log ingestion now runs independently in the background, consuming negligible CPU cycles from the main Python process.
  • High-Performance Queuing: By integrating crossbeam-channel, a lock-free concurrency library in Rust, we achieved an over 10x increase in throughput.

2. ⚡️ Liberation from the GIL

  • I/O-bound operations (database transmission) and data queuing no longer contend for the Python GIL.
  • This ensures that your main application logic (e.g., check.py) runs significantly faster and remains stable even under heavy traffic loads.

3. 🛠️ Modern Build System (Maturin)

  • We have adopted Maturin, the standard tool for building hybrid Python/Rust packages.
  • Migrated from setuptools-rust to maturin to ensure build stability and simplify the release process.
  • Seamless Installation: Users can simply run pip install vectorwave to automatically download and install the optimized pre-compiled Rust binary (Wheel). No Rust toolchain installation is required for end-users.

📋 Changelog

Features & Improvements

  • feat(core): Implemented vectorwave_core crate using PyO3 and Maturin.
  • feat(batch): Updated WeaviateBatchManager to detect the Rust Core and operate in hybrid mode with a graceful fallback to Python.
  • feat(build): Switched build backend to maturin in pyproject.toml and removed legacy setup.py.

CI/CD

  • ci(release): Updated GitHub Actions workflows to automatically build and upload Wheels for Linux, Windows, and macOS to PyPI upon release.

Refactoring

  • refactor: Renamed internal method _flush_batch to _flush_batch_core for better clarity.
  • refactor: Restructured package layout to integrate vectorwave.vectorwave_core submodule.

📦 Installation & Upgrade

Existing users can upgrade to the latest version via pip:

pip install --upgrade vectorwave

Contributors

  • @Cozymori (Junyeong Gim) - Rust Porting & Architecture Design

Contributors

Cozymori
Assets 2
Loading