v1.2.0

✨ Highlights

• New SemanticChunker with advance peak detection, skip_window support, group embedding support and better thresholding! The new semantic chunker requires much less tuning for consistently sized chunks.

# Import the chunker
from chonkie import SemanticChunker

# Initialize the chunker
chunker = SemanticChunker(chunk_size=512)

# define the text
text = ...

# CHONK!
chunk = chunker(text)

• Support for MongoDB via MongoDBHandshake!

from chonkie import MongoDBHandshake, SemanticChunker

# Initialize the handshake
handshake = MongoDBHandshake(uri="mongodb://localhost:8080", db_name="my_documents")

# Create some chunks
chunker = SemanticChunker()
chunks = chunker.chunk("Chonkie loves to chonk your texts!")

# Write chunks to MongoDB
handshake.write(chunks)

• Support for Pinecone via PineconeHandshake!

from chonkie import PineconeHandshake, SemanticChunker

# Initialize the handshake
handshake = PineconeHandshake(api_key=...)

# Create some chunks
chunker = SemanticChunker()
chunks = chunker.chunk("Chonkie loves to chonk your texts!")

# Write chunks to Pinecone
handshake.write(chunks)

• Support for Weaviate via WeaviateHandshake!

from chonkie import WeaviateHandshake, SemanticChunker

# Initialize the handshake
handshake = WeaviateHandshake(url=url, api_key=...)

# Create some chunks
chunker = SemanticChunker()
chunks = chunker.chunk("Chonkie loves to chonk your texts!")

# Write chunks to Weaviate
handshake.write(chunks)

What's Changed

• Fix: Update README — Absolute path for logo by @chonk-lain in #235
• Update README.md — Add recent integrations to the list by @chonknick in #237
• docs: fix url by @superpung in #240
• Support weaviate by @xucailiang in #204
• Fix: Remove failing weaviate cloud tests from CI/CD by @chonknick in #246
• Fix: Add _truncate to the OpenAIEmbddings (#239) by @chonknick in #247
• Feat: Add PineconeHandshake by @chonk-lain in #243
• docs: updated handshakes overview and move changelog to Python SDK by @sahibpreetsingh12 in #255
• Feat: Add MongoDBHandshake by @chonk-lain in #253
• Feat: Add better SemanticChunker with peak finding, window similarities and more! by @chonknick in #251
• Migrate the cookbook to chonkie-inc/cookbook by @chonknick in #258
• Docs: update mongodb docs by @chonk-lain in #259

New Contributors

• @chonk-lain made their first contribution in #235
• @superpung made their first contribution in #240
• @xucailiang made their first contribution in #204
• @sahibpreetsingh12 made their first contribution in #255

Full Changelog: v1.1.2...v1.2.0