Skip to content

Pathik 0.2.17

Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 27 Mar 01:29
· 17 commits to main since this release
v0.2.17
a6fd70b

Kafka Streaming Feature Changelog (v0.2.17)

Added Support for Kafka Streams

  • Kafka Integration: Added ability to stream crawled content directly to Kafka topics
  • Content Type Selection: Stream HTML-only, Markdown-only, or both content types
  • Multi-user Support: Added session tracking with the -session flag for multi-user environments
  • Topic Configuration: Custom topic support via the -topic flag
  • Parallel Crawling: Maintained parallel crawling capability with Kafka streaming
  • Fixed issues from the python build in Pre-Release 0.2.16

Documentation & Examples

  • Added comprehensive documentation for Kafka features in README
  • Created example Kafka consumers in Go and Python in the examples directory
  • Added Docker Compose setup for easy local Kafka development
  • Documented multi-user and multi-URL workflows

Technical Improvements

  • Added proper error handling for Kafka connectivity issues
  • Support for authentication (SASL/PLAIN) and TLS
  • Environment variable configuration via .env file
  • Unified CLI experience across both Go binary and Python wrapper

This update enables real-time processing pipelines for crawled content and simplifies integration with streaming data workflows.

Assets 7
Loading