Stream Sifter

A small library for efficient filtering of streaming data sources.

Features

• Custom rule-based filters
• Real-time data processing
• Pluggable filter modules

Installation

pip install stream-sifter

Basic Usage

from stream_sifter.core import StreamProcessor
from stream_sifter.filters import BaseFilter

class MyFilter(BaseFilter):
    def filter(self, item):
        return item.get('value', 0) > 10

processor = StreamProcessor([MyFilter()])
input_data = [
    {'value': 5},
    {'value': 12},
    {'value': 8},
    {'value': 22}
]

for item in processor.filter_stream(input_data):
    print(item)
# Output:
# {'value': 12}
# {'value': 22}

Pluggable Filters Example

You can dynamically load filter plugins:

from stream_sifter.plugins import load_plugin_filter

my_plugin = load_plugin_filter('my_plugin.FilterClass')
processor = StreamProcessor([my_plugin()])

# Filter your data stream as above

Further Documentation

Refer to the API documentation in stream_sifter/filters.py and stream_sifter/plugins.py for more detail on filter implementation and plugins.