Signal Extraction Framework

Modular • Streaming • Inspectable • Reproducible

Build computer-vision pipelines around signals, not monolithic scripts.

Design architectures that survive changing algorithms.

Documentation • PyPI • Benchmarks

---

---

Performance Highlights

Memory (RSS)	Peak Allocations	Runtime Overhead	Realtime Latency
↓ 84%	↓ 98%	1.06×	↓ 74%

---

Recent synthetic benchmarks show that SEF can:

• reduce process memory usage by up to 84% through end-to-end streaming;
• reduce Python peak allocations by up to 98%;
• keep architectural overhead close to a direct procedural implementation (1.06×);
• reduce realtime latency by up to 74% through configurable backpressure policies.

See Benchmarks for reproducible commands, raw CSV files, execution plans, and generated charts.

---

✨ Why SEF

SEF is designed for projects where the pipeline architecture matters as much as the computer-vision algorithm itself.

Instead of hiding the workflow inside monolithic OpenCV scripts, SEF turns every stage into an explicit, inspectable, and reusable component.

Modern computer vision evolves faster than most software architectures.

Models, trackers, and foundation models constantly change, while projects often remain tied to monolithic scripts that are difficult to evolve.

SEF decouples pipeline architecture from individual algorithms.

Algorithms become interchangeable components instead of architectural decisions.

This allows pipelines to evolve over time without rewriting the surrounding system.

SEF provides:

• modular pipeline contracts for acquisition, processing, signal extraction, analysis, and visualization;
• hybrid batch/streaming execution with automatic materialization boundaries;
• plugin-based extensibility through decorators and runtime registries;
• UI-agnostic artifacts reusable from Streamlit, notebooks, CLIs, or future interfaces;
• inspectable execution plans for debugging and reproducibility;
• versioned configurations to reproduce experiments over time.

SEF is not a model-training framework, a dashboard application, or a generic workflow engine.

It is an architectural layer that sits between raw computer-vision libraries and domain-specific applications, making video-to-signal pipelines easier to build, explain, benchmark, and evolve.

Reproducibility First

Every execution produces a reproducible description of the experiment itself.

SEF can automatically generate:

• normalized configurations;
• execution plans;
• execution summaries;
• reproducibility files;
• visual artifacts;
• execution metadata.

Experiments can be reconstructed months later without relying on undocumented scripts.

Use Cases

Use SEF when the pipeline structure matters as much as the individual computer vision model:

• motion tracking and displacement analysis;
• ArUco marker detection and relative motion analysis;
• frame preprocessing, masking, stabilization, and intermediate inspection;
• video-to-signal extraction followed by cleaning and analytics;
• UI-agnostic visual artifact generation for dashboards, notebooks, or CLIs;
• rapid experimentation with custom pipeline components;
• teaching or evaluating modular computer-vision architectures.

SEF is not intended to replace model-training frameworks, annotation platforms, or low-level OpenCV scripts when a single script is enough.

Quick Start

Install the core package:

pip install sef

Install only the adapter stacks you need:

pip install "sef[opencv]"        # OpenCV video/tracking/ArUco components
pip install "sef[visualization]" # Matplotlib visualizers
pip install "sef[ui]"            # Streamlit Studio
pip install "sef[yolo]"          # Ultralytics pose extraction
pip install "sef[pose]"          # COCO pose analyzer helpers
pip install "sef[all]"           # all runtime adapter extras

Create and inspect a starter project:

sef init tracking-demo
sef doctor --config pipeline.yaml
sef validate pipeline.yaml --strict
sef run pipeline.yaml --dry-run --explain

For a dependency-light custom component scaffold:

sef init plugin
sef components inspect sample_count
python -m pytest tests/test_custom_components.py

Python API

The common path is intentionally short: describe a source, add stages, and run. The facade accepts registered plugin names, component classes, component instances, or plain Python callables:

import sef


@sef.processor(
    "grayscale",
    description="Convert one frame image to grayscale.",
    metadata={"domain": "preprocessing", "input": "Frame", "output": "Frame"},
)
def grayscale(image):
    return image.mean(axis=2)


outputs = (
    sef.pipeline("custom-run", include_builtins=True)
    .frames("opencv_buffered", path="videos/input.mp4")
    .process("grayscale")
    .signals("opencv_tracker", tracker_type="MIL", start_box=[100, 200, 50, 80])
    .analyze("vertical_position")
    .run()
)

Comparisons

Tool type	SEF position
OpenCV scripts	SEF adds reusable pipeline contracts, config, execution planning, artifacts, and plugin inspection.
Model frameworks	SEF orchestrates computer-vision pipeline stages; it is not a training framework.
Workflow engines	SEF is domain-specific for frame/signal/visual-artifact pipelines, not a general DAG scheduler.
Dashboard apps	SEF keeps output UI-agnostic so Streamlit, notebooks, CLIs, or future UIs can consume the same artifacts.

Visual Results

Pre-processed video	Processed video

SEF Studio

SEF Studio is the Streamlit application built on top of the same runtime. It is not a separate engine: visual composition, Python facade usage, and versioned config resolve to the same core execution model.

pip install -e ".[ui]"
streamlit run ui/app.py

Project Status

• Experimental, pre-1.0.
• Public API is being documented and hardened.
• Config schema is versioned.
• Optional dependencies are split by feature extras.
• Suitable for research, demos, and framework exploration.
• Not yet presented as production-stable infrastructure.

Core Authors

• Matteo Vittori
• Alejandro Innocenzi

Acknowledgements

Special thanks to:

• Michele Loreti, for guidance and advice throughout the project.
• Tomek Paczkowski, for kindly granting ownership of the sef package name on PyPI.