Repository navigation
Releases
Releases · arturormk/curaflow
Curaflow v1.0.0
Curaflow v1.0.0
First stable release of Curaflow: an incremental, parallel fetch → normalize → build pipeline with hierarchical fanout for web‑curated datasets.
Highlights
-Incremental fetch & build
- HTTP sources (http_json, http_html, http_xml, http_bytes) with ETag / Last‑Modified and content digests.
- Targets rebuilt only when inputs change, with structural diffs stored under .curaflow/diffs/.
- Hierarchical fanout
- Scrape index pages → fan out to tenant/detail pages → fan out further to images/media.
- ODS support
- ods_table source plugin backed by pyexcel-ods3 to read ODS spreadsheets into normalized extraction tables.
- Inspection & tooling
- curaflow table to inspect YAML sources as pretty terminal tables with natural sorting.
- JSON diff helper (curaflow.json_diff) and watch target plugin for compact change summaries.
- Plugin system
- Simple source/target plugin registry, with helpers for HTML sources and QML list‑model targets.
Installation
python -m venv .venv && source .venv/bin/activate
pip install curaflow-1.0.0-py3-none-any.whl # or from PyPI/GitHub asset
curaflow --helpNotes
- Packaged as both a wheel and source distribution.
- CI runs linting, type checking, tests, and packaging checks on each change.