[dataflow-dataset] DataFlow Dataset Build Report (mixed) — 2026-10-09

[github-actions[bot]]

Summary

Built a cleaned, deduplicated dataset from GitHub discussions and PRs.

Metric	Value
Execution mode	mixed
Input records	600
Output (clean) records	597
Retention rate	99.5%
Average text length	2,688 chars
Operators used	CharNumberFilter, HashDeduplicateFilter, python_alpha_ratio_filter

DataFlow handled the supported stages (length filter, hash dedup) while a deterministic Python stage preserved the alpha-ratio contract (AlphaWordsFilter skipped to avoid NLTK downloads).

Records by Source

Source	Count
pull_request	500
discussion	97

Stage Counts

Stage	Records
Input	600
After length filter (50–100,000 chars)	597
After alpha-ratio filter (≥0.25)	597
After dedup	597

Pipeline

1. Normalise — unified JSONL (title + body → text; fields id, source, text, url, author, created_at)
2. Length filter — DataFlow CharNumberFilter (≥50) plus Python ≤100,000 cap
3. Alpha-ratio filter — Python, ≥25% alphabetic
4. Dedup — DataFlow HashDeduplicateFilter

Dataset Artifact

dataset_clean.jsonl was uploaded as a workflow artifact for this run.

Configuration

DataFlow package: open-dataflow==1.0.10 (https://github.com/OpenDCAI/DataFlow)

---

Generated by DataFlow PR & Discussion Dataset Builder — Run #37876940883 on 2026-10-09

🌊 Dataset built by DataFlow PR & Discussion Dataset Builder · copilot · auto · 23 AIC · ⌖ 0.787 AIC · ⊞ 15.4K · ◷

• expires on Oct 15, 2026, 7:01 PM UTC-08:00