🎨 NeMo Data Designer v0.8.0
Data Designer v0.8.0 adds deeper operational visibility and more flexible workflow execution, while improving how Data Designer integrates into larger applications.
- 📊 Operational visibility - monitor generation with a live terminal throughput dashboard, local OpenTelemetry/Prometheus metrics, and opt-in scheduler event JSONL for offline analysis.
- 🗂️ Remote filesystem seeds -
DirectorySeedReaderandFileContentsSeedReadercan now use injectedfsspecfilesystems, enabling existing seed workflows against non-local storage. - 💻 More flexible CLI workflows - Python workflows can receive custom arguments after
--acrosscreate,preview,validate, andcheck-models. Create runs can also load typedRunConfigsettings from YAML. - 🚦 Async scheduler tuning and resilience - configure the concurrent row-group horizon, while deferred retries now block only dependent work and leave healthy model resources available.
- 🧩 Application-friendly logging - preserve application-managed logging, control automatic configuration per instance, and use
reset_logging()without disturbing foreign handlers.
Dev Notes
- 🔗 Pause, Inspect, Resume: Workflow Chaining as a Control Surface - demonstrates named workflow boundaries, human review gates, reusable intermediate artifacts, and downstream resume through a runnable document-review recipe.
- 🖼️ Image Generation for Multimodal Data Pipelines - explores controlled image generation across documents, products, traffic scenes, robotics, medical-style imagery, agriculture, and image editing, with accompanying runnable recipes.
What's Changed
- docs(agentic-ci): fix stale generator class name in engine.md by @github-actions[bot] in #780
- chore(agentic-ci): declare pydantic as direct dep of data-designer-engine by @github-actions[bot] in #752
- docs: enable Fern multi-source by @lbliii in #784
- chore(agentic-ci): declare pandas as direct dep of data-designer-engine by @github-actions[bot] in #705
- chore: Ensure weights sum to non-zero value by @mikeknep in #786
- feat: Support remote filesystem seeds by @mikeknep in #765
- fix: break health check import cycle by @andreatgretel in #781
- chore: make agentic CI review timeout advisory by @andreatgretel in #776
- chore(agentic-ci): declare jinja2 as direct dep of data-designer-engine by @github-actions[bot] in #785
- refactor: remove verified dead code by @nabinchha in #791
- chore: add nvidia-internal deploymentType to telemetry enum by @mckornfield in #769
- docs: add image generation dev note by @nabinchha in #777
- docs: publish recipes with devnotes by @andreatgretel in #794
- feat: chart generation throughput by @eric-tramel in #696
- ci: bump the all-actions group across 1 directory with 5 updates by @dependabot[bot] in #795
- fix: preserve preconfigured logging by @schultzjack in #755
- fix: coerce judge score drift by @schultzjack in #756
- fix: bump vulnerable dependencies by @johnnygreco in #803
- feat(cli): load per-run RunConfig YAML by @eric-tramel in #801
- fix(ci): reduce pull request runtime by @andreatgretel in #806
- chore(agentic-ci): declare pyarrow as a direct engine dependency by @github-actions[bot] in #797
- feat(cli): support Python workflow args by @eric-tramel in #799
- fix: handle terminal UI edge cases by @andreatgretel in #796
- feat: persist scheduler events to JSONL by @eric-tramel in #800
- fix: localize deferred row admission by @eric-tramel in #743
- fix(agentic-ci): improve dependency PR quality by @andreatgretel in #787
- feat: make async row-group horizon configurable by @eric-tramel in #745
- fix: restore Fern fallback live reload by @andreatgretel in #788
- feat: add OpenTelemetry metrics for create jobs by @eric-tramel in #802
- ci: bump the all-actions group with 2 updates by @dependabot[bot] in #812
- fix: bump vulnerable dependencies by @johnnygreco in #813
- test: make request timeout test deterministic by @andreatgretel in #814
- feat: add logging reset support by @nabinchha in #808
- docs: update token badge to 10T+ by @nabinchha in #811
- docs: dev note for workflow chaining by @andreatgretel in #775
- docs: add engine-native record selection plan by @nabinchha in #792
- docs: fix workflow chaining devnote date by @andreatgretel in #815
- chore(agentic-ci): declare pandas, pyarrow, pyyaml as direct interface deps by @github-actions[bot] in #816
- chore: bump cryptography dependency and remove upper bound by @mckornfield in #821
New Contributors
- @schultzjack made their first contribution in #755
Full Changelog: v0.7.0...v0.8.0