docs: expand workflow chaining guidance

[andreatnvidia]

📋 Summary

Updates the workflow chaining docs to reflect newer stage-boundary features, including on_success, resume-aware callback versioning, and row-count changes between stages. The added guidance shows how users can shrink, grow, or custom-upsample data at workflow boundaries instead of resizing a dataset mid-stage.

🔗 Related Issue

N/A

🔄 Changes

• Add a Postprocessing hooks section to the workflow chaining concept page.
• Show a disagreement-filter callback that reads stage parquet output, writes a callback output dataset, and returns it for downstream seeding.
• Clarify when to use output_processors versus on_success and how callback versioning/empty outputs interact with resume.
• Add a Changing row counts between stages section covering shrink/filter and grow/explode patterns.
• Explain ordered seed cycling when a downstream stage requests more rows than the upstream selected output contains.
• Add a custom upsampling callback example that repeats persona rows and adds variant_id for downstream prompt variation.

🧪 Testing

• make check-fern-docs-locally
• .venv/bin/ruff check --fix .
• .venv/bin/ruff format .
• Unit tests added/updated: N/A, docs-only change
• E2E tests added/updated: N/A, docs-only change
• make test passes: N/A, docs-only change

✅ Checklist

• Follows commit message conventions
• Commits are signed off (DCO)
• Architecture docs updated (if applicable): N/A, concept docs only

[github-actions]

Fern preview: https://nvidia-preview-pr-771.docs.buildwithfern.com/nemo/datadesigner

Fern previews include the docs-website version archive with PR changes synced into latest. Notebook tutorials are rendered without execution outputs in previews.

[greptile-apps]

Greptile Summary

Expands the workflow-chaining concept page with two new sections covering postprocessing hooks (on_success callbacks with resume versioning and empty-output handling) and row-count changes at stage boundaries (shrink via filtering, grow via ordered seed cycling, and a custom upsampling callback).

• Adds Postprocessing hooks with a keep_disagreements filter callback, explaining on_success_version as a resume-identity tag and allow_empty=True for zero-row stages.
• Adds Changing row counts between stages with both an implicit cycling example and a repeat_each_persona custom-upsampling callback that assigns variant_id to distinguish repeated rows.

Confidence Score: 5/5

Documentation-only change; no production code is modified.

Both callback examples are correct: the pandas operations in repeat_each_persona produce the described 1,000-row output with valid variant_id values 0–9, and the keep_disagreements filter callback correctly writes a subset parquet directory and returns it for downstream seeding. All API parameters shown are consistent with the existing page. No logic errors or misleading claims were found.

No files require special attention.

Important Files Changed

Filename	Overview
fern/versions/latest/pages/concepts/workflow-chaining.mdx	Adds two new sections — Postprocessing hooks and Changing row counts between stages — with working Python callback examples. The pandas operations and API usage are consistent with the rest of the page; no logic errors found.

_{Reviews (3): Last reviewed commit: "Merge branch 'main' into andreatgretel/d..." | Re-trigger Greptile}

[johnnygreco]

Thanks for putting this together, @andreatgretel!

Summary

This expands the workflow chaining concept page with guidance for on_success callbacks, resume-aware callback versioning, empty callback outputs, and row-count changes between stages. The new docs match the PR intent and line up with the workflow implementation and existing interface tests I checked.

Findings

No findings.

What Looks Good

• The new sections draw a useful distinction between output_processors and on_success, which should help users pick the right stage-boundary tool.
• The row-count guidance matches the workflow behavior: downstream stages inherit the selected upstream output count unless num_records is set explicitly.
• The examples write callback outputs under the stage artifact directory and pair those paths with on_success_version, which is a clear, resume-friendly pattern.

Verdict

Ship it - ready to merge as-is.

---

This review was generated by an AI assistant.