Manage data pipelines: reference clarity, overview links, CLI output examples#4904
Merged
shannonbradshaw merged 2 commits intoviamrobotics:new-docs-sitefrom Apr 13, 2026
Conversation
…ples - reference.md: add a CLI flag column to the Pipeline configuration fields table with an intro explaining the fields are set by the listed CLI flags and SDK parameters. Split the Data source types table so the primary table covers CLI usage and description, with SDK constants in a second table. Replace the confusing "(query only)" label with "not valid on create" and a paragraph explaining that pipeline_sink is a query-time data source type, not a create-time one. Add a CLI label column to the Run statuses table and note that SDK methods return the enum while the CLI prints the friendly label. Promote the "deleting a pipeline deletes the sink" warning to a caution alert and link to the manage page. - overview.md: link the "how pipelines work" list items to examples and the cron schedule / data source types references so readers hit concrete content instead of abstract descriptions. - query-results.md: expand the intro to reference Create a pipeline, explain that the pipeline ID is how you target the sink, and show the viam datapipelines list / describe lookup commands up front. - manage-pipelines.md: add example output for `viam datapipelines list` and `viam datapipelines describe` (verified against rdk/cli/datapipelines.go:45-72 and 171-228). Note that an empty list output means no pipelines in the org, which is not an error. Add a CLI label column to the local Run statuses table and note the SDK-vs-CLI label difference. Promote the "deleting a pipeline" warning to a caution alert. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
✅ Deploy Preview for viam-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
There was a problem hiding this comment.
Pull request overview
Documentation-focused update to the Data Pipelines “Manage data pipelines” section to reduce ambiguity in references and make the CLI experience clearer via concrete output examples and cross-links.
Changes:
- Clarifies pipeline configuration/reference tables by adding CLI flag mappings, splitting wide tables, and adding CLI labels for run statuses.
- Adds concrete cross-links from the overview to relevant reference/examples sections.
- Expands “query results” and “manage pipelines” pages with CLI commands and example outputs (list/describe), plus improved delete warnings.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| docs/data/pipelines/reference.md | Adds CLI flag mappings to config fields; refactors data source/run status tables; upgrades delete warning to caution alert with link. |
| docs/data/pipelines/overview.md | Adds direct links to examples and reference anchors for the “how pipelines work” bullets. |
| docs/data/pipelines/query-results.md | Rewrites intro to emphasize pipeline ID and adds CLI commands to retrieve it, linking to manage page. |
| docs/data/pipelines/manage-pipelines.md | Adds CLI example outputs for list/describe, adds run status CLI labels, and upgrades delete warning to a caution alert. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| | `schedule` | string | Yes | `--schedule` | Cron expression in UTC. Determines both when the pipeline runs and the query time window. See [Cron schedule](#cron-schedule). | | ||
| | `mql_binary` | array | Yes | `--mql` or `--mql-path` | MQL aggregation pipeline as an array of stage objects. See [Supported MQL operators](/data/reference/#supported-mql-operators). | | ||
| | `enable_backfill` | bool | Yes | `--enable-backfill` | Whether to process historical time windows. See [Backfill behavior](#backfill-behavior). | | ||
| | `data_source_type` | enum | No | `--data-source-type` | Data source to query. Default: `standard`. See [Data source types](#data-source-types). | |
Comment on lines
+47
to
+51
| | Type | CLI `--data-source-type` value | Description | | ||
| | ------------- | ------------------------------ | ------------------------------------------------------------------------------------------------- | | ||
| | Standard | `standard` | The raw `readings` collection containing all historical tabular data. Default for new pipelines. | | ||
| | Hot storage | `hotstorage` | The [hot data store](/data/hot-data-store/). A rolling window of recent data, with lower latency. | | ||
| | Pipeline sink | not valid on create | The output of another pipeline. Used when querying results, not when creating a pipeline. | |
Comment on lines
+22
to
+27
| Example output (one line per pipeline, indented by a leading tab in the actual output): | ||
|
|
||
| ```text | ||
| hourly-temp-avg (ID: 64f3a1b2c4d5e6f7a8b9c0d1) [Enabled] [Data Source Type: Standard] | ||
| daily-summary (ID: 64f3a1b2c4d5e6f7a8b9c0d2) [Disabled] [Data Source Type: Hot Storage] | ||
| ``` |
- reference.md: mark data_source_type as Required=Yes to align with create-a-pipeline.md and with the CLI's actual behavior. Verified against rdk/cli/datapipelines.go:99 + 328-342, where dataSourceTypeToProto errors on empty string, making the flag effectively required even though the CLI definition does not set Required: true. - reference.md: put the actual pipeline_sink value string in the Data source types table instead of "not valid on create", and move the create-time restriction into the description so the column contains values consistently. - manage-pipelines.md: drop the "indented by a leading tab" prose from the list example output. The rendered example uses spaces and mentioning tabs here just confuses readers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR B from the Manage Data section review. Cleans up the data pipelines subsection based on reviewer feedback about ambiguous reference wording and missing CLI output examples.
`reference.md`
`overview.md`
`query-results.md`
`manage-pipelines.md`
Test plan
🤖 Generated with Claude Code