feat: Data Streams #2877
Merged
feat: Data Streams #2877
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adding support for: odigos-io/ui-kit#79 --------- Signed-off-by: Ben Elferink <ben.elferink@icloud.com>
…feat/data-streams
…feat/data-streams
fix: update DestinationModal to include updateDestination prop chore: bump @odigos/ui-kit version to 0.0.29
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR introduces end-to-end support for Data Streams (grouping) across the UI, backend services, and GraphQL API.
- Adds a new “Choose Data Stream” UI page and integrates stream context into existing setup steps
- Implements service-layer logic for adding/removing workloads and destinations from named streams
- Extends GraphQL schema, models, and resolvers to expose DataStream operations
Reviewed Changes
Copilot reviewed 52 out of 52 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| frontend/webapp/app/(setup)/choose-stream/page.tsx | New page for selecting/creating data streams |
| frontend/webapp/app/(setup)/choose-sources/page.tsx | Updated Source step to use stream-aware form |
| frontend/webapp/app/(setup)/choose-destination/page.tsx | Updated Destination step to handle stream routing |
| frontend/webapp/app/(overview)/layout.tsx | Wired DataStream CRUD into the overview header actions menu |
| frontend/services/data_stream.go | New service logic for grouping sources/destinations by stream |
| frontend/services/sources.go | Updated source CRUD to include stream labels |
| frontend/services/destinations.go | Updated destination mapping to include DataStreamNames |
| frontend/graph/schema.graphqls | Added DataStream types, inputs, mutations, and Query field |
| frontend/graph/schema.resolvers.go | Implemented DataStreams Query, Update/Delete resolvers |
| frontend/graph/model/models_gen.go | Generated GraphQL model for DataStream |
| frontend/graph/conversions.go | Extended converter to include dataStreamNames on K8sActualSource |
| collector/processors/odigosurltemplateprocessor/README.md | Minor doc formatting fixes |
Comments suppressed due to low confidence (3)
frontend/graph/schema.graphqls:675
- [nitpick] The
idparameter here represents the existing stream name (oldStreamName). Consider renaming it (e.g.oldStreamNameorcurrentStreamName) to avoid confusion with other IDs.
updateDataStream(id: ID!, dataStream: DataStreamInput!): DataStream!
frontend/services/data_stream.go:1
- This file introduces substantial new data-stream logic (
DeleteDestinationOrRemoveStreamName,UpdateSourcesCurrentStreamName, etc.) but lacks accompanying unit tests. Consider adding tests to cover key code paths.
package services
collector/processors/odigosurltemplateprocessor/README.md:128
- The regex example is wrapped with extra spaces and backticks, and the description text mistakenly shows
SA*instead ofSA_. Please correct the code fence and placeholder text to accurately reflect the intended pattern.
+- only digits or special characters - `` ^[\d_\-!@#$%^&*()=+{}\[\]:;"'<>,.?/\\|`~]+$ `` -> `{id}` (`1234`, `123_456`, `0`)
…es webhook (#2908) This PR adds a default data-stream label to Source CRDs, and validates the existence of a label.
…feat/data-streams
RonFed
reviewed
May 26, 2025
Collaborator
There was a problem hiding this comment.
don't think this file should have any diff in this PR
Contributor
Author
There was a problem hiding this comment.
Not sure how that got there, removed
This PR bumps the UI-Kit with several bug fixes. In addition, this PR also fixes: - already instrumented sources not select in the list - sources get clean when editing the ds name
alonkeyval
approved these changes
May 28, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support for Data Streams / Grouping in the UI.
This feature is currently disabled - until we have the Odigos Routing Connector ready.