rust(feat): Adds FlowDescriptor and FlowBuilder to improve performance#396
Merged
Conversation
nathan-sift
requested changes
Nov 21, 2025
nathan-sift
approved these changes
Nov 22, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Creates two new public structs, the
FlowDescriptorand theFlowBuilder. These are intended to abstract the completeFlowConfigandChannelConfigto the minimum required for ingesting data once registered to Sift.Internally, the
FlowDescriptorwill have an index map of somekeythat maps to the index of that channel within the flow; it will also hold the data type of that field to improve validation while building a flow.The
FlowDescriptorcan be created manually, or by converting aFlowConfig. Here, we are using strings as the "key", though the key can be arbitrary, so long as it is unique per channel. For maximum performance, the returned indices for each registered channel should then be used when setting the value in order to avoid a hash lookup when building the flow below:Then to build a flow using the
FlowBuilder, you can either use the "key" to set the value, or the index returned when registering the channel in theFlowDescriptor: