This repository has been archived by the owner on Jan 23, 2023. It is now read-only.
Factor out the streaming SQL transformations into a dedicated Processor #21
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Split the Processor semantics into 3 general purpose processor types - Time Window Aggregation type - existing multi-binder + sidecar UDF (no SQL anymore) - FSQL type - factor out the existing streaming SQL into dedicated processor type which is deployed in a separate POD. - SCS type - Spring Cloud Stream support including all OOTB SCDF apps. Update the Processor CRD to reflect the new Processor semantics. Refactor the Processor K8s Controller the allow configuring and deploying the above 3 processor types. Samples - refactor all samples to reflect the new Processor(s) semantics. - added SCS sample Docs - update samples docs - new SCS sample doc - add Stream Schema page - add architecture sub-section - basic SR concepts info added to the Usage section Add StatefulSet processor support - add processor.spec.replicas:int property to the Processor CRD - used to configure the Deployment or StatefulSet's replica count. - in addition to the Deployment now processor can be deployed as StatefulSet as well. - use the processor.spec.attributes.partitionedInput=true to enable StatefulSet. - Refactor the ProcessorReconciler to support both deployment types. - add streaming-pipeline-ticktock-partitioned.yaml to show how to use manually partitioning (SCS only). - minor doc improvements. Upgrade multibinder Boot to 2.7.0. Add multibinder cloudevents dependencies. Regenerate CDRs java code. Resolves #20
tzolov
force-pushed
the
green-blue-processor-split
branch
from
3bab66b
to
28736bd
Compare
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #20