Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for non-pk steaming read in spark #188

Merged
merged 10 commits into from
Mar 28, 2023
Merged

add support for non-pk steaming read in spark #188

merged 10 commits into from
Mar 28, 2023

Conversation

moresun
Copy link
Contributor

@moresun moresun commented Mar 28, 2023

1、 support non-pk streaming read in spark
Close #186

2、 support state store for lakesoul sink.
Close #187

@dmetasoul01 dmetasoul01 self-assigned this Mar 28, 2023
@dmetasoul01 dmetasoul01 added bug Something isn't working spark spark support into lakesoul labels Mar 28, 2023
@dmetasoul01 dmetasoul01 added this to In progress in Stream Pipeline via automation Mar 28, 2023
@dmetasoul01 dmetasoul01 merged commit d26a34d into main Mar 28, 2023
Stream Pipeline automation moved this from In progress to Done Mar 28, 2023
@dmetasoul01 dmetasoul01 deleted the nonpkSteam branch March 28, 2023 10:08
lypnaruto pushed a commit that referenced this pull request Apr 4, 2023
* spark operator pipeline

* add stream read for non-pk

* support state store when use write stream format lakesoul

* Revert "spark operator pipeline"

This reverts commit 6bc18dd.

* Delete SparkPipeLine.scala

* fix style

---------

Co-authored-by: sunmaosen <738159199.qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working spark spark support into lakesoul
Projects
Development

Successfully merging this pull request may close these issues.

[Spark] add support state store for lakesoul sink [Spark] Add support for non-pk streaming read
2 participants