-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-26649][SS] Add DSv2 noop sink #23631
Conversation
Test build #101596 has finished for PR 23631 at commit
|
Looks clean and reasonable to me. |
Could you also add a simple test to make sure a query can be restarted if a checkpoint location is provided? Otherwise LGTM. |
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/noop/NoopDataSource.scala
Show resolved
Hide resolved
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/noop/NoopStreamSuite.scala
Outdated
Show resolved
Hide resolved
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/noop/NoopStreamSuite.scala
Outdated
Show resolved
Hide resolved
Test build #101635 has finished for PR 23631 at commit
|
Test build #101636 has finished for PR 23631 at commit
|
Test build #101649 has finished for PR 23631 at commit
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
+1, LGTM. Merged to master. |
Thank you, @gaborgsomogyi , @zsxwing , @MaxGekk , @HeartSaVioR . |
Thank you guys! |
## What changes were proposed in this pull request? Noop data source for batch was added in [apache#23471](apache#23471). In this PR I've added the streaming part. ## How was this patch tested? Additional unit tests. Closes apache#23631 from gaborgsomogyi/SPARK-26649. Authored-by: Gabor Somogyi <gabor.g.somogyi@gmail.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
What changes were proposed in this pull request?
Noop data source for batch was added in #23471.
In this PR I've added the streaming part.
How was this patch tested?
Additional unit tests.