Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 APIs #20279

Closed
wants to merge 7 commits into from

Conversation

brkyvz
Copy link
Contributor

@brkyvz brkyvz commented Jan 16, 2018

What changes were proposed in this pull request?

This PR migrates the MemoryStream to DataSourceV2 APIs.

How was this patch tested?

All existing unit tests in streaming.

@SparkQA
Copy link

SparkQA commented Jan 16, 2018

Test build #86192 has finished for PR 20279 at commit 7a0b564.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • class OneHotEncoderEstimator(JavaEstimator, HasInputCols, HasOutputCols, HasHandleInvalid,
  • class OneHotEncoderModel(JavaModel, JavaMLReadable, JavaMLWritable):
  • class HasOutputCols(Params):
  • class DataSourceRDDPartition[T : ClassTag](val index: Int, val readTask: ReadTask[T])
  • class DataSourceRDD[T: ClassTag](

@SparkQA
Copy link

SparkQA commented Jan 16, 2018

Test build #86190 has finished for PR 20279 at commit 50a541b.

  • This patch fails Spark unit tests.
  • This patch does not merge cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jan 17, 2018

Test build #86206 has finished for PR 20279 at commit a81c2ec.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@brkyvz
Copy link
Contributor Author

brkyvz commented Jan 31, 2018

Closed in favor of #20445

@brkyvz brkyvz closed this Jan 31, 2018
@SparkQA
Copy link

SparkQA commented Jan 31, 2018

Test build #86854 has finished for PR 20279 at commit a81c2ec.

  • This patch fails Spark unit tests.
  • This patch does not merge cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jan 31, 2018

Test build #86853 has finished for PR 20279 at commit 9f6c6b9.

  • This patch fails Spark unit tests.
  • This patch does not merge cleanly.
  • This patch adds no public classes.

@brkyvz brkyvz deleted the memv2 branch February 3, 2019 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants