[SPARK-13166][SQL] Remove DataStreamReader/Writer #11062

rxin · 2016-02-03T22:17:07Z

They seem redundant and we can simply use DataFrameReader/Writer. The new usage looks like:

val df = sqlContext.read.stream("...")
val handle = df.write.stream("...")
handle.stop()

tdas · 2016-02-03T22:24:41Z

sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala

@@ -184,6 +185,34 @@ final class DataFrameWriter private[sql](df: DataFrame) {
  }

  /**
+   * Starts the execution of the streaming query, which will continually output results to the given
+   * path as new data arrives.  The returned [[ContinuousQuery]] object can be used to interact with


extra space

SparkQA · 2016-02-04T00:06:22Z

Test build #50693 has finished for PR 11062 at commit 7528abb.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

marmbrus · 2016-02-04T00:09:33Z

LGTM, merging to master.

…derWriterSuite A follow up PR for #11062 because it didn't rename the test suite. Author: Shixiong Zhu <shixiong@databricks.com> Closes #11096 from zsxwing/rename.

[SPARK-13166][SQL] Remove DataStreamReader/Writer

6a397dc

tdas reviewed Feb 3, 2016
View reviewed changes

whitespace change

7528abb

asfgit closed this in 915a753 Feb 4, 2016

zsxwing mentioned this pull request Feb 5, 2016

[SPARK-13166][SQL]Rename DataStreamReaderWriterSuite to DataFrameReaderWriterSuite #11096

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARK-13166][SQL] Remove DataStreamReader/Writer #11062

[SPARK-13166][SQL] Remove DataStreamReader/Writer #11062

rxin commented Feb 3, 2016

tdas Feb 3, 2016

SparkQA commented Feb 4, 2016

marmbrus commented Feb 4, 2016

[SPARK-13166][SQL] Remove DataStreamReader/Writer #11062

[SPARK-13166][SQL] Remove DataStreamReader/Writer #11062

Conversation

rxin commented Feb 3, 2016

tdas Feb 3, 2016

Choose a reason for hiding this comment

SparkQA commented Feb 4, 2016

marmbrus commented Feb 4, 2016