Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-20496][SS] Bug in KafkaWriter Looks at Unanalyzed Plans #17787

Closed
wants to merge 6 commits into from

Conversation

bllchmbrs
Copy link
Contributor

@bllchmbrs bllchmbrs commented Apr 27, 2017

What changes were proposed in this pull request?

This now asserts that a plan has been analyzed before reading the schema.

How was this patch tested?

Existing Tests.

  • New Test Coming Soon

Please review http://spark.apache.org/contributing.html before opening a pull request.

@bllchmbrs bllchmbrs changed the title [SS] Bug in KafkaWriter Looks at Unanalyzed Plans [SPARK-20496][SS] Bug in KafkaWriter Looks at Unanalyzed Plans Apr 27, 2017
@SparkQA
Copy link

SparkQA commented Apr 27, 2017

Test build #76236 has finished for PR 17787 at commit 703513c.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

/* No topic field or topic option */
var writer: StreamingQuery = null
var ex: Exception = null
try {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need for try block

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

}
} finally {
}
assert(ex.getMessage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the error message when you undo your changes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"'writeStream' can be called only on streaming Dataset/DataFrame;"

@SparkQA
Copy link

SparkQA commented Apr 27, 2017

Test build #76239 has finished for PR 17787 at commit ec9e091.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 27, 2017

Test build #76241 has finished for PR 17787 at commit 7f5ecb7.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 27, 2017

Test build #76247 has finished for PR 17787 at commit a3c1556.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 27, 2017

Test build #76248 has finished for PR 17787 at commit 4a42642.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@bllchmbrs bllchmbrs closed this Apr 28, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants