New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
#84: Added Exasol SQL statement builder #88
Conversation
Codecov Report
@@ Coverage Diff @@
## main #88 +/- ##
==========================================
+ Coverage 94.59% 95.76% +1.17%
==========================================
Files 10 11 +1
Lines 407 425 +18
Branches 53 48 -5
==========================================
+ Hits 385 407 +22
+ Misses 22 18 -4
Continue to review full report at Codecov.
|
assert(result === Set((1, "Berlin"))) | ||
test("returns dataframe with greater than filter") { | ||
val df = getDataFrame().filter($"id" > 2).collect() | ||
assert(df.map(r => r.getString(1)) === Seq("Portugal")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe you should also test that the predicate was really pushed down. (For example by checking the database logs)
(just an idea; not sure if required / makes sense)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it is tested in enricher class.
This is before sending the query to the Exasol.
For example:
val df = spark.read
.option("query", "SELECT * FROM people")
.select("name", "surname")
.filter($"id" > 10)
Should send the modified query to Exasol with column selection and where clauses:
SELECT "name", "surname" FROM people WHERE id > 10
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
Co-authored-by: jakobbraun <jakob.braun@posteo.de>
Fixes #14
Fixes #39
Fixes #84