[ZEPPELIN-4522]. Support multiple sql statements for SparkSqlInterpreter#3579
[ZEPPELIN-4522]. Support multiple sql statements for SparkSqlInterpreter#3579zjffdu wants to merge 4 commits intoapache:masterfrom
Conversation
| sc.setLocalProperty("spark.scheduler.pool", context.getLocalProperties().get("pool")); | ||
| sc.setJobGroup(Utils.buildJobGroupId(context), Utils.buildJobDesc(context), false); |
There was a problem hiding this comment.
Does it makes sense to move this outside of the loop? Or at least move some things like, Utils.buildJobGroupId(context), Utils.buildJobDesc(context), ...
| sc.setJobGroup(Utils.buildJobGroupId(context), Utils.buildJobDesc(context), false); | ||
|
|
||
| try { | ||
| Method method = sqlc.getClass().getMethod("sql", String.class); |
| "" + sparkInterpreter.getZeppelinContext().getMaxResult())); | ||
| String result = sparkInterpreter.getZeppelinContext().showData( | ||
| method.invoke(sqlc, sql), maxResult); | ||
| sc.clearJobGroup(); |
There was a problem hiding this comment.
Make sense, and I think we didn't reset the local property(job group is just a local property) in other places, created ZEPPELIN-4523 to track it.
| sc.clearJobGroup(); | ||
| builder.append(result); | ||
| } catch (Exception e) { | ||
| if (Boolean.parseBoolean(getProperty("zeppelin.spark.sql.stacktrace"))) { |
There was a problem hiding this comment.
Maybe output the SQL clause that caused error as part of the error, in addition to exception? I don't remember, if Spark SQL always shows the original SQL, or not.
There was a problem hiding this comment.
Make sense, fixed it
| method.invoke(sqlc, st), maxResult); | ||
| sc.clearJobGroup(); | ||
| return new InterpreterResult(Code.SUCCESS, msg); | ||
| for (String sql : sqls) { |
There was a problem hiding this comment.
what will be in the builder if we'll have empty list of SQLs?
There was a problem hiding this comment.
Empty will be returned if there's no sql, I just added one test case which only 2 sql comments in the paragraph text. Thanks for the review @alexott
alexott
left a comment
alexott
left a comment
There was a problem hiding this comment.
LGTM, except only question on the empty list of SQLs...
zjffdu
force-pushed
the
ZEPPELIN-4522
branch
4 times, most recently
from
fc38622 to
8fee4c3
Compare
zjffdu
force-pushed
the
ZEPPELIN-4522
branch
from
8fee4c3 to
eda5736
Compare
2 participants
What is this PR for?
Use the SqlSplitter in
zeppelin-interpreterto split sql and execute in SparkSqlInterpreter. Nothing changes for the previous single sql statement paragraph. But just multiple result will be displayed for multiple sql statements.What type of PR is it?
[Feature]
Todos
What is the Jira issue?
How should this be tested?
Screenshots (if appropriate)
Questions: