[ZEPPELIN-4692]. zeppelin pyspark doesn't print java output #3696
Conversation
zjffdu
force-pushed
the
ZEPPELIN-4692
branch
5 times, most recently
from
38bf218
to
57e6c16
Compare
| System.setOut(new PrintStream(context.out)); | ||
| System.setErr(new PrintStream(context.out)); | ||
| Utils.printDeprecateMessage(sparkInterpreter.getSparkVersion(), context, properties); | ||
| return super.interpret(st, context); |
There was a problem hiding this comment.
Shouldn't we set pool, job group, etc., like in IPython interpreter?
There was a problem hiding this comment.
There was a problem hiding this comment.
But it's not called anywhere in the context of the PySparkInterpreter - in the whole source code I see only similar function in PythonInterpreter but it's called inside it...
There was a problem hiding this comment.
preCall is a hook before running user code, PySparkInterpreter extends PythonInterpreter. https://github.com/apache/zeppelin/blob/master/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java#L347
https://github.com/apache/zeppelin/blob/master/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java#L384
There was a problem hiding this comment.
Thank you Jeff! I somehow missed the @Override :-(
|
Thanks for the review @alexott will merge if no more comment |
### What is this PR for? The root cause of this issue is that we didn't redirect java output to interpreter output. This PR fix it via redirect java output before interpreting python code in both PySparkInterpreter & IPySparkInterpreter. Unit test is also added to verify this feature. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4692 ### How should this be tested? * Unit test is added, also manually tested it ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? NO Author: Jeff Zhang <zjffdu@apache.org> Closes #3696 from zjffdu/ZEPPELIN-4692 and squashes the following commits: e1a5ead [Jeff Zhang] [ZEPPELIN-4692]. zeppelin pyspark doesn't print java output (cherry picked from commit 6cdcc5b) Signed-off-by: Jeff Zhang <zjffdu@apache.org>
What is this PR for?
The root cause of this issue is that we didn't redirect java output to interpreter output. This PR fix it via redirect java output before interpreting python code in both PySparkInterpreter & IPySparkInterpreter. Unit test is also added to verify this feature.
What type of PR is it?
[Bug Fix]
Todos
What is the Jira issue?
How should this be tested?
Screenshots (if appropriate)
Questions: