New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-20129][Core] JavaSparkContext should use SparkContext.getOrCreate #20347
Conversation
Pulling functionality from apache spark
pull latest from apache spark
Pulling functionality from apache spark
Pulling functionality from apache spark
pull request from apache/master
pull latest from apache spark
pull latest from apache spark
Pull apache spark
pull latest apache spark
Apache spark pull latest
Test build #86456 has finished for PR 20347 at commit
|
Can you please explain why do we need to change to |
@mengxr suggested this in the JIRA originally -- what was the reasoning? It makes some sense, but so does leaving the current behavior, where a constructor calls a constructor. It's a behavior change, albeit a slight one. |
Using |
Yes, you can already get the new semantics here with Yes, probably better to add a new method, or else, decide that it's not worth a new API method just as a shortcut for the above. Maybe that's the right conclusion, unless @mengxr comes back with a particular reason to change the behavior slightly. |
My major concern is that, if there is a existing On the other hand, the following comment copyed from the comment of the class
If that is the case, there should be no active |
@rekhajoshm I think maybe the right resolution here is to do nothing. I haven't heard @mengxr on his old JIRA to make this change. Thank you for chasing down open JIRAs like this of course. |
Thank you @srowen I admire you for doing what you do over all the jira/PR's I have studied, and followed up. |
What changes were proposed in this pull request?
Using SparkContext getOrCreate() instead of recreating new sc in JavaSparkContext.
How was this patch tested?
Existing tests