[SPARK-26788][yarn] Remove SchedulerExtensionService. #23839
Conversation
Since the yarn module is actually private to Spark, this interface was never actually "public". Since it has no use inside of Spark, let's avoid adding a yarn-specific extension that isn't public, and point any potential users are more general solutions (like using a SparkListener).
|
Test build #102515 has started for PR 23839 at commit |
felixcheung
left a comment
felixcheung
left a comment
There was a problem hiding this comment.
is there deprecation note on the yarn doc page?
|
Retest this please. |
I'm not sure it's even mentioned, but will check. If it's not, I'd rather just not call attention to it (it's never been mentioned in javadocs, for example). |
|
No mention of the config key or the class name in the documentation, so I think we're good. |
|
Test build #102556 has finished for PR 23839 at commit
|
|
Unrelated test failures... they're really weird, as if the YARN mini cluster stopped running containers for > 10 min. No idea what happened. |
|
retest this please |
|
Test build #102563 has finished for PR 23839 at commit
|
|
So lots of "approvals", but will anyone actually merge the change? |
|
Go for it, I dont' see any objections and your judgment is obviously solid in this areas. |
|
I know I can merge my own PRs, I'm just questioning this pattern that I see a lot where people approve PRs but nobody merges them. |
|
For me personally it's that I know you can merge them and maybe want to let you pull the trigger as you know best that you don't have anything else you want to do with it, and also maybe which branches it should back port too. I'll merge this one. |
|
(For non-committer PRs, I flag the email for followup in 1-3 days in Gmail, and yes I would then merge it, not just leave it approved indefinitely) |
|
Yup, I think in the same way of @srowen's. |
6 participants
Since the yarn module is actually private to Spark, this interface was never
actually "public". Since it has no use inside of Spark, let's avoid adding
a yarn-specific extension that isn't public, and point any potential users
are more general solutions (like using a SparkListener).