[SPARK-3201][YARN-Stable] Make the yarn-client support the extraJavaOptions

[SaintBacchus]

it's very inconvenient if we want to set the "-X" java opts in the process of "ExecutorLauncher".
So I add it in yarn-client mode.

[AmplabJenkins]

Can one of the admins verify this patch?

[vanzin]

Hi @hzw19900416,

I don't think this is the correct way to do this. Yarn should have separate options for the "client mode AM", and not reuse the driver settings, because the driver will invariably need a lot more resources than the client-mode AM.

Bug SPARK-1953 is the closes I could find to ask for this feature.

[vanzin]

SPARK-1507 is also related (maybe both could be merged into a "create separate options to control the client-mode AM resource allocation request" bug).

[SaintBacchus]

Hi @vanzin ,I agree with you about that "separate options for the 'client mode AM'" is the better way to do so.
And are you doing this job now?
If so, please leave the extra java opts for our users.
Nowadays, it's very inconvenient for me to use the remote debug of ExecutorLauncher

[vanzin]

I'm not working on that issue, so if you'd like to repurpose your patch for that, feel welcome to do so.