[SPARK-3264] Allow users to set executor Spark home in Mesos

[andrewor14]

The executors and the driver may not share the same Spark home. There is currently one way to set the executor side Spark home in Mesos, through setting spark.home. However, this is neither documented nor intuitive. This PR adds a more specific config spark.mesos.executor.home and exposes this to the user.

@liancheng @tnachen

[andrewor14]

test this please

[SparkQA]

QA tests have started for PR 2166 at commit b87965e.

• This patch merges cleanly.

[liancheng]

Should we mark spark.home as deprecated?

[SparkQA]

QA tests have finished for PR 2166 at commit b87965e.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[andrewor14]

spark.home is not documented in the first place, so I think it's fine to leave it

[liancheng]

Then it still would be good to mark it as deprecated in SparkContext.getSparkHome, so that other developers won't rely on it in the future.

[tnachen]

the patch looks good to me, I don't think this requires a cluster test.

[mattf]

lgtm, nice idea

i've been using rpm installed spark, which provides a single version and location on all nodes. however, this will make for a clear path to running multiple versions of spark on a mesos cluster.

[andrewor14]

Thanks, I'm merging this into master and 1.1.

@liancheng spark.home is actually used in a few other places too, though largely in tests. It would be a non-trivial to figure out a good way to deprecate this (unfortunately). I have actually attempted this in a prior PR (#1734) but to no avail. :( I prefer to address that separately.

@tnachen If you have the cycles, would you mind updating running-on-mesos.md to reflect this change? I would do it but I do not have enough expertise or experience in running Spark on Mesos.

@mattf Great to know this helps your use case.