Skip to content

Commit

Permalink
[MINOR][TEST] Expand spark-submit test to allow python2/3 executable
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?

This backports a tiny part of another change:
4bdfda9#diff-3c792ce7265b69b448a984caf629c96bR161
... which just works around the possibility that the local python interpreter is 'python3' or 'python2' when running the spark-submit tests.

I'd like to backport to 2.3 too.

This otherwise prevents this test from passing on my mac, though I have a custom install with brew. But may affect others.

## How was this patch tested?

Existing tests.

Closes #24407 from srowen/Python23check.

Authored-by: Sean Owen <sean.owen@databricks.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
(cherry picked from commit 7f64963)
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
srowen authored and dongjoon-hyun committed Apr 18, 2019
1 parent c1e5824 commit 233646c
Showing 1 changed file with 1 addition and 1 deletion.
Expand Up @@ -148,7 +148,7 @@ public void testPySparkLauncher() throws Exception {

Map<String, String> env = new HashMap<>();
List<String> cmd = buildCommand(sparkSubmitArgs, env);
assertEquals("python", cmd.get(cmd.size() - 1));
assertTrue(Arrays.asList("python", "python2", "python3").contains(cmd.get(cmd.size() - 1)));
assertEquals(
String.format("\"%s\" \"foo\" \"%s\" \"bar\" \"%s\"",
parser.MASTER, parser.DEPLOY_MODE, SparkSubmitCommandBuilder.PYSPARK_SHELL_RESOURCE),
Expand Down

0 comments on commit 233646c

Please sign in to comment.