Skip to content

fix issue with spark-submit with java#18878

Closed
brandonJY wants to merge 1 commit intoapache:masterfrom
brandonJY:patch-1
Closed

fix issue with spark-submit with java#18878
brandonJY wants to merge 1 commit intoapache:masterfrom
brandonJY:patch-1

Conversation

@brandonJY
Copy link
Contributor

What changes were proposed in this pull request?

remove quote when specifying class using spark-submit in command line.
Specifying java class with quote does not work. Suggest to remove it.

How was this patch tested?

Built and the code for this application, ran jekyll locally per docs/README.md.

remove quote when specify class using spark-submit in command line.
@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@srowen
Copy link
Member

srowen commented Aug 8, 2017

Why wouldn't it work? This is interpreted by bash

@brandonJY
Copy link
Contributor Author

Running spark 2.2.0 standalone mode in Ubuntu 16.04 LTS. It interprets classpath with the quote. E.g. for classname=foo, it interprets it as "foo"
Probably it is just a environment issue?

@srowen
Copy link
Member

srowen commented Aug 8, 2017

It works fine for me in, for example:

spark2-submit --class "org.apache.spark.examples.SparkPi" --deploy-mode client --master yarn /opt/cloudera/parcels/SPARK2/lib/spark2/examples/jars/spark-examples_2.11-2.2.0.cloudera1.jar

The thing is the quotes never even make it to the submit script, right? bash interprets them. That's why I can't see how this would be an issue. What's your exact command line?

@HyukjinKwon
Copy link
Member

HyukjinKwon commented Aug 8, 2017

it works fine for me too. Mine was mac.

@brandonJY
Copy link
Contributor Author

Never mind. I found it is actually something in my config. Silly mistake. Sorry for the trouble. Closing this PR now.

@brandonJY brandonJY closed this Aug 9, 2017
@HyukjinKwon
Copy link
Member

Thanks for confirmation.

@brandonJY brandonJY deleted the patch-1 branch January 18, 2018 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants