Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-2558][DOCS] Add --queue example to YARN doc #2218

Closed
wants to merge 2 commits into from

Conversation

kramimus
Copy link
Contributor

Put original YARN queue spark-submit arg description in
running-on-yarn html table and example command line

Put original YARN queue spark-submit arg description in
running-on-yarn html table and example command line
@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@ScrapCodes
Copy link
Member

I just checked, --queue is a valid option in spark submit. And thanks for updating the docs.

Looks good.

@tgravescs
Copy link
Contributor

Jenkins, test this please

@SparkQA
Copy link

SparkQA commented Sep 3, 2014

QA tests have started for PR 2218 at commit f8cda0d.

  • This patch merges cleanly.

@@ -125,6 +125,13 @@ Most of the configs are the same for Spark on YARN as for other deployment modes
the environment of the executor launcher.
</td>
</tr>
<tr>
<td><code>spark.yarn.queue</code></td>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This config is only supported in yarn client mode right now. Its more because of backwards compatibility and being able to pass it around in client mode then anything. In general you can use the spark-submit --queue option. If we want to make it available as a config then we should make it available in both modes.

So I would prefer to leave this config undocumented and tell people to use the --queue option until support for both modes is added.

@SparkQA
Copy link

SparkQA commented Sep 3, 2014

QA tests have finished for PR 2218 at commit f8cda0d.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • protected trait YarnAllocateResponse

@SparkQA
Copy link

SparkQA commented Sep 5, 2014

Can one of the admins verify this patch?

@andrewor14
Copy link
Contributor

ok to test

@andrewor14
Copy link
Contributor

The title of this PR is now a little outdated. Can you update it to include --queue instead of the config?

@SparkQA
Copy link

SparkQA commented Sep 12, 2014

QA tests have started for PR 2218 at commit 4b5d808.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 12, 2014

QA tests have finished for PR 2218 at commit 4b5d808.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@kramimus kramimus changed the title [SPARK-2558][DOCS] Add spark.yarn.queue description to YARN doc [SPARK-2558][DOCS] Add --queue example to YARN doc Sep 12, 2014
@tgravescs
Copy link
Contributor

+1, thanks @kramimus!

@asfgit asfgit closed this in f116f76 Sep 12, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
6 participants