[SPARK-4556] [BUILD] binary distribution assembly can't run in local mode

[srowen]

Add note on building a runnable distribution with make-distribution.sh

[AmplabJenkins]

Merged build triggered.

[AmplabJenkins]

Merged build started.

[SparkQA]

Test build #32822 has started for PR 6186 at commit 9fa7883.

[busbey]

add a closing "for more information on usage run ./make-distribution.sh --help"

[AmplabJenkins]

Merged build triggered.

[AmplabJenkins]

Merged build started.

[SparkQA]

Test build #32824 has started for PR 6186 at commit 4002966.

[SparkQA]

Test build #32822 has finished for PR 6186 at commit 9fa7883.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Merged build finished. Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32822/
Test PASSed.

[SparkQA]

Test build #32824 has finished for PR 6186 at commit 4002966.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Merged build finished. Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32824/
Test PASSed.

[andrewor14]

@pwendell do we want to document the make-distribution script explicitly? I remember there's some discussion on that.

[pwendell]

Yeah I think so - we've had this around for a long time and I use it heavily as part of releases. I've actually made some effort to support backwards compatibility in this script over the years, so IMO it's something I'm prepared to keep supporting.

[vanzin]

This is great, but should we also use the opportunity to get rid of that "bigtop-dist" profile altogether? It generates broken artifacts, as shown in the bug.

[busbey]

Does that profile get used by BigTop for packaging spark downstream?

[vanzin]

Does that profile get used by BigTop for packaging spark downstream?

Good question; but if they use it, they must have a lot of extra code to deal with the broken tarball it generates. But yes, maybe better to leave that for a separate change and give the bigtop guys a heads up.