-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-3787][BUILD] Assembly jar name is wrong when we build with sbt omitting -Dhadoop.version #3046
Conversation
Test build #22675 has finished for PR 3046 at commit
|
Really not an SBT expert here, but this generally LGTM, thanks! |
@pwendell @ScrapCodes Can you review this PR? |
Test build #23140 has finished for PR 3046 at commit
|
…o fix-assembly-jarname-2
Test build #23550 has finished for PR 3046 at commit
|
Test build #24058 has finished for PR 3046 at commit
|
Test build #24837 has finished for PR 3046 at commit
|
Looks good - I'm going to merge this with a slight modification (adding a comment to explain whats going on). |
… omitting -Dhadoop.version This PR is another solution for When we build with sbt with profile for hadoop and without property for hadoop version like: sbt/sbt -Phadoop-2.2 assembly jar name is always used default version (1.0.4). When we build with maven with same condition for sbt, default version for each profile is used. For instance, if we build like: mvn -Phadoop-2.2 package jar name is used hadoop2.2.0 as a default version of hadoop-2.2. Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp> Closes #3046 from sarutak/fix-assembly-jarname-2 and squashes the following commits: 41ef90e [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into fix-assembly-jarname-2 50c8676 [Kousuke Saruta] Merge branch 'fix-assembly-jarname-2' of github.com:sarutak/spark into fix-assembly-jarname-2 52a1cd2 [Kousuke Saruta] Fixed comflicts dd30768 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into fix-assembly-jarname2 f1c90bb [Kousuke Saruta] Fixed SparkBuild.scala in order to read `hadoop.version` property from pom.xml af6b100 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into fix-assembly-jarname c81806b [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into fix-assembly-jarname ad1f96e [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into fix-assembly-jarname b2318eb [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into fix-assembly-jarname 5fc1259 [Kousuke Saruta] Fixed typo. eebbb7d [Kousuke Saruta] Fixed wrong jar name
This PR is another solution for When we build with sbt with profile for hadoop and without property for hadoop version like:
jar name is always used default version (1.0.4).
When we build with maven with same condition for sbt, default version for each profile is used.
For instance, if we build like:
jar name is used hadoop2.2.0 as a default version of hadoop-2.2.