SPARK-1650: Correctly identify maven project version

[rahulsinghaliitd]

Better account for various side-effect outputs while executing
"mvn help:evaluate -Dexpression=project.version"

[AmplabJenkins]

Can one of the admins verify this patch?

[pwendell]

Hm - would be it be less brittle to do this?

VERSION=$(mvn help:evaluate -Dexpression=project.version | grep -v "INFO" | tail -n 1)

That way we don't make any assumptions about the messages that can occur before the version is printed. Otherwise over time we'll have to keep writing rules to match various types of input.

[pwendell]

We could also tighten this up a bit by only asking the version for core:

VERSION=$(mvn help:evaluate -Dexpression=project.version -pl core | grep -v "INFO" | tail -n 1)

[rahulsinghaliitd]

1. Using "tail -n 1" works for all my error cases, will make the change.
2. Wouldn't using the umbrella project be more accurate (considering the fact that we use a fat jar)?

[pwendell]

sure, if you want it's okay to just do the parent. They all have the same version, so it's fine either way.

[pwendell]

Good catch on this - made a small suggestion.

[rahulsinghaliitd]

@pwendell Thanks, I had recently used a maven build and encountered this problem. I even had a local commit to add the maven build to make-distribution.sh but you beat me to it. :)

[rahulsinghaliitd]

I am noob to GitHub and Spark upstream procedures, so confused about what happens when a pull request is accepted.

1. Is the PR merged or rebased on the destination branch?
2. And as in this cases, a second commit is required. Now should I just push a second commit and the two commits will be squashed when the pull request is accepted, or do I need to squash them? If yes how (create a new PR or force modify this one)
3. Lastly, will I need to cherry-pick the commit to 1.0.0. branch or will the core developer who accepts this PR will do it?

[pwendell]

You can just make another commit and then push it to your branch. When we merge the pull request we actually bundle up all of your commits and squash them into a single commit.

We will deal with cherry-picking at the time of commit, you don't need to do that yourself.

[pwendell]

@rahulsinghaliitd I actually just merged this and made the suggested change when merging it. Thanks!

[rahulsinghaliitd]

Thanks @pwendell , I was waiting because I think there maybe more minor problems

1. The current test to detect if "mvn" is installed or not, is not correct.
2. If maven command fails (due to whatever reason), we would end up picking some random error string as the version.