New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAM-1008] Modify jenkins-test script to support Java 8 build. #1026

Merged
merged 2 commits into from Jul 6, 2016

Conversation

Projects
None yet
4 participants
@fnothaft
Member

fnothaft commented Apr 28, 2016

Resolves #1008. As per discussion with @shaneknapp, it is preferable to support builds across Java versions by manually setting JAVA_HOME in a shell script run inside of Jenkins. This is due to certain configuration limitations inside of Jenkins. Currently, we are doing this via a script tracked inside of Jenkins. I would rather roll all of that config into the ./scripts/jenkins-test script so that we have revision control, etc.

@shaneknapp

This comment has been minimized.

Show comment
Hide comment
@shaneknapp

shaneknapp Apr 28, 2016

Contributor

LGTM++

Contributor

shaneknapp commented Apr 28, 2016

LGTM++

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Apr 28, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1222/

Build result: FAILURE

[...truncated 24 lines...]Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.4.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.4.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.3.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.3.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.3.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.5.2,centosADAM-prb ? 2.6.0,2.10,1.4.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.4.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.4.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.4.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.5.2,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.5.2,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Apr 28, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1222/

Build result: FAILURE

[...truncated 24 lines...]Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.4.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.4.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.3.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.3.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.3.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.5.2,centosADAM-prb ? 2.6.0,2.10,1.4.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.4.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.4.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.4.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.5.2,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.5.2,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Apr 28, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1223/
Test PASSed.

AmplabJenkins commented Apr 28, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1223/
Test PASSed.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Apr 28, 2016

Member

Are folks OK with me merging this/can I get a merge?

Member

fnothaft commented Apr 28, 2016

Are folks OK with me merging this/can I get a merge?

@shaneknapp

This comment has been minimized.

Show comment
Hide comment
@shaneknapp

shaneknapp Apr 28, 2016

Contributor

you can merge it only if you don't forget to change the jenkins build.
remove those bits you added to the jenkins-test script. :)

On Thu, Apr 28, 2016 at 2:28 PM, Frank Austin Nothaft <
notifications@github.com> wrote:

Are folks OK with me merging this/can I get a merge?


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1026 (comment)

Contributor

shaneknapp commented Apr 28, 2016

you can merge it only if you don't forget to change the jenkins build.
remove those bits you added to the jenkins-test script. :)

On Thu, Apr 28, 2016 at 2:28 PM, Frank Austin Nothaft <
notifications@github.com> wrote:

Are folks OK with me merging this/can I get a merge?


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1026 (comment)

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Apr 28, 2016

Member

@shaneknapp oh, of course! I don't want to change the Jenkins build though until this is right about to merge (i.e., I will change the Jenkins build immediately before merging this).

Member

fnothaft commented Apr 28, 2016

@shaneknapp oh, of course! I don't want to change the Jenkins build though until this is right about to merge (i.e., I will change the Jenkins build immediately before merging this).

@shaneknapp

This comment has been minimized.

Show comment
Hide comment
@shaneknapp

shaneknapp Apr 28, 2016

Contributor

i took a closer look and have some comments... mostly bash "style" stuff
to clean things up.

On Thu, Apr 28, 2016 at 3:58 PM, Frank Austin Nothaft <
notifications@github.com> wrote:

@shaneknapp https://github.com/shaneknapp oh, of course! I don't want
to change the Jenkins build though until this is right about to merge
(i.e., I will change the Jenkins build immediately before merging this).


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1026 (comment)

Contributor

shaneknapp commented Apr 28, 2016

i took a closer look and have some comments... mostly bash "style" stuff
to clean things up.

On Thu, Apr 28, 2016 at 3:58 PM, Frank Austin Nothaft <
notifications@github.com> wrote:

@shaneknapp https://github.com/shaneknapp oh, of course! I don't want
to change the Jenkins build though until this is right about to merge
(i.e., I will change the Jenkins build immediately before merging this).


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1026 (comment)

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Apr 28, 2016

Member

@shaneknapp that'd be great. Do you want to drop them inline on the PR? Alternatively, if you want to send a commit against my branch, that'd be great too.

Member

fnothaft commented Apr 28, 2016

@shaneknapp that'd be great. Do you want to drop them inline on the PR? Alternatively, if you want to send a commit against my branch, that'd be great too.

@shaneknapp

This comment has been minimized.

Show comment
Hide comment
@shaneknapp

shaneknapp Apr 28, 2016

Contributor

yeah, there's a bunch that needs to be done... i'll start at it for a bit and then post some thoughts and inline comments.

Contributor

shaneknapp commented Apr 28, 2016

yeah, there's a bunch that needs to be done... i'll start at it for a bit and then post some thoughts and inline comments.

Show outdated Hide outdated scripts/jenkins-test
@@ -2,81 +2,107 @@
set -e -x
export JAVA_HOME=/usr/java/jdk1.8.0_60

This comment has been minimized.

@shaneknapp

shaneknapp Apr 28, 2016

Contributor

put all of your variable decs up here, rather than spreading them throughout the script. and only use export for variables that need to be visible to subprocs (JAVA_HOME, PATH, etc).

@shaneknapp

shaneknapp Apr 28, 2016

Contributor

put all of your variable decs up here, rather than spreading them throughout the script. and only use export for variables that need to be visible to subprocs (JAVA_HOME, PATH, etc).

Show outdated Hide outdated scripts/jenkins-test
# Copy the jar into our temp space for testing
cp -r . "$ADAM_TMP_DIR"
popd
ADAM_TMP_DIR="$(mktemp -d -t "adamTestXXXXXXX")"

This comment has been minimized.

@shaneknapp

shaneknapp Apr 28, 2016

Contributor

extraneous quotes

@shaneknapp

shaneknapp Apr 28, 2016

Contributor

extraneous quotes

Show outdated Hide outdated scripts/jenkins-test
fi
which mvn
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

This comment has been minimized.

@shaneknapp

shaneknapp Apr 28, 2016

Contributor

extraneous quotes

@shaneknapp

shaneknapp Apr 28, 2016

Contributor

extraneous quotes

Show outdated Hide outdated scripts/jenkins-test
fi
which mvn
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
PROJECT_ROOT="$DIR/.."
VERSION="$(grep "<version>" "$PROJECT_ROOT/pom.xml" | head -2 | tail -1 | sed 's/ *<version>//g' | sed 's/<\/version>//g')"

This comment has been minimized.

@shaneknapp

shaneknapp Apr 28, 2016

Contributor

extraneous quotes around $(...)

@shaneknapp

shaneknapp Apr 28, 2016

Contributor

extraneous quotes around $(...)

Show outdated Hide outdated scripts/jenkins-test
pushd "$ADAM_TMP_DIR"

This comment has been minimized.

@shaneknapp

shaneknapp Apr 28, 2016

Contributor

extraneous quotes

@shaneknapp

shaneknapp Apr 28, 2016

Contributor

extraneous quotes

@shaneknapp

This comment has been minimized.

Show comment
Hide comment
@shaneknapp

shaneknapp Apr 28, 2016

Contributor

ill just say that you have a lot of extraneous quotes, and inconsistent variable declaration. a good, easy to read standard looks like this:

FOO="${BAR}.baz.zomgwtf"

for STDOUT declared vars:

FOO=$(command)

curly braces around vars are optional, depending on legibility.

bonus points for 'set -x -e'. :)

clean up the variable decs and quotes, and i'll take a second look.

Contributor

shaneknapp commented Apr 28, 2016

ill just say that you have a lot of extraneous quotes, and inconsistent variable declaration. a good, easy to read standard looks like this:

FOO="${BAR}.baz.zomgwtf"

for STDOUT declared vars:

FOO=$(command)

curly braces around vars are optional, depending on legibility.

bonus points for 'set -x -e'. :)

clean up the variable decs and quotes, and i'll take a second look.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Apr 29, 2016

Member

+1 from me, after cleanup

Member

heuermh commented Apr 29, 2016

+1 from me, after cleanup

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh May 18, 2016

Member

@fnothaft bump, would be nice to have in place for #989

Member

heuermh commented May 18, 2016

@fnothaft bump, would be nice to have in place for #989

@heuermh heuermh modified the milestone: 0.20.0 Jun 5, 2016

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Jul 6, 2016

Member

Updated to resolve review comments. Also, added commit to address #827.

Member

fnothaft commented Jul 6, 2016

Updated to resolve review comments. Also, added commit to address #827.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jul 6, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1307/

Build result: FAILURE

GitHub pull request #1026 of commit 782075b automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > /home/jenkins/git2/bin/git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1026/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains ec177bd # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1026/merge^{commit} # timeout=10Checking out Revision ec177bd (origin/pr/1026/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f ec177bda52cc845b436dfdbbd43facfbf6e41cbeFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Jul 6, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1307/

Build result: FAILURE

GitHub pull request #1026 of commit 782075b automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > /home/jenkins/git2/bin/git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1026/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains ec177bd # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1026/merge^{commit} # timeout=10Checking out Revision ec177bd (origin/pr/1026/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f ec177bda52cc845b436dfdbbd43facfbf6e41cbeFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

fnothaft added some commits Apr 28, 2016

[ADAM-1008] Modify jenkins-test script to support Java 8 build.
Resolves #1008. As per discussion with @shaneknapp, it is preferable to support
builds across Java versions by manually setting JAVA_HOME in a shell script run
inside of Jenkins. This is due to certain configuration limitations inside of
Jenkins. Currently, we are doing this via a script tracked inside of Jenkins.
I would rather roll all of that config into the ./scripts/jenkins-test script
so that we have revision control, etc.
@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jul 6, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1310/
Test PASSed.

AmplabJenkins commented Jul 6, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1310/
Test PASSed.

@shaneknapp

This comment has been minimized.

Show comment
Hide comment
@shaneknapp

shaneknapp Jul 6, 2016

Contributor

looks great!

Contributor

shaneknapp commented Jul 6, 2016

looks great!

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Jul 6, 2016

Member

LGTM

Member

heuermh commented Jul 6, 2016

LGTM

@heuermh heuermh merged commit 4ba0b62 into bigdatagenomics:master Jul 6, 2016

1 check passed

default Merged build finished.
Details
@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Jul 6, 2016

Member

Thank you, @fnothaft!

Member

heuermh commented Jul 6, 2016

Thank you, @fnothaft!

@fnothaft fnothaft deleted the fnothaft:jenkins-script branch Jul 6, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment