New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAM-1875] Add ability to timeout a piped command. #1881

Merged
merged 1 commit into from Jan 25, 2018

Conversation

4 participants
@fnothaft
Member

fnothaft commented Jan 22, 2018

Resolves #1875. Adds an optional timeout parameter to the GenomicRDD.pipe function. If this is provided, a command will be killed (but without failing the Spark task) if it runs for longer than the timeout parameter.

@fnothaft fnothaft added this to the 0.24.0 milestone Jan 22, 2018

@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 22, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2583/

Build result: FAILURE

[...truncated 7 lines...] > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1881/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains ad7c7b0 # timeout=10Checking out Revision ad7c7b0 (origin/pr/1881/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f ad7c7b00783c944ba654722149860e2b9f23a48dFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.7.3,2.11,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.10,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.1,centosADAM-prb ? 2.7.3,2.11,2.2.1,centos completed with result FAILUREADAM-prb ? 2.6.2,2.10,2.2.1,centos completed with result FAILUREADAM-prb ? 2.7.3,2.10,2.2.1,centos completed with result FAILUREADAM-prb ? 2.6.2,2.11,2.2.1,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@coveralls

This comment has been minimized.

coveralls commented Jan 22, 2018

Coverage Status

Coverage decreased (-0.04%) to 82.779% when pulling 5a01aa7 on fnothaft:issues/1875-timeout-pipe into 4223f56 on bigdatagenomics:master.

@coveralls

This comment has been minimized.

coveralls commented Jan 22, 2018

Coverage Status

Coverage decreased (-0.07%) to 82.707% when pulling db40096 on fnothaft:issues/1875-timeout-pipe into c5b4ebc on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 22, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2584/
Test PASSed.

@@ -454,21 +454,27 @@ trait GenomicRDD[T, U <: GenomicRDD[T, U]] extends Logging {
* @param environment A map containing environment variable/value pairs to set
* in the environment for the newly created process. Default is empty.
* @param flankSize Number of bases to flank each command invocation by.
* @param optTimeout An optional parameter specifying how long to let a single

This comment has been minimized.

@heuermh

heuermh Jan 22, 2018

Member

Add doc that this is in seconds, or alternatively allow the caller to specify the TimeUnit

[ADAM-1875] Add ability to timeout a piped command.
Resolves #1875. Adds an optional timeout parameter to the `GenomicRDD.pipe`
function. If this is provided, a command will be killed (but without failing the
Spark task) if it runs for longer than the timeout parameter.
@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 24, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2596/
Test PASSed.

@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 24, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2597/

Build result: FAILURE

[...truncated 7 lines...] > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1881/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 3352cc2 # timeout=10Checking out Revision 3352cc2 (origin/pr/1881/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 3352cc22b09aa8cd2364a44121701ebc138954f8First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.3,2.11,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.10,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.1,centosADAM-prb ? 2.7.3,2.11,2.2.1,centos completed with result SUCCESSADAM-prb ? 2.6.2,2.10,2.2.1,centos completed with result FAILUREADAM-prb ? 2.7.3,2.10,2.2.1,centos completed with result SUCCESSADAM-prb ? 2.6.2,2.11,2.2.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@fnothaft

This comment has been minimized.

Member

fnothaft commented Jan 25, 2018

Jenkins, test this please.

@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 25, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2598/
Test PASSed.

@heuermh heuermh merged commit adff336 into bigdatagenomics:master Jan 25, 2018

1 of 2 checks passed

Codacy/PR Quality Review Not so good... This pull request quality could be better.
Details
default Merged build finished.
Details
@heuermh

This comment has been minimized.

Member

heuermh commented Jan 25, 2018

Thank you, @fnothaft

@heuermh heuermh added this to Completed in Release 0.24.0 Feb 10, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment