New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAM-891] Mark SparkContext as @transient. #894

Merged
merged 1 commit into from Dec 11, 2015

Conversation

Projects
None yet
4 participants
@fnothaft
Member

fnothaft commented Dec 2, 2015

Closes #891. In org.bdgenomics.adam.rdd.ADAMContext, the val sparkContext:
SparkContext is not marked as @transient, which causes serialization issues.
SparkContexts are not serializable and should only be called from the driver.

[ADAM-891] Mark SparkContext as @transient.
Closes #891. In org.bdgenomics.adam.rdd.ADAMContext, the val sparkContext:
SparkContext is not marked as @transient, which causes serialization issues.
SparkContexts are not serializable and should only be called from the driver.
@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Dec 2, 2015

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1024/

Build result: FAILURE

GitHub pull request #894 of commit a4be852 automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/894/merge^{commit} # timeout=10 > git branch -a --contains 55dfb83 # timeout=10 > git rev-parse remotes/origin/pr/894/merge^{commit} # timeout=10Checking out Revision 55dfb83 (origin/pr/894/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 55dfb8363337194e825c5ddfbebecadffd197becFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.4.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.4.1,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Dec 2, 2015

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1024/

Build result: FAILURE

GitHub pull request #894 of commit a4be852 automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/894/merge^{commit} # timeout=10 > git branch -a --contains 55dfb83 # timeout=10 > git rev-parse remotes/origin/pr/894/merge^{commit} # timeout=10Checking out Revision 55dfb83 (origin/pr/894/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 55dfb8363337194e825c5ddfbebecadffd197becFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.4.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.4.1,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Dec 2, 2015

Member

Should ADAMContext be serializable?

Member

ryan-williams commented Dec 2, 2015

Should ADAMContext be serializable?

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Dec 2, 2015

Member

Jenkins, retest this please.

Member

heuermh commented Dec 2, 2015

Jenkins, retest this please.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Dec 2, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1025/
Test PASSed.

AmplabJenkins commented Dec 2, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1025/
Test PASSed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Dec 2, 2015

Member

LGTM, ok to merge via Merge pull request button?

Member

heuermh commented Dec 2, 2015

LGTM, ok to merge via Merge pull request button?

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Dec 3, 2015

Member

@ryan-williams I think it needs to be serializable as we have closures in the class that need to be serialized (e.g., for BAM->ADAM conversion). It might make sense to pull the things that need to be serializable out into another object, but I digress.

Member

fnothaft commented Dec 3, 2015

@ryan-williams I think it needs to be serializable as we have closures in the class that need to be serialized (e.g., for BAM->ADAM conversion). It might make sense to pull the things that need to be serializable out into another object, but I digress.

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Dec 3, 2015

Member

Makes sense, gtk, thx

Member

ryan-williams commented Dec 3, 2015

Makes sense, gtk, thx

heuermh added a commit that referenced this pull request Dec 11, 2015

@heuermh heuermh merged commit 486354e into bigdatagenomics:master Dec 11, 2015

1 check passed

default Merged build finished.
Details
@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Dec 11, 2015

Member

Thanks!

Member

heuermh commented Dec 11, 2015

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment