New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add flatten command. #630

Merged
merged 1 commit into from Mar 27, 2015

Conversation

Projects
None yet
3 participants
@tomwhite
Member

tomwhite commented Mar 26, 2015

This adds a 'flatten' command to the CLI for removing nesting from ADAM files in Parquet format. This is useful for querying ADAM data using tools like Impala, which can't currently read nested fields.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Mar 26, 2015

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/651/

Build result: FAILURE

GitHub pull request #630 of commit 610604d automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/630/merge^{commit} # timeout=10 > git branch -a --contains 4bf33a3 # timeout=10 > git rev-parse remotes/origin/pr/630/merge^{commit} # timeout=10Checking out Revision 4bf33a3 (origin/pr/630/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 4bf33a3d68b15fc0a6e0aca062b47916536443a7First time build. Skipping changelog.Triggering ADAM-prb ? 1.0.4,centosTriggering ADAM-prb ? 2.2.0,centosTriggering ADAM-prb ? 2.3.0,centosADAM-prb ? 1.0.4,centos completed with result FAILUREADAM-prb ? 2.2.0,centos completed with result FAILUREADAM-prb ? 2.3.0,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/651/

Build result: FAILURE

GitHub pull request #630 of commit 610604d automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/630/merge^{commit} # timeout=10 > git branch -a --contains 4bf33a3 # timeout=10 > git rev-parse remotes/origin/pr/630/merge^{commit} # timeout=10Checking out Revision 4bf33a3 (origin/pr/630/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 4bf33a3d68b15fc0a6e0aca062b47916536443a7First time build. Skipping changelog.Triggering ADAM-prb ? 1.0.4,centosTriggering ADAM-prb ? 2.2.0,centosTriggering ADAM-prb ? 2.3.0,centosADAM-prb ? 1.0.4,centos completed with result FAILUREADAM-prb ? 2.2.0,centos completed with result FAILUREADAM-prb ? 2.3.0,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Mar 26, 2015

Member

Looks great @tomwhite ! Can you just run format-source and push an update?

Member

fnothaft commented Mar 26, 2015

Looks great @tomwhite ! Can you just run format-source and push an update?

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Mar 27, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/658/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/658/
Test PASSed.

@tomwhite

This comment has been minimized.

Show comment
Hide comment
@tomwhite

tomwhite Mar 27, 2015

Member

That seemed to fix it. Thanks for taking a look @fnothaft

Member

tomwhite commented Mar 27, 2015

That seemed to fix it. Thanks for taking a look @fnothaft

fnothaft added a commit that referenced this pull request Mar 27, 2015

@fnothaft fnothaft merged commit 562518f into bigdatagenomics:master Mar 27, 2015

1 check passed

default Merged build finished.
Details
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Mar 27, 2015

Member

Merged! Thanks @tomwhite!

Member

fnothaft commented Mar 27, 2015

Merged! Thanks @tomwhite!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment