New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does the nested record Flattener drop Maps/Arrays? #1244

Closed
fnothaft opened this Issue Nov 6, 2016 · 2 comments

Comments

Projects
None yet
2 participants
@fnothaft
Member

fnothaft commented Nov 6, 2016

I saw this when doing a cleanup pass over org.bdgenomics.adam.util. Specifically, look at the following lines:

This is somewhat problematic as some of our nested schemas (specifically, Genotype) use Array fields, and many use Maps.

CC @tomwhite. Do you know if this code is still in use? If not, it might make more sense to drop the code instead of patching the code. I think we were using it in Eggo to flatten schemas for Hive/Impala/Spark SQL before those tools supported nested schemas, but I think those tools all support nested schemas now.

@fnothaft fnothaft added the question label Nov 6, 2016

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Nov 8, 2016

Member

Ping @tomwhite @laserson for comment.

Member

fnothaft commented Nov 8, 2016

Ping @tomwhite @laserson for comment.

fnothaft added a commit to fnothaft/adam that referenced this issue Dec 9, 2016

fnothaft added a commit to fnothaft/adam that referenced this issue Dec 9, 2016

fnothaft added a commit to fnothaft/adam that referenced this issue Dec 9, 2016

@bpburns

This comment has been minimized.

Show comment
Hide comment
@bpburns

bpburns Dec 19, 2016

Any update on this? It looks like you made a commit removing some of the code, but the utility is still present at the path you reference in the first message for the issue.

bpburns commented Dec 19, 2016

Any update on this? It looks like you made a commit removing some of the code, but the utility is still present at the path you reference in the first message for the issue.

fnothaft added a commit to fnothaft/adam that referenced this issue Dec 22, 2016

fnothaft added a commit to fnothaft/adam that referenced this issue Dec 29, 2016

fnothaft added a commit to fnothaft/adam that referenced this issue Jan 5, 2017

@heuermh heuermh closed this in c31724f Jan 5, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment