Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does the nested record Flattener drop Maps/Arrays? #1244

Closed
fnothaft opened this issue Nov 6, 2016 · 2 comments
Closed

Does the nested record Flattener drop Maps/Arrays? #1244

fnothaft opened this issue Nov 6, 2016 · 2 comments
Labels

Comments

@fnothaft
Copy link
Member

@fnothaft fnothaft commented Nov 6, 2016

I saw this when doing a cleanup pass over org.bdgenomics.adam.util. Specifically, look at the following lines:

This is somewhat problematic as some of our nested schemas (specifically, Genotype) use Array fields, and many use Maps.

CC @tomwhite. Do you know if this code is still in use? If not, it might make more sense to drop the code instead of patching the code. I think we were using it in Eggo to flatten schemas for Hive/Impala/Spark SQL before those tools supported nested schemas, but I think those tools all support nested schemas now.

@fnothaft fnothaft added the question label Nov 6, 2016
@fnothaft
Copy link
Member Author

@fnothaft fnothaft commented Nov 8, 2016

Ping @tomwhite @laserson for comment.

fnothaft added a commit to fnothaft/adam that referenced this issue Dec 9, 2016
fnothaft added a commit to fnothaft/adam that referenced this issue Dec 9, 2016
fnothaft added a commit to fnothaft/adam that referenced this issue Dec 9, 2016
@bpburns
Copy link

@bpburns bpburns commented Dec 19, 2016

Any update on this? It looks like you made a commit removing some of the code, but the utility is still present at the path you reference in the first message for the issue.

fnothaft added a commit to fnothaft/adam that referenced this issue Dec 22, 2016
fnothaft added a commit to fnothaft/adam that referenced this issue Dec 29, 2016
fnothaft added a commit to fnothaft/adam that referenced this issue Jan 5, 2017
@heuermh heuermh closed this in c31724f Jan 5, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.