Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Build info/version info inside ADAM-generated files #188
Also, I may have missed some previous discussions on how we do this, but I recently converted hg19 to a Parquet file of ADAMNucleotideConfigFragments. It seems there's no way to recover the reference version information - or am I missing something? The AVRO record contig fields don't store this. Can we shove it in the Parquet metadata somewhere?
Once we upgrade to Parquet 1.6.0, we'll be able to read/write arbitrary metadata much more easily. We can easily drop the version info (introduced in #138) into the metadata to help with debugging.
The upgrade to 1.6.0 is going well but three tests are failing because of issues with predicates (UnboundRecordFilter).