New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

save full DNS response #1

Open
mwullink opened this Issue Dec 16, 2015 · 2 comments

Comments

Projects
None yet
1 participant
@mwullink
Contributor

mwullink commented Dec 16, 2015

Currently the answer, authority and additional sections are not stored.

Modify the storage format to use nested datatypes to include these missing sections in the same row as the other DNS query/response attributes.

@mwullink

This comment has been minimized.

Show comment
Hide comment
@mwullink

mwullink Oct 10, 2016

Contributor

Talked to the Impala developers and they said that Impala support for writing nested Parquet data is not on the roadmap for the near future.

Contributor

mwullink commented Oct 10, 2016

Talked to the Impala developers and they said that Impala support for writing nested Parquet data is not on the roadmap for the near future.

@mwullink

This comment has been minimized.

Show comment
Hide comment
@mwullink

mwullink Aug 10, 2017

Contributor

Impala now supports reading nested Parquet data but cannot write new Parquet data.
ENTRADA uses Impala to compact Parquet files from staging into new files in the queries.
so we cannot use nesting yet unless we do the compacting with new code. this is much more work.
best would be to wait for Impala write support.

Contributor

mwullink commented Aug 10, 2017

Impala now supports reading nested Parquet data but cannot write new Parquet data.
ENTRADA uses Impala to compact Parquet files from staging into new files in the queries.
so we cannot use nesting yet unless we do the compacting with new code. this is much more work.
best would be to wait for Impala write support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment