Skip to content
This repository was archived by the owner on May 10, 2024. It is now read-only.

PARQUET-583: Parquet to Thrift schema conversion#87

Closed
xhochy wants to merge 2 commits into
apache:masterfrom
xhochy:parquet-583
Closed

PARQUET-583: Parquet to Thrift schema conversion#87
xhochy wants to merge 2 commits into
apache:masterfrom
xhochy:parquet-583

Conversation

@xhochy
Copy link
Copy Markdown
Member

@xhochy xhochy commented Apr 16, 2016

Depends on #86

Comment thread src/parquet/schema/types.h Outdated
public:
virtual ~Visitor() {}

virtual void Visit(const Node* node) = 0;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this should be made Visit(Node* node) to differentiate with ConstVisitor?

@wesm
Copy link
Copy Markdown
Member

wesm commented Apr 17, 2016

This looks good to me outside of the minor comments. The way we're handling the not-always-set SchemaElement fields bums me out a bit, but if they can be counted on to always be -1 if not set it does make testing easier. Pretty open-minded about improving this aspect of the schema code generally

@xhochy
Copy link
Copy Markdown
Member Author

xhochy commented Apr 18, 2016

For the not-always-set SchemaElement fields, we probably need a way in SchemaElement to track if they were set (which we could also define as value == -1). Then we could decide consistently in the to-thrift-schema-conversion if we should write them out.

@wesm
Copy link
Copy Markdown
Member

wesm commented Apr 18, 2016

If the struct is zeroed out, you can use the element.__isset for this

@xhochy
Copy link
Copy Markdown
Member Author

xhochy commented Apr 19, 2016

Yes, we have that in Thrift. Probably I got it wrong in my last comment ;) What I'm missing is that we haven't got something similar to .__isset in PrimitiveNode to track if we have set one of length, precision, scale.

But to get this PR to a close: Should I only call element->__set_type_length if length != -1 or keep the code for as it is? (this is the only remaining issue here or am I missing something?)

@wesm
Copy link
Copy Markdown
Member

wesm commented Apr 19, 2016

Keeping things as is works for me right now, we can iterate on the optional field stuff later.

@wesm
Copy link
Copy Markdown
Member

wesm commented Apr 19, 2016

+1, merging

@asfgit asfgit closed this in 49a5c1a Apr 19, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants