-
Notifications
You must be signed in to change notification settings - Fork 58
Allow key value metadata to be set after writing rows #399
Allow key value metadata to be set after writing rows #399
Conversation
41a15b5
to
0be014a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we add these methods to the GenericWriter
type as well?
writer.go
Outdated
// keys. This may create incompatibilities with other parquet libraries, or may | ||
// cause some key/value pairs to be lost when open parquet files written with | ||
// repeated keys. We can revisit this decision if it ever becomes a blocker. | ||
func (w *Writer) KeyValueMetadata(key, value string) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about renaming to Set*
to better communicate that the previous value will be overwritten?
We could also imagine having an Add*
method for cases where the application needs to associated multiple values to a key.
func (w *Writer) KeyValueMetadata(key, value string) { | |
func (w *Writer) SetKeyValueMetadata(key, value string) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated in the latest commit.
0be014a
to
98a5424
Compare
Latest commit adds the method to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your contributions!
This adds a
SetKeyValueMetadata
method to the writer to allow key value file metadata to be updated after the writer is created. This allows for writing metadata that might be derived from the rows without having to buffer all the rows before creating the writer.Fixes #397.