Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support MessagePackEachRow import format #64261

Open
altexy opened this issue May 23, 2024 · 0 comments
Open

Support MessagePackEachRow import format #64261

altexy opened this issue May 23, 2024 · 0 comments
Labels

Comments

@altexy
Copy link

altexy commented May 23, 2024

I offer in addition to JSONEachRow support the similar format, but using sequence of MessagePack map objects.

The existing MsgPack format is schema-less, just sequence of MessagePack encoded values.

BSONEachRow is great, but don't have full UInt64 support, because internally support only Int64.
At the moment only RowBinaryWithNames format supports widely used UInt64 type, I'm talking about row-oriented self-describing binary formats. But it has a different usage pattern, because it has a header and multiple rows.

I want to have a format like JSONEachRow which make possible to change the fields set on the fly, but binary.

IMO need to support input_format_skip_unknown_fields flag

@altexy altexy added the feature label May 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant