Skip to content

Blog post about changes to write wide columns, large string columns #10073

@alamb

Description

@alamb

@adriangb has made some great contributions to the Rust parquet writer to support several usecases where parquet writers typically struggle:

I think it would be amazing to write a blog post explaining the challenge and the solutions

Challenge 1: Oversized pages for large binary/string columns

Challenge 2: RAM buffering requirements when writing large Row Groups

The idea is to write a blog in https://arrow.apache.org/blog/ (source in https://github.com/apache/arrow-site) that explains the challenges and how we solved it with Software Engineering rather than a new file format

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions