Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C#] Create implementation of ARROW-300 / IPC record batch body buffer compression #24834

Closed
Tracked by #24829
asfimport opened this issue May 2, 2020 · 2 comments · Fixed by #39871
Closed
Tracked by #24829

[C#] Create implementation of ARROW-300 / IPC record batch body buffer compression #24834

asfimport opened this issue May 2, 2020 · 2 comments · Fixed by #39871
Assignees
@adamreeve
Labels
Component: C# Type: task
Milestone
16.0.0

Comments

@asfimport
Copy link
Collaborator

Reporter: Wes McKinney / @wesm

Note: This issue was originally created as ARROW-8675. Please see the migration documentation for further details.
@asfimport asfimport mentioned this issue Jan 11, 2023
Open
7 tasks
@adamreeve
Copy link
Contributor

I'd like to work on this. IPC decompression support was added in #32240, but compression support has not yet been added to the .NET library.

@adamreeve
Copy link
Contributor

take

@github-actions github-actions bot mentioned this issue Feb 1, 2024
Merged
CurtHagenlocher pushed a commit that referenced this issue Feb 7, 2024
@adamreeve
GH-24834: [C#] Support writing compressed IPC data (#39871)
c9f6e04 
### Rationale for this change

This allows using compression when writing IPC streams and files with the Arrow .NET library

### What changes are included in this PR?

* Adds a compress method to the `ICompressionCodec` interface and implements this for Zstd and LZ4Frame in the `Apache.Arrow.Compression` package
* Adds new compression related options to `IpcOptions`
* Implements buffer compression in `ArrowStreamWriter`

### Are these changes tested?

Yes, new unit tests have been added

### Are there any user-facing changes?

Yes, this is a new user-facing feature and the `status.rst` and `csharp/README` files have been updated

* Closes: #24834

Authored-by: Adam Reeve <adreeve@gmail.com>
Signed-off-by: Curt Hagenlocher <curt@hagenlocher.org>
@CurtHagenlocher CurtHagenlocher added this to the 16.0.0 milestone Feb 7, 2024
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
@adamreeve @dgreiss
apacheGH-24834: [C#] Support writing compressed IPC data (apache#39871)
292a051 
### Rationale for this change

This allows using compression when writing IPC streams and files with the Arrow .NET library

### What changes are included in this PR?

* Adds a compress method to the `ICompressionCodec` interface and implements this for Zstd and LZ4Frame in the `Apache.Arrow.Compression` package
* Adds new compression related options to `IpcOptions`
* Implements buffer compression in `ArrowStreamWriter`

### Are these changes tested?

Yes, new unit tests have been added

### Are there any user-facing changes?

Yes, this is a new user-facing feature and the `status.rst` and `csharp/README` files have been updated

* Closes: apache#24834

Authored-by: Adam Reeve <adreeve@gmail.com>
Signed-off-by: Curt Hagenlocher <curt@hagenlocher.org>
zanmato1984 pushed a commit to zanmato1984/arrow that referenced this issue Feb 28, 2024
@adamreeve @zanmato1984
apacheGH-24834: [C#] Support writing compressed IPC data (apache#39871)
2b03da9 
### Rationale for this change

This allows using compression when writing IPC streams and files with the Arrow .NET library

### What changes are included in this PR?

* Adds a compress method to the `ICompressionCodec` interface and implements this for Zstd and LZ4Frame in the `Apache.Arrow.Compression` package
* Adds new compression related options to `IpcOptions`
* Implements buffer compression in `ArrowStreamWriter`

### Are these changes tested?

Yes, new unit tests have been added

### Are there any user-facing changes?

Yes, this is a new user-facing feature and the `status.rst` and `csharp/README` files have been updated

* Closes: apache#24834

Authored-by: Adam Reeve <adreeve@gmail.com>
Signed-off-by: Curt Hagenlocher <curt@hagenlocher.org>
thisisnic pushed a commit to thisisnic/arrow that referenced this issue Mar 8, 2024
@adamreeve @thisisnic
apacheGH-24834: [C#] Support writing compressed IPC data (apache#39871)
ce539e8 
### Rationale for this change

This allows using compression when writing IPC streams and files with the Arrow .NET library

### What changes are included in this PR?

* Adds a compress method to the `ICompressionCodec` interface and implements this for Zstd and LZ4Frame in the `Apache.Arrow.Compression` package
* Adds new compression related options to `IpcOptions`
* Implements buffer compression in `ArrowStreamWriter`

### Are these changes tested?

Yes, new unit tests have been added

### Are there any user-facing changes?

Yes, this is a new user-facing feature and the `status.rst` and `csharp/README` files have been updated

* Closes: apache#24834

Authored-by: Adam Reeve <adreeve@gmail.com>
Signed-off-by: Curt Hagenlocher <curt@hagenlocher.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@adamreeve adamreeve

Labels
Component: C# Type: task
Projects
None yet
Milestone
16.0.0
Development

Successfully merging a pull request may close this issue.

GH-24834: [C#] Support writing compressed IPC data adamreeve/arrow
3 participants
@adamreeve @CurtHagenlocher @asfimport