Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Java] Implement RecordBatch IPC buffer compression from ARROW-300 #24831

Closed
Tracked by #24829
asfimport opened this issue May 2, 2020 · 9 comments
Closed
Tracked by #24829

[Java] Implement RecordBatch IPC buffer compression from ARROW-300 #24831

asfimport opened this issue May 2, 2020 · 9 comments

Comments

@asfimport
Copy link

asfimport commented May 2, 2020

Reporter: Wes McKinney / @wesm
Assignee: Liya Fan / @liyafan82

Related issues:

Note: This issue was originally created as ARROW-8672. Please see the migration documentation for further details.

@asfimport
Copy link
Author

Liya Fan / @liyafan82:
@wesm This issue looks interesting. May I try to provide a patch for it?

@asfimport
Copy link
Author

Wes McKinney / @wesm:
Sure please feel free

@asfimport
Copy link
Author

Wes McKinney / @wesm:
Since this involves introducing dependencies into the Java build this seems too risky for the next few days

@asfimport
Copy link
Author

Antoine Pitrou / @pitrou:
Is there any plan to implement ZStandard compression as well?

@asfimport
Copy link
Author

Liya Fan / @liyafan82:
@pitrou We have already supported it in ARROW-12110.
I have linked that issue as a sub-work item of this.

@asfimport
Copy link
Author

Antoine Pitrou / @pitrou:
Great, so this issue can be closed I supposed?

@asfimport
Copy link
Author

Liya Fan / @liyafan82:
Almost. I think there are still some performance tuning and clean-up work (ARROW-12163, ARROW-12115, ARROW-11901) to do before we can close it?

I have linked those issues as well.

@asfimport
Copy link
Author

Antoine Pitrou / @pitrou:
If you want, but this issue was really about making buffer compression available. Performance improvements are a separate concern, IMHO.

@asfimport
Copy link
Author

Liya Fan / @liyafan82:
Make sense.
So it should be reasonable to close this issue now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants