Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ZSTD compression #367

Merged
merged 5 commits into from
May 28, 2024
Merged

Add ZSTD compression #367

merged 5 commits into from
May 28, 2024

Conversation

byian
Copy link
Contributor

@byian byian commented Mar 2, 2024

To resolve #35

  • Added zstd v1.5.5 sources to contrib
  • New cmake option WITH_SYSTEM_ZSTD to choose between building and using the system installed library
  • Compression and decompression are implemented using the Simple API (so there is no choice of compression level)
  • Updated tests

@byian
Copy link
Contributor Author

byian commented Mar 4, 2024

@Enmk Could you please trigger CI for this?

case static_cast<uint8_t>(CompressionMethodByte::NONE): {

throw CompressionError("compression method not defined" + std::to_string((method)));
}
Copy link
Collaborator

@Enmk Enmk Mar 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please handle invalid\unsupported compression methods too:

Suggested change
}
}
default:
throw CompressionError("Unknown or unsupported compression method " + std::to_string((method)));

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Collaborator

@Enmk Enmk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some modifications required

};

// see DB::CompressionMethodByte from src/Compression/CompressionInfo.h of ClickHouse project
enum class CompressionMethodByte : uint8_t {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that a part of the public API that users should know about? If not, please move away from the headers to the cpp.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved to cpp

@@ -3,6 +3,8 @@
#include "query.h"
#include "exceptions.h"

#include "base/compressed.h"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this is not the best decision: clients are not expected to rely on stuff in base, at least explicitly. I think it is best to move the CompressionMethod declaration back to client.h

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@byian byian requested a review from Enmk April 7, 2024 10:29
Copy link
Collaborator

@Enmk Enmk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Enmk Enmk merged commit 9d23bf5 into ClickHouse:master May 28, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

zSTD Support
2 participants