-
Notifications
You must be signed in to change notification settings - Fork 545
output: azure_blob: compression with zstd update #2309
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
WalkthroughDocumentation updates to the Azure Blob output: Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: defaults Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
📜 Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
pipeline/outputs/azure_blob.md(1 hunks)
🔇 Additional comments (1)
pipeline/outputs/azure_blob.md (1)
29-30: Well-documented compression options and inheritance behavior.The documentation clearly explains:
- Both
compress(transport-level) andcompress_blob(blob-level) compression support, with zstd now available- How
compress_blobinherits the codec fromcompresswhen set, establishing clear precedence- The incompatibility constraint with
blob_type = appendblobThese clarifications significantly improve the documentation. Once the underlying code merges in fluent-bit/fluent-bit#11202, verify that the documented behavior matches the implementation.
Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
| | `compress` | Sets payload compression in network transfer. Supported value: `gzip` | _none_ | | ||
| | `compress_blob` | Enables GZIP compression in the final `blockblob` file. This option isn't compatible when `blob_type` = `appendblob`. | _none_ | | ||
| | `compress` | Sets payload compression in network transfer. Supported values: `gzip`, `zstd`. | _none_ | | ||
| | `compress_blob` | Enables compression in the final `blockblob` file. When enabled without `compress`, it uses GZIP; if `compress` is also set, it inherits that codec. This option isn't compatible when `blob_type` = `appendblob`. | _none_ | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does isn't compatible mean here? Does it error out or silently disable or something else?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This wasn't changed, fluent-bit will error out as the two are not compatible, so fluent-bot will not be able to know what to do.
see https://github.com/fluent/fluent-bit/blob/master/plugins/out_azure_blob/azure_blob_conf.c#L669
Document the new zstd option for Azure Blob network compression and explain how compress_blob inherits its codec, clarifying precedence between transport and blob-level compression.
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.