Skip to content

Conversation

@nberlee
Copy link

@nberlee nberlee commented Dec 19, 2025

Document the new zstd option for Azure Blob network compression and explain how compress_blob inherits its codec, clarifying precedence between transport and blob-level compression.

⚠️ Heads-up: the code enabling Zstandard support is still under review in fluent/fluent-bit#11202, so please keep this docs PR in waiting-on-code-merge until that lands.

Summary by CodeRabbit

  • Documentation
    • Updated Azure Blob output documentation with expanded compression options, now supporting gzip and zstd codecs.
    • Clarified compression behavior including default GZIP application and codec inheritance based on settings.
    • Added compatibility notes regarding compression and blob type configurations.

✏️ Tip: You can customize this high-level summary in your review settings.

@nberlee nberlee requested review from a team as code owners December 19, 2025 13:34
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 19, 2025

Walkthrough

Documentation updates to the Azure Blob output: compress now documents both gzip and zstd; compress_blob expanded to describe final blockblob compression behavior, inheritance from compress, defaulting to GZIP when unset, and incompatibility with blob_type = appendblob.

Changes

Cohort / File(s) Change Summary
Azure Blob Output Documentation
pipeline/outputs/azure_blob.md
Expanded compress docs to list gzip and zstd; expanded compress_blob description to explain final blockblob compression behavior, inheritance from compress, default GZIP when compress is unset, and note incompatibility with blob_type = appendblob.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

  • Focus review on pipeline/outputs/azure_blob.md to verify wording accuracy and consistency with existing docs/specs.

Poem

🐰 I nibble on docs beneath the moonlight,
Zstd and gzip twinkle, cozy and bright,
Blobs learn to tuck in their compressed beds,
Clear notes guide bytes to soft, sleepy threads,
Hooray — documentation hops into sight! 🥕✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically identifies the main change: adding zstd compression support to the Azure Blob output documentation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

📜 Recent review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 749e1f9 and 5d1f7e5.

📒 Files selected for processing (1)
  • pipeline/outputs/azure_blob.md (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • pipeline/outputs/azure_blob.md

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7fb7ce2 and 749e1f9.

📒 Files selected for processing (1)
  • pipeline/outputs/azure_blob.md (1 hunks)
🔇 Additional comments (1)
pipeline/outputs/azure_blob.md (1)

29-30: Well-documented compression options and inheritance behavior.

The documentation clearly explains:

  • Both compress (transport-level) and compress_blob (blob-level) compression support, with zstd now available
  • How compress_blob inherits the codec from compress when set, establishing clear precedence
  • The incompatibility constraint with blob_type = appendblob

These clarifications significantly improve the documentation. Once the underlying code merges in fluent-bit/fluent-bit#11202, verify that the documented behavior matches the implementation.

Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
| `compress` | Sets payload compression in network transfer. Supported value: `gzip` | _none_ |
| `compress_blob` | Enables GZIP compression in the final `blockblob` file. This option isn't compatible when `blob_type` = `appendblob`. | _none_ |
| `compress` | Sets payload compression in network transfer. Supported values: `gzip`, `zstd`. | _none_ |
| `compress_blob` | Enables compression in the final `blockblob` file. When enabled without `compress`, it uses GZIP; if `compress` is also set, it inherits that codec. This option isn't compatible when `blob_type` = `appendblob`. | _none_ |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does isn't compatible mean here? Does it error out or silently disable or something else?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wasn't changed, fluent-bit will error out as the two are not compatible, so fluent-bot will not be able to know what to do.
see https://github.com/fluent/fluent-bit/blob/master/plugins/out_azure_blob/azure_blob_conf.c#L669

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants