Skip to content

docs: pipeline: outputs: kafka: document schema registry support - #2640

Merged
patrick-stephens merged 3 commits into
fluent:masterfrom
eschabell:erics_kafka_reg_options_updates
Aug 7, 2026
Merged

docs: pipeline: outputs: kafka: document schema registry support#2640
patrick-stephens merged 3 commits into
fluent:masterfrom
eschabell:erics_kafka_reg_options_updates

Conversation

@eschabell

@eschabell eschabell commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Document the schema_registry_* parameters for resolving Avro schemas from a Confluent Schema Registry, including subject and ID based resolution, basic and bearer authentication, multiple registry endpoints with failover, and the accepted dotted key spellings.

Note, covers code merges without corresponding docs PR.

Summary by CodeRabbit

  • Documentation
    • Added configuration guidance for Kafka outputs using Confluent Schema Registry.
    • Documented authentication, schema selection, framing, registry URLs, endpoint failover, caching, retries, and runtime schema resolution.
    • Added YAML and INI configuration examples.

Document the schema_registry_* parameters for resolving Avro schemas
from a Confluent Schema Registry, including subject and ID based
resolution, basic and bearer authentication, multiple registry
endpoints with failover, and the accepted dotted key spellings.

Note, covers code merges without corresponding docs PR.

Signed-off-by: Eric D. Schabell <eric@schabell.org>
@eschabell eschabell self-assigned this Aug 1, 2026
@eschabell eschabell added the waiting-on-review Waiting on a review from mainteners label Aug 1, 2026
@eschabell
eschabell requested a review from a team as a code owner August 1, 2026 09:00
@eschabell eschabell added the 5.0.9 label Aug 1, 2026
@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 8b5bfc86-0534-46f8-87cc-70ae82de06eb

📥 Commits

Reviewing files that changed from the base of the PR and between ff07462 and a3ede50.

📒 Files selected for processing (1)
  • pipeline/outputs/kafka.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • pipeline/outputs/kafka.md

📝 Walkthrough

Walkthrough

The Kafka output documentation adds Confluent Schema Registry settings, runtime Avro schema resolution behavior, authentication and failover details, caching and retry information, and YAML/INI configuration examples.

Changes

Kafka Schema Registry

Layer / File(s) Summary
Schema Registry configuration and resolution
pipeline/outputs/kafka.md
Documents authentication, Confluent framing, subject and version selection, registry URLs, schema lookup precedence, caching, retries, endpoint failover, aliases, and YAML/INI examples.

Estimated code review effort: 1 (Trivial) | ~5 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the documentation of Kafka Schema Registry support.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@eschabell

Copy link
Copy Markdown
Collaborator Author

@patrick-stephens ready for review!

Comment thread pipeline/outputs/kafka.md Outdated
@eschabell eschabell added 5.1 and removed 5.0.9 labels Aug 6, 2026
…or schema registry

Signed-off-by: Eric D. Schabell <eric@schabell.org>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pipeline/outputs/kafka.md`:
- Around line 242-246: Update the authenticated Schema Registry examples around
schema_registry_url to use https:// endpoints, including the additional example,
so schema_registry_http_user and schema_registry_http_passwd credentials are not
sent over HTTP.
- Around line 221-225: Update the high-availability description near the schema
registry configuration to state that endpoint failover applies only to retry
attempts during the initial schema fetch, before the schema is successfully
resolved and cached. Remove the implication that the plugin performs later
schema resolutions or resumes from the last successful endpoint afterward.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6223267c-a668-4532-97c1-a448b8b63ab2

📥 Commits

Reviewing files that changed from the base of the PR and between 0eb06a6 and ff07462.

📒 Files selected for processing (1)
  • pipeline/outputs/kafka.md

Comment thread pipeline/outputs/kafka.md Outdated
Comment thread pipeline/outputs/kafka.md Outdated
…security

  - Scope endpoint failover to the initial schema fetch only; the schema
    is cached for the plugin's lifetime and the registry isn't re-contacted
  - Use https:// in the authenticated Schema Registry example so basic
    auth credentials aren't shown going out over plain HTTP

Signed-off-by: Eric D. Schabell <eric@schabell.org>
@patrick-stephens
patrick-stephens merged commit ed1aac3 into fluent:master Aug 7, 2026
8 checks passed
patrick-stephens pushed a commit that referenced this pull request Aug 7, 2026
* docs: pipeline: outputs: kafka: document schema registry support

Document the schema_registry_* parameters for resolving Avro schemas
from a Confluent Schema Registry, including subject and ID based
resolution, basic and bearer authentication, multiple registry
endpoints with failover, and the accepted dotted key spellings.

Note, covers code merges without corresponding docs PR.

Signed-off-by: Eric D. Schabell <eric@schabell.org>

* docs: pipeline: outputs: kafka: note Avro encoder build requirement for schema registry

Signed-off-by: Eric D. Schabell <eric@schabell.org>

* docs: pipeline: outputs: kafka: fix schema registry doc accuracy and security

  - Scope endpoint failover to the initial schema fetch only; the schema
    is cached for the plugin's lifetime and the registry isn't re-contacted
  - Use https:// in the authenticated Schema Registry example so basic
    auth credentials aren't shown going out over plain HTTP

Signed-off-by: Eric D. Schabell <eric@schabell.org>

---------

Signed-off-by: Eric D. Schabell <eric@schabell.org>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

5.1 waiting-on-review Waiting on a review from mainteners

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants