Skip to content

fix: use --normalize flag value when registering schemas#3332

Merged
David Novicki (david-novicki) merged 5 commits intoconfluentinc:mainfrom
david-novicki:CLI-3689
Apr 24, 2026
Merged

fix: use --normalize flag value when registering schemas#3332
David Novicki (david-novicki) merged 5 commits intoconfluentinc:mainfrom
david-novicki:CLI-3689

Conversation

@david-novicki
Copy link
Copy Markdown
Member

@david-novicki David Novicki (david-novicki) commented Apr 23, 2026

Checklist

  • I have successfully built and used a custom CLI binary, without linter issues from this PR.
  • I have clearly specified in the What section below whether this PR applies to Confluent Cloud, Confluent Platform, or both.
  • I have verified this PR in Confluent Cloud pre-prod or production environment, if applicable.
  • I have verified this PR in Confluent Platform on-premises environment, if applicable.
  • I have attached manual CLI verification results or screenshots in the Test & Review section below.
  • I have added appropriate CLI integration or unit tests for any new or updated commands and functionality.
  • I confirm that this PR introduces no breaking changes or backward compatibility issues.
  • I have indicated the potential customer impact if something goes wrong in the Blast Radius section below.
  • I have put checkmarks below confirming that the feature associated with this PR is enabled in:
    • Confluent Cloud prod
    • Confluent Cloud stag
    • Confluent Platform
    • Check this box if the feature is enabled for certain organizations only

Note on "no breaking changes": This is a bug fix that restores the documented default behavior of --normalize (false). However, any client that was implicitly relying on the previous (undocumented) always-normalize behavior without passing --normalize will observe a change. This is described in detail in the Default behavior after this fix and Blast Radius sections below.

What

Applies to both Confluent Cloud and Confluent Platform.

Bug fix

pkg/schemaregistry.RegisterSchemaWithAuth was passing a hard-coded true as the normalize argument to client.Register, so every schema registration went through the Schema Registry normalization path regardless of what the user set for --normalize. Callers populate RegisterSchemaConfigs.Normalize from the flag (or zero value), but that value was being dropped at the call site.

-response, err := client.Register(schemaCfg.Subject, request, true)
+response, err := client.Register(schemaCfg.Subject, request, schemaCfg.Normalize)

The Normalize field is also now set explicitly at the internal/kafka/command_topic_produce.go call sites per review feedback, so no caller silently relies on the struct zero value.

New flag: confluent kafka topic produce --normalize

To keep kafka topic produce symmetrical with schema-registry schema create, a --normalize flag is now available on the produce command. Before this PR, the produce command had no user-facing way to control normalization — registration during produce was implicitly forced on by the bug above, and would have been implicitly off after the bug fix. Exposing the flag directly is both clearer and preserves user control. Flag default is false, matching schema create.

Default behavior after this fix

The --normalize flag is declared with a default of false on both commands, so after this PR:

  • confluent schema-registry schema create without --normalize sends normalize=false — schemas are registered without normalization by default.
  • confluent schema-registry schema create --normalize sends normalize=true.
  • confluent kafka topic produce without --normalize likewise sends normalize=false.
  • confluent kafka topic produce --normalize sends normalize=true.

This is a visible change from the previous (buggy) behavior, which always sent normalize=true regardless of the flag (on schema create) or ignored normalization entirely from user control (on topic produce).

Blast Radius

  • Customers using confluent schema-registry schema create (Cloud and on-prem) will now get the un-normalized registration behavior by default, matching the flag's documented default of false.
  • Customers who were relying on the (undocumented) always-normalize behavior to register schemas without passing --normalize will see a behavior change. They can restore the prior behavior by passing --normalize.
  • confluent kafka topic produce schema-registration paths are likewise affected: they now send normalize=false by default instead of the previous forced true. Users who want normalization on produce can now opt in via the new --normalize flag.
  • No impact on read paths, listing, or any other schema-registry subcommand.

References

Test & Review

Unit tests added (two):

  1. pkg/schemaregistry/schema_test.goTestRegisterSchemaWithAuth_ForwardsNormalizeFlag. Stands up an httptest Schema Registry and asserts that the outbound normalize query parameter matches RegisterSchemaConfigs.Normalize for both true and false. Verified this test fails on the pre-fix code (hard-coded true) and passes on the fixed code — a genuine regression test for the bug.
  2. internal/kafka/command_topic_produce_test.goTestProduceCommand_NormalizeFlag. Constructs the produce command via newProduceCommand and asserts the --normalize flag is registered with the expected name, default (false), description, and round-trips correctly via cmd.Flags().GetBool. Verified this test fails when the flag declaration is removed from the command — also a genuine regression test.

Help-output golden fixtures updated:

  • test/fixtures/output/kafka/topic/produce-help.golden
  • test/fixtures/output/kafka/topic/produce-help-onprem.golden

These are consumed by test/help_test.go's TestHelp, which auto-walks the command tree and diffs --help output against the goldens. The new --normalize line is inserted right after --schema-id-header to match cobra's output order.

Other verification:

  • Built a local CLI binary with go build -o confluent ./cmd/confluent — build clean. Confirmed ./confluent schema-registry schema create --help and ./confluent kafka topic produce --help both render the --normalize flag correctly.
  • Ran the full unit-test target locally (go test -timeout 0 -count=1 $(go list ./... | grep -v .../test)). All packages touched by this PR (pkg/schemaregistry, internal/kafka, internal/schema-registry) pass, including both new tests. The only failure is a pre-existing, unrelated flake in pkg/flink/internal/controller (details below).
  • Ran go vet on the changed packages (pkg/schemaregistry/..., internal/kafka/..., cmd/confluent/...) — no issues. (Did not run the full golangci-lint suite locally; that will run in CI.)
  • Not yet executed against Cloud or on-prem clusters — would like a reviewer's guidance on whether Schema Registry cluster verification is required for this size of fix, or whether the unit tests + CI are sufficient.

Known pre-existing CI failure (unrelated to this PR)

Semaphore CI (ci/semaphoreci/pr: Confluent CLI) is reporting a failure on this PR. I reproduced the failure locally by running the full unit-test target ($(go list ./... | grep -v .../test)) and traced it to a single pre-existing, unrelated test:

  • Failing test: pkg/flink/internal/controller.TestInteractiveOutputControllerTestSuite/TestCloseTableViewOnUserInput
  • Failure: panic: close of nil channel originating in github.com/gdamore/tcell/v2/tscreen.go:586tview.(*Application).Stop when the test goroutine sends an Escape/CtrlQ key event. Looks like a Go 1.26 / tcell 2.7.4 / tview interaction when no real TTY is attached.

Evidence it is pre-existing and not caused by this PR:

  1. Reproduces identically on origin/main — I checked out the main version of pkg/flink/internal/controller/ and re-ran the test in isolation; same panic, same stack, same line.
  2. This PR does not touch any Flink / TUI / tcell / tview code — only pkg/schemaregistry/schema.go, pkg/schemaregistry/schema_test.go, internal/kafka/command_topic_produce.go, internal/kafka/command_topic_produce_test.go, and the two kafka/topic/produce-help*.golden fixtures.
  3. Recently merged PRs on main show the same CI failure — e.g. PR Update github.com/confluentinc/confluent-kafka-go/v2 to v2.14.1 and add back #3325 #3327 was merged with ci/semaphoreci/pr: FAILURE, so the team has been merging past this flake.
  4. All packages actually touched by this PR (pkg/schemaregistry, internal/kafka, internal/schema-registry) pass unit tests locally.

Proposed path: merge this PR through the same route other recent merges took (the flake is not a regression from this change), and track the flink controller test separately. Happy to file that ticket if one doesn't already exist.

RegisterSchemaWithAuth hardcoded `true` for the normalize parameter
since confluentinc#2449 (schema-registry-sdk-go upgrade), causing every schema
registration to send normalize=true regardless of the --normalize flag.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 23, 2026 16:18
@david-novicki David Novicki (david-novicki) requested a review from a team as a code owner April 23, 2026 16:18
@confluent-cla-assistant
Copy link
Copy Markdown

🎉 All Contributor License Agreements have been signed. Ready to merge.
Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes schema registration so the Schema Registry normalize query parameter follows the CLI’s --normalize flag (instead of always forcing normalization on), aligning behavior with the documented default.

Changes:

  • Forward RegisterSchemaConfigs.Normalize from RegisterSchemaWithAuth into client.Register(...) to honor the caller’s normalization choice.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

}

response, err := client.Register(schemaCfg.Subject, request, true)
response, err := client.Register(schemaCfg.Subject, request, schemaCfg.Normalize)
Copy link

Copilot AI Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RegisterSchemaWithAuth now forwards schemaCfg.Normalize. Any existing callers that do not explicitly set RegisterSchemaConfigs.Normalize will now register schemas with normalization disabled (Go zero value), which is a behavior change from the previous hard-coded true. Please audit/update call sites to set Normalize intentionally (even if defaulting to false) so behavior doesn’t depend on an omitted field.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — addressed in fabd0a9: both command_topic_produce.go call sites now set Normalize: false explicitly instead of relying on the struct zero value. The only other caller (command_schema_create.go) was already passing the flag value explicitly.

}

response, err := client.Register(schemaCfg.Subject, request, true)
response, err := client.Register(schemaCfg.Subject, request, schemaCfg.Normalize)
Copy link

Copilot AI Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are existing integration tests for schema-registry schema create, but none appear to assert that the normalize query parameter matches the --normalize flag. Adding an integration/unit test that fails when the flag value isn’t forwarded would prevent regressions of this bug.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — added in 443d924: pkg/schemaregistry/schema_test.go / TestRegisterSchemaWithAuth_ForwardsNormalizeFlag stands up an httptest Schema Registry and asserts the outbound normalize query parameter matches RegisterSchemaConfigs.Normalize for both true and false. Verified it fails on the pre-fix hard-coded true.

…WithAuth

Covers the regression fixed in 8578b04: a hard-coded `true` in
RegisterSchemaWithAuth caused the `normalize` query parameter to
always be `true` regardless of `RegisterSchemaConfigs.Normalize`.

The test stands up an httptest Schema Registry and asserts the
outbound `normalize` query parameter matches the struct value for
both `true` and `false`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per PR review: now that RegisterSchemaWithAuth forwards
schemaCfg.Normalize instead of hard-coding true, callers that omit
the field silently get the Go zero value. Both topic-produce call
sites (cloud and on-prem) have no --normalize flag surface, so make
their intent explicit with Normalize: false rather than relying on
struct zero values. No user-visible behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Matches the existing --normalize flag on `confluent schema-registry
schema create`. Previously, schema registration during `kafka topic
produce` had no user-controllable path for schema normalization — it
was implicitly forced on by the bug fixed earlier in this PR, and
after that fix was implicitly off. The flag now lets users opt in to
normalization in both the Cloud and on-prem produce flows (default:
false, matching `schema create`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `TestProduceCommand_NormalizeFlag` that constructs the produce
command via newProduceCommand and asserts the --normalize flag is
registered with the expected name, default (false), description, and
round-trips correctly via cmd.Flags().GetBool. Verified the test
fails when the flag declaration is removed.

Also updates the produce-help and produce-help-onprem golden fixtures
consumed by the TestHelp integration suite so the auto-generated help
output matches after adding the new flag.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@david-novicki David Novicki (david-novicki) merged commit 0131aad into confluentinc:main Apr 24, 2026
1 of 2 checks passed
airlock-confluentinc Bot pushed a commit that referenced this pull request Apr 25, 2026
Drive-by: srsdk was in the "default" group alongside testify; gci wants
it in its own "github.com/confluentinc/" group per .golangci.yml.
Pre-existing from #3332; unblocks CI here.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants