Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-15017 Fix snapshot load in dual write mode for Quotas and SCRAM #13757

Merged

Conversation

pprovenzano
Copy link
Contributor

@pprovenzano pprovenzano commented May 24, 2023

Create lists of all the objects in the snapshot and add to the lists the objects that
are only in ZK. Then reverse migrate as appropriate.

Added test to validate for SCRAM and Quota

More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.

Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

…the objects that

are only in ZK. Then reverse migrate as appropriate
@mumrah
Copy link
Contributor

mumrah commented May 26, 2023

The jira for this is KAFKA-15017 btw

@pprovenzano pprovenzano changed the title [WIP] Fix snapshot load during dual write. KAFKA-15017 Fix snapshot load in dual write mode for Quotas and SCRAM May 31, 2023
@mumrah mumrah added the kraft label May 31, 2023
Copy link
Contributor

@mumrah mumrah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pprovenzano, LGTM

@mumrah mumrah merged commit 731c8c9 into apache:trunk May 31, 2023
1 check failed
mumrah pushed a commit that referenced this pull request Jun 1, 2023
… SCRAM (#13757)

This patch fixes the case where a ClientQuota or SCRAM credential was added in KRaft, but not written back to ZK. This missed write only occurred when handling a KRaft snapshot. If the changed quota was processed in a metadata delta (which is the typical case), it would be written to ZK.

Reviewers: David Arthur <mumrah@gmail.com>
emissionnebula added a commit to confluentinc/kafka that referenced this pull request Jun 16, 2023
…tream-3.5

* commit 'c2f6f29ca6e1306ac77ec726bac4cd09bd1aa80b': (76 commits)
  KAFKA-15019: Improve handling of broker heartbeat timeouts (apache#13759)
  KAFKA-15003: Fix ZK sync logic for partition assignments (apache#13735)
  MINOR: Add 3.5 upgrade steps for ZK and KRaft (apache#13792)
  KAFKA-15010 ZK migration failover support (apache#13758)
  KAFKA-15017 Fix snapshot load in dual write mode for ClientQuotas and SCRAM  (apache#13757)
  MINOR: Update LICENSE-binary following snappy upgrade (apache#13791)
  Upgrade to snappy v1.1.10.0 (apache#13786)
  KAFKA-15004: Fix configuration dual-write during migration (apache#13767)
  KAFKA-8713: JsonConverter replace.null.with.default should prevent emitting default for Struct fields (apache#13781)
  KAFKA-14996: Handle overly large user operations on the kcontroller (apache#13742)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants