Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: don't enable non-strict retention by default #16077

Merged
merged 1 commit into from
Jan 12, 2024

Conversation

dotnwat
Copy link
Member

@dotnwat dotnwat commented Jan 11, 2024

In #15996 a change was made that enabled space management and non-strict retention mode when upgrading (unlike v23.1 -> v23.2 upgrade in which the features were disabled by default). It also maintained the behavior that new clusters would have the features enabled (the same behavior as a new v23.2 cluster). Only part of this behavior is desired, so this PR reverts the behavior for strict retention setting.

A new 23.3 cluster will have the same behavior as a new 23.2 cluster

  • sm=true
  • strict_retention=false

An upgrade sequence 23.1, 23.2, 23.2 will have

  • sm=true
  • strict_retention=true

NOTE: this requires the same workaround that we deployed for v23.2 to make upgrade options sticky. That is, .retention_local_strict_override needs to be set in upgraded clusters for the option to be sticky.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.3.x
  • v23.2.x
  • v23.1.x

Release Notes

Improvements

  • This PR partially reverts the change such that strict retention remains enabled after upgrade unless it had been explicitly disabled before the upgrade.

Comment on lines 1968 to 1973
# in 23.2 space management exists, but is disabled by default for
# upgraded clusters.
self._upgrade(wipe_cache, self.intermediate_version)
self._check_value_everywhere("space_management_enable", False)
self._check_value_everywhere("retention_local_strict", True)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about clusters that started on 23.2, where retention_local_strict is False because that's the default? We're intentionally flipping that off -- is that right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you asking whether retention_local_string stays False after a 23.2 -> 23.3 upgrade in this case?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think the issue here is two fold. first is that this keeps strict retention enabled for new installations, which isn't what we want. maybe it's just one fold

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kind of, I'm pointing out that the default in 23.2 is False, and it isn't quite captured by the commit message that this is an intentional flip in behavior in upgrade.

I chatted with Noah a bit, and it seems like this blanket setting of retention_local_strict=True isn't what we want, and we should perhaps revert the removal of the legacy default decorator. Otherwise, it's a bit convoluted to describe the upgrade behavior of this config with this change as is.

new 23.3 cluster (strict=True): I think the desired behavior is actually strict=False
23.2 -> 23.3 (strict=False -> strict=True): I don't think this is what we want either. It should leave things as strict=False moving forward
23.1 -> 23.2 -> 23.3 (strict=True -> strict=True -> strict=True): this is okay

@dotnwat dotnwat marked this pull request as draft January 11, 2024 21:35
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
@dotnwat dotnwat marked this pull request as ready for review January 12, 2024 00:16
@dotnwat dotnwat requested a review from andrwng January 12, 2024 00:23
@dotnwat dotnwat merged commit 3c8e9ff into redpanda-data:dev Jan 12, 2024
22 checks passed
@vbotbuildovich
Copy link
Collaborator

/backport v23.3.x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants