Skip to content

Conversation

@guswynn
Copy link
Contributor

@guswynn guswynn commented Mar 15, 2024

Mostly copied #25513, but with 2 differences:

  • Arc<ConfigSet> because in storage rendering we clone the StorageConfiguration struct for convenience.
  • We maintain a StorageParameters in memory in the storage controller and each worker in the storage state, as opposed to copying all the fields into another struct. In this case, I just left dyncfg_updates None.

Motivation

  • This PR adds a known-desirable feature.

Checklist

  • This PR has adequate test coverage / QA involvement has been duly considered.
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • This PR includes the following user-facing behavior changes:

@guswynn guswynn requested a review from a team March 15, 2024 19:05
@guswynn guswynn requested review from a team and danhhz as code owners March 15, 2024 19:05
@guswynn guswynn requested a review from ParkMyCar March 15, 2024 19:05
Copy link
Contributor

@danhhz danhhz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yesssssss

enable_dependency_read_hold_asserts: config.enable_dependency_read_hold_asserts(),
user_storage_managed_collections_batch_duration: config
.user_storage_managed_collections_batch_duration(),
dyncfg_updates: Some(config.dyncfg_updates()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this makes the copy in PersistParameters above redundant, we should probably remove that one

        persist: PersistParameters {
            config_updates: config.dyncfg_updates(),
        },

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to delete this AND PersistParameters entirely!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh that's awesome!

Copy link
Contributor

@danhhz danhhz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@guswynn guswynn requested a review from a team as a code owner March 15, 2024 20:34
Copy link
Contributor

@rjobanp rjobanp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some nits to leave some better in-code comments explaining the semantics of when we populate the dyncfg_updates field and when we don't

@guswynn
Copy link
Contributor Author

guswynn commented Mar 18, 2024

@rjobanp added some comments! merging!

@guswynn guswynn enabled auto-merge March 18, 2024 15:50
@guswynn guswynn merged commit 910ba8f into MaterializeInc:main Mar 18, 2024
@guswynn guswynn deleted the storage-dyncfg branch March 18, 2024 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants