Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config, sql: completing partial zone configs #32059

Closed

Conversation

ridwanmsharif
Copy link

@ridwanmsharif ridwanmsharif commented Oct 31, 2018

With cascading zone configs, as explained in the RFC it is possible
to get into situations where the zones find themselves in an invalid
state as a result of changes made to parent zone configs. This addresses
that issue by writing fields in tandem in the partial zone.

Release note (sql change): Setting only one bound for the range sizes or
setting per-replica constraints will implicitly set other fields in the
zone config for future validation purposes.

@ridwanmsharif ridwanmsharif requested review from benesch and a team October 31, 2018 19:12
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@benesch
Copy link
Contributor

benesch commented Oct 31, 2018

Just to double-check my understanding, if I run

ALTER TABLE f CONFIGURE ZONE USING range_min_bytes = 4MB
ALTER RANGE default CONFIGURE ZONE USING range_max_bytes = 128MB

the update to the default max bytes won't propagate to f, right? I think I'd find this quite surprising as a user. I was expecting that

ALTER TABLE f CONFIGURE ZONE USING range_min_bytes = 4MB

would return an error, and we'd require users to do something like:

ALTER TABLE f CONFIGURE ZONE USING range_min_bytes = 4MB, range_max_bytes = COPY FROM PARENT

Thoughts? If there was a discussion on this somewhere, apologies! I'd also be curious to get @a-robinson's and @knz's thoughts if they haven't weighed in already.

Copy link
Author

@ridwanmsharif ridwanmsharif left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ALTER TABLE f CONFIGURE ZONE USING range_min_bytes = 4MB, range_max_bytes = COPY FROM PARENT

This is essentially what is happening here but implicitly (which I prefer). The issue of the changes to the default max bytes not applying to the child zone will exist regardless, unless we write validation code that validates all descendants when validating a given zone when making a change. There we have a penalty for sure. We talked about this being a problem in the RFC and thought that having these two pairs written in tandem would avoid problems later on fairly easily and it'd me appropriate for most cases. That was a while ago, we may have changed our mind since then. I think the reason we moved away from the descendants validation idea was also in part due to the complexity that would add (in addition to the time penalty), so this + some documentation of what the new behavior under cascading looks like, seemed preferable.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained

With cascading zone configs, as explained in the RFC it is possible
to get into situations where the zones find themselves in an invalid
state as a result of changes made to parent zone configs. This addresses
that issue by writing fields in tandem in the partial zone.

Release note (sql change): Setting only one bound for the range sizes or
setting per-replica constraints will implicitly set other fields in the
zone config for future validation purposes.
@benesch
Copy link
Contributor

benesch commented Oct 31, 2018

This is essentially what is happening here but implicitly (which I prefer)

Yes, exactly—it's the implicitness that scares me. To be clear I'm very much in favor of a solution that doesn't involve descendent validation. I just think that we need to force users to explicitly indicate that they understand when they set field A that they're setting B along with it.

Copy link
Author

@ridwanmsharif ridwanmsharif left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessarily disagreeing but to be clear, you'd rather have the user figure out what the range_max_bytes are from the default and only then apply a change to them in tandem manually? Also, what about for per-replica constraints?
when someone does something like:
ALTER ... CONFIGURE ZONE USING constraints = '{+region=west: 2, +region=east: 3}'"
at this point we just solidify the num_replicas that was being applied to the zone.

So in this case, later on when a parent's num_replicas value reduces to < 5, we are rest assured the value for this zone is still >= 5 because of the implicit setting before. Are you saying you want the client to now have to do this instead?:
ALTER ... CONFIGURE ZONE USING num_replicas = num_ge_5, constraints = '{+region=west: 2, +region=east: 3}'"

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained

@benesch
Copy link
Contributor

benesch commented Oct 31, 2018

Are you saying you want the client to now have to do this instead?:
ALTER ... CONFIGURE ZONE USING num_replicas = num_ge_5, constraints = '{+region=west: 2, +region=east: 3}'"

Yes, exactly. That way the user will hopefully realize that they are overriding both values. And in the common case where you only intend to set one of the values and want to copy the other from the parent, I think COPY FROM PARENT is a reasonable choice. So in your example:

ALTER ... CONFIGURE ZONE USING num_replicas = COPY FROM PARENT, constraints = '{+region=west: 2, +region=east: 3}'"

But before you go down this road I definitely want to hear what @knz and @a-robinson think.

@ridwanmsharif
Copy link
Author

Taking a second stab at this in #32861. We're not moving forward with this approach.

@ridwanmsharif ridwanmsharif deleted the zone-validation branch December 5, 2018 19:15
craig bot pushed a commit that referenced this pull request Dec 12, 2018
32861: sql: add validation to cascading zone configs r=ridwanmsharif a=ridwanmsharif

I found a few hours free and so took a second stab at #32059. Thought it'd be fun and new to try adding something to SQL for this, so @knz please feel free to tell me if what I've done here makes no sense. 

There are to be two changes to be made: 
1) Allow for the `COPY FROM PARENT` command so users can do something like:
` ALTER ... CONFIGURE ZONE USING range_min_bytes = ..., range_max_bytes = COPY FROM PARENT`

2) Add validation to the zone configs so certain fields are written in tandem. The fields are:
      - `NumReplicas` and `Constraints` (Only when per-replica constraints are in use)
      - `RangeMinBytes` and `RangeMaxBytes`
      - `LeasePreferences` and `Constraints`

Each of the changes are contained in their own commits.

Co-authored-by: Ridwan Sharif <ridwan@cockroachlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants