Skip to content

Commit

Permalink
*: audit operation names in new schema changer
Browse files Browse the repository at this point in the history
This PR made non-functional changes related to new schema changer ops:
1. Rename all status changing operation on Index and Column to comform
to the pattern: `Make[StatusA][Index/Column][StatusB]`

2. Rename a few constant/field/function names to be consistent with the
principle of "being explicit":
  2.1. Rename operation name `XxxGcXxx` to `XxxGCXxx`;
  2.2. Rename field name in operations `DescID` to `DescriptorID`;
  2.3. Rename descpb.DELETE_AND_WRITE_ONLY to descpb.WRITE_ONLY;
  2.4. Rename sql.RunningStatusDeleteAndWriteOnly to sql.RunningStatusWriteOnly

3. Seveal comments enhencement

4. Rename `DELETE_AND_WRITE_ONLY` to `WRITE_ONLY` in some files under
docs/.

Release note: None
  • Loading branch information
Xiang-Gu committed Sep 22, 2022
1 parent 30ba7c3 commit b31d532
Show file tree
Hide file tree
Showing 363 changed files with 2,946 additions and 2,938 deletions.
14 changes: 7 additions & 7 deletions docs/RFCS/20200909_transactional_schema_changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ interesting cases that will get more attention later.

When dropping a column, the column is logically dropped
immediately when the transaction which is performing the drop commits; the
column goes from `PUBLIC`->`DELETE_AND_WRITE_ONLY`->`DELETE_ONLY`->`DROPPED`.
column goes from `PUBLIC`->`WRITE_ONLY`->`DELETE_ONLY`->`DROPPED`.
The important thing to note is that the writing transaction uses its provisional
view of the table descriptor. The following is fine:

Expand Down Expand Up @@ -227,7 +227,7 @@ Ideally, at this point, the ongoing transaction could treat c1 as a `PUBLIC`
column and could perform reads and writes against it. Let's say that at the
outset `foo` is at version 1 (`foo@v1`). Recall from the [online schema change
RFC] and the [F1 schema change paper] that the steps to adding a column are
`DELETE_ONLY`->`DELETE_AND_WRITE_ONLY`->(maybe backfill)->`PUBLIC`.
`DELETE_ONLY`->`WRITE_ONLY`->(maybe backfill)->`PUBLIC`.

In this proposal, during the execution of the above statement we'd first perform
a single-step schema change in a [child transaction](#child-transactions)
Expand All @@ -242,7 +242,7 @@ transactions to wait for all leases on the old version to be dropped (see
Once the child transaction commits and has evidence that all leases on all nodes
are up-to-date, the statement execution can continue. At this point, a new child
transaction can write and commit the table descriptor which has the column in
`DELETE_AND_WRITE_ONLY` with a pending mutation to perform a backfill. Once all
`WRITE_ONLY` with a pending mutation to perform a backfill. Once all
nodes have adopted this latest version, a backfill of the new index can be
performed. So long as the column does not become `PUBLIC` before the user
transaction commits, other transactions won't observe its side effects (if there
Expand Down Expand Up @@ -290,7 +290,7 @@ described by the descriptor which will be committed. The backfill process
determines which rows to rewrite based on their MVCC timestamp. Specifically
it will only backfill rows which have an MVCC timestamp earlier than the
ModificationTime (i.e. commit timestamp as of [#40581]) which moved the newly
added column to `DELETE_AND_WRITE_ONLY`.
added column to `WRITE_ONLY`.

```sql
postgres=> COMMIT;
Expand Down Expand Up @@ -648,7 +648,7 @@ partition "ALTER TABLE Implementation" {
#LightBlue:Commit child transaction;
#LightBlue:Wait for one version using new child transaction;
}
partition "Move column to DELETE_AND_WRITE_ONLY" {
partition "Move column to WRITE_ONLY" {
#LightBlue:Read descriptor using new child transaction;
note right
The version really should not have changed unless the
Expand All @@ -661,7 +661,7 @@ partition "ALTER TABLE Implementation" {
concurrent with transactional schema changes.
end note
#LightBlue:Write new descriptor version with
`DELETE_AND_WRITE_ONLY` column and index;
`WRITE_ONLY` column and index;
#LightBlue:Wait for one version using new child transaction;
}
partition "Perform backfill" {
Expand All @@ -677,7 +677,7 @@ partition "ALTER TABLE Implementation" {
partition "Prepare user transaction for next statement" {
:Push the user transaction to a timestamp above the read timestamp
at which all nodes were on a version where the new index and column
are in `DELETE_AND_WRITE_ONLY`;
are in `WRITE_ONLY`;
:Acquire new lease on current version of descriptor;
:Synthesize a descriptor which has the old primary index in
`DELETE_AND_ WRITE_ONLY` and the newly backfilled index as the
Expand Down
4 changes: 2 additions & 2 deletions pkg/ccl/backupccl/backup_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6003,7 +6003,7 @@ func getMockTableDesc(
for _, addingIndex := range addingIndexes {
mutationID++
mockTableDescriptor.Mutations = append(mockTableDescriptor.Mutations, descpb.DescriptorMutation{
State: descpb.DescriptorMutation_DELETE_AND_WRITE_ONLY,
State: descpb.DescriptorMutation_WRITE_ONLY,
Direction: descpb.DescriptorMutation_ADD,
Descriptor_: &descpb.DescriptorMutation_Index{Index: &addingIndex},
MutationID: mutationID,
Expand All @@ -6012,7 +6012,7 @@ func getMockTableDesc(
for _, droppingIndex := range droppingIndexes {
mutationID++
mockTableDescriptor.Mutations = append(mockTableDescriptor.Mutations, descpb.DescriptorMutation{
State: descpb.DescriptorMutation_DELETE_AND_WRITE_ONLY,
State: descpb.DescriptorMutation_WRITE_ONLY,
Direction: descpb.DescriptorMutation_DROP,
Descriptor_: &descpb.DescriptorMutation_Index{Index: &droppingIndex},
MutationID: mutationID,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ func SetLocalityRegionalByRow(desc catalog.TableDescriptor) catalog.TableDescrip
// Yes, this does modify an immutable.
func AddColumnDropBackfillMutation(desc catalog.TableDescriptor) catalog.TableDescriptor {
desc.TableDesc().Mutations = append(desc.TableDesc().Mutations, descpb.DescriptorMutation{
State: descpb.DescriptorMutation_DELETE_AND_WRITE_ONLY,
State: descpb.DescriptorMutation_WRITE_ONLY,
Direction: descpb.DescriptorMutation_DROP,
Descriptor_: &descpb.DescriptorMutation_Column{Column: MakeColumnDesc(desc.GetNextColumnID() - 1)},
})
Expand All @@ -84,7 +84,7 @@ func AddColumnDropBackfillMutation(desc catalog.TableDescriptor) catalog.TableDe
func AddNewColumnBackfillMutation(desc catalog.TableDescriptor) catalog.TableDescriptor {
desc.TableDesc().Mutations = append(desc.TableDesc().Mutations, descpb.DescriptorMutation{
Descriptor_: &descpb.DescriptorMutation_Column{Column: MakeColumnDesc(desc.GetNextColumnID())},
State: descpb.DescriptorMutation_DELETE_AND_WRITE_ONLY,
State: descpb.DescriptorMutation_WRITE_ONLY,
Direction: descpb.DescriptorMutation_ADD,
MutationID: 0,
Rollback: false,
Expand All @@ -96,7 +96,7 @@ func AddNewColumnBackfillMutation(desc catalog.TableDescriptor) catalog.TableDes
// Yes, this does modify an immutable.
func AddPrimaryKeySwapMutation(desc catalog.TableDescriptor) catalog.TableDescriptor {
desc.TableDesc().Mutations = append(desc.TableDesc().Mutations, descpb.DescriptorMutation{
State: descpb.DescriptorMutation_DELETE_AND_WRITE_ONLY,
State: descpb.DescriptorMutation_WRITE_ONLY,
Direction: descpb.DescriptorMutation_ADD,
Descriptor_: &descpb.DescriptorMutation_PrimaryKeySwap{PrimaryKeySwap: &descpb.PrimaryKeySwap{}},
})
Expand All @@ -107,7 +107,7 @@ func AddPrimaryKeySwapMutation(desc catalog.TableDescriptor) catalog.TableDescri
// Yes, this does modify an immutable.
func AddNewIndexMutation(desc catalog.TableDescriptor) catalog.TableDescriptor {
desc.TableDesc().Mutations = append(desc.TableDesc().Mutations, descpb.DescriptorMutation{
State: descpb.DescriptorMutation_DELETE_AND_WRITE_ONLY,
State: descpb.DescriptorMutation_WRITE_ONLY,
Direction: descpb.DescriptorMutation_ADD,
Descriptor_: &descpb.DescriptorMutation_Index{Index: &descpb.IndexDescriptor{}},
})
Expand All @@ -118,7 +118,7 @@ func AddNewIndexMutation(desc catalog.TableDescriptor) catalog.TableDescriptor {
// Yes, this does modify an immutable.
func AddDropIndexMutation(desc catalog.TableDescriptor) catalog.TableDescriptor {
desc.TableDesc().Mutations = append(desc.TableDesc().Mutations, descpb.DescriptorMutation{
State: descpb.DescriptorMutation_DELETE_AND_WRITE_ONLY,
State: descpb.DescriptorMutation_WRITE_ONLY,
Direction: descpb.DescriptorMutation_DROP,
Descriptor_: &descpb.DescriptorMutation_Index{Index: &descpb.IndexDescriptor{}},
})
Expand Down
8 changes: 4 additions & 4 deletions pkg/ccl/schemachangerccl/testdata/end_to_end/create_index
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ upsert descriptor #104
version: 4
mutationId: 1
- state: DELETE_ONLY
+ state: DELETE_AND_WRITE_ONLY
+ state: WRITE_ONLY
name: t1
nextColumnId: 4
...
Expand Down Expand Up @@ -233,7 +233,7 @@ upsert descriptor #104
version: 4
mutationId: 1
- state: MERGING
+ state: DELETE_AND_WRITE_ONLY
+ state: WRITE_ONLY
- direction: ADD
index:
...
Expand Down Expand Up @@ -326,7 +326,7 @@ upsert descriptor #104
- - money
- version: 4
- mutationId: 1
- state: DELETE_AND_WRITE_ONLY
- state: WRITE_ONLY
- - direction: ADD
- index:
- createdExplicitly: true
Expand All @@ -337,7 +337,7 @@ upsert descriptor #104
...
version: 4
mutationId: 1
- state: DELETE_AND_WRITE_ONLY
- state: WRITE_ONLY
+ state: DELETE_ONLY
name: t1
nextColumnId: 4
Expand Down
14 changes: 7 additions & 7 deletions pkg/ccl/schemachangerccl/testdata/explain/create_index
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ Schema change plan for CREATE INDEX ‹id1› ON ‹defaultdb›.‹public›.
│ ├── 1 element transitioning toward TRANSIENT_ABSENT
│ │ └── ABSENT → DELETE_ONLY TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 0, SourceIndexID: 1}
│ └── 10 Mutation operations
│ ├── MakeAddedIndexBackfilling {"IsSecondaryIndex":true}
│ ├── MakeAbsentIndexBackfilling {"IsSecondaryIndex":true}
│ ├── AddIndexPartitionInfo {"Partitioning":{"IndexID":2,"TableID":104}}
│ ├── MakeAddedTempIndexDeleteOnly {"IsSecondaryIndex":true}
│ ├── MakeAbsentTempIndexDeleteOnly {"IsSecondaryIndex":true}
│ ├── AddColumnToIndex {"ColumnID":1,"IndexID":3,"TableID":104}
│ ├── AddColumnToIndex {"ColumnID":2,"IndexID":3,"Ordinal":1,"TableID":104}
│ ├── AddColumnToIndex {"ColumnID":3,"IndexID":3,"Kind":2,"TableID":104}
Expand All @@ -43,7 +43,7 @@ Schema change plan for CREATE INDEX ‹id1› ON ‹defaultdb›.‹public›.
│ │ ├── 1 element transitioning toward TRANSIENT_ABSENT
│ │ │ └── DELETE_ONLY → WRITE_ONLY TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 0, SourceIndexID: 1}
│ │ └── 3 Mutation operations
│ │ ├── MakeAddedIndexDeleteAndWriteOnly {"IndexID":3,"TableID":104}
│ │ ├── MakeDeleteOnlyIndexWriteOnly {"IndexID":3,"TableID":104}
│ │ ├── SetJobStateOnDescriptor {"DescriptorID":104}
│ │ └── UpdateSchemaChangerJob {"RunningStatus":"PostCommitPhase ..."}
│ ├── Stage 2 of 7 in PostCommitPhase
Expand Down Expand Up @@ -81,7 +81,7 @@ Schema change plan for CREATE INDEX ‹id1› ON ‹defaultdb›.‹public›.
│ ├── 1 element transitioning toward PUBLIC
│ │ └── WRITE_ONLY → VALIDATED SecondaryIndex:{DescID: 104, IndexID: 2, ConstraintID: 0, TemporaryIndexID: 3, SourceIndexID: 1}
│ └── 1 Validation operation
│ └── ValidateUniqueIndex {"IndexID":2,"TableID":104}
│ └── ValidateIndex {"IndexID":2,"TableID":104}
└── PostCommitNonRevertiblePhase
├── Stage 1 of 2 in PostCommitNonRevertiblePhase
│ ├── 2 elements transitioning toward PUBLIC
Expand All @@ -91,15 +91,15 @@ Schema change plan for CREATE INDEX ‹id1› ON ‹defaultdb›.‹public›.
│ │ └── WRITE_ONLY → TRANSIENT_DELETE_ONLY TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 0, SourceIndexID: 1}
│ └── 5 Mutation operations
│ ├── SetIndexName {"IndexID":2,"Name":"id1","TableID":104}
│ ├── MakeDroppedIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── MakeAddedSecondaryIndexPublic {"IndexID":2,"TableID":104}
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── MakeValidatedSecondaryIndexPublic {"IndexID":2,"TableID":104}
│ ├── SetJobStateOnDescriptor {"DescriptorID":104}
│ └── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"PostCommitNonRev..."}
└── Stage 2 of 2 in PostCommitNonRevertiblePhase
├── 1 element transitioning toward TRANSIENT_ABSENT
│ └── TRANSIENT_DELETE_ONLY → TRANSIENT_ABSENT TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 0, SourceIndexID: 1}
└── 4 Mutation operations
├── CreateGcJobForIndex {"IndexID":3,"TableID":104}
├── CreateGCJobForIndex {"IndexID":3,"TableID":104}
├── MakeIndexAbsent {"IndexID":3,"TableID":104}
├── RemoveJobStateFromDescriptor {"DescriptorID":104}
└── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"all stages compl..."}
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ └── PUBLIC → ABSENT IndexPartitioning:{DescID: 104, IndexID: 3}
└── 7 Mutation operations
├── LogEvent {"TargetStatus":1}
├── CreateGcJobForIndex {"IndexID":2,"TableID":104}
├── CreateGCJobForIndex {"IndexID":2,"TableID":104}
├── MakeIndexAbsent {"IndexID":2,"TableID":104}
├── CreateGcJobForIndex {"IndexID":3,"TableID":104}
├── CreateGCJobForIndex {"IndexID":3,"TableID":104}
├── MakeIndexAbsent {"IndexID":3,"TableID":104}
├── RemoveJobStateFromDescriptor {"DescriptorID":104}
└── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"all stages compl..."}
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,17 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 3, IndexID: 3}
│ │ └── PUBLIC → ABSENT IndexPartitioning:{DescID: 104, IndexID: 3}
│ └── 6 Mutation operations
│ ├── MakeDroppedIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── LogEvent {"TargetStatus":1}
│ ├── CreateGcJobForIndex {"IndexID":2,"TableID":104}
│ ├── CreateGCJobForIndex {"IndexID":2,"TableID":104}
│ ├── MakeIndexAbsent {"IndexID":2,"TableID":104}
│ ├── SetJobStateOnDescriptor {"DescriptorID":104}
│ └── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"PostCommitNonRev..."}
└── Stage 2 of 2 in PostCommitNonRevertiblePhase
├── 1 element transitioning toward ABSENT
│ └── DELETE_ONLY → ABSENT TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 0, SourceIndexID: 1}
└── 4 Mutation operations
├── CreateGcJobForIndex {"IndexID":3,"TableID":104}
├── CreateGCJobForIndex {"IndexID":3,"TableID":104}
├── MakeIndexAbsent {"IndexID":3,"TableID":104}
├── RemoveJobStateFromDescriptor {"DescriptorID":104}
└── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"all stages compl..."}
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,17 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 3, IndexID: 3}
│ │ └── PUBLIC → ABSENT IndexPartitioning:{DescID: 104, IndexID: 3}
│ └── 6 Mutation operations
│ ├── MakeDroppedIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── LogEvent {"TargetStatus":1}
│ ├── CreateGcJobForIndex {"IndexID":2,"TableID":104}
│ ├── CreateGCJobForIndex {"IndexID":2,"TableID":104}
│ ├── MakeIndexAbsent {"IndexID":2,"TableID":104}
│ ├── SetJobStateOnDescriptor {"DescriptorID":104}
│ └── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"PostCommitNonRev..."}
└── Stage 2 of 2 in PostCommitNonRevertiblePhase
├── 1 element transitioning toward ABSENT
│ └── DELETE_ONLY → ABSENT TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 0, SourceIndexID: 1}
└── 4 Mutation operations
├── CreateGcJobForIndex {"IndexID":3,"TableID":104}
├── CreateGCJobForIndex {"IndexID":3,"TableID":104}
├── MakeIndexAbsent {"IndexID":3,"TableID":104}
├── RemoveJobStateFromDescriptor {"DescriptorID":104}
└── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"all stages compl..."}
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,17 @@ Schema change plan for rolling back CREATE INDEX ‹id1› ON ‹defaultdb›.pu
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 104, ColumnID: 3, IndexID: 3}
│ │ └── PUBLIC → ABSENT IndexPartitioning:{DescID: 104, IndexID: 3}
│ └── 6 Mutation operations
│ ├── MakeDroppedIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── MakeWriteOnlyIndexDeleteOnly {"IndexID":3,"TableID":104}
│ ├── LogEvent {"TargetStatus":1}
│ ├── CreateGcJobForIndex {"IndexID":2,"TableID":104}
│ ├── CreateGCJobForIndex {"IndexID":2,"TableID":104}
│ ├── MakeIndexAbsent {"IndexID":2,"TableID":104}
│ ├── SetJobStateOnDescriptor {"DescriptorID":104}
│ └── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"PostCommitNonRev..."}
└── Stage 2 of 2 in PostCommitNonRevertiblePhase
├── 1 element transitioning toward ABSENT
│ └── DELETE_ONLY → ABSENT TemporaryIndex:{DescID: 104, IndexID: 3, ConstraintID: 0, SourceIndexID: 1}
└── 4 Mutation operations
├── CreateGcJobForIndex {"IndexID":3,"TableID":104}
├── CreateGCJobForIndex {"IndexID":3,"TableID":104}
├── MakeIndexAbsent {"IndexID":3,"TableID":104}
├── RemoveJobStateFromDescriptor {"DescriptorID":104}
└── UpdateSchemaChangerJob {"IsNonCancelable":true,"RunningStatus":"all stages compl..."}
Loading

0 comments on commit b31d532

Please sign in to comment.