Refactor the soft delete design #3866

nisiyong · 2021-08-01T15:09:00Z

Which issue(s) this PR fixes:

Fixes #3700

Brief changelog

These CHANGES need to modify MySQL schemas. Upgrade Apollo with steps as follow:

Execute the following SQL scripts to add the deletedAt column
- scripts/sql/delta/v190-v200/apolloconfigdb-v190-v200.sql
- scripts/sql/delta/v190-v200/apolloportaldb-v190-v200.sql
Deploy apollo-configservice
Deploy apollo-adminservice
Deploy apollo-portal
Execute the following SQL scripts to add the unique index
- scripts/sql/delta/v190-v200/apolloconfigdb-v190-v200-after.sql
- scripts/sql/delta/v190-v200/apolloportaldb-v190-v200-after.sql

If you were using Flyway manage your database schemas, upgrade Apollo with steps as follow:

Execute the following maven commands to add the deletedAt column
- mvn -N -Pconfigdb -Dflyway.target=2.0.0 flyway:migrate
- mvn -N -Pportaldb -Dflyway.target=2.0.0 flyway:migrate
Deploy apollo-configservice
Deploy apollo-adminservice
Deploy apollo-portal
Execute the following maven commands to add the unique index
- mvn -N -Pconfigdb flyway:migrate
- mvn -N -Pportaldb flyway:migrate

Useful SQL scripts (Optional)

Scripts to check whether there is any duplicate data

Before you add unique index, you could use the following SQLs to check whether there are duplicate data in your database.

-- ApolloConfigDB data unique check 
select `AppId`,`Secret`,count(*) from `AccessKey` where `IsDeleted`=0 group by `AppId`,`Secret` having count(*) > 1;
select `AppId`,count(*) from `App` where `IsDeleted`=0 group by `AppId` having count(*) > 1;
select `AppId`,`Name`, count(*) from `AppNamespace` where `IsDeleted`=0 group by `AppId`,`Name` having count(*) > 1;
select `AppId`,`Name`,count(*) from `Cluster` where `IsDeleted`=0 group by `AppId`,`Name` having count(*) > 1;
select `AppId`,`ClusterName`,`NamespaceName`,count(*) from `Namespace` where `IsDeleted`=0 group by `AppId`,`ClusterName`,`NamespaceName` having count(*) > 1;
select `NamespaceId`,count(*) from `NamespaceLock` where `IsDeleted`=0 group by `NamespaceId` having count(*) > 1;
select `ReleaseKey`,count(*) from `Release` where `IsDeleted`=0 group by `ReleaseKey` having count(*) > 1;
select `Key`,`Cluster`,count(*) from `ServerConfig` where `IsDeleted`=0 group by `Key`,`Cluster` having count(*) > 1;

-- ApolloPortalDB data unique check 
select `AppId`,count(*) from `App` where `IsDeleted`=0 group by `AppId` having count(*) > 1;
select `AppId`,`Name`, count(*) from `AppNamespace` where `IsDeleted`=0 group by `AppId`,`Name` having count(*) > 1;
select `AppId`,count(*) from `Consumer` where `IsDeleted`=0 group by `AppId` having count(*) > 1;
select `ConsumerId`,`RoleId`,count(*) from `ConsumerRole` where `IsDeleted`=0 group by `ConsumerId`,`RoleId` having count(*) > 1;
select `Token`,count(*) from `ConsumerToken` where `IsDeleted`=0 group by `Token` having count(*) > 1;
select `UserId`,`AppId`,count(*) from `Favorite` where `IsDeleted`=0 group by `UserId`,`AppId` having count(*) > 1;
select `TargetId`,`PermissionType`,count(*) from `Permission` where `IsDeleted`=0 group by `TargetId`,`PermissionType` having count(*) > 1;
select `RoleName`,count(*) from `Role` where `IsDeleted`=0 group by `RoleName` having count(*) > 1;
select `RoleId`,`PermissionId`,count(*) from `RolePermission` where `IsDeleted`=0 group by `RoleId`,`PermissionId` having count(*) > 1;
select `Key`,count(*) from `ServerConfig` where `IsDeleted`=0 group by `Key` having count(*) > 1;
select `UserId`,`RoleId`,count(*) from `UserRole` where `IsDeleted`=0 group by `UserId`,`RoleId` having count(*) > 1;
select `Username`,count(*) from `Users` group by `Username` having count(*) > 1;

Scripts to rollback the unique indices

The following SQLs could rollback the unqiue indices if you want to degrade your application, and if you want to upgrade later, your could execute scripts/sql/delta/v190-v200/*-v190-v200-after.sql again.

-- Restore indices before 2.0.0 with ApolloConfigDB 
Use ApolloConfigDB;
ALTER TABLE `AccessKey`
    ADD INDEX `AppId` (`AppId`(191)),
    DROP INDEX `UK_AppId_Secret_DeletedAt`;
ALTER TABLE `App`
    ADD INDEX `AppId` (`AppId`(191)),
    DROP INDEX `UK_AppId_DeletedAt`;
ALTER TABLE `AppNamespace`
    ADD INDEX `IX_AppId` (`AppId`),
    DROP INDEX `UK_AppId_Name_DeletedAt`;
ALTER TABLE `Cluster`
    ADD INDEX `IX_AppId_Name` (`AppId`,`Name`),
    DROP INDEX `UK_AppId_Name_DeletedAt`;
ALTER TABLE `Namespace`
    ADD INDEX `AppId_ClusterName_NamespaceName` (`AppId`(191),`ClusterName`(191),`NamespaceName`(191)),
    DROP INDEX `UK_AppId_ClusterName_NamespaceName_DeletedAt`;
ALTER TABLE `NamespaceLock`
    ADD UNIQUE INDEX `IX_NamespaceId` (`NamespaceId`),
    DROP INDEX`UK_NamespaceId_DeletedAt`;
ALTER TABLE `Release`
    ADD INDEX `IX_ReleaseKey` (`ReleaseKey`),
    DROP INDEX `UK_ReleaseKey_DeletedAt`;
ALTER TABLE `ServerConfig`
    ADD INDEX `IX_Key` (`Key`),
    DROP INDEX `UK_Key_Cluster_DeletedAt`;

-- Restore indices before 2.0.0 with ApolloPortalDB
Use ApolloPortalDB;
ALTER TABLE `App`
    ADD INDEX `AppId` (`AppId`(191)),
    DROP INDEX `UK_AppId_DeletedAt`;
ALTER TABLE `AppNamespace`
    ADD INDEX `IX_AppId` (`AppId`),
    DROP INDEX `UK_AppId_Name_DeletedAt`;
ALTER TABLE `Consumer`
    ADD INDEX `AppId` (`AppId`(191)),
    DROP INDEX `UK_AppId_DeletedAt`;
ALTER TABLE `ConsumerRole`
    ADD INDEX `IX_ConsumerId_RoleId` (`ConsumerId`,`RoleId`),
    DROP INDEX `UK_ConsumerId_RoleId_DeletedAt`;
ALTER TABLE `ConsumerToken`
    ADD UNIQUE INDEX `IX_Token` (`Token`),
    DROP INDEX `UK_Token_DeletedAt`;
ALTER TABLE `Favorite`
    ADD INDEX `IX_UserId` (`UserId`),
    DROP INDEX `UK_UserId_AppId_DeletedAt`;
ALTER TABLE `Permission`
    ADD INDEX `IX_TargetId_PermissionType` (`TargetId`(191),`PermissionType`),
    DROP INDEX `UK_TargetId_PermissionType_DeletedAt`;
ALTER TABLE `Role`
    ADD INDEX `IX_RoleName` (`RoleName`(191)),
    DROP INDEX `UK_RoleName_DeletedAt`;
ALTER TABLE `RolePermission`
    ADD INDEX `IX_RoleId` (`RoleId`),
    DROP INDEX `UK_RoleId_PermissionId_DeletedAt`;
ALTER TABLE `ServerConfig`
    ADD INDEX `IX_Key` (`Key`),
    DROP INDEX `UK_Key_DeletedAt`;
ALTER TABLE `UserRole`
    ADD INDEX `IX_UserId_RoleId` (`UserId`,`RoleId`),
    DROP INDEX `UK_UserId_RoleId_DeletedAt`;
ALTER TABLE `Users`
    DROP INDEX `UK_Username`;

Follow this checklist to help us incorporate your contribution quickly and easily:

Read the Contributing Guide before making this pull request.
Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
Write necessary unit tests to verify the code.
Run mvn clean test to make sure this pull request doesn't break anything.
Update the CHANGES log.

apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/entity/Item.java

codecov-commenter · 2021-08-09T16:33:35Z

Codecov Report

Merging #3866 (25616e6) into master (b1aba93) will increase coverage by 0.01%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master    #3866      +/-   ##
============================================
+ Coverage     52.54%   52.56%   +0.01%     
- Complexity     2633     2635       +2     
============================================
  Files           486      488       +2     
  Lines         15244    15253       +9     
  Branches       1577     1577              
============================================
+ Hits           8010     8017       +7     
- Misses         6678     6682       +4     
+ Partials        556      554       -2

Impacted Files	Coverage Δ
...m/ctrip/framework/apollo/biz/entity/AccessKey.java	`58.33% <ø> (ø)`
...a/com/ctrip/framework/apollo/biz/entity/Audit.java	`52.94% <ø> (ø)`
...com/ctrip/framework/apollo/biz/entity/Cluster.java	`58.82% <ø> (ø)`
.../com/ctrip/framework/apollo/biz/entity/Commit.java	`5.55% <ø> (ø)`
...p/framework/apollo/biz/entity/GrayReleaseRule.java	`100.00% <ø> (ø)`
...va/com/ctrip/framework/apollo/biz/entity/Item.java	`88.88% <ø> (ø)`
...m/ctrip/framework/apollo/biz/entity/Namespace.java	`61.11% <ø> (ø)`
...m/ctrip/framework/apollo/biz/entity/Privilege.java	`8.33% <ø> (ø)`
...com/ctrip/framework/apollo/biz/entity/Release.java	`82.14% <ø> (ø)`
...ip/framework/apollo/biz/entity/ReleaseHistory.java	`72.41% <ø> (ø)`
... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b1aba93...25616e6. Read the comment docs.

nisiyong · 2021-08-09T16:42:51Z

All checks have passed!!! Unbelievable! I almost gave up.😂😂😂

@nobodyiam
I have fixed the CI, you could take a look first.
Next, we should sort out the UNIQUE INDEX. Good night.

nobodyiam · 2021-08-10T01:19:23Z

All checks have passed!!! Unbelievable! I almost gave up.😂😂😂

@nobodyiam
I have fixed the CI, you could take a look first.
Next, we should sort out the UNIQUE INDEX. Good night.

Congratulations! This is a remarkable milestone achieved!
BTW, it seems com.ctrip.framework.apollo.common.jpa.H2Function is only used in tests so we could move it to test package?

nisiyong · 2021-08-10T14:50:31Z

BTW, it seems com.ctrip.framework.apollo.common.jpa.H2Function is only used in tests so we could move it to test package?

As another module use this Class, we can't move it to the test package in apollo-common.

if you insist, we have two options:

Duplicate this Class in each module test package.

Create a new module apollo-test, move this class in this module, and another module depend on this module with test scope.

<dependency>
    <groupId>com.ctrip.framework.apollo</groupId>
    <artifactId>apollo-test</artifactId>
    <scope>test</scope>
</dependency>

Cause only one class, I prefer to keep it in main package.

apollo-configservice/src/main/resources/import.sql

nobodyiam · 2021-08-11T00:31:14Z

BTW, it seems com.ctrip.framework.apollo.common.jpa.H2Function is only used in tests so we could move it to test package?

As another module use this Class, we can't move it to the test package in apollo-common.

if you insist, we have two options:
Duplicate this Class in each module test package.
Create a new module apollo-test, move this class in this module, and another module depend on this module with test scope.
<dependency>
    <groupId>com.ctrip.framework.apollo</groupId>
    <artifactId>apollo-test</artifactId>
    <scope>test</scope>
</dependency>
Cause only one class, I prefer to keep it in main package.

Thanks for the explanation, I think it's ok to leave it in apollo-common module.

nobodyiam · 2021-09-04T11:24:01Z

@nisiyong would you please also update the CHANGES.md?
@apolloconfig/committers This is a big pull request but I think it's very important as it solves the long-standing problem that we can't apply unique index due to soft delete mechanism. Please help to review and share your idea, thx!

nisiyong · 2021-09-05T02:14:16Z

After completing the unique index DDL, this PR now is ready. Thanks @nobodyiam give me a lot of suggestions.
Cause this PR will bring an important mechanism, I hope some of @apolloconfig/committers could review it.
Issue #3700 could show you the full design, and this PR description shows how to upgrade apollo.

nobodyiam · 2021-09-05T08:16:25Z

I exported the demo site data(access code: yu77) and tried to verify the upgrade steps. However, I met 2 errors when applying the apolloconfigdb-v1_9_0-v1_10_0-after.sql. We need to take a look and see how we can prevent these errors in real production environment.

Duplicate entry 'SampleApp-default-TEST1.kkk-20190227165728-eef1805d89d606f5-0' for key 'UK_AppId_ClusterName_NamespaceName_BranchName_DeletedAt'
It seems there are some dirty data existing for gray release rules

select AppId, ClusterName, NamespaceName, BranchName from GrayReleaseRule group by AppId, ClusterName, NamespaceName, BranchName, DeletedAt having count(1) > 1;

Notice we use the findTopByXXX query in the programs, which explains why it still work with these dirty data.

Duplicate entry '4118--0' for key 'UK_NamespaceId_Key_DeletedAt'
the configuration data has 2 items with key as empty string, which looks like the designed behavior...

Besides the empty key items, there are also some dirty data existing for items:

select NamespaceId, Key, DeletedAt, count(1) from Item where Key != '' group by NamespaceId, Key, DeletedAt having count(1) > 1

scripts/sql/delta/v1_9_0-v1_10_0/apolloconfigdb-v1_9_0-v1_10_0-after.sql

scripts/sql/delta/v1_9_0-v1_10_0/apolloportaldb-v1_9_0-v1_10_0-after.sql

nisiyong · 2021-09-05T15:06:55Z

Duplicate entry 'SampleApp-default-TEST1.kkk-20190227165728-eef1805d89d606f5-0' for key 'UK_AppId_ClusterName_NamespaceName_BranchName_DeletedAt'

In this case, we can find another DML to repair the duplicate dirty data. If it is difficult to do, we can temporarily give up adding a unique index for this table, due to the application logic layer adatping this case.

Duplicate entry '4118--0' for key 'UK_NamespaceId_Key_DeletedAt'

Because empty key is reasonable, I think maybe the unique index should consider the LineNum column.

nobodyiam · 2021-09-06T00:26:02Z

Duplicate entry 'SampleApp-default-TEST1.kkk-20190227165728-eef1805d89d606f5-0' for key 'UK_AppId_ClusterName_NamespaceName_BranchName_DeletedAt'

In this case, we can find another DML to repair the duplicate dirty data. If it is difficult to do, we can temporarily give up adding a unique index for this table, due to the application logic layer adatping this case.

Maybe we could write a program to fix these dirty data? Before the user applies the apolloconfigdb-v1_9_0-v1_10_0-after.sql?

Duplicate entry '4118--0' for key 'UK_NamespaceId_Key_DeletedAt'

Because empty key is reasonable, I think maybe the unique index should consider the LineNum column.

Yeah, this sounds reasonable.

nisiyong · 2021-09-21T06:41:37Z

Duplicate entry 'SampleApp-default-TEST1.kkk-20190227165728-eef1805d89d606f5-0' for key 'UK_AppId_ClusterName_NamespaceName_BranchName_DeletedAt'

@nobodyiam
Shall we consider add the ReleaseId field in unique keys on this table GrayReleaseRule?

nisiyong · 2021-09-21T06:45:58Z

Post some SQLs to check unique data before upgrade.

-- ApolloConfigDB data unique check 
select `AppId`,`Secret`,count(*) from `AccessKey` where `IsDeleted`=0 group by `AppId`,`Secret` having count(*) > 1;
select `AppId`,count(*) from `App` where `IsDeleted`=0 group by `AppId` having count(*) > 1;
select `AppId`,`Name`, count(*) from `AppNamespace` where `IsDeleted`=0 group by `AppId`,`Name` having count(*) > 1;
select `AppId`,`Name`,count(*) from `Cluster` where `IsDeleted`=0 group by `AppId`,`Name` having count(*) > 1;
select `AppId`,`ClusterName`,`NamespaceName`,count(*) from `Namespace` where `IsDeleted`=0 group by `AppId`,`ClusterName`,`NamespaceName` having count(*) > 1;
select `NamespaceId`,count(*) from `NamespaceLock` where `IsDeleted`=0 group by `NamespaceId` having count(*) > 1;
select `ReleaseKey`,count(*) from `Release` where `IsDeleted`=0 group by `ReleaseKey` having count(*) > 1;
select `Key`,`Cluster`,count(*) from `ServerConfig` where `IsDeleted`=0 group by `Key`,`Cluster` having count(*) > 1;

-- ApolloPortalDB data unique check 
select `AppId`,count(*) from `App` where `IsDeleted`=0 group by `AppId` having count(*) > 1;
select `AppId`,`Name`, count(*) from `AppNamespace` where `IsDeleted`=0 group by `AppId`,`Name` having count(*) > 1;
select `AppId`,count(*) from `Consumer` where `IsDeleted`=0 group by `AppId` having count(*) > 1;
select `ConsumerId`,`RoleId`,count(*) from `ConsumerRole` where `IsDeleted`=0 group by `ConsumerId`,`RoleId` having count(*) > 1;
select `Token`,count(*) from `ConsumerToken` where `IsDeleted`=0 group by `Token` having count(*) > 1;
select `UserId`,`AppId`,count(*) from `Favorite` where `IsDeleted`=0 group by `UserId`,`AppId` having count(*) > 1;
select `TargetId`,`PermissionType`,count(*) from `Permission` where `IsDeleted`=0 group by `TargetId`,`PermissionType` having count(*) > 1;
select `RoleName`,count(*) from `Role` where `IsDeleted`=0 group by `RoleName` having count(*) > 1;
select `RoleId`,`PermissionId`,count(*) from `RolePermission` where `IsDeleted`=0 group by `RoleId`,`PermissionId` having count(*) > 1;
select `Key`,count(*) from `ServerConfig` where `IsDeleted`=0 group by `Key` having count(*) > 1;
select `UserId`,`RoleId`,count(*) from `UserRole` where `IsDeleted`=0 group by `UserId`,`RoleId` having count(*) > 1;
select `Username`,count(*) from `Users` where `IsDeleted`=0 group by `Username` having count(*) > 1;

nobodyiam · 2021-09-23T00:52:34Z

Duplicate entry 'SampleApp-default-TEST1.kkk-20190227165728-eef1805d89d606f5-0' for key 'UK_AppId_ClusterName_NamespaceName_BranchName_DeletedAt'

@nobodyiam
Shall we consider add the ReleaseId field in unique keys on this table GrayReleaseRule?

I think adding a ReleaseId field in unique keys doesn't make much sense. As one gray release(AppId+ClusterName+NamespaceName+BranchName) would only target for one releaseId. If there are 2 releaseid for one gray release, only one will be effective.

nisiyong · 2021-12-30T13:00:28Z

Maybe we could write a program to fix these dirty data?

We can do this, but where to trigger this logic when the user upgrades apollo.

nisiyong · 2022-03-12T10:13:18Z

@nobodyiam

I have removed the unique index on tables Item and GrayReleaseRule, and also update SQL scripts for docker-quick-stark and flyway. Please take a look.

nobodyiam

This looks great to me!
BTW, shall we also add some upgrade instructions for those flyway users in this pr's changelog? e.g. first migrate to 2.0.0 and then after services are deployed, migrate to 2.0.1? It looks like this could be achieved by mvn -N -Pportaldb -Dflyway.target=2.0.0 flyway:migrate but I haven't verified it.

apollo-adminservice/src/test/resources/import.sql

apollo-biz/src/test/resources/import.sql

apollo-common/src/main/java/com/ctrip/framework/apollo/common/jpa/H2Function.java

.../main/java/com/ctrip/framework/apollo/common/jpa/SqlFunctionsMetadataBuilderContributor.java

apollo-configservice/src/test/resources/import.sql

apollo-portal/src/test/resources/import.sql

scripts/sql/delta/v190-v200/apolloconfigdb-v190-v200-after.sql

scripts/sql/delta/v190-v200/apolloconfigdb-v190-v200.sql

scripts/sql/delta/v190-v200/apolloportaldb-v190-v200-after.sql

scripts/sql/delta/v190-v200/apolloportaldb-v190-v200.sql

nisiyong · 2022-03-14T15:31:14Z

mvn -N -Pportaldb -Dflyway.target=2.0.0 flyway:migrate

I test the flyway.target option in my local MySQL with an empty schema. It works fine! And the Flyway changelog has been updated. Please take a look.

Also provide my local console output log for you, if you need it.

➜  apollo git:(refactor-soft-delete) mvn -N -Pportaldb -Dflyway.target=2.0.0 flyway:migrate
Running `/Users/nisiyong/CodeProjects/apollo/mvnw`...
[INFO] Scanning for projects...
[INFO] 
[INFO] -----------------< com.ctrip.framework.apollo:apollo >------------------
[INFO] Building Apollo 2.0.0-SNAPSHOT
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] 
[INFO] --- flyway-maven-plugin:5.2.4:migrate (default-cli) @ apollo ---
[INFO] Flyway Community Edition 5.2.4 by Boxfuse
[INFO] Database: jdbc:mysql://localhost:3306 (MySQL 5.7)
[INFO] Creating schema `ApolloPortalDB` ...
[INFO] Creating Schema History table: `ApolloPortalDB`.`flyway_schema_history`
[INFO] Current version of schema `ApolloPortalDB`: null
[INFO] Migrating schema `ApolloPortalDB` to version 1.0.0 - initialization
[WARNING] DB: Changing sql mode 'NO_AUTO_CREATE_USER' is deprecated. It will be removed in a future release. (SQL State: HY000 - Error Code: 3090)
[WARNING] DB: Changing sql mode 'NO_AUTO_CREATE_USER' is deprecated. It will be removed in a future release. (SQL State: HY000 - Error Code: 3090)
[INFO] Migrating schema `ApolloPortalDB` to version 1.1.1 - extend appId
[INFO] Migrating schema `ApolloPortalDB` to version 1.1.2 - extend username
[INFO] Migrating schema `ApolloPortalDB` to version 1.1.3 - add preferred username
[INFO] Migrating schema `ApolloPortalDB` to version 1.1.4 - delegating-password-encoder
[INFO] Migrating schema `ApolloPortalDB` to version 1.1.5 - jdbc-session
[INFO] Migrating schema `ApolloPortalDB` to version 2.0.0 - add-column-deletedat
[INFO] Successfully applied 7 migrations to schema `ApolloPortalDB` (execution time 00:01.445s)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.565 s
[INFO] Finished at: 2022-03-14T23:30:03+08:00
[INFO] ------------------------------------------------------------------------
➜  apollo git:(refactor-soft-delete) mvn -N -Pportaldb flyway:migrate                      
Running `/Users/nisiyong/CodeProjects/apollo/mvnw`...
[INFO] Scanning for projects...
[INFO] 
[INFO] -----------------< com.ctrip.framework.apollo:apollo >------------------
[INFO] Building Apollo 2.0.0-SNAPSHOT
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] 
[INFO] --- flyway-maven-plugin:5.2.4:migrate (default-cli) @ apollo ---
[INFO] Flyway Community Edition 5.2.4 by Boxfuse
[INFO] Database: jdbc:mysql://localhost:3306 (MySQL 5.7)
[INFO] Successfully validated 9 migrations (execution time 00:00.077s)
[INFO] Current version of schema `ApolloPortalDB`: 2.0.0
[INFO] Migrating schema `ApolloPortalDB` to version 2.0.1 - add-unique-key
[INFO] Successfully applied 1 migration to schema `ApolloPortalDB` (execution time 00:00.255s)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.455 s
[INFO] Finished at: 2022-03-14T23:30:16+08:00
[INFO] ------------------------------------------------------------------------

nobodyiam

This looks great to me!
As this is a major change, I think it's better to have another approval.

nobodyiam

LGTM

nisiyong · 2022-03-16T03:04:42Z

@Anilople @lepdou
Do you have time to help with a review recently?

lepdou

@nisiyong @nobodyiam

I have reviewed all changes. I checked as follows:

upgrade steps
all table's unique key
all delete method has updated deleteAt field

It's great and look good to me.

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/entity/po/Permission.java

nobodyiam

ready to merge

nisiyong · 2022-04-05T08:39:52Z

@nobodyiam

I just added some useful SQL scripts to the PR description for users who might need them when upgrading Apollo. Please take a look.

nobodyiam · 2022-04-05T10:50:23Z

@nisiyong looks great!

nisiyong requested review from nobodyiam, JaredTan95 and kezhenxu94 August 1, 2021 15:09

nobodyiam reviewed Aug 4, 2021

View reviewed changes

apollo-biz/src/main/java/com/ctrip/framework/apollo/biz/entity/Item.java Outdated Show resolved Hide resolved

nisiyong force-pushed the refactor-soft-delete branch from 3c2d891 to a89c02b Compare August 9, 2021 16:30

nobodyiam reviewed Aug 11, 2021

View reviewed changes

apollo-configservice/src/main/resources/import.sql Outdated Show resolved Hide resolved

nobodyiam added this to the 1.10.0 milestone Aug 21, 2021

nisiyong force-pushed the refactor-soft-delete branch from 03cf95f to c692f8f Compare August 29, 2021 13:47

nisiyong removed the request for review from kezhenxu94 September 4, 2021 02:28

nisiyong force-pushed the refactor-soft-delete branch from c692f8f to d7d81a7 Compare September 4, 2021 03:05

nisiyong force-pushed the refactor-soft-delete branch from e49082c to a232c63 Compare September 4, 2021 16:22

nisiyong changed the title ~~WIP: Refactor the soft delete design~~ Refactor the soft delete design Sep 4, 2021

nobodyiam reviewed Sep 5, 2021

View reviewed changes

scripts/sql/delta/v1_9_0-v1_10_0/apolloconfigdb-v1_9_0-v1_10_0-after.sql Outdated Show resolved Hide resolved

nobodyiam reviewed Sep 5, 2021

View reviewed changes

scripts/sql/delta/v1_9_0-v1_10_0/apolloportaldb-v1_9_0-v1_10_0-after.sql Outdated Show resolved Hide resolved

nisiyong force-pushed the refactor-soft-delete branch from 132fc43 to 81a3733 Compare November 17, 2021 15:24

nisiyong force-pushed the refactor-soft-delete branch from 81a3733 to 123228a Compare December 30, 2021 12:54

nisiyong added 4 commits March 12, 2022 18:10

fix: add Cluster Column in unique keys on ApolloConfigDB.ServerConfig

c112121

fix: move sql to v190-v200 and polish DML

2a5b718

feat: remove unique index DDL on GrayReleaseRule and Item

82cbdb5

feat: update sql scripts for docker-quick-stark and flyway

2690297

nisiyong force-pushed the refactor-soft-delete branch from 5376008 to 2690297 Compare March 12, 2022 10:10

nobodyiam reviewed Mar 13, 2022

View reviewed changes

nisiyong requested a review from nobodyiam March 14, 2022 14:16

nobodyiam approved these changes Mar 15, 2022

View reviewed changes

nisiyong requested review from a team and removed request for JaredTan95 March 15, 2022 03:46

nobodyiam previously approved these changes Mar 16, 2022

View reviewed changes

Merge branch 'master' into refactor-soft-delete

de69f3d

nisiyong requested review from lepdou and Anilople March 16, 2022 03:01

lepdou previously approved these changes Mar 17, 2022

View reviewed changes

Merge branch 'master' into refactor-soft-delete

25616e6

lepdou dismissed stale reviews from nobodyiam and themself via 25616e6 March 17, 2022 10:46

Anilople reviewed Mar 17, 2022

View reviewed changes

apollo-portal/src/main/java/com/ctrip/framework/apollo/portal/entity/po/Permission.java Show resolved Hide resolved

Merge branch 'master' into refactor-soft-delete

ed480c5

nobodyiam approved these changes Mar 19, 2022

View reviewed changes

nobodyiam merged commit 0223121 into apolloconfig:master Mar 19, 2022

github-actions bot locked and limited conversation to collaborators Mar 19, 2022

nisiyong deleted the refactor-soft-delete branch March 19, 2022 09:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor the soft delete design #3866

Refactor the soft delete design #3866

nisiyong commented Aug 1, 2021 •

edited by nobodyiam

codecov-commenter commented Aug 9, 2021 •

edited

nisiyong commented Aug 9, 2021

nobodyiam commented Aug 10, 2021

nisiyong commented Aug 10, 2021

nobodyiam commented Aug 11, 2021

nobodyiam commented Sep 4, 2021

nisiyong commented Sep 5, 2021

nobodyiam commented Sep 5, 2021 •

edited

nisiyong commented Sep 5, 2021

nobodyiam commented Sep 6, 2021

nisiyong commented Sep 21, 2021 •

edited

nisiyong commented Sep 21, 2021 •

edited

nobodyiam commented Sep 23, 2021

nisiyong commented Dec 30, 2021

nisiyong commented Mar 12, 2022

nobodyiam left a comment

nisiyong commented Mar 14, 2022

nobodyiam left a comment

nobodyiam left a comment

nisiyong commented Mar 16, 2022

lepdou left a comment •

edited

nobodyiam left a comment

nisiyong commented Apr 5, 2022 •

edited

nobodyiam commented Apr 5, 2022

Refactor the soft delete design #3866

Refactor the soft delete design #3866

Conversation

nisiyong commented Aug 1, 2021 • edited by nobodyiam

Which issue(s) this PR fixes:

Brief changelog

Useful SQL scripts (Optional)

Scripts to check whether there is any duplicate data

Scripts to rollback the unique indices

codecov-commenter commented Aug 9, 2021 • edited

Codecov Report

nisiyong commented Aug 9, 2021

nobodyiam commented Aug 10, 2021

nisiyong commented Aug 10, 2021

nobodyiam commented Aug 11, 2021

nobodyiam commented Sep 4, 2021

nisiyong commented Sep 5, 2021

nobodyiam commented Sep 5, 2021 • edited

nisiyong commented Sep 5, 2021

nobodyiam commented Sep 6, 2021

nisiyong commented Sep 21, 2021 • edited

nisiyong commented Sep 21, 2021 • edited

nobodyiam commented Sep 23, 2021

nisiyong commented Dec 30, 2021

nisiyong commented Mar 12, 2022

nobodyiam left a comment

Choose a reason for hiding this comment

nisiyong commented Mar 14, 2022

nobodyiam left a comment

Choose a reason for hiding this comment

nobodyiam left a comment

Choose a reason for hiding this comment

nisiyong commented Mar 16, 2022

lepdou left a comment • edited

Choose a reason for hiding this comment

nobodyiam left a comment

Choose a reason for hiding this comment

nisiyong commented Apr 5, 2022 • edited

nobodyiam commented Apr 5, 2022

nisiyong commented Aug 1, 2021 •

edited by nobodyiam

codecov-commenter commented Aug 9, 2021 •

edited

nobodyiam commented Sep 5, 2021 •

edited

nisiyong commented Sep 21, 2021 •

edited

nisiyong commented Sep 21, 2021 •

edited

lepdou left a comment •

edited

nisiyong commented Apr 5, 2022 •

edited