reduce the execution time of go test -race for package ddl

[zimulala]

Development Task

Update: 2021-05-07 17:35 CST

$ make failpoint-enable
$ cd ddl
$ GO111MODULE=on go test -v -vet=off -p 5 -timeout 20m -race  

The execution time of more than 5s test cases in the different test suite are listed as the following:

suiteName.testCaseName	execTime
testDDLSerialSuite.TestCancelJob	5.112s
testSerialSuite.TestCreateTableNoBlock	5.217s
testDBSuite2.TestTableLock	5.238s
testColumnSuite.TestColumn	5.524s
testDBSuite4.TestCancelDropPrimaryKey	5.686s
testDBSuite4.TestAnonymousIndex	5.772s
testDBSuite2.TestLock	5.883s
testStateChangeSuite.TestDDLIfExists	6.034s
testStateChangeSuite.TestDropNotNullColumn	6.065s
testIntegrationSuite9.TestMultiRegionGetTableEndHandle	6.171s
testIntegrationSuite2.TestUniqueKeyNullValueClusterIndex	6.200s
testDDLSuite.TestParallelDDL	6.247s
testIntegrationSuite5.TestAlterTableAddPartitionByList	6.483s
testIntegrationSuite1.TestPartitionAddIndex	6.705s
testIntegrationSuite4.TestAlterIndexVisibility	6.737s
testDBSuite1.TestAlterTableAlterPartition	6.739s
testSequenceSuite.TestSequenceAsDefaultValue	6.880s
testIntegrationSuite5.TestAlterTableAddPartition	7.090s
testStateChangeSuite.TestParallelAlterModifyColumnAndAddPK	7.114s
testStateChangeSuite.TestParallelAlterSchemaCharsetAndCollate	7.460s
testDBSuite3.TestCancelDropColumn	7.757s
testIntegrationSuite9.TestAutoRandomChangeFromAutoInc	7.758s
testIntegrationSuite6.TestAddColumnTooMany	8.138s
testDBSuite3.TestCancelDropColumns	8.190s
testDBSuite5.TestCancelTruncateTable	8.214s
testColumnTypeChangeSuite.TestColumnTypeChangeFromDateTimeTypeToOthers	8.248s
testDBSuite4.TestCancelAddIndex1	8.486s
testIntegrationSuite3.TestChangingCharsetToUtf8	9.469s
testIntegrationSuite5.TestAlterTableDropPartition	9.651s
testDBSuite3.TestAddAnonymousIndex	9.661s
testDBSuite5.TestAddIndexForGeneratedColumn	9.766s
testColumnTypeChangeSuite.TestColumnTypeChangeFromStringToOthers	10.079s
testPartitionSuite.TestAddPartitionReplicaBiggerThanTiFlashStores	10.297s
testIntegrationSuite3.TestCreateTableWithAutoIdCache	10.331s
testIntegrationSuite3.TestAlterColumn	10.710s
testDBSuite4.TestComment	10.725s
testColumnTypeChangeSuite.TestColumnTypeChangeFromNumericToOthers	11.209s
testIntegrationSuite5.TestPartitionAddPrimaryKey	11.225s
testDBSuite2.TestAddUniqueIndexRollback	11.411s
testDBSuite2.TestDropIndex	11.670s
testDDLTableSplitSuite.TestTableSplit	11.825s
testDBSuite7.TestAddIndexWithPK	11.935s
testIntegrationSuite3.TestChangeColumnPosition	12.433s
testIntegrationSuite1.TestCreateTableWithListColumnsPartition	12.692s
testIntegrationSuite4.TestExchangePartitionTableCompatiable	12.846s
testDBSuite5.TestAddIndex5	13.004s
testDBSuite1.TestAddPrimaryKeyRollback2	13.150s
testIntegrationSuite3.TestAlterAlgorithm	13.268s
testDBSuite3.TestFKOnGeneratedColumns	13.521s
testIntegrationSuite9.TestPrimaryKey	13.547s
testIntegrationSuite2.TestPartitionDropPrimaryKey	14.480s
testDBSuite5.TestAlterPrimaryKey	15.104s
testDBSuite1.TestCharacterSetInColumns	15.189s
testDBSuite6.TestDropColumn	15.283s
testDBSuite4.TestAddIndex4	15.536s
testStateChangeSuite.TestShowIndex	15.593s
testSerialDBSuite.TestModifyColumnnReorgInfo	15.831s
testIntegrationSuite9.TestInvisibleIndex	16.795s
testDBSuite1.TestAddIndex1	16.838s
testDBSuite2.TestAddIndex2	18.057s
testIntegrationSuite7.TestDuplicateErrorMessage	18.187s
testColumnTypeChangeSuite.TestColumnTypeChangeFromJsonToOthers	18.957s
testSequenceSuite.TestShowCreateSequence	19.774s
testDBSuite3.TestAddIndex3	19.844s
testIntegrationSuite4.TestPartitionCancelAddIndex	20.191s
testDBSuite5.TestAddPrimaryKeyRollback1	20.680s
testDBSuite1.TestColumn	22.858s
testDBSuite4.TestAddPrimaryKey4	23.818s
testSerialDBSuite.TestCommitTxnWithIndexChange	25.053s
testIntegrationSuite4.TestChangingTableCharset	25.201s
testDBSuite1.TestAddPrimaryKey1	28.116s
testIntegrationSuite4.TestAddPartitionTooManyPartitions	30.405s
testDBSuite2.TestAddPrimaryKey2	31.668s
testIntegrationSuite6.TestCreateTableTooLarge	36.040s
testIntegrationSuite1.TestCreateTableWithListPartition	38.099s
testDBSuite3.TestAddPrimaryKey3	38.724s
testIntegrationSuite3.TestMultiRegionGetTableEndHandle	42.431s
testSequenceSuite.TestSequenceFunction	42.713s
testIntegrationSuite2.TestPartitionCancelAddPrimaryKey	48.912s
testDBSuite3.TestCancelAddIndex	52.152s
testDBSuite5.TestCancelAddPrimaryKey	53.672s
testIntegrationSuite5.TestBackwardCompatibility	78.994s
testIntegrationSuite4.TestIndexOnMultipleGeneratedColumn	90.607s
testDBSuite1.TestModifyColumnTime	95.554s
testIntegrationSuite1.TestDisableTablePartition	147.742s

There are some test cases we may refine further:

suiteName.testCaseName	execTime	ranking	creator
testDBSuite1.TestModifyColumnTime	95.554s	1	@jianzhiyao
testIntegrationSuite4.TestIndexOnMultipleGeneratedColumn	90.607s	1	@jianzhiyao
testIntegrationSuite5.TestBackwardCompatibility	78.994s	1	@zimulala
testDBSuite5.TestCancelAddPrimaryKey	53.672s	1
testDBSuite3.TestCancelAddIndex	52.152s	1
testIntegrationSuite2.TestPartitionCancelAddPrimaryKey	48.912s	1
testSequenceSuite.TestSequenceFunction	42.713s	1	@AilinKid
testIntegrationSuite3.TestMultiRegionGetTableEndHandle	42.431s	1
testDBSuite3.TestAddPrimaryKey3	38.724s	2
testIntegrationSuite1.TestCreateTableWithListPartition	38.099s	1	@crazycs520
testIntegrationSuite6.TestCreateTableTooLarge	36.040s	1
testDBSuite2.TestAddPrimaryKey2	31.668s	1
testIntegrationSuite4.TestAddPartitionTooManyPartitions	30.405s	1

Archived 2021-04-25

Recently optimized PR:

• quickly process the next job after processing the cancelled DDL job ddl: quickly process the next job after processing the cancelled DDL job #24146
  Before this PR:
  Executing go test takes 196.455s in the DDL package.
  After this PR:
  Executing go test takes 134.877s in the DDL package.

[xhebox]

@zimulala Actually, I am wonering skip -race for some of the tests. The patch from @jianzhiyao may improve the speed but will consume more memory. TestModifyColumnTime is 8s vs 30s on my PC without race or with race.

[zimulala]

@xhebox Some tests can skip '-race', and which tests you are working on can be explained here. Avoid duplication of work.

[AilinKid]

One question: why separate tests into small cases can reduce the execution time/memory usage (run parallelly? / mem resue?).

[xhebox]

One question: why separate tests into small cases can reduce the execution time/memory usage (run parallelly? / mem resue?).

Maybe mem will increase, but concurrency does help.

[AilinKid]

One question: why separate tests into small cases can reduce the execution time/memory usage (run parallelly? / mem resue?).

Maybe mem will increase, but concurrency does help.

Seems worthy

[zimulala]

Before CI using PingCAP-QE/ci#194.

branch	go test ddl package with race( exec go test -v -vet=off -p 5 -timeout 20m -race locally)	CI (Race Test Chunk #7 ddl-other)
before this PR(#24704)	566s	6m 13s
after this PR(#24704)	505s	6m33s

[zimulala]

After CI using PingCAP-QE/ci#194. Ci (Race Test Chunk #7 ddl-other) is not accurate, so there is no comparison here.

branch	CI (Race Test Chunk #7 ddl-other) in local
before this PR(#24704)	375s
after this PR	267s

[zimulala]

After PR #24780:

branch	go test ddl package with race( exec go test -v -vet=off -p 5 -timeout 20m -race locally)	CI (Race Test Chunk #7 ddl-other)	CI (Race Test Chunk #7 ddl-other) in local
before this PR(#24704)	505s	6m 22s	379s
after this PR (#24780)	405s	2m 45s	199s

[zimulala]

The time of CI (Race Test Chunk #7 ddl-other) is reduced to 3min, and this issue can be closed first.