ddl: add a channel to limit multiple DDL jobs writing at the same time (#14342)

[sre-bot]

cherry-pick #14342 to release-3.0

---

What problem does this PR solve?

When performing DDL operations on multiple connections at the same time, you may encounter a "Write conflict" error.

What is changed and how it works?

Adding a channel on a single TiDB to limit multiple DDL jobs writing to the DDL job list at the same time results in "Write conflict".

Check List

Tests

• Benchmark test

func addCreateTableJob(ctx sessionctx.Context, d *ddl, dbInfo *model.DBInfo, tblInfo *model.TableInfo) *model.Job {
	job := &model.Job{...}
	task := &limitJobTask{job, make(chan error)}
	d.limitJobCh <- task
	err := <-task.err
...
	return job
}

func BenchmarkAddDDLs(b *testing.B) {
...
	taskCnt := 1
	count := 200
	tblInfos := make([]*model.TableInfo, 0, count*taskCnt)
	for i := 0; i < count*taskCnt; i++ {
		tblInfo := testTableInfo(d, fmt.Sprintf("t_%d", i), 2)
		tblInfos = append(tblInfos, tblInfo)
	}
	wg := sync.WaitGroup{}
	runDDLsFunc := func(start, end int) {
		defer wg.Done()
		for i := start; i < end; i++ {
			addCreateTableJob(ctx, d, dbInfo, tblInfos[i])
		}
	}
	
	b.ResetTimer()
	wg.Add(taskCnt)
	for i := 0; i < taskCnt; i++ {
		go runDDLsFunc(i*count, (i+1)*count)
	}
	wg.Wait()
	b.StopTimer()
}

WC is the error of write confilct.

taskCnt	count	spend time(after)	WC count(after)	spend time(before)	WC count(before)
1	200	0.117ns	7	0115ns	6
3	200	0.254ns	18	1.092	147
5	200	0.419s	41	10.387	1210
5	400	1.894s	148	68.745	3509

Related changes

• Need to cherry-pick to the release branch

Release note

• Adding a channel on a single TiDB to limit multiple DDL jobs writing to the DDL job list at the same time results in "Write conflict".

[sre-bot]

/run-all-tests

[zimulala]

PTAL @djshow832 @AilinKid

[djshow832]

LGTM

[AilinKid]

LGTM

[sre-bot]

/run-all-tests

[sre-bot]

@sre-bot merge failed.

[zimulala]

/run-integration-ddl-test

[zimulala]

/run-all-tests