fast DDL sort path change to support multiple fast DDL jobs running simultaneously #52596
Labels
affects-8.1
component/ddl
This issue is related to DDL of TiDB.
severity/moderate
type/bug
The issue is confirmed as a bug.
Enhancement
After walk the fast DDL sort path, there should be functionality that a) must remove all folders that won't be used in future, maybe caused by DDL owner switch and has finished this job. And b) optionally remove not running folders to release space for other tasks.
To support a),
ddl
can maintain a memory cache of allprocessing = 1
jobs oftidb_ddl_job
table, if a folder's job ID does not exist in the cache, we can know it's finished by other DDL owner. The cache has some similarity withrunningJobs
which is the picked running job of this DDL owner, maybe we can reuse that structure.To support b), we can reuse the
runningJobs
to know if the job is running. And limit the caller goroutine to not concurrently call b) and modifyrunningJobs
The text was updated successfully, but these errors were encountered: