scylla start very slowly, Spend a lot of time for Loading repair history

[zey1996]

Installation details
Scylla version: 5.2.11-arm64
Cluster size: 4 Node
OS: CentOS
Hardware details (for performance issues) Delete if unneeded
Platform: kubernetes containerd
Hardware:
memory=320G
cpu:

Architecture:          aarch64
Byte Order:            Little Endian
CPU(s):                128
On-line CPU(s) list:   0-127
Thread(s) per core:    1
Core(s) per socket:    64
Socket(s):             2
NUMA node(s):          4
Model:                 0
CPU max MHz:           2600.0000
CPU min MHz:           200.0000
BogoMIPS:              200.00
L1d cache:             64K
L1i cache:             64K
L2 cache:              512K
L3 cache:              32768K
NUMA node0 CPU(s):     0-31
NUMA node1 CPU(s):     32-63
NUMA node2 CPU(s):     64-95
NUMA node3 CPU(s):     96-127
Flags:                 fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp asimdfhm

Disks: 4 SSD, raid0

---

I use scylla-manager for repair. and I use tombstone_gc = {'mode':'repair'} on my table.
First the cluster runs for several days. and I want to rolling restart my cluster.
But I found scylla is very slow to start. It takes more than an hour
and i check the log, found this:

Looks like scylla is loading repair_history.
Then I check the system.repair_history, found out that this table has millions of records.
I try to learn the source code of scylla, but I don't found some code for clean this table,
I guess this table is used by gc. but it makes scylla start too slowly.
How can I fix it? or maybe I can do somethink to clean this table?

[mykaul]

@asias - thoughts?

[MyByte0]

@asias - thoughts?
That two point I found.

1. The repair_service::load_history() has get_tables_metadata().for_each_table_gently, I use get_tables_metadata().parallel_for_each_table replace. Is that ok?
2. There is no logic for table repair_history. Maybe need add ttl for this?

[mykaul]

Now that #16927 is in - what's left here?

[zey1996]

Now that #16927 is in - what's left here?

#16927
It will make the records load faster. but I think we should control the num of records.
#17103

[mykaul]

I see there's work still on #17103 - I assume it might miss 6.0, shall I defer this to 6.1?

[asias]

Fixed by 99b7ccf. Closing.