[Bug] Concurrent modify syncStateSet and Mark synchronizing frequently when shrink

[echooymxq]

Before Creating the Bug Report

• I found a bug, not just asking a question, which should be created in GitHub Discussions.

• I have searched the GitHub Issues and GitHub Discussions of this repository and believe that this is not a duplicate.

• I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ.

Runtime platform environment

Linux

RocketMQ version

develop

JDK Version

openjdk8

Describe the Bug

rocketmq/store/src/main/java/org/apache/rocketmq/store/ha/autoswitch/AutoSwitchHAService.java

Lines 288 to 295 in cc16a1b

	// If the slaveBrokerId is in syncStateSet but not in connectionCaughtUpTimeTable,
	// it means that the broker has not connected.
	for (Long slaveBrokerId : newSyncStateSet) {
	if (!this.connectionCaughtUpTimeTable.containsKey(slaveBrokerId)) {
	newSyncStateSet.remove(slaveBrokerId);
	isSyncStateSetChanged = true;
	}
	}

1. The newSyncStateSet foreach maybe ConcurrentModificationException and cause the checkSyncStateSetTask stop forever.

2. The connectionCaughtUpTimeTable always not contain masterBrokerId, it will cause mark synchronizing frequently when shrink syncStateSet.

Steps to Reproduce

Use the latest develop branch code, just run in controller mode.

What Did You Expect to See?

No error happen or No shrink if the syncStateSet no change.

What Did You See Instead?

1. shrink replication task stoped.

Additional Context

No response

[RongtongJin]

@echooymxq Good catch！