ConcurrentModificationException for the TaskGarbageCollectionStage

[alirezazamani]

In the async TaskGarbageCollectionStage, since the workflow context can be changes in other threads (main pipeline), while looping through all of the contexts we might get the exception below:

377558 [TaskJobPurgeWorker-CLUSTER_TestTaskRebalancerParallel] ERROR org.apache.helix.controller.pipeline.AbstractAsyncBaseStage - Failed to process TASK::TaskGarbageCollectionStage asynchronously
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1442)
at java.util.HashMap$EntryIterator.next(HashMap.java:1476)
at java.util.HashMap$EntryIterator.next(HashMap.java:1474)
at org.apache.helix.task.TaskUtil.workflowGarbageCollection(TaskUtil.java:1054)
at org.apache.helix.controller.stages.TaskGarbageCollectionStage.execute(TaskGarbageCollectionStage.java:55)
at org.apache.helix.controller.pipeline.AbstractAsyncBaseStage$1.run(AbstractAsyncBaseStage.java:47)
at org.apache.helix.controller.GenericHelixController$3.handleEvent(GenericHelixController.java:605)
at org.apache.helix.controller.GenericHelixController$3.handleEvent(GenericHelixController.java:601)
at org.apache.helix.common.DedupEventProcessor.run(DedupEventProcessor.java:42)