RepoMetadataScheduler stops on/after Problems with MongoDB

[pluralistix]

We've seen problems with the scheduled metadata-refreshs after having problems (e.g. not talking to master and retries used up) with the underlying MongoDB.

[oli99sc]

I will check the code. To help narrow down the issue, could you pelase provide a stacktrace in the logs which could give a hint about where the problem occured.

[oli99sc]

The RepoMetadataScheduler has a scheduled method which ensures (or tries to) that there is a RepoMetadataGeneratorJob for each scheduled repository. This class keeps track of scheduled jobs in a HashMap. The RepoMetadataGeneratorJobs are scheduled with a fixed delay in a ScheduledExecutorService. Unfortunately the documentation of ScheduledExecutorService says "If any execution of the task encounters an exception, subsequent executions are suppressed."
Temporary problems with the connection to the mongodb instance can leads to an inconsisten state of the HashMap as it still contains jobs, that are no longer executed.

I am now working on a fix

[oli99sc]

commit includes testcase to reproduce error condition and prove fix helps