Config Server pulling repo in each request

[fjvieira]

In the Config Server, every time it tries to fullfil a GET request for a GIT repository, it calls refresh method of JGitEnvironmentRepository, that results in a PULL to GIT repo, what takes a lot of time and can generate issues in case of many concurrent requests. Maybe it should be better to have a timeout threshold and a scheduled task to do it.

[rlynch]

At my company we have a similar problem and are working on a PR which will use messaging to proactively update the locally cloned repos on a git update and almost entirely eliminate the need for a pull request during a confit get request. We want to do some internal testing and hardening before submitting it here but look for it soon. We are not committers so there is no guarantee that it will ever be accepted in SCCS so don't run your business counting on this but we will have something for the community to review in the near future.

[spencergibb]

@rlynch how different is that from the monitor functionality?

[fjvieira]

@spencergibb The monitor functionality is nice. I adapted the monitor solution to use SQS/SNS instead of Cloud Bus and it works very well. My point is that should be a way of overpassing what I described. Nowadays the average response time is more than 1 second. And considering that health check of each Config Client do a GET to print the list of resources, it is an eternity. I disabled this health check in my clients and the response time of /health now is less than 20ms.

[rlynch]

Not much, only difference is it updates the configuration servers instead of the clients. Then we can "mostly" remove the call to git during the findOne method. This also allows us to reduce the locking on the findOne method which really hindered scalability for us. It's coded up but still needs to be tested so it only works in unit test land at this point.

[marcellodesales]

@fjvieira that's definitely a great finding for sure! We were asked to investigate the health check as well, but we considered not using it... I've been working with @rlynch and his patch should help help with your requirement 👍

[kuipercm]

@rlynch (How) Can we help? This is a feature that would be most welcome in our environment as well. The load and response time caused by the git pull is too much at the moment, so avoiding git pulls would be very helpful.

[asarkar]

I've a different thought about this. If we could add caching to config server then the constant Git pull could be reduced. This would have been real easy to do using Spring AOP if JGitEnvironmentRepository.refresh wasn't a private method. AFAIK, Spring AOP can't proxy private methods. Perhaps a small PR can change that.
Of course, using the monitor eliminates the pull totally but at the cost of an added dependency on a queue. And I don't think the monitor works for multiple instances of config server without additional code.

[ghost]

Is it possible to turn off or disable the git pull automatic change update for config server instance as this is as we want to pull the changes only On demand for config server from git. Please let me know if there is a any setting exists already to turn on or off git pull.

[ryanjbaxter]

I believe that is what this enhancement request is for

[ChiragMoradiya]

I assume this has been solved with spring.cloud.config.server.git.refreshRate configuration property.