Run git gc periodically

[ikhoon]

Motivation:

Central Dogma git repository does not run git gc automatically.
git gc will save disk storage and improve performance.

Modifications:

• Add RepositoryGarbageCollectionPlugin that schedules GC with a Quartz cron expression
• Add RepositoryGarbageCollectionConfig in order to configure a GC schedule and the minimum number of new commits for the GC
• Add GitGcRevision that reads and writes the last revision when GC was run.

Result:

• You can now schedule git gc with a Quartz cron expression and the minimum number of new commits.

  "repositoryGarbageCollection": {
    "minNumNewCommits": 1000,
    "schedule": "0 0 * * * ?"
  }

• Fixes Periodic garbage collection of Git repositories #264

[codecov]

Codecov Report

Merging #564 (c215202) into master (091993d) will decrease coverage by 0.27%.
The diff coverage is 49.14%.

@@             Coverage Diff              @@
##             master     #564      +/-   ##
============================================
- Coverage     70.10%   69.83%   -0.28%     
- Complexity     3271     3302      +31     
============================================
  Files           329      332       +3     
  Lines         13030    13219     +189     
  Branches       1405     1427      +22     
============================================
+ Hits           9135     9231      +96     
- Misses         3018     3097      +79     
- Partials        877      891      +14     

Impacted Files	Coverage Δ	Complexity Δ
...ecorp/centraldogma/server/CentralDogmaBuilder.java	52.30% <0.00%> (-1.67%)	23.00 <0.00> (ø)
...internal/storage/repository/RepositoryWrapper.java	29.41% <0.00%> (-1.84%)	8.00 <0.00> (ø)
...al/storage/repository/cache/CachingRepository.java	85.04% <0.00%> (-1.62%)	35.00 <0.00> (ø)
...raldogma/server/storage/repository/Repository.java	66.96% <ø> (ø)	33.00 <0.00> (ø)
...internal/storage/repository/git/GitRepository.java	79.17% <33.33%> (-1.06%)	151.00 <0.00> (ø)
...nal/storage/RepositoryGarbageCollectionPlugin.java	49.35% <49.35%> (ø)	15.00 <15.00> (?)
...internal/storage/repository/git/GitGcRevision.java	51.66% <51.66%> (ø)	10.00 <10.00> (?)
...ogma/server/RepositoryGarbageCollectionConfig.java	88.88% <88.88%> (ø)	4.00 <4.00> (?)
...necorp/centraldogma/server/CentralDogmaConfig.java	80.95% <100.00%> (+0.26%)	45.00 <6.00> (+1.00)
...erver/internal/storage/PurgeSchedulingService.java	73.56% <0.00%> (-3.45%)	16.00% <0.00%> (-3.00%)
... and 7 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 091993d...c215202. Read the comment docs.

[minwoox]

Looks awesome!
Left some nit comments. 😄

[trustin]

We should not perform GC when the gain is not large enough, e.g. we shouldn't perform GC when only a few hundred commits were added since the last GC. We will otherwise see a large amount of disk I/O every day.

[ikhoon]

We should not perform GC when the gain is not large enough, e.g. we shouldn't perform GC when only a few hundred commits were added since the last GC. We will otherwise see a large amount of disk I/O every day.

Good suggestion! The GC will run after 200 commits were added since the last GC. The 200 was referenced in https://docs.gitlab.com/ee/administration/housekeeping.html#manual-housekeeping

[trustin]

I think that's implementation detail which might not be the case in jGit. How about trying to run GC against a large repository using jGit to make sure it doesn't thrash disk on every GC?

[ikhoon]

How about trying to run GC against a large repository using jGit to make sure it doesn't thrash disk on every GC?

That makes sense. Only a few repositories have mega commits which are suitable GC targets.

[trustin]

Running GC at least once is a good idea in my opinion though, because it will reduce the number of files in the data directory, by packing multiple objects into a single pack file.

[minwoox]

Thanks a lot, @ikhoon!
Could you also update the documentation, please?

[ikhoon]

Could you also update the documentation, please?

The documentation meant the PR description? Then I did.

[minwoox]

The documentation meant the PR description? Then I did.

What I meant was this page. 😄
https://line.github.io/centraldogma/setup-configuration.html

[ikhoon]

The documentation meant the PR description? Then I did.

What I meant was this page. 😄
https://line.github.io/centraldogma/setup-configuration.html

https://github.com/line/centraldogma/pull/564/files#diff-650dc6b906f91adeda4a61b5c71aa8c9b5680d6266d9d7179072579e7d5a4cb5

Is there anything I am missing? 😅

[minwoox]

Is there anything I am missing?

Yeah, letting me know about it. 🤣

[minwoox]

Thanks, @ikhoon!