-
Notifications
You must be signed in to change notification settings - Fork 117
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Run git gc periodically #564
Conversation
Motivation: Central Dogma git repository does not run git gc automatically. git gc will save disk storage and improve performance. Modifications: - Add `GarbageCollectingServicePlugin` that run git gc every day Result: - Central Dogma now run git gc everyday. - Fixes line#264
Codecov Report
@@ Coverage Diff @@
## master #564 +/- ##
============================================
- Coverage 70.10% 69.83% -0.28%
- Complexity 3271 3302 +31
============================================
Files 329 332 +3
Lines 13030 13219 +189
Branches 1405 1427 +22
============================================
+ Hits 9135 9231 +96
- Misses 3018 3097 +79
- Partials 877 891 +14
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks awesome!
Left some nit comments. 😄
...n/java/com/linecorp/centraldogma/server/internal/storage/GarbageCollectingServicePlugin.java
Outdated
Show resolved
Hide resolved
...n/java/com/linecorp/centraldogma/server/internal/storage/GarbageCollectingServicePlugin.java
Outdated
Show resolved
Hide resolved
We should not perform GC when the gain is not large enough, e.g. we shouldn't perform GC when only a few hundred commits were added since the last GC. We will otherwise see a large amount of disk I/O every day. |
Good suggestion! The GC will run after 200 commits were added since the last GC. The 200 was referenced in https://docs.gitlab.com/ee/administration/housekeeping.html#manual-housekeeping |
I think that's implementation detail which might not be the case in jGit. How about trying to run GC against a large repository using jGit to make sure it doesn't thrash disk on every GC? |
That makes sense. Only a few repositories have mega commits which are suitable GC targets. |
Running GC at least once is a good idea in my opinion though, because it will reduce the number of files in the data directory, by packing multiple objects into a single pack file. |
.../linecorp/centraldogma/server/internal/storage/RepositoryGarbageCollectingServicePlugin.java
Outdated
Show resolved
Hide resolved
.../linecorp/centraldogma/server/internal/storage/RepositoryGarbageCollectingServicePlugin.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot, @ikhoon!
Could you also update the documentation, please?
...ain/java/com/linecorp/centraldogma/server/internal/storage/repository/git/GitGcRevision.java
Outdated
Show resolved
Hide resolved
...ain/java/com/linecorp/centraldogma/server/internal/storage/repository/git/GitGcRevision.java
Outdated
Show resolved
Hide resolved
The documentation meant the PR description? Then I did. |
What I meant was this page. 😄 |
Is there anything I am missing? 😅 |
Yeah, letting me know about it. 🤣 |
Thanks, @ikhoon! |
Motivation:
Central Dogma git repository does not run git gc automatically.
git gc will save disk storage and improve performance.
Modifications:
RepositoryGarbageCollectionPlugin
that schedules GC with a Quartz cron expressionRepositoryGarbageCollectionConfig
in order to configure a GC schedule and the minimum number of new commits for the GCGitGcRevision
that reads and writes the last revision when GC was run.Result: