Prevents multiple Git mirroring jobs from running in the same directory#1269
Prevents multiple Git mirroring jobs from running in the same directory#1269
Conversation
Motivation: A Git mirroring task creates a local directory derived from the remote URL and fetches upstream updates. https://github.com/line/centraldogma/blob/4c617bb2af047e4eb4e9090e75fa71d8460f7e49/server-mirror-git/src/main/java/com/linecorp/centraldogma/server/internal/mirror/AbstractGitMirror.java#L143-L146 Because the tasks runs asynchronously, multiple fetch operations can end up running the same directory if the sample remote URL is used across multiple mirror configurations. https://github.com/line/centraldogma/blob/4c617bb2af047e4eb4e9090e75fa71d8460f7e49/server/src/main/java/com/linecorp/centraldogma/server/internal/mirror/MirrorSchedulingService.java#L313-L314 In an internal use case, a single Git repository was used to configure 36 mirroring jobs, each targeting a different branch. As a result, the Git internal structure or state could be corrupted with the following exception: ```java Caused by: org.eclipse.jgit.api.errors.TransportException: Expected ACK/NAK, found EOF at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:249) at com.linecorp.centraldogma.server.internal.mirror.AbstractGitMirror.fetchRemoteHeadAndGetCommitId(AbstractGitMirror.java:657) at com.linecorp.centraldogma.server.internal.mirror.AbstractGitMirror.mirrorRemoteToLocal(AbstractGitMirror.java:361) ... 15 common frames omitted Caused by: org.eclipse.jgit.errors.TransportException: Expected ACK/NAK, found EOF at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:458) at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:351) at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:343) ``` Modifications: - Use a unique mirroring directory per Git mirroring job so that each job gets its own Git directory. Result: Fixed a race condition that could occur when multiple Git mirroring jobs shared the same directory for a remote URL
📝 WalkthroughWalkthroughUpdates repository directory naming in AbstractGitMirror to prevent collisions by using localRepo parent name, localRepo name, and id instead of deriving from remote URL. Adds validation in integration tests to verify mirror directories are created with valid HEAD references. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@server-mirror-git/src/main/java/com/linecorp/centraldogma/server/internal/mirror/AbstractGitMirror.java`:
- Around line 139-142: The directory naming using dirName =
localRepo().parent().name() + '-' + localRepo().name() + '-' + id() in
AbstractGitMirror can collide when hyphens appear in project/repo names; change
the scheme to produce a collision-safe key (e.g., compute a stable digest/hash
of the tuple [localRepo().parent().name(), localRepo().name(), id()] or encode
each component with an unambiguous separator or percent-encoding) and use that
digest/encoded string when constructing repoDir = new File(workDir, ...),
ensuring uniqueness across different (project, repo, mirrorId) combinations.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 23f6685f-449b-4554-bde8-4866dd7d1b60
📒 Files selected for processing (2)
it/mirror/src/test/java/com/linecorp/centraldogma/it/mirror/git/GitMirrorIntegrationTest.javaserver-mirror-git/src/main/java/com/linecorp/centraldogma/server/internal/mirror/AbstractGitMirror.java
Motivation:
A Git mirroring task creates a local directory derived from the remote URL and fetches upstream updates.
centraldogma/server-mirror-git/src/main/java/com/linecorp/centraldogma/server/internal/mirror/AbstractGitMirror.java
Lines 143 to 146 in 4c617bb
Because the tasks runs asynchronously, multiple fetch operations can end up running in the same directory if the sample remote URL is used across multiple mirror configurations.
centraldogma/server/src/main/java/com/linecorp/centraldogma/server/internal/mirror/MirrorSchedulingService.java
Lines 313 to 314 in 4c617bb
As a result, the Git internal structure or state could be corrupted with the following exception:
Modifications:
Result:
Fixed a race condition that could occur when multiple Git mirroring jobs shared the same directory for a remote URL