Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GC task may crash if it encounters an invalid repository #21605

Closed
mpeter50 opened this issue Oct 26, 2022 · 3 comments · Fixed by #22422
Closed

GC task may crash if it encounters an invalid repository #21605

mpeter50 opened this issue Oct 26, 2022 · 3 comments · Fixed by #22422
Labels
Milestone

Comments

@mpeter50
Copy link
Contributor

Description

Months ago I had a few migrations that failed for some reason.
I don't mean the follow up syncs, but the initial sync.

A few days ago I started using the garbage collect task to regain some storage space, and I've found these repositories by them stopping the whole GC task.
This means when it gets to one of these repositories, it will stop and won't continue with the remaining ones.

Here is a log excerpt I've saved for reference, from when I've encountered this issue:

2022/10/25 20:49:41 .../repository/check.go:84:func1() [E] [63582888-2] Repository garbage collection failed for &{332 5 Migrations <nil> git_access git_access   4 https://gitlab.freedesktop.org/pipewire/pipewire/-/wikis/git_access main 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 true true false false <nil> 1 map[] map[] [] <nil> false 0 <nil> false 0 0 <nil> <nil> false false [] default  1647821608 1647821608}. Stdout:
    Error: exit status 128 - fatal: not a git repository (or any parent up to mount point /var/lib)
    Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
     - fatal: not a git repository (or any parent up to mount point /var/lib)
    Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

On taking a closer look, these repositories had a size of 0 bytes according to the repo list on the management interface (gitea.lan/admin/repos), and their directories were ... not empty, but incomplete, like one of them only had the hooks directory, and possibly only part of that even.

I've deleted these repositories since then, and have re-migrated them. GC now does not crash at them.
On that, I think I had difficulties in doing that, as (if I remember correctly) it didn't work through the page with the delete button that is shown when I browse to an invalid repository, but it did work from the above mentioned repo list on the management interface.
Unfortunately I don't have logs about this. I really should set up some central log collection service..

Gitea Version

v1.17.3

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

2.36.2

Operating System

Raspbian

How are you running Gitea?

Gitea runs in a Docker container, which was self-built from this repository according to the Docker.rootless Dockerfile.

Database

MySQL

@lunny lunny added this to the 1.17.4 milestone Oct 26, 2022
@lunny lunny modified the milestones: 1.17.4, 1.17.5 Dec 21, 2022
@strk
Copy link
Member

strk commented Jan 12, 2023

I have a similar problem, that is GC task stops on first error. I would think that GC task should continue to next repository rather than stopping. If you see a good use case for stopping the whole thing maybe it could be a configuration setting, but I personally don't see a good reason to stop on first error.

@delvh
Copy link
Member

delvh commented Jan 12, 2023

The problem most likely comes from the

for i,v := range <something> {
  if err := function(v); err != nil {
    return err
  }
}

pattern which is used everywhere in Gitea.
I do agree that it probably makes the most sense to ignore an error in this case.

zeripath added a commit to zeripath/gitea that referenced this issue Jan 13, 2023
The current code propagates all errors up to the iteration step meaning that
a single malformed repo will prevent GC of other repos.

This PR simply stops that propagation.

Fix go-gitea#21605

Signed-off-by: Andrew Thornton <art27@cantab.net>
@zeripath
Copy link
Contributor

So the majority of the cron tasks don't propagate errors in this way. I'm not sure why this function was different.

lunny pushed a commit that referenced this issue Jan 13, 2023
The current code propagates all errors up to the iteration step meaning
that a single malformed repo will prevent GC of other repos.

This PR simply stops that propagation.

Fix #21605

Signed-off-by: Andrew Thornton <art27@cantab.net>

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
zeripath added a commit to zeripath/gitea that referenced this issue Jan 13, 2023
Backport go-gitea#22422

The current code propagates all errors up to the iteration step meaning
that a single malformed repo will prevent GC of other repos.

This PR simply stops that propagation.

Fix go-gitea#21605

Signed-off-by: Andrew Thornton <art27@cantab.net>
jolheiser pushed a commit that referenced this issue Jan 13, 2023
Backport #22422

The current code propagates all errors up to the iteration step meaning
that a single malformed repo will prevent GC of other repos.

This PR simply stops that propagation.

Fix #21605

Signed-off-by: Andrew Thornton <art27@cantab.net>
@go-gitea go-gitea locked and limited conversation to collaborators May 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants