Mirror of an Azure Repo fails with LFS objects

[dandaolrian]

Description

We were trying to use gitea as a mirror of an Azure repo with LFS objects using the standard "git" New migration option and the LFS download failed with the following in the UI

Migrating from https://dev.azure.com/****/_git/lfs_test failed.
StoreMissingLfsObjectsInRepository: unexpected EOF

Debugging it we found this was as we need
Authenticate with a Bearer token for the Download
With Azure we need "application/vnd.git-lfs" - if this isn't set we just get the JSON definition of the LFS object

If we add both of those to the Download function in transferadaper.go we were able to set up a mirror

// Download reads the download location and downloads the data.
func (a *BasicTransferAdapter) Download(ctx context.Context, l *Link) (io.ReadCloser, error) {
    req, err := createRequest(ctx, http.MethodGet, l.Href, l.Header, nil)
    req.Header.Set("Accept", "application/vnd.git-lfs")
    req.Header.Set("Authorization", "Bearer <token>")

I did have a look at how the token could be passed through as it's not available in the function and I didn't see a nice option

Last place we have migration.MigrateOptions is in MigrateRepositoryGitData in services\repository\migrate.go and the only options I saw were:

• Passing it down to StoreMissingLfsObjectsInRepository in modules\repository\repo.go but that would cascade through a lot of functions
• Add it to lfs.NewClient services\repository\migrate.go:179 - then it could be consumed directly in transferadaper.go

And also we'd obviously need the extra header. Or would this work will other git repo's ?

Maybe a custom migration for Azure makes the most sense

Gitea Version

1.26.0+dev-3-g8ad2a538da

Can you reproduce the bug on the Gitea demo site?

Yes

Log Gist

https://gist.github.com/dandaolrian/eac864b3c5b5f00d17033dc558da4282#file-gitea_azure_lfs_mirror-txt

Screenshots

No response

Git Version

git version 2.51.0.windows.1

Operating System

Win11 (for proof of concept)

How are you running Gitea?

We currently in a proof of concept stage - we've built ourselves as we wanted to understand why it was failing

Database

SQLite