Migration from https git with basic authentication failed

[zpericic]

Gitea Version

1.15.9

Git Version

2.33.1

Operating System

Fedora 35

How are you running Gitea?

git clone https://github.com/go-gitea/gitea
git checkout v1.15.9
TAGS="bindata" make build

Running using systemd service "/etc/systemd/system/gitea.service"

[Unit]
Description=Gitea HTTP Server
After=network.target

[Service]
Type=simple
Environment=HOME=/var/gitea GITEA_WORK_DIR=/var/gitea/gitea
WorkingDirectory=/var/gitea/gitea
ExecStart=/var/gitea/gitea/gitea web
Restart=always

[Install]
WantedBy=multi-user.target

Database

PostgreSQL

Can you reproduce the bug on the Gitea demo site?

Yes

Log Gist

No response

Description

Hi,

I have been trying to migrate some repository from CGIT based http service with basic authentication enabled which always results with:

"modules/task/task.go:54:handle() [E] Run task failed: Authentication failed: Clone: exit status 128 - fatal: Authentication failed for 'https://example.git/myrepo.git"

Log reveals command used:

...dules/git/command.go:118:RunInDirTimeoutEnvFullPipelineFunc() [D] /usr/bin/git -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= clone --mirror --quiet -- https://username:password@example.com/myrepo.git /srv/gitea/work/repo/test/myrepo.git

It also fails If I use command directly:

# /usr/bin/git -c credential.helper= -c protocol.version=2 -c filter.lfs.required= -c filter.lfs.smudge= -c filter.lfs.clean= clone --mirror --quiet -- https://username:password@example.com/myrepo.git /srv/gitea/work/repo/test/myrepo.git
fatal: Authentication failed for  https://example.com/myrepo.git

As it seams, git is not using URL embedded password. I have no clue if this should work with http basic authentication nor if it is only broke/deprecated in newer git versions.

I have create patch d389b8a8fb196f37a222ffdbac3d6d7ff509eb78 which resolves my problem.

Patch extracts password from URL and use "credential.helper" git config to provide password for git command.

-c credential.helper='/bin/echo -e "username=user\npassword=password\n"'

This approach should work for other migrations. It works with github private repos but didn't tested with all migration providers.

Any thoughts? Should this be implement in other places?

Screenshots

No response

[Gusted]

As it seams, git is not using URL embedded password. I have no clue if this should work with http basic authentication nor if it is only broke/deprecated in newer git versions.

Well, it could be that cgit(I don't have a really good understanding of how it works) doesn't support the HTTP basic auth. You could check this by doing a simplified version of this command: git clone https://username:password@example.com/myrepo.git and see if that works. If it doesn't... we should somehow use that kind of workaround by using the credential.helper, as that should work in all cases.

[zpericic]

Well, it could be that cgit(I don't have a really good understanding of how it works) doesn't support the HTTP basic auth. You could check this by doing a simplified version of this command: git clone https://username:password@example.com/myrepo.git and see if that works. If it doesn't... we should somehow use that kind of workaround by using the credential.helper, as that should work in all cases.

Runing git clone in shell with embedded password doesn't work. It won't even call GIT_ASKPASS or system defined credential.helper.

# git clone https://username:password@example.com/myrepo.git
Cloning into 'myrepo'...
fatal: Authentication failed for  'https://example.com/myrepo.git'

If I remove password from URL it works correctly, etc. ask for password via defined GIT_ASKPASS or credential.helper. Both GTK popup and console prompt are working correctly.

# git clone https://username@example.com/myrepo.git
Cloning into 'myrepo'...
Password for 'https://example.com/myrepo.git':
Fetching objects...

Embedding password in URL could be deprecated but I could not find any reference. Git internally use curl so it naturally accepts password in ~/.netrc but this is not option for gitea.

Google say that people did have problems HTTP basic authentication, most of problems are resolved using GIT_ASKPASS and credential.helper but this is for console use.

After further investigation:
Cgit is not involved in any authentication because repository is protected via Apache mod_auth_gssapi. And after some packet sniff I found problem.

Here is sequence:

1. Run "git clone https://username:password@example.com/myrepo.git".
2. Apache responds with:

HTTP/1.1 401 Unauthorized
WWW-Authenticate: Negotiate
WWW-Authenticate: Basic realm="Some realm"

3. After 401 git should respond with "Authorization" but instead git resend original request without required field.
4. Apache responds again with same 401.
   5.a. For request with embedded password git resign with authentication failed.
   5.b. If no password is embedded git tries to get password and continue with correct "Authorization" header.

As it seams, there is bug in git. In step 3 git should respond with required header. I'll make new issue for git.

But as there are many broken git implementation we should implement credential.helper approach as it handle everything. Even with git bugfix, a lot of time will pass before it comes to life...

[Gusted]

Wow, thanks for the extensive comment.

Yeah that indeed seems like a case that GIT should be handling, I think we should just revert now to use the credential.helper workaround.

[zpericic]

Yeap.

We should patch other files which interface external repository. Namely, services/mirror/mirror_pull.go and services/mirror/mirror_push.go.

But I'am not sure where is remote password stored. I found it in repository origin URL and that could be problem because password should be removed from URL and passed only via credential.helper while running pull/push.

So workaround should look like:

1. Take URL from repository
2. Extract password
3. Store URL without password in config
4. Do git command
5. Restore URL with password for feature operations

Another solution would be:

1. Take URL from repository
2. Extract password
3. Store URL without password in config
4. Store password in separate file
5. Use password file with credential.helper

[jcatrysse]

This is still an issue when for example importing from Team Foundation Server. Passing credentials on the command line is not working, and can only be done using the Credentials Helper.

I see that a #18172 has been created, but it has been closed again. This would really help us out.
For the moment we cannot do any imports from within the Gitea web interface and all is running from command line.