Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docker] git clone via SSH fails #5894

Closed
fb33 opened this issue Sep 30, 2022 · 25 comments
Closed

[Docker] git clone via SSH fails #5894

fb33 opened this issue Sep 30, 2022 · 25 comments
Labels
bug Issues that are considered to be bugs docker About Docker topics

Comments

@fb33
Copy link
Contributor

fb33 commented Sep 30, 2022

Hi,

Since I updated the source code from August 19 to September 29 for my docker ort image, I've some troubles during the scan.

Now, when I'm running ORT, I add -u ort:ort, but I'm facing with some errors during git clone due to the user.

[2022-09-30T15:32:59.676Z] ________ _____________________
[2022-09-30T15:32:59.676Z] \_____  \\______   \__    ___/ the OSS Review Toolkit, version DOCKER-SNAPSHOT.
[2022-09-30T15:32:59.676Z]  /   |   \|       _/ |    |
[2022-09-30T15:32:59.676Z] /    |    \    |   \ |    |    Running 'scan' under Java 11.0.16.1 on Linux with
[2022-09-30T15:32:59.676Z] \_______  /____|_  / |____|    3 CPUs and a maximum of 10240 MiB of memory.
[2022-09-30T15:32:59.676Z]         \/       \/
[2022-09-30T15:32:59.676Z] Environment variables:
[2022-09-30T15:32:59.676Z] ORT_CONFIG_DIR = /home/ort/.ort/config
[2022-09-30T15:32:59.676Z] ORT_DATA_DIR = /home/ort/.ort
[2022-09-30T15:32:59.676Z] JAVA_HOME = /opt/java/openjdk
[2022-09-30T15:32:59.676Z] ANDROID_HOME = /opt/android-sdk
[2022-09-30T15:32:59.676Z] 
[...]
[2022-09-30T15:33:34.121Z] 15:33:31.596 [main] INFO  org.ossreviewtoolkit.scanner.PathScanner - Scanning 'Maven:com.fasterxml:classmate:1.5.1' in thread 'main' (2 of 79)
[2022-09-30T15:33:34.121Z] 15:33:31.599 [main] INFO  org.ossreviewtoolkit.downloader.Downloader - Trying to download 'Maven:com.fasterxml:classmate:1.5.1' sources to '/tmp/ort-ScanCode16026999469159078050/Maven/com.fasterxml/classmate/1.5.1' from VCS...
[2022-09-30T15:33:34.121Z] 15:33:31.601 [main] INFO  org.ossreviewtoolkit.downloader.Downloader - Using processed VcsInfo(type=Git, url=ssh://git@github.com/FasterXML/java-classmate.git, revision=classmate-1.5.1, path=). Original was VcsInfo(type=Git, url=git@github.com:FasterXML/java-classmate.git, revision=classmate-1.5.1, path=).
[2022-09-30T15:33:34.121Z] 15:33:31.601 [main] INFO  org.ossreviewtoolkit.downloader.Downloader - Detected VCS type 'Git' from type name 'Git'.
[2022-09-30T15:33:34.121Z] 15:33:31.610 [main] INFO  org.ossreviewtoolkit.downloader.VersionControlSystem - Adding Git revision 'classmate-1.5.1' (taken from package metadata) as a candidate.
[2022-09-30T15:33:34.381Z] 15:33:34.286 [main] INFO  org.ossreviewtoolkit.downloader.VersionControlSystem - No Git revision for package 'classmate' and version '1.5.1' found: IOException: Unable to list the remote tags.
[2022-09-30T15:33:34.381Z] Caused by: TransportException: ssh://git@github.com/FasterXML/java-classmate.git: Auth fail
[2022-09-30T15:33:34.381Z]     Caused by: TransportException: ssh://git@github.com/FasterXML/java-classmate.git: Auth fail
[2022-09-30T15:33:34.381Z]         Caused by: JSchException: Auth fail
[2022-09-30T15:33:34.381Z] 15:33:34.287 [main] INFO  org.ossreviewtoolkit.downloader.VersionControlSystem - Trying revision candidate 'classmate-1.5.1' (1 of 1)...
[2022-09-30T15:33:34.381Z] 15:33:34.287 [main] INFO  org.ossreviewtoolkit.downloader.vcs.Git - Trying to fetch only revision 'classmate-1.5.1' with depth limited to 50.
[2022-09-30T15:33:34.381Z] 15:33:34.292 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git fetch --depth 50 origin classmate-1.5.1' in '/tmp/ort-ScanCode16026999469159078050/Maven/com.fasterxml/classmate/1.5.1'...
[2022-09-30T15:33:34.382Z] 15:33:34.319 [main] INFO  org.ossreviewtoolkit.downloader.vcs.Git - Could not fetch only revision 'classmate-1.5.1': IOException: Running 'git fetch --depth 50 origin classmate-1.5.1' in '/tmp/ort-ScanCode16026999469159078050/Maven/com.fasterxml/classmate/1.5.1' failed with exit code 128:
[2022-09-30T15:33:34.382Z] No user exists for uid 1000
[2022-09-30T15:33:34.382Z] fatal: Could not read from remote repository.
[2022-09-30T15:33:34.382Z] 
[2022-09-30T15:33:34.382Z] Please make sure you have the correct access rights
[2022-09-30T15:33:34.382Z] and the repository exists.

Any idea ?

@sschuberth
Copy link
Member

Any idea ?

The Dockerfile has been completely rewritten. If you run into trouble, you might want to use docker/legacy/Dockerfile as a work-around for the time being.

@sschuberth sschuberth added bug Issues that are considered to be bugs docker About Docker topics labels Sep 30, 2022
@sschuberth sschuberth changed the title [Docker] git clone [Docker] git clone via SSH fails Sep 30, 2022
@tsteenbe
Copy link
Member

tsteenbe commented Oct 6, 2022

@fb33 could you post the exact series of Docker commands you are using to do a ORT scan of a public (open source) project. Looking for "steps to reproduce" the issue.

@heliocastro
Copy link
Contributor

@fb33 How ( and where ) are you storing the ssh credentials ?
The new container runs as a regular user. If you build it by default, the container user is ort, so the credentials need to be there.

Then i do recommend you try on the docker runtime do something similar like this:

docker run --mount type=bind,source="$(pwd)"/.ssh,target=/home/ort/.ssh,readonly ort

This of course is the most basic bind type, you can be more restrict if your environment need, like labeling, selinux, etc..

@fb33
Copy link
Contributor Author

fb33 commented Oct 6, 2022

Hi,

The implementation is a bit complexe due to the our CI process. ORT usage is drived by a jenkins pipeline in a docker in docker context.
But I can resume it, in few steps :

1/ launch ORT container :
docker run -t -d --entrypoint -e DOCKER_DAEMON_HOST=**** -v /home/jenkins/agent/workspace/_oss/audit-ort-manually/project@tmp/config1233094375676195490tmp:/etc/passwd -u ort:ort --cpus=3 -v$HOME/.ssh:/ort/.ssh -w /project -v /home/jenkins/agent/workspace/_oss/audit-ort-manually/project:/project -v /home/jenkins/agent/workspace/_oss/audit-ort-manually/ort_home:/ort/.ort -v /home/jenkins/agent/workspace/_oss/audit-ort-manually/npm_logs:/ort/.npm/_logs -e ORT_HTTP_USERNAME=**** -e ORT_HTTP_PASSWORD=**** -e JAVA_OPTS=-Xms512M -Xmx10g -XX:MaxMetaspaceSize=6g -Dkotlinx.coroutines.io.parallelism=10 -Dkotlin.daemon.jvm.options='-Xmx8g' -Dsvnkit.ssh2.key=/ort/.ssh/id_rsa -Dsvnkit.ssh2.username=**** -w ******** -e SVN_PASSWORD=**** -e SVN_USER=**** -e WORKSPACE=/home/jenkins/agent/workspace/_oss/audit-ort-manually docker.docker-registry.lectra.com/ort-lectra:node16.14.2_npm8.5.0_scancode30.1.0-TEST cat

2/ connect in container and prepare ssh context : (here some logs)

Running ssh-add (command line suppressed)
Identity added: /home/jenkins/agent/workspace/_oss/audit-ort-manually/project@tmp/private_key_16533079177528635265.key 
Running ssh-add (command line suppressed)
Identity added: /home/jenkins/agent/workspace/_oss/audit-ort-manually/project@tmp/private_key_5471863660807846091.key 
[ssh-agent] Started.
[Pipeline] {
[Pipeline] configFileProvider
provisioning config files...
copy managed file [eu_autobuild_svc_git_config] to file:/home/jenkins/agent/workspace/_oss/audit-ort-manually/project@tmp/config6445647360620459756tmp
copy managed file [known_hosts] to file:/home/jenkins/agent/workspace/_oss/audit-ort-manually/project@tmp/config9475434430233463539tmp
[Pipeline] {
[Pipeline] sh
mkdir -p /home/ort/.ssh
[Pipeline] sh
cp /home/jenkins/agent/workspace/_oss/audit-ort-manually/project@tmp/config9475434430233463539tmp /home/ort/.ssh/known_hosts
[Pipeline] sh
cp /home/jenkins/agent/workspace/_oss/audit-ort-manually/project@tmp/config6445647360620459756tmp /home/ort/.gitconfig

3/ exec ORT command
/usr/bin/ort --info --config /project/ort.conf scan --skip-excluded -f JSON -i /project/reports/analyzer/analyzer-result.yml -o /project/reports/scanner

@heliocastro
Copy link
Contributor

Here:

-u ort:ort --cpus=3 -v$HOME/.ssh:/ort/.ssh

First -u is unecessary unless you are running with a complete different user
Second, should not be /home/ort/.ssh ? Is mounting /ort/.ssh

@fb33
Copy link
Contributor Author

fb33 commented Oct 7, 2022

@heliocastro 👍
You are right!
my bad, I quickly replace root by ort...
I'll test with the good folder

@fb33
Copy link
Contributor Author

fb33 commented Oct 7, 2022

So, I go a step further.
But now I've this error:

14:39:11  12:39:08.560 [main] INFO  org.ossreviewtoolkit.scanner.PathScanner - Finished scanning 'Maven:com.auth0:java-jwt:3.8.1' in thread 'main' (1 of 79).
14:39:11  12:39:08.561 [main] INFO  org.ossreviewtoolkit.scanner.PathScanner - Scanning 'Maven:com.fasterxml:classmate:1.5.1' in thread 'main' (2 of 79)
14:39:11  12:39:08.562 [main] INFO  org.ossreviewtoolkit.downloader.Downloader - Trying to download 'Maven:com.fasterxml:classmate:1.5.1' sources to '/tmp/ort-ScanCode5781155196184446427/Maven/com.fasterxml/classmate/1.5.1' from VCS...
14:39:11  12:39:08.564 [main] INFO  org.ossreviewtoolkit.downloader.Downloader - Using processed VcsInfo(type=Git, url=ssh://git@github.com/FasterXML/java-classmate.git, revision=classmate-1.5.1, path=). Original was VcsInfo(type=Git, url=git@github.com:FasterXML/java-classmate.git, revision=classmate-1.5.1, path=).
14:39:11  12:39:08.564 [main] INFO  org.ossreviewtoolkit.downloader.Downloader - Detected VCS type 'Git' from type name 'Git'.
14:39:11  12:39:08.572 [main] INFO  org.ossreviewtoolkit.downloader.VersionControlSystem - Adding Git revision 'classmate-1.5.1' (taken from package metadata) as a candidate.
14:39:11  12:39:11.398 [main] INFO  org.ossreviewtoolkit.downloader.VersionControlSystem - No Git revision for package 'classmate' and version '1.5.1' found: IOException: Unable to list the remote tags.
14:39:11  Caused by: TransportException: ssh://git@github.com/FasterXML/java-classmate.git: Auth fail
14:39:11      Caused by: TransportException: ssh://git@github.com/FasterXML/java-classmate.git: Auth fail
14:39:11          Caused by: JSchException: Auth fail
14:39:11  12:39:11.399 [main] INFO  org.ossreviewtoolkit.downloader.VersionControlSystem - Trying revision candidate 'classmate-1.5.1' (1 of 1)...
14:39:11  12:39:11.399 [main] INFO  org.ossreviewtoolkit.downloader.vcs.Git - Trying to fetch only revision 'classmate-1.5.1' with depth limited to 50.
14:39:11  12:39:11.404 [main] INFO  org.ossreviewtoolkit.utils.common.ProcessCapture - Running 'git fetch --depth 50 origin classmate-1.5.1' in '/tmp/ort-ScanCode5781155196184446427/Maven/com.fasterxml/classmate/1.5.1'...
14:39:12  12:39:12.301 [main] INFO  org.ossreviewtoolkit.downloader.vcs.Git - Could not fetch only revision 'classmate-1.5.1': IOException: Running 'git fetch --depth 50 origin classmate-1.5.1' in '/tmp/ort-ScanCode5781155196184446427/Maven/com.fasterxml/classmate/1.5.1' failed with exit code 128:
14:39:12  git@github.com: Permission denied (publickey).
14:39:12  fatal: Could not read from remote repository.
14:39:12  
14:39:12  Please make sure you have the correct access rights
14:39:12  and the repository exists.

Ort Git try to clone the project with SSH, but github seems to authorize clone only via https or git cli on "anonymous mode"

I've got the same error, if I try to clone the project on my ubuntu via ssh.

@heliocastro
Copy link
Contributor

@fb33 Can you please paste the very first lines of ort run ( the ones where the ASCII ORT appears ? I'm pretty sure that ort is running with a complete different user then not finding yours .ssh files

@fb33
Copy link
Contributor Author

fb33 commented Oct 11, 2022

here are the lines :

14:38:34  ________ _____________________
14:38:34  \_____  \\______   \__    ___/ the OSS Review Toolkit, version DOCKER-SNAPSHOT.
14:38:34   /   |   \|       _/ |    |
14:38:34  /    |    \    |   \ |    |    Running 'scan' under Java 11.0.16.1 on Linux with
14:38:34  \_______  /____|_  / |____|    3 CPUs and a maximum of 10240 MiB of memory.
14:38:34          \/       \/
14:38:34  Environment variables:
14:38:34  ORT_CONFIG_DIR = /home/ort/.ort/config
14:38:34  ORT_DATA_DIR = /home/ort/.ort
14:38:34  JAVA_HOME = /opt/java/openjdk
14:38:34  ANDROID_HOME = /opt/android-sdk

@fb33
Copy link
Contributor Author

fb33 commented Oct 11, 2022

I think, with my previous version of ORT docker image (build in august), the VCS url was with HTTPS and not with git SSH.

@heliocastro
Copy link
Contributor

If was https, and not ssh, and you are behind proxy, that would explain most of everything
ssh clones over proxy environment need something like corskscrew
Good example: https://gist.github.com/ozbillwang/005bd1dfc597a2f3a00148834ad3e551

@fb33
Copy link
Contributor Author

fb33 commented Oct 13, 2022

Since 4c57907
with

if (url.startsWith("git://")) {
url = "https://${url.removePrefix("git://")}"
}

it should fix my problem ! :)

@sschuberth
Copy link
Member

it should fix my problem ! :)

Feel free to close this issue then after you were able to verify the fix 😉

@fb33
Copy link
Contributor Author

fb33 commented Oct 14, 2022

Hi @sschuberth and @fviernau

I think a case is missing in the process to "Stop using the unauthenticated Git protocol in VCS processed"

I made a PR : #5951

@sschuberth
Copy link
Member

I made a PR : #5951

I've commented over there.

@schvvarzekatze
Copy link

I still get the ssh error in the scanner step for all repositories cloned by ssh. I reproduced it for one sample dependency:

11:48:18.202 [DefaultDispatcher-worker-3] INFO  org.ossreviewtoolkit.downloader.VersionControlSystem - Adding Git revision 'jasypt-1.9.3' (taken from package metadata) as a candidate.
11:48:20.375 [DefaultDispatcher-worker-3] INFO  org.ossreviewtoolkit.downloader.VersionControlSystem - No Git revision for package 'jasypt' and version '1.9.3' found: IOException: Unable to list the remote tags.
Caused by: TransportException: ssh://git@github.com/jasypt/jasypt.git: Auth fail
    Caused by: TransportException: ssh://git@github.com/jasypt/jasypt.git: Auth fail
        Caused by: JSchException: Auth fail
11:48:20.378 [DefaultDispatcher-worker-3] INFO  org.ossreviewtoolkit.scanner.provenance.DefaultPackageProvenanceResolver - Trying revision candidate 'jasypt-1.9.3' (1 of 1).
11:48:20.388 [DefaultDispatcher-worker-3] INFO  org.ossreviewtoolkit.downloader.vcs.Git - Trying to fetch only revision 'jasypt-1.9.3' with depth limited to 50.
11:48:28.833 [DefaultDispatcher-worker-3] INFO  org.ossreviewtoolkit.downloader.vcs.Git - Could not fetch only revision 'jasypt-1.9.3': TransportException: ssh://git@github.com/jasypt/jasypt.git: Auth fail
Caused by: TransportException: ssh://git@github.com/jasypt/jasypt.git: Auth fail
    Caused by: JSchException: Auth fail
11:48:28.833 [DefaultDispatcher-worker-3] INFO  org.ossreviewtoolkit.downloader.vcs.Git - Falling back to fetching all refs with depth limited to 50.
11:48:30.613 [DefaultDispatcher-worker-3] INFO  org.ossreviewtoolkit.downloader.vcs.Git - Could not fetch with only a depth of 50: TransportException: ssh://git@github.com/jasypt/jasypt.git: Auth fail
Caused by: TransportException: ssh://git@github.com/jasypt/jasypt.git: Auth fail
    Caused by: JSchException: Auth fail
11:48:30.614 [DefaultDispatcher-worker-3] INFO  org.ossreviewtoolkit.downloader.vcs.Git - Falling back to fetch everything including tags.
11:48:32.375 [DefaultDispatcher-worker-3] WARN  org.ossreviewtoolkit.downloader.vcs.Git - Failed to fetch everything: TransportException: ssh://git@github.com/jasypt/jasypt.git: Auth fail
Caused by: TransportException: ssh://git@github.com/jasypt/jasypt.git: Auth fail
    Caused by: JSchException: Auth fail
11:48:32.376 [DefaultDispatcher-worker-3] INFO  org.ossreviewtoolkit.scanner.provenance.DefaultPackageProvenanceResolver - Could not resolve revision candidate 'jasypt-1.9.3': TransportException: ssh://git@github.com/jasypt/jasypt.git: Auth fail
Caused by: TransportException: ssh://git@github.com/jasypt/jasypt.git: Auth fail
    Caused by: JSchException: Auth fail
11:48:32.442 [DefaultDispatcher-worker-3] INFO  org.ossreviewtoolkit.scanner.provenance.DefaultPackageProvenanceResolver - Could not resolve VCS for 'Maven:org.jasypt:jasypt:1.9.3': IOException: Could not resolve revision for package 'Maven:org.jasypt:jasypt:1.9.3' with VcsInfo(type=Git, url=ssh://git@github.com/jasypt/jasypt.git, revision=jasypt-1.9.3, path=):
        Could not resolve revision candidate 'jasypt-1.9.3': TransportException: ssh://git@github.com/jasypt/jasypt.git: Auth fail
Caused by: TransportException: ssh://git@github.com/jasypt/jasypt.git: Auth fail
    Caused by: JSchException: Auth fail

When I change the vcs_processed to

          type: "Git"
          url: "https://github.com/jasypt/jasypt.git"
          revision: "jasypt-1.9.3"
          path: ""

the scan is without the error. I used the docker image of ort updated today.

What could be a quick fix in my case to succesfully scan all repositories cloned by ssh?

Thank you very much.

@sschuberth
Copy link
Member

What could be a quick fix in my case to succesfully scan all repositories cloned by ssh?

In contrast to HTTP(S), you cannot clone unauthenticated via SSH. That means, you have to provide credentials when cloning via SSH. So, when cloning via SSH from GitHub, you need to have an account on GitHub that you can use and authenticate with. And ideally, the credentials should be provided by an SSH agent; then JSch used by JGit will pick them up.

@limod
Copy link

limod commented Oct 26, 2022

Hi! I also had problemes with ssh + auth fail and added a SSH Key for our private repository via the SSH-Agent. While testing i mounted my local ssh-agent into docker: -v $SSH_AUTH_SOCK:/ssh-agent --env SSH_AUTH_SOCK=/ssh-agent

This resolved the Auth fail exception (Thanks!) but now i get the following error:

13:40:53.725 [DefaultDispatcher-worker-50] INFO  org.ossreviewtoolkit.downloader.vcs.Git - Could not fetch only revision '2a872f23d36cfb2c44b1c725c44a9ad4b37f6838': TransportException: ssh://git@git.xx.com/xx/xx.git: remote hung up unexpectedly
Caused by: TransportException: ssh://git@git.xx.com/xx/xx.git: remote hung up unexpectedly
    Caused by: TranslationBundleLoadingException: Loading of translation bundle failed for [org.eclipse.jgit.internal.transport.ssh.jsch.JSchText, en_US]
        Caused by: MissingResourceException: Can't find bundle for base name org.eclipse.jgit.internal.transport.ssh.jsch.JSchText, locale en_US

Do you have any idea what causes this error?

@sschuberth
Copy link
Member

sschuberth commented Oct 26, 2022

Loading of translation bundle failed for [org.eclipse.jgit.internal.transport.ssh.jsch.JSchText, en_US]

See this for some background information. @limod, do you use a custom ORT distribution by any chance, like a custom-built Fat JAR?

@limod
Copy link

limod commented Oct 26, 2022

@sschuberth I do not use a custom ORT distribution. I cloned the repository (on 19.10.22) and executed docker build -t ort .

@sschuberth
Copy link
Member

sschuberth commented Oct 26, 2022

@limod despite the ordering in the stacktrace, it seems that remote hung up unexpectedly is the root cause, and Can't find bundle for base name is just a follow-up error as JGit fails to find a translation for some JSch-related message. I've prepared a fix to JGit for that, but like I said, that's likely not the root cause.

@limod
Copy link

limod commented Nov 3, 2022

Sorry for the delay, the remote hung up unexpectedly has resolved, seemed that there was a real issue while testing with the repository. I ended in a Invalid Key Exception but there is already an issue for this error: #6024

@sschuberth
Copy link
Member

I ended in a Invalid Key Exception but there is already an issue for this error: #6024

... and a potential solution 😉 @limod could you also please give #6030 a try in this regard?

@sschuberth
Copy link
Member

@limod could you also please give #6030 a try in this regard?

I'm confident that the merged #6030 solves this issue. If not, feel free to reopen.

lucamilanesio pushed a commit to GerritCodeReview/jgit that referenced this issue Nov 7, 2022
The paths needs to include "ssh" to match the class's package name. This
resolves

Caused by: TransportException: ssh://git@git.xx.com/xx/xx.git: remote hung up unexpectedly
    Caused by: TranslationBundleLoadingException: Loading of translation bundle failed for [org.eclipse.jgit.internal.transport.ssh.jsch.JSchText, en_US]
        Caused by: MissingResourceException: Can't find bundle for base name org.eclipse.jgit.internal.transport.ssh.jsch.JSchText, locale en_US

Also see [1] for reference.

[1]: oss-review-toolkit/ort#5894 (comment)

Change-Id: Ie27b9fc1cdd1d83f8123821be42e65da59ecf49d
Signed-off-by: Sebastian Schuberth <opensource@schuberth.dev>
@sschuberth
Copy link
Member

@limod despite the ordering in the stacktrace, it seems that remote hung up unexpectedly is the root cause, and Can't find bundle for base name is just a follow-up error as JGit fails to find a translation for some JSch-related message. I've prepared a fix to JGit for that, but like I said, that's likely not the root cause.

FYI, this specific issue has been fixed with #6148.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issues that are considered to be bugs docker About Docker topics
Projects
None yet
Development

No branches or pull requests

6 participants