Unable to use ssh gitlab connection for on prem gitlab #1322

balchua · 2019-03-26T13:17:40Z

We wanted to use argocd with our on prem gitlab. Our gitlab is using self signed cert, so decided to use ssh instead.

But i keep getting this error ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

If anyone have an idea on how to get past this one.

The text was updated successfully, but these errors were encountered:

jessesuen · 2019-03-26T18:17:49Z

For private git repos over SSH, you need to either:

add the repo using --insecure-ignore-host-key
customize the argocd image such that the public ssh key of your gitlab is known to Argo CD. This involves running ssh-keyscan against your private git repo and appending it to /etc/ssh/ssh_known_hosts. You should be able to do this using a volume mount or rebuilding the image.

For private git repos over HTTPS, you need to customize the argocd image servers cert bundle under /etc/ssl/certs. We don't currently have a insecure-tls-skip-verify option for HTTPS at the moment. Adding to /etc/ssl/certs should be possible using a volume mount.

balchua · 2019-03-26T21:44:16Z

@jessesuen thanks. I will try the insecure-ignore-host-key. With regards to self signed https, i was able to mount the cert to /etc/ssl/certs and add the repo. But started failing when i try to create an app.
I get the same error as this.
#1171

jessesuen · 2019-03-26T23:04:47Z

Did you update all services (argocd-server, argocd-repo-server, argocd-application-controller) with the cert bundle? I think we may need all of them to have the change.

If that doesn't work, we can dup this bug to #1171 to figure out why self-signed HTTPS does not work. One user did report adding their cert bundle to /etc/ssl/certs resolved it for them.

balchua · 2019-03-26T23:49:04Z

Aaa nope. Only added it to argocd-server. Curious to know which component is managing the addition of application? argocd-server or argocd-repo-server? Thanks

jessesuen · 2019-03-27T04:59:01Z

argocd-server tests the repo before storing it to verify it is valid, so it is needed there. argocd-repo-server is responsible for cloning the repo and rendering the manifests. I don't recall if it is truly necessary in argocd-application-controller.

balchua · 2019-03-28T03:24:43Z

I tried the insecure-ignore-host-key didn't work i still get ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain error. But mounting the certificate on /etc/ssl/certs/ to both the argocd-server and argocd-repo-server worked.

mjasion · 2019-04-02T15:55:34Z

I have similar issue with private repository on selfhosted Gitlab :(
Error which I am getting on adding project:

application spec is invalid: InvalidSpecError: No credentials available for source repository and repository is not publicly accessible: error creating SSH agent: "SSH agent requested but SSH_AUTH_SOCK not-specified"

My config map:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
data:
  repositories: |
    - insecureIgnoreHostKey: true
      sshPrivateKeySecret:
        key: sshPrivateKey
        name: repo-argocd-get-started-1347344391
      url: ssh://git@gitlab.example.com/marcin.jasion/argocd-get-started.git

And Log from argocd-server pod:

time="2019-04-02T15:38:22Z" level=info msg="received unary call /application.ApplicationService/Create" grpc.method=Create grpc.request.claims="{\"iat\":1554217511,\"iss\":\"argocd\",\"nbf\":1554217511,\"sub\":\"admin\"}" grpc.request.content="application:<TypeMeta:<kind:\"\" apiVersion:\"\" > metadata:<name:\"argo-get-started\" generateName:\"\" namespace:\"\" selfLink:\"\" uid:\"\" resourceVersion:\"\" generation:0 creationTimestamp:<0001-01-01T00:00:00Z> clusterName:\"\" > spec:<source:<repoURL:\"git@gitlab.example.com:marcin.jasion/argocd-get-started.git\" path:\"/\" targetRevision:\"master\" directory:<recurse:false jsonnet:<> > > destination:<server:\"https://kubernetes.default.svc\" namespace:\"default\" > project:\"default\" syncPolicy:<automated:<prune:true > > > status:<sync:<status:\"\" comparedTo:<source:<repoURL:\"\" path:\"\" targetRevision:\"\" > destination:<server:\"\" namespace:\"\" > > revision:\"\" > health:<status:\"\" message:\"\" > reconciledAt:<0001-01-01T00:00:00Z> observedAt:<0001-01-01T00:00:00Z> sourceType:\"\" > > " grpc.service=application.ApplicationService grpc.start_time="2019-04-02T15:38:22Z" span.kind=server system=grpc
time="2019-04-02T15:38:22Z" level=info msg="finished unary call with code InvalidArgument" error="rpc error: code = InvalidArgument desc = application spec is invalid: InvalidSpecError: No credentials available for source repository and repository is not publicly accessible: error creating SSH agent: \"SSH agent requested but SSH_AUTH_SOCK not-specified\"" grpc.code=InvalidArgument grpc.method=Create grpc.service=application.ApplicationService grpc.start_time="2019-04-02T15:38:22Z" grpc.time_ms=10.44 span.kind=server system=grpc

I am using minikube with kubernetes 1.14.0

balchua · 2019-04-03T00:08:28Z

For now im mounting the certs to the pods and use https instead of ssh. Though i still would want to use ssh access in the future.

andreyvelich · 2019-04-03T00:34:17Z

I got the same problem with private GitHub repository. I specified public key for the private repo and after run this command:

argocd repo add git@github.com:andreyvelich/test-argocd.git --ssh-private-key-path=<Path to my Private Key>

I got error:

FATA[0000] rpc error: code = Unauthenticated desc = invalid session: signature is invalid

andreyvelich · 2019-04-03T18:38:31Z

I got the problem. I need to make argocd login in the command line before run this command.
Right now, I have added my private github repo to the ArgoCD.

balchua · 2019-04-03T23:21:42Z

I will give it a shot again.

alexmt · 2019-04-23T21:43:23Z

@balchua . Based on logs from #1322 (comment) it looks like --ssh-private-key-path was not specified. Can you please confirm that you specified both --ssh-private-key-path and --insecure-ignore-host-key flags?

balchua · 2019-04-23T22:00:15Z

@alexmt will try this one and let you know. Thanks for looking into this.

gaby · 2019-04-30T00:08:23Z

Is there a way of doing this without having to use argocd cli or modying the default deployment yamls? The CA for my git repo is trusted on the host the node is running, yet the container has 0 awareness of this, it's only getting the regular CA's I dont know from where.

I tested the latest 1.0-RC and this still happens. After initial deployment changing the Deployments in my cluster results in the UI always saying I have the wrong password... And theres no way to change the password in the ArgoCD UI.

Timoses · 2019-05-22T06:37:34Z

Back on the topic of ssh connection:

I also get:

argocd repo add my_user@myrepo:myport/~my_user/kubernetes.git --insecure-ignore-host-key --ssh-private-key-path ./id_rsa
FATA[0005] rpc error: code = Unknown desc = ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

When I do

kubectl cp ./id_rsa argocd-server-XXX-YYY:/home/argocd/.ssh/id_rsa
kubectl cp ./id_rsa.pub argocd-server-XXX-YYY:/home/argocd/.ssh/id_rsa.pub

kubectl exec -it argocd-server-XXX-YYY sh
    # succeeds (uses ssh key)
    git clone ssh://my_user@myrepo:myport/~my_user/kubernetes.git

    # asks for password ('ssh://' removed from front of repo url)
    git clone my_user@myrepo:myport/~my_user/kubernetes.git

Is there anything else I can investigate or supply to get to the cause?

EDIT:
This hinted me to update to v1.0.0 and it worked ; ).

alexec · 2019-06-01T00:39:02Z

Next step: create a kustomize application that has a private remote base.

alexec · 2019-06-03T15:55:08Z

@balchua where are you on this issue?

balchua · 2019-06-03T21:53:08Z

Sorry guys i haven't updated this issue. Its all good for me now. Im closing this one. Thanks!!

Kampe · 2020-06-22T19:53:08Z

I see this issue myself when using SSH + declarative setup, what was the fix @balchua?

HerHde · 2022-01-19T01:40:12Z

Since I ran into the same problem today while trying to add a private repo from my GitLab instance via SSH (AND host key verification) in a declarative way to my ArgoCD running in Minikube, I'll share my solution:

Generate a SSH keypair for Argo without a password (ssh-keygen -t ed25519 -N '' -C argo@minikube -f ed25519_delme)
Add it as a (read only) deploy key to the GitLab repo

From the declarative doc, take the argocd-repositories.yaml manifest, add the git SSH clone url, the sshPrivateKey (from ed25519_delme), adapt name and remove username and password

apiVersion: v1
kind: Secret
metadata:
  name: gitlab-argo-examples
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  url: git@git.example.org:mine/gitlab-argo-examples.git
  sshPrivateKey: |
    -----BEGIN OPENSSH PRIVATE KEY-----
    ... taken from ed25519_delme ...
    -----END OPENSSH PRIVATE KEY-----

Append the host key fingerprints from ssh-keyscan git.example.org 2> /dev/null to the argocd-ssh-known-hosts-cm.yaml manifest ssh_known_hosts block, as well as adding the ArgoCD namespace:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/name: argocd-ssh-known-hosts-cm
    app.kubernetes.io/part-of: argocd
  name: argocd-ssh-known-hosts-cm
  namespace: argocd
data:
  ssh_known_hosts: |
    bitbucket.org ssh-rsa AAAAblablabla...
    [...]
    git.example.org ssh-ed25519 AAAA...
    git.example.org ssh-rsa AAAABANANAAAAAA...

kubectl apply -f argocd-ssh-known-hosts-cm.yaml and kubectl apply -f argocd-repositories.yaml. Now the ArgoCD web UI should report a 🗹 Successful connection to the repo and work.

I hope this helps anyone ;-)

marcellodesales · 2022-01-25T00:57:44Z

🎉 Multiple Private Repos in Gitlab - Shared SSH Key - `Prefixed ssh://`

First define the shared secret using ssh:// repos for pattern matching the URLs
Declare the repo in a project that matches the repo substring

🔧 Shared Gitlab Secret for an Org

apiVersion: v1
kind: Secret
metadata:
  name: argocd-gitlab-private-repo-creds
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repo-creds
stringData:
  type: git
  url: ssh://git@gitlab.com/xyz
  sshPrivateKey: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIJKAIBAAKCAgEA4MlrODdd+t3vNPGnel1lUA7FgTikVmK9sTDF+IMPaD32ChME

🔧 Declare the app with URL (substring)

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: xyz-payment-aws-sae1-ppd-dev
  namespace: argocd
spec:
  destination:
    namespace: xyz-aws-sae1-ppd-dev
    server: https://kubernetes.default.svc
  project: default
  source:
    repoURL: ssh://git@gitlab.com/xyz/services-deploy/payment-service-deploy.git
    path: env/aws-sae1-ppd-dev
    targetRevision: HEAD
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true
    - Validate=false
    - ApplyOutOfSyncOnly=true

alexmt added the bug Something isn't working label Mar 26, 2019

alexmt added this to the v0.13 milestone Mar 26, 2019

jessesuen removed the bug Something isn't working label Mar 26, 2019

jessesuen removed this from the v0.13 milestone Mar 26, 2019

andreyvelich mentioned this issue Apr 9, 2019

Unable to create an app from private repository #1411

Closed

jessesuen added this to the v1.1 milestone Jun 1, 2019

jessesuen added the bug Something isn't working label Jun 1, 2019

balchua closed this as completed Jun 3, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to use ssh gitlab connection for on prem gitlab #1322

Unable to use ssh gitlab connection for on prem gitlab #1322

balchua commented Mar 26, 2019

jessesuen commented Mar 26, 2019

balchua commented Mar 26, 2019

jessesuen commented Mar 26, 2019

balchua commented Mar 26, 2019

jessesuen commented Mar 27, 2019

balchua commented Mar 28, 2019

mjasion commented Apr 2, 2019

balchua commented Apr 3, 2019

andreyvelich commented Apr 3, 2019 •

edited

Loading

andreyvelich commented Apr 3, 2019

balchua commented Apr 3, 2019

alexmt commented Apr 23, 2019

balchua commented Apr 23, 2019

gaby commented Apr 30, 2019

Timoses commented May 22, 2019 •

edited

Loading

alexec commented Jun 1, 2019

alexec commented Jun 3, 2019

balchua commented Jun 3, 2019

Kampe commented Jun 22, 2020 •

edited

Loading

HerHde commented Jan 19, 2022 •

edited

Loading

marcellodesales commented Jan 25, 2022

Unable to use ssh gitlab connection for on prem gitlab #1322

Unable to use ssh gitlab connection for on prem gitlab #1322

Comments

balchua commented Mar 26, 2019

jessesuen commented Mar 26, 2019

balchua commented Mar 26, 2019

jessesuen commented Mar 26, 2019

balchua commented Mar 26, 2019

jessesuen commented Mar 27, 2019

balchua commented Mar 28, 2019

mjasion commented Apr 2, 2019

balchua commented Apr 3, 2019

andreyvelich commented Apr 3, 2019 • edited Loading

andreyvelich commented Apr 3, 2019

balchua commented Apr 3, 2019

alexmt commented Apr 23, 2019

balchua commented Apr 23, 2019

gaby commented Apr 30, 2019

Timoses commented May 22, 2019 • edited Loading

alexec commented Jun 1, 2019

alexec commented Jun 3, 2019

balchua commented Jun 3, 2019

Kampe commented Jun 22, 2020 • edited Loading

HerHde commented Jan 19, 2022 • edited Loading

marcellodesales commented Jan 25, 2022

🎉 Multiple Private Repos in Gitlab - Shared SSH Key - Prefixed ssh://

🔧 Shared Gitlab Secret for an Org

🔧 Declare the app with URL (substring)

andreyvelich commented Apr 3, 2019 •

edited

Loading

Timoses commented May 22, 2019 •

edited

Loading

Kampe commented Jun 22, 2020 •

edited

Loading

HerHde commented Jan 19, 2022 •

edited

Loading

🎉 Multiple Private Repos in Gitlab - Shared SSH Key - `Prefixed ssh://`