Support secured devfile registries / index.jsons in odo

[johnmcollier]

/kind user-story
/area devfile

User Story

Currently, odo supports devfile registries that are running behind publicly signed certificates and don't require any authentication. As a user, I want to be able to use odo with devfile registries / index.jsons that may be behind self-signed certs and/or require authentication, via a username/password or personal access token.

Acceptance Criteria

• odo should be able to work with devfile registries / indexes that use self-signed certificates (such as ones running behind a corporate firewall), using a flag (such as --insecure)
• odo should allow the user to enter credentials to access a secured devfile registry if needed
  • Examples being an index.json on a private github repo, Github enterprise, etc

This item only handles the secure devfile registry and download of the devfile from the secure registry. It does not handle the sample application being stored under a secure repo.

CC @elsony @deboer-tim

[kadel]

• odo should be able to work with devfile registries / indexes that use self-signed certificates (such as ones running behind a corporate firewall)

Self-signed certificates are insecure. By default, odo needs to fail if a self-signed or otherwise invalid or untrusted certificate is used.
If there is a need to support self-signed certificates, we need to make it optional and odo needs to always show a big warning that the certificate is self-signed or trusted.

odo should allow the user to enter credentials to access a secured devfile registry if needed

• Examples being an index.json on a private github repo, Github enterprise, etc

we should define how devfile registry will look like first. Is it going to be always a git repository? Simple HTTP(S) server serving static files? Or should we define some API for it?

/triage needs-information

[deboer-tim]

I agreed that self-signed certs are by definition insecure and should fail by default, but I do hit them regularly enough in dev systems. IMHO there should be an option to accept them, e.g. similar to "curl --insecure".

I also agree we should discuss where/how the registry can be hosted, and what the client requirements are. This should be a discussion along with other devfile registry clients so I've just opened an issue to discuss: devfile/api#39.

We have an internal security team that is recommending support for several scenarios. There's a meeting later today, we can try to distill this and provide more specific examples of the environments that we think it should support.

[adisky]

The current picture for a devfile registry is not much clear, We could assume that in any form, we need to provide authentication parameters to odo,

As with the current code odo access registry when it downloads devfile.yaml, in odo create or while odo catalog list components. Two approach i can think of is

• Provide authentication params while adding registry odo registry add <registry-name> <username> <password>, and odo would create a secret in the cluster for registry auth parms, and save the secret reference in preference.yaml. While accessing the registry, odo would use the secret ref to get auth params from the cluster.

• Provide authentication params flag to each command odo create --registry "name=<registry-name> username=<user-name> password=<password>", but this we cannot do with odo catalog list components.

[kadel]

It might be a little bit early for this. We still don't know how the registry will be secured.

[adisky]

It might be a little bit early for this. We still don't know how the registry will be secured.

Agreed it is still early to decide, lets wait for registry structure to be finalized.

[johnmcollier]

@kadel @adisky The devfile registry will be secured with an oauth provider on Kube, and access granted via a service account token.

[elsony]

There 3 scenarios that we need to consider cover for accessing secure devfile repos:

1. GitHub public repo:

• Clients authenticate with Git credentials or personal access token (PAT)
• TLS achieved with Git public CA signed cert in client trust store

2. GitHub Enterprise:

• Clients authenticate with GHE credentials or PAT
• TLS achieved with GHE public CA signed cert in client trust store

3. Hosted in cluster

• Clients authenticate with Service account token
• TLS achieved with ingress gateway CA signed cert in client trust store

[GeekArthur]

Couple of things we need to confirm before creating the design docs:

1. After we collect the credentials (username/password or PAT), how do we use those credentials to access the secure GitHub URL that hosts registry resource (eg. index.json and devfile.yaml). For PAT support, we might need to append the PAT to the HTTPS request, but need a way to support username/password as well.
2. Where we store those credentials? Does the credential storage needs extra credential as well?

• Mac: we can store them in keychain
• Windows: we can store them in somewhere like credential manager

3. We should discuss the odo design such as CLI design, arguments, flags, usage, examples for secure registry support, the odo design should consider the following stages:

• Collecting credentials
• Storing credentials
• Using credentials

[GeekArthur]

Draft Design Doc for Secure Registry Support

Table of contents

• Problem Statement
• Terminology
• Proposed Design
• Related Issues

Problem Statement

Currently odo only supports registry that is hosted by the platform that has publicly signed certificate, we should support secure registry so that user is able to store the confidential files to the registry and let registry host on the platform with certificate in user's trust store and need authentication on user side to access the platform.

Terminology

Registry: registry is the place that stores index file (index.json) and devfile (devfile.yaml) so that user can catalog and create devfile component from the registry. The registry itself can be hosted on GitHub (GitHub-hosted registry) or Cluster (Cluster-hosted registry)

Authentication method (Credential):

• Username/Password: usually user has full access to the resource with Username/Password authentication method.
• Personal Access Token (PAT): this is the recommended authentication method as user can grant limited resource access to the personal access token to make the resource access more secure.

Proposed Design

Support Scenarios:
In order to make secure registry support feature more clear and specific, we should support the following scenarios:

1. GitHub-hosted registry
   1. GitHub public:
   • Clients authenticate with GitHub personal access token
   • TLS achieved with Git public CA signed certification in client trust store
   2. GitHub Enterprise:
   • Clients authenticate with GHE personal access token
   • TLS achieved with GHE public CA signed certification in client trust store
2. Cluster-hosted registry
   • Clients authenticate with service account token
   • TLS achieved with ingress gateway CA signed cert in client trust store

Context:

1. Given GitHub is going to depreciate basic authentication with username/password https://developer.github.com/changes/2020-02-14-deprecating-password-auth/, we have to only support personal access token authentication method for GitHub-hosted registry scenario.
2. For cluster-hosted registry, the registry architecture would be creating a NGINX server to host the secure registry, then create a ingress gateway for the NGNIX server to let client access.

Work flow to access secure registry:

1. Collect Credential
   Currently we have odo registry add <registry name> <registry URL> and odo registry update <registry name> <registry URL> to add and update registry accordingly. Regarding the CLI design, we can implement the following CLI design to support collecting credential:

• odo registry add <registry name> <registry URL> --token <token>
• odo registry update <registry name> <registry URL> --token <token>

2. Store Credential
   We can use third-party package keyring(https://github.com/zalando/go-keyring) to help store user's credential, this package is platform agnostic, which means it can automatically use the existing keyring instance on the platform, for example:

• Mac: Implementation depends on the /usr/bin/security binary for interfacing with the OS X keychain.
• Linux: Implementation depends on the Secret Service dbus interface, which is provided by GNOME Keyring.
• Windows: Windows Credential Manager support by using the lib https://github.com/danieljoos/wincred.

3. Use Credential
   We can still use the built-in package net/http with adding token to the request header, sample code:

token := "123abc"
bearer := "Bearer " + token
req, err := http.NewRequest("GET", url, nil)
if err != nil {
	log.Println(err)
}
req.Header.Add("Authorization", bearer)

4. Delete Credential
   If multiple secure registries share the same credential, odo registry delete <registry name> will delete the credential from keyring instance once the last secure registry using that credential has been deleted.

Related issues

Dynamic registry support: #2940
Performance improvement for odo catalog list components: #3112

[GeekArthur]

Design proposal: #3329

[j-c-berger]

@GeekArthur, is there any ID work needed for this issue? Thanks!

[GeekArthur]

@GeekArthur, is there any ID work needed for this issue? Thanks!

@j-c-berger Yes, there are some ID related work but that can start once the feature is done, currently the feature is still in design proposal stage. Also I think the ID work should be done by Red Hat ID team as it's odo feature, or that should be done by you?

[j-c-berger]

@GeekArthur, thanks for getting back. I see how it's a good idea for Red Hat ID to cover the doc work since it's an odo feature.

My ID team has several meetings lined up in the next couple days. ID will discuss who should take up the doc work for this issue then get back to you. Thanks!

[GeekArthur]

@GeekArthur, thanks for getting back. I see how it's a good idea for Red Hat ID to cover the doc work since it's an odo feature.

My ID team has several meetings lined up in the next couple days. ID will discuss who should take up the doc work for this issue then get back to you. Thanks!

Okay, thanks! FYI, usually for each odo release, Red Hat ID team is gonna review and work on the documentation if it's needed.

[elsony]

One thing to note that this item only covers secure devfile repo and download of the devfile. The support of the sample applications coming from a stack in a secured repo is not covered as part of this item.

[GeekArthur]

/remove-triage needs-information

[GeekArthur]

/triage ready

[GeekArthur]

One thing to note that this item only covers secure devfile repo and download of the devfile. The support of the sample applications coming from a stack in a secured repo is not covered as part of this item.

Issue is open to track support downloading starter project from secure repository/host: #3567