Supporting Private GitHub repos and other artifact stores

[endzyme]

Fixes #684 (potentially)
Related issue: #684

Overview

Hi all, I am looking for some initial implementation feedback on supporting private GitHub repository releases as well as a few other potential artifact repositories.

The intent here is to support some way of running custom commands, which return an artifact file via stdout. This could solve #684 by running something like gh api -H Accept:application/octet-stream /repos/kubernetes-sigs/krew/releases/assets/55894121 to download artifacts from a private GitHub repo's release assets. It could also allow people to have their on private krew indexes which rely upon local machine tooling to get the artifacts. I'm thinking example commands like aws s3 cp s3://my-fancy-private-bucket/my-cool-krew-plugin.tgz -. This type of support could separate the concerns of auth to the local machine running the installation.

I initially wanted to implement this as an S3 SDK scheme but decided it would be better to enable more options via running commands on the local machine. Happy to discuss alternative approaches here too.

Early feedback request on the design

Keep in mind my testing and work is pretty rough and just an initial concept.

1. Should this implementation bloat the uri spec key to support other "schemes" (such as file:// and cmd://) or should it really be a separate key in the plugin manifest spec? (Providing string like cmd://run-my-super-sweet-script -i thing -o otherthing just feels kinda not ideal.)

2. Should this leverage temp files on the file system to "buffer" the stdout from the "artifact download command"? I initially attempted to use exec.Command() and exec.Command().Stdout() but hit issues with the pipe closing before the stream could be read (appears to be how exec.Command() is intended to function).

3. Is this introducing too much of a security risk, allowing manifests to effectively execute arbitrary code via krew when installing plugins?

Below is an example plugin manifest using this PR to give an example of how this is written right now.

apiVersion: krew.googlecontainertools.github.com/v1alpha2
kind: Plugin
metadata:
  name: testfile
spec:
  version: "v0.0.1"
  homepage: https://foobar.com/
  shortDescription: Throwin testfiles
  caveats: |
    Something something testfile

  platforms:
  - uri: cmd://gh api -H Accept:application/octet-stream /repos/kubernetes-sigs/krew/releases/assets/55894121
    sha256: 5df32eaa0e888a2566439c4ccb2ef3a3e6e89522f2f2126030171e2585585e4f
    bin: testfile
    files:
    - from: ./krew-linux_amd64
      to: testfile
    selector:
      matchExpressions:
      - key: "os"
        operator: "In"
        values:
        - linux
        - darwin

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: endzyme
Once this PR has been reviewed and has the lgtm label, please assign chriskim06 for approval by writing /assign @chriskim06 in a comment. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Welcome @endzyme!

It looks like this is your first PR to kubernetes-sigs/krew 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/krew has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[ahmetb]

Is it ok we start with an issue detailing how will it look like instead of reviewing the code and discussing this on a PR?

The main concern for me right now is "can we find an alternative way to reference private artifacts + authenticate that'll work for most users who need privately fetch packages, and how does that look like for users and package maintainers to set up".

[endzyme]

Sure, I can close this out.