Read Git credentials from Vault

[Southclaws]

Currently, there are two ways to securely use Git to pull config repos and target repos:

1. ssh
2. basic auth

SSH is really awkward to automate declaratively, so I generally avoid it. Basic auth works well as GitHub/Lab both provide machine tokens for automated access.

The issue is, you need to pass these tokens in as environment variables. It also means propagating them through to the target config which makes multiple tokens awkward.

So, pico should instead just load all credentials from Vault.

And drop SSH support, it's not worth maintaining for the aforementioned reason.

---

General architecture:

Now:

• Init
• Ping vault
• Clone config repo
• Get secrets for targets and up targets

Future:

• Init
• Ping vault
• Acquire bootstrapping credentials - user/pass for config repo
• Clone config repo
• Get secrets for target repo credentials, clone target repos, up targets

This also means that there needs to be some form of naming convention for the secrets in Vault so they map to repositories. Maybe based on regex?

[Southclaws]

Given the following configuration:

Configs in GitLab: pico run https://gitlab.com/ns/config

A target on GitHub and BitBucket

T({
  name: "app",
  url: "https://github.com/ns/repo",
  up: "./run"
});
T({
  name: "app",
  url: "https://bitbucket.org/ns/repo",
  up: "./run"
});

There are three sets of credentials to acquire. They may have the following names in Vault:

• pico-gitlab-auth
• pico-github-auth
• pico-bitbucket-auth

To close this issue, there needs to be some logic that maps from the above configuration to these Vault secrets.

---

[ADRFranklin]

Why not have some global config in the main file which can be used to setup what these values could be.

For example:

G({
    "respository" {
        "gitlab": {
            "ssh": false,                
            "url": "https://${GITLAB_USERNAME}:${GITLAB_TOKEN}@gitlab.com",
            "path": "${VAULT_PATH}/core_deployment"
        },
        "bitbucket": {
            "ssh": true,
            "ssh_use_vault": true,
            "path": "/path" // since `ssh_use_vault` is true, this is the vault path
        },
        "github": {
            "ssh": true,
            "ssh_use_vault": false,
            "path": "~/.ssh/github.com_rsa" // this would be a volume in docker
        }        
    }
});

T({
  name: "app",
  repo: "github:/my_name/my_repo",
  up: "./run"
});

T({
  name: "app",
  repo: "bitbucket:/my_org/new_repo",
  up: "./run"
});

T({
  name: "app",
  repo: "gitlab:/my_name/my_repo",
  up: "./run"
});

I might have made a mistake in the way I did it, I didn't check it syntax issues, but you get the idea. I think it might also make sense to make use of Vault's SSH storage, but be able to use that to also automate it using SSH and if it was done via pico, it could make it extremely simple.

[Southclaws]

Yes I was thinking something like this. Instead of changing the repo key I'd introduce an auth key that pointed to some additionally declared resource.

[Southclaws]

T({
  name: "app",
  url: "https://github.com/ns/repo",
  auth: "github_01",
  up: "./run"
});
T({
  name: "app",
  url: "https://bitbucket.org/ns/repo",
  auth: "bb_01",
  up: "./run"
});

A({
  name: "github_01",
  source: "vault",
  path: "pico_auth",
  method: "http",
  user_key: "GITHUB_USERNAME",
  pass_key: "GITHUB_PASSWORD"
});

Would read from /secret/pico_auth the keys for username/password. (Assuming /secret is where the KV engine is mounted).

[Southclaws]

I have a feeling I will regret the single-letter function names in configuration files...

[ADRFranklin]

haha probably, but I do like the concept. It could work out nicely.

[Southclaws]

It suddenly got a whole lot more complex:

Currently, all the targets are bundled together into a single gitwatch instance with a single auth method (either nil or ssh). This means different auth methods for different targets won't work simply with the current setup.

I think it will be easier to update gitwatch to support per-repo settings such as auth instead of messing around with clustering targets with shared auth, firing multiple gitwatch instances and multiplexing their channels.