Support username/password credentials stored as JSON formatted secret

[fknittel]

Feature Request

Some of our secrets are stored in JSON format (example from aws cli):

{
  "ARN": "arn:aws:secretsmanager:us-west-2:123456789012:secret:MyTestDatabaseSecret-a1b2c3",
  "Name": "MyTestDatabaseSecret",
  "VersionId": "EXAMPLE1-90ab-cdef-fedc-ba987EXAMPLE",
  "SecretString": "{\n  \"username\":\"david\",\n  \"password\":\"BnQw&XDWgaEeT9XGTT29\"\n}\n",
  "VersionStages": [
    "AWSPREVIOUS"
  ],
  "CreatedDate": 1523477145.713
}

To access this type of secret from Jenkins, we currently duplicate them in the format supported by this plugin for username/password secrets ("raw" secret with the tags jenkins:credentials:type = usernamePassword and jenkins:credentials:username = david).

We would like to avoid this duplication, as it is currently a manual step for us and we need to keep the duplicated secrets manually in sync during password rotations.

Suggestion

Support a set of additional tags for the jenkins:credentials:type = usernamePassword type, which allows the plugin to read the correct JSON fields:
jenkins:credentials:usernameField = username
jenkins:credentials:passwordField = password
This would allow any JSON formatted secret and support any possible field names.

To clarify, the following JSON secret:

  "SecretString": "{\n  \"user\":\"david\",\n  \"pass\":\"BnQw&XDWgaEeT9XGTT29\"\n}\n",

would have the following tags attached:
jenkins:credentials:type = usernamePassword
jenkins:credentials:usernameField = user
jenkins:credentials:passwordField = pass

Alternative

A simpler, less flexible alternative would be to add a new credentials type specifically for the common JSON format with username and password fields. Possibly jenkins:credentials:type = jsonUsernamePassword.

[fknittel]

Just noticed, that you document a work-around under advanced usage using a string secret type and readJSON. Unfortunately this approach does not work in places where we cannot add custom credential post-processing.

[chriskilding]

Which places are you unable to add custom credential post-processing for? I ask because our teams at work that use JSON values for credentials have been able to achieve what they want with the readJson snippet, from the readme.

[fknittel]

We use a few Jenkins plugins that we configure via CasC yaml, which need direct references to credentialIds. (One recent example which triggered me to create this issue: The influxdb plugin migrated from plain username/password to credentialId.)

We could probably replace all global plugin configuration with job-local configuration (using the readJson approach), so it's not impossible to do, but it feels like a work-around for a rather common case?

[chriskilding]

The reason this is unfortunately not possible is that the username field has to be supplied when credentials are listed, as well as when they are used in a job. The credentials list is populated with a ListSecrets call (the result is cached for 5 mins). This is to ensure that the performance is reasonable, and to avoid unnecessarily exposing secret values (which GetSecretValue would do). The only way to supply non-secret fields like the username in a way that ListSecrets can show them, is to store those fields as AWS tags on the secret.

I get where you're coming from - it would be nice if different tools would share secrets more easily - but alas the credentials API doesn't provide wiggle room here.

[fknittel]

@chriskilding Hm, thanks very much for your analysis. What do you think of the following compromise:

If all our json secrets additionally had a jenkins:credentials:username tag duplicating the username found in the username json field, we would still be able to use the same secret entry for multiple tools (and also use the regular password rotation lambdas). While the password value changes regularly, the username value is mostly a constant, so we don't really lose much from a maintenance stand-point.

The jenkins plugin would use the regular username tag for ListSecrets and only need to parse the json for GetSecretValue.
(The json mode could be activated by specifying some sort of jenkins:credentials:passwordFieldName tag.)