Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Secrets providers API, initial TextFile and EnvironmentVariable provider implementations #887

Merged
merged 8 commits into from
Sep 10, 2021

Conversation

glennmatthews
Copy link
Contributor

Fixes: #

  • Add model documentation for the Secret class
  • Add SecretsProvider abstract base class defining the interface for retrieving a specific secret from a specific secrets provider
  • Add EnvironmentVariableSecretsProvider and TextFileSecretsProvider concrete implementations
  • Secret.value property now retrieves the value from the specified SecretsProvider with the specified parameters
  • Add ability for plugins to register additional SecretsProvider subclasses
    • Add plugins documentation of this feature
    • Dummy plugin now includes a simple (insecure!) example ConstantValueSecretsProvider
  • Improved Secret edit view
    • "provider" selection is now a select menu listing available secrets providers, instead of a free-text field
    • Provider parameters can still be inputted as raw JSON, but specific SecretsProvider classes can now provide a ParametersForm that will be displayed as an alternative to the JSON text field
      • Added a view endpoint and custom jquery code to retrieve the appropriate ParametersForm when selecting or changing providers
      • Added JS to keep the JSON field and parameters form in sync with one another

TODO:

  • Identify what if any hard-coded and/or configurable restrictions we want to place on EnvironmentVariableSecretsProvider and TextFileSecretsProvider to reduce potential security risks of allowing a user to access environment variables or filesystem files that they shouldn't actually be permitted to access.

image

image

image

Copy link
Contributor

@jathanism jathanism left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The form does not appear to be presenting on create of a new secret, no matter the . If I create a new environment-variable new secret, the form doesn't show, but if I use JSON then go to edit the form, I am able to then use the form to update the value.

nautobot/extras/secrets/__init__.py Show resolved Hide resolved
@glennmatthews
Copy link
Contributor Author

The form does not appear to be presenting on create of a new secret, no matter the . If I create a new environment-variable new secret, the form doesn't show, but if I use JSON then go to edit the form, I am able to then use the form to update the value.

Good catch! Probably on the last N iterations of working on this I was just working with existing secrets and forgot to test the new-secret flow with the latest code. Will fix.

Copy link
Contributor

@jathanism jathanism left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registration comment/concern. Not quite asking for changes just yet, but another area where I'm curious about your line of thinking.

nautobot/extras/apps.py Show resolved Hide resolved
@itdependsnetworks
Copy link
Contributor

Presumably this is in support of #541, which did initially state that:

Nautobot needs a supported set of UI and APIs for working with secrets (Git tokens, device credentials, OAuth2 parameters, SSoT integration credentials, etc.).

In support of the highlighted "device credentials", I was expecting there would be some kind of context and/or accommodations for devices, where a given device would be presented as a configurable parameter as obj. I can totally understand if it has been descoped, but I am strong advocate to include this feature. If it is de-scoped, would be great to know either way though.

@glennmatthews
Copy link
Contributor Author

In support of the highlighted "device credentials", I was expecting there would be some kind of context and/or accommodations for devices, where a given device would be presented as a configurable parameter as obj. I can totally understand if it has been descoped, but I am strong advocate to include this feature. If it is de-scoped, would be great to know either way though.

The current idea is that rather than having a dynamic parameter in the Secret definition itself, we will provide the ability to associate a specific Secret with a single device or a set of devices (using the config-context pattern). This does mean that if you have a different set of credentials for each device, you will need to create a distinct Secret record per device (rather than a single Secret definition that somehow resolves to a different secret value per device), but in the (hopefully more common) case where you have a shared service account for a whole group of devices, you will be able to create a single Secret and share it across that set of devices. Does this adequately address your concern?

@itdependsnetworks
Copy link
Contributor

You are addressing the more common, however, there is certainly examples of per device, I actually had a customer today tell us that the there is a new requirement for the user/pass to be per device. I am not a fan of the config-context pattern, as is a bit of magic that will not be obvious to users. I also think that with the obj context, one can still have the same capability by example.

@glennmatthews
Copy link
Contributor Author

Good points, thanks Ken. We should discuss this some more. Need to make sure that any solution we come up with isn't too tightly coupled with Devices specifically (as that was a key shortcoming of the old feature). I'm also hesitant about the usability/painfulness of potentially needing something like a mixture of Jinja2 templating with JSON if we need to have dynamic parameters to the lookup.

@glennmatthews
Copy link
Contributor Author

Merging this into the gfm-secrets branch in its current state, will continue to iterate and refine before the final code drop.

@glennmatthews glennmatthews merged commit a94d140 into gfm-secrets Sep 10, 2021
@glennmatthews glennmatthews deleted the gfm-secrets-retrieval-environment branch September 10, 2021 14:13
glennmatthews added a commit that referenced this pull request Nov 15, 2021
… Groups (#868)

* Initial model, UI, and REST API for Secrets

* Secrets providers API, initial TextFile and EnvironmentVariable provider implementations (#887)

* Add Secret.value property, add EnvironmentVariable provider, add dummy-plugin Constant provider, add tests

* Add TextFileSecretProvider

* Add docs

* Improve display of secret providers in the UI

* Refactor SecretsProvider registration to use the Nautobot registry instead of python entry_points

* Refactor slightly

* Add ability for secrets providers to define an HTML form for parameter inputs

* Fix default value for JSONField and add error handling in JS

* Add username_secret and token_secret support to GitRepository

* Docs updates

* Review feedback - add description field, etc.

* Revise secrets docs; add SecretError exceptions instead of returning None on various failures

* One of these days I'll remember to run flake8 before pushing

* Review comments

* SecretsGroup feature (#1042)

* WIP

* More WIP

* WIP remove SecretType model

* Such WIP. Wow

* WIP: working secretsgroup-edit UI

* More WIP

* Change Category/Meaning to Access Type/Secret Type

* Add SecretsGroup key to Device model; get tests passing

* Add test coverage for REST API and filters

* Add SecretsGroup view tests

* Linting fixes

* Docs updates

* Cleanup leftover SecretType cruft

* Update nautobot/docs/user-guides/git-data-source.md

Co-authored-by: Jathan McCollum <jathan@gmail.com>

* Fix egregious issues

Co-authored-by: Jathan McCollum <jathan@gmail.com>

* Support Jinja2 templating of secret parameters (#1058)

* Support Jinja2 templating of secret parameters

* Add secrets providers to plugin detail view

* Doc updates

* Include SecretsGroupAssociation in GraphQL

* Move 'Secrets' to a top-level menu

* Don't try to sort `SecretsProvider` class objects in plugin config features registry (#1065)

* Fix TypeError when trying to sort `SecretsProvider` class objects
* Don't sort `secrets_providers` when added to features.

* Add release-note content for Secrets

* Update nautobot/extras/views.py

Co-authored-by: John Anderson <lampwins@gmail.com>

* Change FK to SecretsGroup behavior to SET_NULL

* Use render_jinja2() in rendered_parameters()

Co-authored-by: Jathan McCollum <jathan@gmail.com>
Co-authored-by: John Anderson <lampwins@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants