Feature: Secrets integration, including Secrets Providers and Secrets Groups

[glennmatthews]

Towards: #541

This is the initial implementation of a generic/abstract Secret data model, including basic UI, REST API, and generic testing. At this time it doesn't include support for any concrete secret provider (environment variables, Hashicorp Vault, AWS Secrets Manager, etc.); my plan is to do incremental PRs into this feature branch as I proceed for ease of incremental reviewing. This PR, as it currently stands, is essentially all of the generic "boilerplate" needed to add a new, functional data model to Nautobot - it's a fair bit of code but there shouldn't be anything "surprising" in here so it should be a pretty quick/easy review.

[hellerve]

Comment for me: This seems to be the only page where we use Markdown ticks? I see that there are other forms in this file that do the same thing, but it doesn’t seem to be universal. Which way if preferred?

[glennmatthews]

Fair question. I don't think we have an official position on this yet. In general I'd like to see a lot of improvement to our docstrings (many of the forms in this file don't even have docstrings at all) and that's maybe something we should revisit then.

[hellerve]

This looks reasonable to me as of now, bearing in mind that I know next to nothing of the actual spec for this.

[jathanism]

Great start. My only gripe right now is that parameters are required and depending on the provider, that might not always be the case? I assume you've already considered that?

[glennmatthews]

I can't think of a case offhand where we'd have a "singleton" provider that has no configurable parameters (because it only ever returns a single specific secret?). If you can think of one, I'm happy to reconsider this part of the design. :-)

[jathanism]

I can't think of a case offhand where we'd have a "singleton" provider that has no configurable parameters (because it only ever returns a single specific secret?). If you can think of one, I'm happy to reconsider this part of the design. :-)

Nah, just covering bases. :)

[dgarros]

@glennmatthews great work

With this implementation is it possible to have difference secret per device, or group of devices ?

For example, as Nelly the Network Engineer, I want to be have a different login/password for my Cisco devices and my Arista devices or between my Console devices and my network devices etc ...

[glennmatthews]

With this implementation is it possible to have difference secret per device, or group of devices ?

That's the next piece of the design that we're actively thinking about. :-) Attaching a secret to an individual device is useful but definitely doesn't scale as well as being able to define secrets for groups of devices. We've talked about maybe building something atop config-contexts as an existing model in this area, but it's not definite yet.

[dgarros]

That's the next piece of the design that we're actively thinking about. :-) Attaching a secret to an individual device is useful but definitely doesn't scale as well as being able to define secrets for groups of devices. We've talked about maybe building something atop config-contexts as an existing model in this area, but it's not definite yet.

As a first step, a SecretsProvider class could define a model/content type and expect an object of this type to be passed when get_value_for_secret is called.

Something like that.

class DeviceSpecificSecretsProvider(SecretsProvider):
    """
    Example of a plugin-provided SecretsProvider that could return a value different per device or group of devices
    """
    model = "dcim.device" 

    @classmethod
    def get_value_for_secret(cls, secret, context=None):
        """
        Return a different secret based on the name of the device.
        """
        return secret.parameters.get(context.name)

[glennmatthews]

This should be nearly feature-complete at this point - please review. :-)

Possible remaining TODOs:

• Improve GraphQL representation of SecretsGroupAssociation "through" table
• Improve REST API for creating/updating SecretsGroup's contained SecretsGroupAssociations
• Move "Secrets" menu items out of "Extensibility" into a top-level "Secrets" menu
• Author release-notes content

Devices and Git Repositories can now use Secrets Groups:

[glennmatthews]

Improved GraphQL representation to allow for inspection of the "through" model:

[glennmatthews]

[jathanism]

Ship this beautiful beast!

[lampwins]

This looks awesome! Just a couple of comments/questions.

[lampwins]

Perhaps a premature optimization, but given the relative specificity defined here, should this be its own model to allow people to define their own?

[glennmatthews]

The hope was that we could keep the design simpler by not introducing two additional data models here. Additionally, they kind of need to be constants because we need to be able to look them up in some fashion - if the user can dynamically define which AccessType means "http(s)" (or even not define that type at all), how can a GitRepository look up the HTTP(S) secret that it needs?

[lampwins]

That's fair, but it is important to note we already have this problem with status, related to the imbedded logic for slacc and container. Not saying we should perpetuate that problem, but something to think about.