Updated docs and added configuration for S3 objectstore upload of media files

[paepke]

Docs and configuration examples how to use an S3 backend for uploading media files

Fixes:

#1814

[jeremystretch]

Any work involving reference to external systems must take special caution to avoid proprietary naming. For instance, we would not introduce a setting named S3_STORAGE_ENABLED as S3 is the brand name of an Amazon product. We want to avoid this because introducing support for additional products would necessitate introducing parallel settings. Instead, settings should use generic names, and any services or libraries requiring specific keys should be transparent to NetBox.

For example, instead of:

AWS_ACCESS_KEY_ID = ''
AWS_SECRET_ACCESS_KEY = ''
AWS_S3_ENDPOINT_URL = ''

We might do something like this:

STORAGE_PARAMETERS = {
    'AWS_ACCESS_KEY_ID': '',
    'AWS_SECRET_ACCESS_KEY': '',
    'AWS_S3_ENDPOINT_URL': '',
}

These arguments can then be passed through to the storage library. This ensures that we can support all current and future settings pertinent to the library without any additional code changes to NetBox.

(I'm not particular suggesting STORAGE_PARAMETERS as a setting name, just illustrating how we can wrap service-specific keys under a generic label.)

[paepke]

I was thinking over the weekend about your proposal. In the first place it sounds very good, but I'm unsure how to accomplish that in a maintenance friendly way.
The different django-storage backends have very different constants how they are configured like "AWS_", "GS_" or "AZURE_*". Possible plugins to django-storage or future versions will probably follow this naming structure.
There are these solutions with come to my mind:

1. Copy over the values from the STORAGE_PARAMETERS dict via locals().update(STORAGE_PARAMETERS). This looks kinda messy in my eyes
2. reassigning the statics by naming them - this would lead to a big block of if/else constructs
3. writing an additional wrapper for the django-storages plugin. This seems a bit of over the top just for settings the variables in the settings.py correctly
4. Your ideas?

S3 is not only a product name, its a standard which a lot of other implementations follow and they call themselves "S3 Compliant". Thats why it was ok for me in the first place to use this kind of naming. Additionally the AWS_ prefix is widely used in the documentation of the services which provides a S3 api.
Anyway I highly appreciate your feedback and I'm willing to develop this further to have an good configuration which makes it easy for the user and the codebase to keep this feature maintainable.

[jeremystretch]

Closing this out for now as it's stalled. Let's continue the discussion in #1814 where it's more visible.

[nward]

@paepke commenting here to alert you that I've discussed my approaches for this in #1814. I think we have similar thoughts on this stuff. I have a PoC of my ideas here https://github.com/SearchLightNZ/netbox/tree/configurable_image_storage - this is a more generic implementation than just "S3" and just django-storages, and is of course compatible with those.