AWS_PROFILE, credential_process

[OJFord]

Output of restic version

restic 0.10.0 compiled with go1.15.2 on linux/amd64

What should restic do differently? Which functionality do you think we should add?

It would be nice if standard AWS credential configuration was used, as in not just the same env var names, but actually using boto the AWS SDK, or otherwise respecting AWS_PROFILE and the credential_process option in the same way.

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sourcing-external.html

What are you trying to do? What problem would this solve?

Presently a credential_process cannot be used to authenticate with S3, and an S3-compatible cloud storage cannot be used by using the S3 backend with the appropriate AWS_PROFILE in the environment (and endpoint specified to -r).

On a system already configured to work that way it's at best inconvenient to need a different mechanism, and at worst I imagine there are (corporate) environments that would simply not allow it as a result, since it's been decided to require a credential_process that obtains the secret from some store that also records access for auditing purposes, for example.

Did restic help you today? Did it make you happy in any way?

Yes! Cleanest and most nicely documented solution I've found; no weird novel terminology to learn for the sake of it, and I like the tags and that anything can then be backed up to the same repo. It's so intuitive that coming from something else it actually took me some time to grok it, if that makes any sense. I'm not yet sure why 'directories' are needed as well (one system's directory may be another location elsewhere, but they both represent the same thing and backup to the same repo?) but perhaps it will become clear. Like the look of it a lot and look forward to using it more!

[OJFord]

Sorry if reference to boto was confusing - I had thought AWS called all its SDKs that, but actually it seems it's just the Python one. Go's is the much more apt aws/aws-sdk-go.

Is there a reason restic needs to use minio-go instead? The AWS SDK surely supports non-AWS endpoints, in fact it appears to require specifying it AWS or not (I think that's a recent-ish S3 API change actually): https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#GetObjectInput

(And I've checked, it does support credential_process.)

[MichaelEischer]

The switch to the minio library was made in #366. Looks like at that time the AWS SDK didn't support non-AWS endpoints.

I can't really judge right now what the implications of switching to the official AWS SDK libraries would be, both in terms of functionality and compatibility.