[s3] Allow the use of AWS profiles for access

* Add AWS_S3_SESSION_PROFILE setting that allows using an AWS profile as
an alternative to static credentials.

* Change unused variable to underscore

* Updated AUTHORS

* Make session_profile an attribute of S3Boto3Storage.

* Clarify options for specifying AWS credentials

* Factor out boto3 Session creation into it's own method

* Add comment for _create_session

Co-authored-by: Dan Hook <dhook@irobot.com>

AUTHORS

@@ -45,6 +45,7 @@ By order of apparition, thanks:
     * Taras Petriichuk (Dropbox write_mode option)
     * Zoe Liao (S3 docs)
     * Jonathan Ehwald
+    * Dan Hook
 
 
 Extra thanks to Marty for adding this in Django,

docs/backends/amazon-S3.rst

@@ -28,15 +28,29 @@ If you want to use something like `ManifestStaticFilesStorage`_ then you must in
 
     STATICFILES_STORAGE = 'storages.backends.s3boto3.S3ManifestStaticStorage'
 
-``AWS_ACCESS_KEY_ID``
-    Your Amazon Web Services access key, as a string.
+There are several different methods for specifying the AWS credentials used to create the S3 client.  In the order that ``S3Boto3Storage``
+searches for them:
 
-``AWS_SECRET_ACCESS_KEY``
-    Your Amazon Web Services secret access key, as a string.
+#. ``AWS_S3_SESSION_PROFILE``
+#. ``AWS_S3_ACCESS_KEY_ID`` and ``AWS_S3_SECRET_KEY_ID``
+#. ``AWS_ACCESS_KEY_ID`` and ``AWS_SECRET_ACCESS_KEY``
+#. The environment variables AWS_S3_ACCESS_KEY_ID and AWS_S3_SECRET_ACCESS_KEY
+#. The environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
+#. Use Boto3's default session
+
+``AWS_S3_SESSION_PROFILE``
+    The AWS profile to use instead of ``AWS_ACCESS_KEY_ID`` and ``AWS_SECRET_ACCESS_KEY``. All configuration information
+    other than the key id and secret key is ignored in favor of the other settings specified below.
 
 .. note::
+      If this is set, then it is a configuration error to also set ``AWS_S3_ACCESS_KEY_ID`` and ``AWS_S3_SECRET_ACCESS_KEY``.
+      ``AWS_ACCESS_KEY_ID`` and ``AWS_SECRET_ACCESS_KEY`` are ignored
 
-      If ``AWS_ACCESS_KEY_ID`` and ``AWS_SECRET_ACCESS_KEY`` are not set, boto3 internally looks up IAM credentials.
+``AWS_S3_ACCESS_KEY_ID or AWS_ACCESS_KEY_ID``
+    Your Amazon Web Services access key, as a string.
+
+``AWS_S3_SECRET_ACCESS_KEY or AWS_SECRET_ACCESS_KEY``
+    Your Amazon Web Services secret access key, as a string.
 
 ``AWS_STORAGE_BUCKET_NAME``
     Your Amazon Web Services storage bucket name, as a string.

storages/backends/s3boto3.py

@@ -294,9 +294,19 @@ def get_default_settings(self):
         else:
             cloudfront_signer = None
 
+        s3_access_key_id = setting('AWS_S3_ACCESS_KEY_ID')
+        s3_secret_access_key = setting('AWS_S3_SECRET_ACCESS_KEY')
+        s3_session_profile = setting('AWS_S3_SESSION_PROFILE')
+        if (s3_access_key_id or s3_secret_access_key) and s3_session_profile:
+            raise ImproperlyConfigured(
+                'AWS_S3_SESSION_PROFILE should not be provided with '
+                'AWS_S3_ACCESS_KEY_ID and AWS_S3_SECRET_ACCESS_KEY'
+            )
+
         return {
             "access_key": setting('AWS_S3_ACCESS_KEY_ID', setting('AWS_ACCESS_KEY_ID')),
             "secret_key": setting('AWS_S3_SECRET_ACCESS_KEY', setting('AWS_SECRET_ACCESS_KEY')),
+            "session_profile": setting('AWS_S3_SESSION_PROFILE'),
             "file_overwrite": setting('AWS_S3_FILE_OVERWRITE', True),
             "object_parameters": setting('AWS_S3_OBJECT_PARAMETERS', {}),
             "bucket_name": setting('AWS_STORAGE_BUCKET_NAME'),
@@ -342,12 +352,9 @@ def __setstate__(self, state):
     def connection(self):
         connection = getattr(self._connections, 'connection', None)
         if connection is None:
-            session = boto3.session.Session()
+            session = self._create_session()
             self._connections.connection = session.resource(
                 's3',
-                aws_access_key_id=self.access_key,
-                aws_secret_access_key=self.secret_key,
-                aws_session_token=self.security_token,
                 region_name=self.region_name,
                 use_ssl=self.use_ssl,
                 endpoint_url=self.endpoint_url,
@@ -356,6 +363,22 @@ def connection(self):
             )
         return self._connections.connection
 
+    def _create_session(self):
+        """
+        If a user specifies a profile name and this class obtains access keys
+        from another source such as environment variables,we want the profile
+        name to take precedence.
+        """
+        if self.session_profile:
+            session = boto3.Session(profile_name=self.session_profile)
+        else:
+            session = boto3.Session(
+                    aws_access_key_id=self.access_key,
+                    aws_secret_access_key=self.secret_key,
+                    aws_session_token=self.security_token
+            )
+        return session
+
     @property
     def bucket(self):
         """

tests/test_s3boto3.py

@@ -52,6 +52,13 @@ def test_clean_name_windows(self):
         path = self.storage._clean_name("path\\to\\somewhere")
         self.assertEqual(path, "path/to/somewhere")
 
+    def test_s3_session(self):
+        settings.AWS_S3_SESSION_PROFILE = "test_profile"
+        with mock.patch('boto3.Session') as mock_session:
+            storage = s3boto3.S3Boto3Storage()
+            _ = storage.connection
+            mock_session.assert_called_once_with(profile_name="test_profile")
+
     def test_pickle_with_bucket(self):
         """
         Test that the storage can be pickled with a bucket attached