Version 2

* Remove signing view Widgets are now signed when rendered. This behavior saves one call to the application server. * Add support for `accept` attribute Limit upload policy MIME-Type in `accept` attribute. * Add support for `multiple` file uploads * Remove `django-appconf` dependency * Remove progress bar * Add middleware to allow seamless Django integration * Add automatical `FileField.widget` overwrite * Add tests * Reduce file size * Reduce asset size
codingjoe · Aug 30, 2017 · f2c9d9c · f2c9d9c
1 parent f3fb3b5
commit f2c9d9c
Show file tree

Hide file tree

Showing 20 changed files with 439 additions and 564 deletions.
diff --git a/README.md b/README.md
@@ -1,72 +1,101 @@
 # django-s3file
 
-
 A lightweight file upload input for Django and Amazon S3.
 
+_less than 200 lines and no dependencies_
+
 [![PyPi Version](https://img.shields.io/pypi/v/django-s3file.svg)](https://pypi.python.org/pypi/django-s3file/)
 [![Build Status](https://travis-ci.org/codingjoe/django-s3file.svg?branch=master)](https://travis-ci.org/codingjoe/django-s3file)
-[![Code Health](https://landscape.io/github/codingjoe/django-s3file/master/landscape.svg?style=flat)](https://landscape.io/github/codingjoe/django-s3file/master)
 [![Test Coverage](https://coveralls.io/repos/codingjoe/django-s3file/badge.svg?branch=master)](https://coveralls.io/r/codingjoe/django-s3file)
-[![Code health](https://scrutinizer-ci.com/g/codingjoe/django-s3file/badges/quality-score.svg?b=master)](https://scrutinizer-ci.com/g/codingjoe/django-s3file/?branch=master)
 [![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://raw.githubusercontent.com/codingjoe/django-s3file/master/LICENSE)
-[![Join the chat at https://gitter.im/codingjoe/django-s3file](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/codingjoe/django-s3file?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
-
 
 ## Features
 
- - Pure JavaScript (no jQuery)
- - Python 2 and 3 support
- - Auto swapping based on your environment
- - Pluggable as it returns a simple django file, just like native input
- - Easily extensible (authentication, styles)
-
+*   lightweight: less 200 lines
+*   no JavaScript or Python dependencies (no jQuery)
+*   Python 3 and 2 support
+*   Auto swapping based on your environment
+*   works just like the buildin
 
 ## Installation
 
-Just install S3file using `pip` or `easy_install`.
+Just install S3file using `pip`.
+
 ```bash
 pip install django-s3file
 ```
-Don't forget to add `s3file` to the `INSTALLED_APPS`.
 
+Add the S3File app and middleware in your settings:
 
-## Usage
-
-### Simple integrations
+```python
 
-Include s3file's URLs in your URL root.
+INSTALLED_APPS = (
+    '...',
+    's3file',
+    '...',
+)
 
-**urls.py**
-```python
-urlpatterns = patterns(
-    ...
-    url(r'^s3file/', include('s3file.urls')),
+MIDDLEWARE_CLASSES = (
+    '...',
+    's3file.middleware.S3FileMiddleware',
+    '...',
 )
 ```
 
+## Usage
+
+By default S3File will replace Django's `FileField` widget,
+but you can also specify the widget manually and pass custom attributes.
+
+### Simple integrations
+
 **forms.py**
+
 ```python
-from s3file.forms import AutoFileInput
+from django import forms
+from django.db import models
+from s3file.forms import S3FileInput
+
+
+class ImageModel(models.Model):
+    file = models.FileField(upload_to='path/to/files')
 
-class MyModelForm(forms.ModelForm):
 
+class MyModelForm(forms.ModelForm):
     class Meta:
-        model = MyModel
-        fields = ('my_file_field')
+        model = ImageModel
+        fields = ('file',)
         widgets = {
-            'my_file_field': AutoFileInput
+            'file': S3FileInput(attrs={'accept': 'image/*'})
         }
 ```
 **Done!** No really, that's all that needs to be done.
 
 
-### Setting up the CORS policy on your AWS S3 Bucket
+### Setting up the AWS S3 bucket
+
+### Upload folder
+
+S3File uploads to a single folder. Files are later moved by Django when
+they are saved to the `upload_to` location.
+
+It is recommended to [setup expiration][aws-s3-lifecycle-rules] for that folder, to ensure that
+old and unused don't add up and produce costs.
+
+[aws-s3-lifecycle-rules]: http://docs.aws.amazon.com/AmazonS3/latest/dev/intro-lifecycle-rules.html
+
+The default folder name is: `tmp/s3file`
+You can change it by changing the `S3FILE_UPLOAD_PATH` setting.
+
+### CORS policy
+
+You will need to allow `POST` from all origins.
+Just add the following to your CORS policy. 
 
 ```xml
 <CORSConfiguration>
     <CORSRule>
         <AllowedOrigin>*</AllowedOrigin>
-        <AllowedMethod>PUT</AllowedMethod>
         <AllowedMethod>POST</AllowedMethod>
         <AllowedMethod>GET</AllowedMethod>
         <MaxAgeSeconds>3000</MaxAgeSeconds>
@@ -75,32 +104,30 @@ class MyModelForm(forms.ModelForm):
 </CORSConfiguration>
 ```
 
+### Uploading multiple files
 
-### Advanced usage examples
+Django does have limited [support to uploaded multiple files][uploading-multiple-files].
+S3File fully supports this feature. The custom middleware makes ensure that files
+are accessible via `request.FILES`, even tho they have been uploaded to AWS S3 directly
+and not to you Django application server.
 
-#### Authentication
-The signing endpoint supports CSRF by default but does not require a authenticated user.
-This and other behavior can be easily added by inheriting from the view.
+[uploading-multiple-files]: https://docs.djangoproject.com/en/1.11/topics/http/file-uploads/#uploading-multiple-files
 
-**views.py**
-```python
-from s3file.views import S3FileView
-from braces.views import LoginRequiredMixin
+### Security and Authentication
 
-class MyS3FileView(LoginRequiredMixin, S3FileView):
-    pass
-```
+django-s3file does not require any authentication setup. Files can only be uploaded
+to AWS S3 by users who have access to the form where the file upload is requested.
 
-Now don't forget to change the URLs.
+You can further limit user data using the [`accept`][att_input_accept]-attribute.
+The specified MIME-Type will be enforced in the AWS S3 policy as well, for enhanced
+server side protection.
 
-**urls.py**
-```python
-urlpatterns = patterns(
-    ...
-    url('^s3file/sign',
-        MyS3FileView.as_view(), name='s3file-sign'),
-)
-```
+S3File uses a strict policy and signature to grand clients permission to upload
+files to AWS S3. This signature expires based on Django's
+[`SESSION_COOKIE_AGE`][setting-SESSION_COOKIE_AGE] setting.
+
+[setting-SESSION_COOKIE_AGE]: https://docs.djangoproject.com/en/1.11/ref/settings/#std:setting-SESSION_COOKIE_AGE
+[att_input_accept]: https://www.w3schools.com/tags/att_input_accept.asp
 
 ## License
 

diff --git a/s3file/__init__.py b/s3file/__init__.py
@@ -0,0 +1 @@
+default_app_config = 's3file.apps.S3FileConfig'
diff --git a/s3file/apps.py b/s3file/apps.py
@@ -0,0 +1,16 @@
+from django.apps import AppConfig
+from django.conf import settings
+
+
+class S3FileConfig(AppConfig):
+    name = 's3file'
+    verbose_name = "S3File"
+
+    def ready(self):
+        from django.forms import FileField
+
+        if hasattr(settings, 'AWS_SECRET_ACCESS_KEY') \
+                and settings.AWS_SECRET_ACCESS_KEY:
+            from .forms import S3FileInput
+
+            FileField.widget = S3FileInput
diff --git a/s3file/conf.py b/s3file/conf.py
diff --git a/s3file/forms.py b/s3file/forms.py
@@ -1,16 +1,17 @@
-# -*- coding:utf-8 -*-
-from __future__ import unicode_literals
-
+import hashlib
+import hmac
+import json
 import logging
+import os
+import uuid
+from base64 import b64encode
 
-from django.core.files.storage import default_storage
-from django.core.urlresolvers import reverse_lazy
-from django.forms.widgets import (
-    FILE_INPUT_CONTRADICTION, CheckboxInput, ClearableFileInput
-)
-from django.utils.safestring import mark_safe
-
-from .conf import settings
+from django.conf import settings
+from django.forms.widgets import ClearableFileInput
+from django.utils.encoding import force_text
+from django.utils.functional import cached_property
+from django.utils.six import binary_type
+from django.utils.timezone import datetime, timedelta
 
 logger = logging.getLogger('s3file')
 
@@ -19,65 +20,93 @@ class S3FileInput(ClearableFileInput):
     """FileInput that uses JavaScript to directly upload to Amazon S3."""
 
     needs_multipart_form = False
-    signing_url = reverse_lazy('s3file-sign')
-    template = (
-        '<div class="s3file" data-policy-url="{policy_url}">'
-        '{input}'
-        '<input name="{name}" type="hidden" />'
-        '<div class="progress progress-striped active">'
-        '<div class="progress-bar" />'
-        '</div>'
-        '</div>'
-    )
-
-    def render(self, name, value, attrs=None):
-        parent_input = super(S3FileInput, self).render(name, value, attrs=None)
-        parent_input = parent_input.replace('name="{}"'.format(name), '')
-        output = self.template.format(
-            policy_url=self.signing_url,
-            input=parent_input,
-            name=name,
-        )
-        return mark_safe(output)
-
-    def is_initial(self, value):
-        return super(S3FileInput, self).is_initial(value)
-
-    def value_from_datadict(self, data, files, name):
-        filename = data.get(name, None)
-        if not self.is_required and CheckboxInput().value_from_datadict(
-                data, files, self.clear_checkbox_name(name)):
-            if filename:
-                # If the user contradicts themselves (uploads a new file AND
-                # checks the "clear" checkbox), we return a unique marker
-                # object that FileField will turn into a ValidationError.
-                return FILE_INPUT_CONTRADICTION
-            # False signals to clear any existing value, as opposed to just None
-            return False
-        if not filename:
-            return None
+    mime_type = None
+
+    def __init__(self, attrs=None):
+        self.expires = timedelta(seconds=settings.SESSION_COOKIE_AGE)
+        self.access_key = settings.AWS_ACCESS_KEY_ID
+        self.secret_access_key = settings.AWS_SECRET_ACCESS_KEY
+        self.bucket_name = settings.AWS_STORAGE_BUCKET_NAME
+        self.upload_path = getattr(settings, 'S3FILE_UPLOAD_PATH', os.path.join('tmp', 's3fine'))
+        super(S3FileInput, self).__init__(attrs=attrs)
         try:
-            upload = default_storage.open(filename)
-        except IOError:
-            logger.exception('File "%s" could not be found.', filename)
-            return False
+            self.mime_type = self.attrs['accept']
+        except KeyError:
+            pass
+
+    def get_expiration_date(self):
+        expiration_date = datetime.utcnow() + self.expires
+        return expiration_date.strftime('%Y-%m-%dT%H:%M:%S.000Z')
+
+    def build_attrs(self, *args, **kwargs):
+        attrs = super(S3FileInput, self).build_attrs(*args, **kwargs)
+        defaults = {
+            'data-policy': force_text(self.get_policy()),
+            'data-signature': self.get_signature(),
+            'data-key': self.upload_folder,
+            'data-s3-url': 'https://s3.amazonaws.com/%s' % self.bucket_name,
+            'data-AWSAccessKeyId': self.access_key,
+        }
+        defaults.update(attrs)
+        try:
+            defaults['class'] += ' s3file'
+        except KeyError:
+            defaults['class'] = 's3file'
+        return defaults
+
+    def get_secret_access_key(self):
+        return binary_type(self.secret_access_key.encode('utf-8'))
+
+    def get_policy(self):
+        policy = {
+            "expiration": self.get_expiration_date(),
+            "conditions": self.get_conditions(),
+        }
+        policy_json = json.dumps(policy)
+        policy_json = policy_json.replace('\n', '').replace('\r', '')
+        return b64encode(binary_type(policy_json.encode('utf-8')))
+
+    def get_conditions(self):
+        conditions = [
+            {"bucket": self.bucket_name},
+            ["starts-with", "$key", self.upload_folder],
+            {"success_action_status": "201"},
+        ]
+        if self.mime_type:
+            top_type, sub_type = self.mime_type.split('/', 1)
+            if sub_type == '*':
+                conditions.append(["starts-with", "$Content-Type", "%s/" % top_type])
+            else:
+                conditions.append({"Content-Type": self.mime_type})
         else:
-            return upload
+            conditions.append(["starts-with", "$Content-Type", ""])
+
+        return conditions
+
+    @cached_property
+    def upload_folder(self):
+        return os.path.join(
+            self.upload_path,
+            uuid.uuid4().hex,
+        )
+
+    def get_signature(self):
+        """
+        Return S3 upload signature.
+
+        :rtype: dict
+        """
+        policy_object = self.get_policy()
+        signature = hmac.new(
+            self.get_secret_access_key(),
+            policy_object,
+            hashlib.sha1
+        ).digest()
+
+        return force_text(b64encode(signature))
 
     class Media:
         js = (
             's3file/js/s3file.js',
 
         )
-        css = {
-            'all': (
-                's3file/css/s3file.css',
-            )
-        }
-
-
-if hasattr(settings, 'AWS_SECRET_ACCESS_KEY') \
-        and settings.AWS_SECRET_ACCESS_KEY:
-    AutoFileInput = S3FileInput
-else:
-    AutoFileInput = ClearableFileInput