Merge d37ee31 into fc26cca

README.md

@@ -1,72 +1,102 @@
 # django-s3file
 
-
 A lightweight file upload input for Django and Amazon S3.
 
 [![PyPi Version](https://img.shields.io/pypi/v/django-s3file.svg)](https://pypi.python.org/pypi/django-s3file/)
 [![Build Status](https://travis-ci.org/codingjoe/django-s3file.svg?branch=master)](https://travis-ci.org/codingjoe/django-s3file)
-[![Code Health](https://landscape.io/github/codingjoe/django-s3file/master/landscape.svg?style=flat)](https://landscape.io/github/codingjoe/django-s3file/master)
 [![Test Coverage](https://coveralls.io/repos/codingjoe/django-s3file/badge.svg?branch=master)](https://coveralls.io/r/codingjoe/django-s3file)
-[![Code health](https://scrutinizer-ci.com/g/codingjoe/django-s3file/badges/quality-score.svg?b=master)](https://scrutinizer-ci.com/g/codingjoe/django-s3file/?branch=master)
 [![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://raw.githubusercontent.com/codingjoe/django-s3file/master/LICENSE)
-[![Join the chat at https://gitter.im/codingjoe/django-s3file](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/codingjoe/django-s3file?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
-
 
 ## Features
 
- - Pure JavaScript (no jQuery)
- - Python 2 and 3 support
- - Auto swapping based on your environment
- - Pluggable as it returns a simple django file, just like native input
- - Easily extensible (authentication, styles)
-
+*   lightweight: less 200 lines
+*   no JavaScript or Python dependencies (no jQuery)
+*   Python 3 and 2 support
+*   auto enabled based on your environment
+*   works just like the build-in
 
 ## Installation
 
-Just install S3file using `pip` or `easy_install`.
+Just install S3file using `pip`.
+
 ```bash
 pip install django-s3file
 ```
-Don't forget to add `s3file` to the `INSTALLED_APPS`.
 
+Add the S3File app and middleware in your settings:
 
-## Usage
-
-### Simple integrations
+```python
 
-Include s3file's URLs in your URL root.
+INSTALLED_APPS = (
+    '...',
+    's3file',
+    '...',
+)
 
-**urls.py**
-```python
-urlpatterns = patterns(
-    ...
-    url(r'^s3file/', include('s3file.urls')),
+MIDDLEWARE = (
+    '...',
+    's3file.middleware.S3FileMiddleware',
+    '...',
 )
 ```
 
+## Usage
+
+By default S3File will replace Django's `FileField` widget,
+but you can also specify the widget manually and pass custom attributes.
+
+The `FileField`'s widget is only than automatically replaced when the
+`DEFAULT_FILE_STORAGE` setting is set to `django-storages`' `S3Boto3Storage`.
+
+### Simple integrations
+
 **forms.py**
+
 ```python
-from s3file.forms import AutoFileInput
+from django import forms
+from django.db import models
+from s3file.forms import S3FileInput
+
+
+class ImageModel(models.Model):
+    file = models.FileField(upload_to='path/to/files')
 
-class MyModelForm(forms.ModelForm):
 
+class MyModelForm(forms.ModelForm):
     class Meta:
-        model = MyModel
-        fields = ('my_file_field')
+        model = ImageModel
+        fields = ('file',)
         widgets = {
-            'my_file_field': AutoFileInput
+            'file': S3FileInput(attrs={'accept': 'image/*'})
         }
 ```
 **Done!** No really, that's all that needs to be done.
 
 
-### Setting up the CORS policy on your AWS S3 Bucket
+### Setting up the AWS S3 bucket
+
+### Upload folder
+
+S3File uploads to a single folder. Files are later moved by Django when
+they are saved to the `upload_to` location.
+
+It is recommended to [setup expiration][aws-s3-lifecycle-rules] for that folder, to ensure that
+old and unused file uploads don't add up and produce costs.
+
+[aws-s3-lifecycle-rules]: http://docs.aws.amazon.com/AmazonS3/latest/dev/intro-lifecycle-rules.html
+
+The default folder name is: `tmp/s3file`
+You can change it by changing the `S3FILE_UPLOAD_PATH` setting.
+
+### CORS policy
+
+You will need to allow `POST` from all origins.
+Just add the following to your CORS policy. 
 
 ```xml
 <CORSConfiguration>
     <CORSRule>
         <AllowedOrigin>*</AllowedOrigin>
-        <AllowedMethod>PUT</AllowedMethod>
         <AllowedMethod>POST</AllowedMethod>
         <AllowedMethod>GET</AllowedMethod>
         <MaxAgeSeconds>3000</MaxAgeSeconds>
@@ -75,32 +105,30 @@ class MyModelForm(forms.ModelForm):
 </CORSConfiguration>
 ```
 
+### Uploading multiple files
 
-### Advanced usage examples
+Django does have limited [support to uploaded multiple files][uploading-multiple-files].
+S3File fully supports this feature. The custom middleware makes ensure that files
+are accessible via `request.FILES`, even thogh they have been uploaded to AWS S3 directly
+and not to your Django application server.
 
-#### Authentication
-The signing endpoint supports CSRF by default but does not require a authenticated user.
-This and other behavior can be easily added by inheriting from the view.
+[uploading-multiple-files]: https://docs.djangoproject.com/en/1.11/topics/http/file-uploads/#uploading-multiple-files
 
-**views.py**
-```python
-from s3file.views import S3FileView
-from braces.views import LoginRequiredMixin
+### Security and Authentication
 
-class MyS3FileView(LoginRequiredMixin, S3FileView):
-    pass
-```
+django-s3file does not require any authentication setup. Files can only be uploaded
+to AWS S3 by users who have access to the form where the file upload is requested.
 
-Now don't forget to change the URLs.
+You can further limit user data using the [`accept`][att_input_accept]-attribute.
+The specified MIME-Type will be enforced in the AWS S3 policy as well, for enhanced
+server side protection.
 
-**urls.py**
-```python
-urlpatterns = patterns(
-    ...
-    url('^s3file/sign',
-        MyS3FileView.as_view(), name='s3file-sign'),
-)
-```
+S3File uses a strict policy and signature to grant clients permission to upload
+files to AWS S3. This signature expires based on Django's
+[`SESSION_COOKIE_AGE`][setting-SESSION_COOKIE_AGE] setting.
+
+[setting-SESSION_COOKIE_AGE]: https://docs.djangoproject.com/en/1.11/ref/settings/#std:setting-SESSION_COOKIE_AGE
+[att_input_accept]: https://www.w3schools.com/tags/att_input_accept.asp
 
 ## License
 

s3file/__init__.py

@@ -0,0 +1 @@
+default_app_config = 's3file.apps.S3FileConfig'

s3file/apps.py

@@ -0,0 +1,20 @@
+from django.apps import AppConfig
+
+try:
+    from storages.backends.s3boto3 import S3Boto3Storage
+except ImportError:
+    from storages.backends.s3boto import S3BotoStorage as S3BotoStorage
+
+
+class S3FileConfig(AppConfig):
+    name = 's3file'
+    verbose_name = 'S3File'
+
+    def ready(self):
+        from django.forms import FileField
+        from django.core.files.storage import default_storage
+
+        if isinstance(default_storage, S3Boto3Storage):
+            from .forms import S3FileInput
+
+            FileField.widget = S3FileInput

s3file/forms.py

@@ -1,16 +1,11 @@
-# -*- coding:utf-8 -*-
-from __future__ import unicode_literals
-
 import logging
+import os
+import uuid
 
+from django.conf import settings
 from django.core.files.storage import default_storage
-from django.core.urlresolvers import reverse_lazy
-from django.forms.widgets import (
-    FILE_INPUT_CONTRADICTION, CheckboxInput, ClearableFileInput
-)
-from django.utils.safestring import mark_safe
-
-from .conf import settings
+from django.forms.widgets import ClearableFileInput
+from django.utils.functional import cached_property
 
 logger = logging.getLogger('s3file')
 
@@ -19,65 +14,70 @@ class S3FileInput(ClearableFileInput):
     """FileInput that uses JavaScript to directly upload to Amazon S3."""
 
     needs_multipart_form = False
-    signing_url = reverse_lazy('s3file-sign')
-    template = (
-        '<div class="s3file" data-policy-url="{policy_url}">'
-        '{input}'
-        '<input name="{name}" type="hidden" />'
-        '<div class="progress progress-striped active">'
-        '<div class="progress-bar" />'
-        '</div>'
-        '</div>'
-    )
+    mime_type = None
 
-    def render(self, name, value, attrs=None):
-        parent_input = super(S3FileInput, self).render(name, value, attrs=None)
-        parent_input = parent_input.replace('name="{}"'.format(name), '')
-        output = self.template.format(
-            policy_url=self.signing_url,
-            input=parent_input,
-            name=name,
-        )
-        return mark_safe(output)
+    def __init__(self, attrs=None):
+        self.expires = settings.SESSION_COOKIE_AGE
+        self.upload_path = getattr(settings, 'S3FILE_UPLOAD_PATH', os.path.join('tmp', 's3file'))
+        super(S3FileInput, self).__init__(attrs=attrs)
+        try:
+            self.mime_type = self.attrs['accept']
+        except KeyError:
+            pass
 
-    def is_initial(self, value):
-        return super(S3FileInput, self).is_initial(value)
+    @property
+    def bucket_name(self):
+        return default_storage.bucket.name
 
-    def value_from_datadict(self, data, files, name):
-        filename = data.get(name, None)
-        if not self.is_required and CheckboxInput().value_from_datadict(
-                data, files, self.clear_checkbox_name(name)):
-            if filename:
-                # If the user contradicts themselves (uploads a new file AND
-                # checks the "clear" checkbox), we return a unique marker
-                # object that FileField will turn into a ValidationError.
-                return FILE_INPUT_CONTRADICTION
-            # False signals to clear any existing value, as opposed to just None
-            return False
-        if not filename:
-            return None
+    @property
+    def client(self):
+        return default_storage.connection.meta.client
+
+    def build_attrs(self, *args, **kwargs):
+        attrs = super(S3FileInput, self).build_attrs(*args, **kwargs)
+        response = self.client.generate_presigned_post(
+            self.bucket_name, os.path.join(self.upload_folder, '${filename}'),
+            Conditions=self.get_conditions(),
+            ExpiresIn=self.expires,
+        )
+        defaults = {
+            'data-fields-%s' % key: value
+            for key, value in response['fields'].items()
+        }
+        defaults['data-url'] = response['url']
+        defaults.update(attrs)
         try:
-            upload = default_storage.open(filename)
-        except IOError:
-            logger.exception('File "%s" could not be found.', filename)
-            return False
+            defaults['class'] += ' s3file'
+        except KeyError:
+            defaults['class'] = 's3file'
+        return defaults
+
+    def get_conditions(self):
+        conditions = [
+            {"bucket": self.bucket_name},
+            ["starts-with", "$key", self.upload_folder],
+            {"success_action_status": "201"},
+        ]
+        if self.mime_type:
+            top_type, sub_type = self.mime_type.split('/', 1)
+            if sub_type == '*':
+                conditions.append(["starts-with", "$Content-Type", "%s/" % top_type])
+            else:
+                conditions.append({"Content-Type": self.mime_type})
         else:
-            return upload
+            conditions.append(["starts-with", "$Content-Type", ""])
+
+        return conditions
+
+    @cached_property
+    def upload_folder(self):
+        return os.path.join(
+            self.upload_path,
+            uuid.uuid4().hex,
+        )
 
     class Media:
         js = (
             's3file/js/s3file.js',
 
         )
-        css = {
-            'all': (
-                's3file/css/s3file.css',
-            )
-        }
-
-
-if hasattr(settings, 'AWS_SECRET_ACCESS_KEY') \
-        and settings.AWS_SECRET_ACCESS_KEY:
-    AutoFileInput = S3FileInput
-else:
-    AutoFileInput = ClearableFileInput

s3file/middleware.py

@@ -0,0 +1,24 @@
+import os
+
+from django.core.files.storage import default_storage
+
+try:
+    from django.utils.deprecation import MiddlewareMixin
+except ImportError:
+    MiddlewareMixin = object
+
+
+class S3FileMiddleware(MiddlewareMixin):
+    @staticmethod
+    def get_files_from_storage(paths):
+        """Return S3 file where the name does not include the path."""
+        for path in paths:
+            f = default_storage.open(path)
+            f.name = os.path.basename(path)
+            yield f
+
+    def process_request(self, request):
+        file_fields = request.POST.getlist('s3file', [])
+        for field_name in file_fields:
+            paths = request.POST.getlist(field_name, [])
+            request.FILES.setlist(field_name, list(self.get_files_from_storage(paths)))