Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Fixed #9535 -- Added a reference guide for file upload classes.

  • Loading branch information...
commit c8c2b8a6382f255b4f0b8296906bdef8eebb5809 1 parent 36de292
@coder9042 coder9042 authored timgraham committed
View
1  docs/ref/files/index.txt
@@ -10,3 +10,4 @@ File handling
file
storage
+ uploads
View
244 docs/ref/files/uploads.txt
@@ -0,0 +1,244 @@
+==================================
+Uploaded Files and Upload Handlers
+==================================
+
+.. module:: django.core.files.uploadedfile
+ :synopsis: Classes representing uploaded files.
+
+Uploaded files
+==============
+
+.. class:: UploadedFile
+
+During file uploads, the actual file data is stored in :attr:`request.FILES
+<django.http.HttpRequest.FILES>`. Each entry in this dictionary is an
+``UploadedFile`` object (or a subclass) -- a simple wrapper around an uploaded
+file. You'll usually use one of these methods to access the uploaded content:
+
+.. method:: UploadedFile.read()
+
+ Read the entire uploaded data from the file. Be careful with this method:
+ if the uploaded file is huge it can overwhelm your system if you try to
+ read it into memory. You'll probably want to use ``chunks()`` instead; see
+ below.
+
+.. method:: UploadedFile.multiple_chunks(chunk_size=None)
+
+ Returns ``True`` if the uploaded file is big enough to require reading in
+ multiple chunks. By default this will be any file larger than 2.5 megabytes,
+ but that's configurable; see below.
+
+.. method:: UploadedFile.chunks(chunk_size=None)
+
+ A generator returning chunks of the file. If ``multiple_chunks()`` is
+ ``True``, you should use this method in a loop instead of ``read()``.
+
+ In practice, it's often easiest simply to use ``chunks()`` all the time.
+ Looping over ``chunks()`` instead of using ``read()`` ensures that large
+ files don't overwhelm your system's memory.
+
+Here are some useful attributes of ``UploadedFile``:
+
+.. attribute:: UploadedFile.name
+
+ The name of the uploaded file (e.g. ``my_file.txt``).
+
+.. attribute:: UploadedFile.size
+
+ The size, in bytes, of the uploaded file.
+
+.. attribute:: UploadedFile.content_type
+
+ The content-type header uploaded with the file (e.g. :mimetype:`text/plain`
+ or :mimetype:`application/pdf`). Like any data supplied by the user, you
+ shouldn't trust that the uploaded file is actually this type. You'll still
+ need to validate that the file contains the content that the content-type
+ header claims -- "trust but verify."
+
+.. attribute:: UploadedFile.content_type_extra
+
+ .. versionadded:: 1.7
+
+ A dictionary containing extra parameters passed to the ``content-type``
+ header. This is typically provided by services, such as Google App Engine,
+ that intercept and handle file uploads on your behalf. As a result your
+ handler may not receive the uploaded file content, but instead a URL or
+ other pointer to the file. (see `RFC 2388`_ section 5.3).
+
+ .. _RFC 2388: http://www.ietf.org/rfc/rfc2388.txt
+
+.. attribute:: UploadedFile.charset
+
+ For :mimetype:`text/*` content-types, the character set (i.e. ``utf8``)
+ supplied by the browser. Again, "trust but verify" is the best policy here.
+
+.. note::
+
+ Like regular Python files, you can read the file line-by-line simply by
+ iterating over the uploaded file:
+
+ .. code-block:: python
+
+ for line in uploadedfile:
+ do_something_with(line)
+
+ However, *unlike* standard Python files, :class:`UploadedFile` only
+ understands ``\n`` (also known as "Unix-style") line endings. If you know
+ that you need to handle uploaded files with different line endings, you'll
+ need to do so in your view.
+
+Subclasses of ``UploadedFile`` include:
+
+.. class:: TemporaryUploadedFile
+
+ A file uploaded to a temporary location (i.e. stream-to-disk). This class
+ is used by the
+ :class:`~django.core.files.uploadhandler.TemporaryFileUploadHandler`. In
+ addition to the methods from :class:`UploadedFile`, it has one additional
+ method:
+
+.. method:: TemporaryUploadedFile.temporary_file_path()
+
+ Returns the full path to the temporary uploaded file.
+
+.. class:: InMemoryUploadedFile
+
+ A file uploaded into memory (i.e. stream-to-memory). This class is used
+ by the :class:`~django.core.files.uploadhandler.MemoryFileUploadHandler`.
+
+Built-in upload handers
+=======================
+
+.. module:: django.core.files.uploadhandler
+ :synopsis: Django's handlers for file uploads.
+
+Together the :class:`MemoryFileUploadHandler` and
+:class:`TemporaryFileUploadHandler` provide Django's default file upload
+behavior of reading small files into memory and large ones onto disk. They
+are located in ``django.core.files.uploadhandler``.
+
+.. class:: MemoryFileUploadHandler
+
+File upload handler to stream uploads into memory (used for small files).
+
+.. class:: TemporaryFileUploadHandler
+
+Upload handler that streams data into a temporary file using
+:class:`~django.core.files.uploadedfile.TemporaryUploadedFile`.
+
+.. _custom_upload_handlers:
+
+Writing custom upload handlers
+==============================
+
+.. class:: FileUploadHandler
+
+All file upload handlers should be subclasses of
+``django.core.files.uploadhandler.FileUploadHandler``. You can define upload
+handlers wherever you wish.
+
+Required methods
+~~~~~~~~~~~~~~~~
+
+Custom file upload handlers **must** define the following methods:
+
+.. method:: FileUploadHandler.receive_data_chunk(raw_data, start)
+
+ Receives a "chunk" of data from the file upload.
+
+ ``raw_data`` is a byte string containing the uploaded data.
+
+ ``start`` is the position in the file where this ``raw_data`` chunk
+ begins.
+
+ The data you return will get fed into the subsequent upload handlers'
+ ``receive_data_chunk`` methods. In this way, one handler can be a
+ "filter" for other handlers.
+
+ Return ``None`` from ``receive_data_chunk`` to short-circuit remaining
+ upload handlers from getting this chunk. This is useful if you're
+ storing the uploaded data yourself and don't want future handlers to
+ store a copy of the data.
+
+ If you raise a ``StopUpload`` or a ``SkipFile`` exception, the upload
+ will abort or the file will be completely skipped.
+
+.. method:: FileUploadHandler.file_complete(file_size)
+
+ Called when a file has finished uploading.
+
+ The handler should return an ``UploadedFile`` object that will be stored
+ in ``request.FILES``. Handlers may also return ``None`` to indicate that
+ the ``UploadedFile`` object should come from subsequent upload handlers.
+
+Optional methods
+~~~~~~~~~~~~~~~~
+
+Custom upload handlers may also define any of the following optional methods or
+attributes:
+
+.. attribute:: FileUploadHandler.chunk_size
+
+ Size, in bytes, of the "chunks" Django should store into memory and feed
+ into the handler. That is, this attribute controls the size of chunks
+ fed into ``FileUploadHandler.receive_data_chunk``.
+
+ For maximum performance the chunk sizes should be divisible by ``4`` and
+ should not exceed 2 GB (2\ :sup:`31` bytes) in size. When there are
+ multiple chunk sizes provided by multiple handlers, Django will use the
+ smallest chunk size defined by any handler.
+
+ The default is 64*2\ :sup:`10` bytes, or 64 KB.
+
+.. method:: FileUploadHandler.new_file(field_name, file_name, content_type, content_length, charset, content_type_extra)
+
+ Callback signaling that a new file upload is starting. This is called
+ before any data has been fed to any upload handlers.
+
+ ``field_name`` is a string name of the file ``<input>`` field.
+
+ ``file_name`` is the unicode filename that was provided by the browser.
+
+ ``content_type`` is the MIME type provided by the browser -- E.g.
+ ``'image/jpeg'``.
+
+ ``content_length`` is the length of the image given by the browser.
+ Sometimes this won't be provided and will be ``None``.
+
+ ``charset`` is the character set (i.e. ``utf8``) given by the browser.
+ Like ``content_length``, this sometimes won't be provided.
+
+ ``content_type_extra`` is extra information about the file from the
+ ``content-type`` header. See :attr:`UploadedFile.content_type_extra
+ <django.core.files.uploadedfile.UploadedFile.content_type_extra>`.
+
+ This method may raise a ``StopFutureHandlers`` exception to prevent
+ future handlers from handling this file.
+
+ .. versionadded:: 1.7
+
+ The ``content_type_extra`` parameter was added.
+
+.. method:: FileUploadHandler.upload_complete()
+
+ Callback signaling that the entire upload (all files) has completed.
+
+.. method:: FileUploadHandler.handle_raw_input(input_data, META, content_length, boundary, encoding)
+
+ Allows the handler to completely override the parsing of the raw
+ HTTP input.
+
+ ``input_data`` is a file-like object that supports ``read()``-ing.
+
+ ``META`` is the same object as ``request.META``.
+
+ ``content_length`` is the length of the data in ``input_data``. Don't
+ read more than ``content_length`` bytes from ``input_data``.
+
+ ``boundary`` is the MIME boundary for this request.
+
+ ``encoding`` is the encoding of the request.
+
+ Return ``None`` if you want upload handling to continue, or a tuple of
+ ``(POST, FILES)`` if you want to return the new data structures suitable
+ for the request directly.
View
32 docs/ref/request-response.txt
@@ -132,7 +132,7 @@ All attributes should be considered read-only, unless stated otherwise below.
A dictionary-like object containing all uploaded files. Each key in
``FILES`` is the ``name`` from the ``<input type="file" name="" />``. Each
- value in ``FILES`` is an :class:`UploadedFile` as described below.
+ value in ``FILES`` is an :class:`~django.core.files.uploadedfile.UploadedFile`.
See :doc:`/topics/files` for more information.
@@ -334,36 +334,6 @@ Methods
process(element)
-UploadedFile objects
-====================
-
-.. class:: UploadedFile
-
-
-Attributes
-----------
-
-.. attribute:: UploadedFile.name
-
- The name of the uploaded file.
-
-.. attribute:: UploadedFile.size
-
- The size, in bytes, of the uploaded file.
-
-Methods
-----------
-
-.. method:: UploadedFile.chunks(chunk_size=None)
-
- Returns a generator that yields sequential chunks of data.
-
-.. method:: UploadedFile.read(num_bytes=None)
-
- Read a number of bytes from the file.
-
-
-
QueryDict objects
=================
View
349 docs/topics/http/file-uploads.txt
@@ -64,57 +64,88 @@ something like::
Notice that we have to pass :attr:`request.FILES <django.http.HttpRequest.FILES>`
into the form's constructor; this is how file data gets bound into a form.
-Handling uploaded files
------------------------
+Here's a common way you might handle an uploaded file::
-.. class:: UploadedFile
+ def handle_uploaded_file(f):
+ with open('some/file/name.txt', 'wb+') as destination:
+ for chunk in f.chunks():
+ destination.write(chunk)
- The final piece of the puzzle is handling the actual file data from
- :attr:`request.FILES <django.http.HttpRequest.FILES>`. Each entry in this
- dictionary is an ``UploadedFile`` object -- a simple wrapper around an uploaded
- file. You'll usually use one of these methods to access the uploaded content:
+Looping over ``UploadedFile.chunks()`` instead of using ``read()`` ensures that
+large files don't overwhelm your system's memory.
- .. method:: read()
+There are a few other methods and attributes available on ``UploadedFile``
+objects; see :class:`UploadedFile` for a complete reference.
- Read the entire uploaded data from the file. Be careful with this
- method: if the uploaded file is huge it can overwhelm your system if you
- try to read it into memory. You'll probably want to use ``chunks()``
- instead; see below.
+Handling uploaded files with a model
+------------------------------------
- .. method:: multiple_chunks()
+If you're saving a file on a :class:`~django.db.models.Model` with a
+:class:`~django.db.models.FileField`, using a :class:`~django.forms.ModelForm`
+makes this process much easier. The file object will be saved to the location
+specified by the :attr:`~django.db.models.FileField.upload_to` argument of the
+corresponding :class:`~django.db.models.FileField` when calling
+``form.save()``::
- Returns ``True`` if the uploaded file is big enough to require
- reading in multiple chunks. By default this will be any file
- larger than 2.5 megabytes, but that's configurable; see below.
+ from django.http import HttpResponseRedirect
+ from django.shortcuts import render
+ from .forms import ModelFormWithFileField
- .. method:: chunks()
+ def upload_file(request):
+ if request.method == 'POST':
+ form = ModelFormWithFileField(request.POST, request.FILES)
+ if form.is_valid():
+ # file is saved
+ form.save()
+ return HttpResponseRedirect('/success/url/')
+ else:
+ form = ModelFormWithFileField()
+ return render(request, 'upload.html', {'form': form})
- A generator returning chunks of the file. If ``multiple_chunks()`` is
- ``True``, you should use this method in a loop instead of ``read()``.
+If you are constructing an object manually, you can simply assign the file
+object from :attr:`request.FILES <django.http.HttpRequest.FILES>` to the file
+field in the model::
- In practice, it's often easiest simply to use ``chunks()`` all the time;
- see the example below.
+ from django.http import HttpResponseRedirect
+ from django.shortcuts import render
+ from .forms import UploadFileForm
+ from .models import ModelWithFileField
- .. attribute:: name
+ def upload_file(request):
+ if request.method == 'POST':
+ form = UploadFileForm(request.POST, request.FILES)
+ if form.is_valid():
+ instance = ModelWithFileField(file_field=request.FILES['file'])
+ instance.save()
+ return HttpResponseRedirect('/success/url/')
+ else:
+ form = UploadFileForm()
+ return render(request, 'upload.html', {'form': form})
- The name of the uploaded file (e.g. ``my_file.txt``).
+Upload Handlers
+===============
- .. attribute:: size
+.. currentmodule:: django.core.files.uploadhandler
- The size, in bytes, of the uploaded file.
+When a user uploads a file, Django passes off the file data to an *upload
+handler* -- a small class that handles file data as it gets uploaded. Upload
+handlers are initially defined in the :setting:`FILE_UPLOAD_HANDLERS` setting,
+which defaults to::
-There are a few other methods and attributes available on ``UploadedFile``
-objects; see `UploadedFile objects`_ for a complete reference.
+ ("django.core.files.uploadhandler.MemoryFileUploadHandler",
+ "django.core.files.uploadhandler.TemporaryFileUploadHandler",)
-Putting it all together, here's a common way you might handle an uploaded file::
+Together :class:`MemoryFileUploadHandler` and
+:class:`TemporaryFileUploadHandler` provide Django's default file upload
+behavior of reading small files into memory and large ones onto disk.
- def handle_uploaded_file(f):
- with open('some/file/name.txt', 'wb+') as destination:
- for chunk in f.chunks():
- destination.write(chunk)
+You can write custom handlers that customize how Django handles files. You
+could, for example, use custom handlers to enforce user-level quotas, compress
+data on the fly, render progress bars, and even send data to another storage
+location directly without storing it locally. See :ref:`custom_upload_handlers`
+for details on how you can customize or completely replace upload behavior.
-Looping over ``UploadedFile.chunks()`` instead of using ``read()`` ensures that
-large files don't overwhelm your system's memory.
+.. _modifying_upload_handlers_on_the_fly:
Where uploaded data is stored
-----------------------------
@@ -132,8 +163,7 @@ like ``/tmp/tmpzfp6I6.upload``. If an upload is large enough, you can watch this
file grow in size as Django streams the data onto disk.
These specifics -- 2.5 megabytes; ``/tmp``; etc. -- are simply "reasonable
-defaults". Read on for details on how you can customize or completely replace
-upload behavior.
+defaults" which can be customized as described in the next section.
Changing upload handler behavior
--------------------------------
@@ -184,134 +214,7 @@ There are a few settings which control Django's file upload behavior:
:setting:`FILE_UPLOAD_HANDLERS`
The actual handlers for uploaded files. Changing this setting allows
complete customization -- even replacement -- of Django's upload
- process. See `upload handlers`_, below, for details.
-
- Defaults to::
-
- ("django.core.files.uploadhandler.MemoryFileUploadHandler",
- "django.core.files.uploadhandler.TemporaryFileUploadHandler",)
-
- Which means "try to upload to memory first, then fall back to temporary
- files."
-
-Handling uploaded files with a model
-------------------------------------
-
-If you're saving a file on a :class:`~django.db.models.Model` with a
-:class:`~django.db.models.FileField`, using a :class:`~django.forms.ModelForm`
-makes this process much easier. The file object will be saved to the location
-specified by the :attr:`~django.db.models.FileField.upload_to` argument of the
-corresponding :class:`~django.db.models.FileField` when calling
-``form.save()``::
-
- from django.http import HttpResponseRedirect
- from django.shortcuts import render
- from .forms import ModelFormWithFileField
-
- def upload_file(request):
- if request.method == 'POST':
- form = ModelFormWithFileField(request.POST, request.FILES)
- if form.is_valid():
- # file is saved
- form.save()
- return HttpResponseRedirect('/success/url/')
- else:
- form = ModelFormWithFileField()
- return render(request, 'upload.html', {'form': form})
-
-If you are constructing an object manually, you can simply assign the file
-object from :attr:`request.FILES <django.http.HttpRequest.FILES>` to the file
-field in the model::
-
- from django.http import HttpResponseRedirect
- from django.shortcuts import render
- from .forms import UploadFileForm
- from .models import ModelWithFileField
-
- def upload_file(request):
- if request.method == 'POST':
- form = UploadFileForm(request.POST, request.FILES)
- if form.is_valid():
- instance = ModelWithFileField(file_field=request.FILES['file'])
- instance.save()
- return HttpResponseRedirect('/success/url/')
- else:
- form = UploadFileForm()
- return render(request, 'upload.html', {'form': form})
-
-
-``UploadedFile`` objects
-========================
-
-In addition to those inherited from :class:`~django.core.files.File`, all
-``UploadedFile`` objects define the following methods/attributes:
-
-.. attribute:: UploadedFile.content_type
-
- The content-type header uploaded with the file (e.g. :mimetype:`text/plain`
- or :mimetype:`application/pdf`). Like any data supplied by the user, you
- shouldn't trust that the uploaded file is actually this type. You'll still
- need to validate that the file contains the content that the content-type
- header claims -- "trust but verify."
-
-.. attribute:: UploadedFile.content_type_extra
-
- .. versionadded:: 1.7
-
- A dictionary containing extra parameters passed to the ``content-type``
- header. This is typically provided by services, such as Google App Engine,
- that intercept and handle file uploads on your behalf. As a result your
- handler may not receive the uploaded file content, but instead a URL or
- other pointer to the file. (see `RFC 2388`_ section 5.3).
-
- .. _RFC 2388: http://www.ietf.org/rfc/rfc2388.txt
-
-.. attribute:: UploadedFile.charset
-
- For :mimetype:`text/*` content-types, the character set (i.e. ``utf8``)
- supplied by the browser. Again, "trust but verify" is the best policy here.
-
-.. attribute:: UploadedFile.temporary_file_path()
-
- Only files uploaded onto disk will have this method; it returns the full
- path to the temporary uploaded file.
-
-.. note::
-
- Like regular Python files, you can read the file line-by-line simply by
- iterating over the uploaded file:
-
- .. code-block:: python
-
- for line in uploadedfile:
- do_something_with(line)
-
- However, *unlike* standard Python files, :class:`UploadedFile` only
- understands ``\n`` (also known as "Unix-style") line endings. If you know
- that you need to handle uploaded files with different line endings, you'll
- need to do so in your view.
-
-Upload Handlers
-===============
-
-When a user uploads a file, Django passes off the file data to an *upload
-handler* -- a small class that handles file data as it gets uploaded. Upload
-handlers are initially defined in the :setting:`FILE_UPLOAD_HANDLERS` setting,
-which defaults to::
-
- ("django.core.files.uploadhandler.MemoryFileUploadHandler",
- "django.core.files.uploadhandler.TemporaryFileUploadHandler",)
-
-Together the ``MemoryFileUploadHandler`` and ``TemporaryFileUploadHandler``
-provide Django's default file upload behavior of reading small files into memory
-and large ones onto disk.
-
-You can write custom handlers that customize how Django handles files. You
-could, for example, use custom handlers to enforce user-level quotas, compress
-data on the fly, render progress bars, and even send data to another storage
-location directly without storing it locally.
-
-.. _modifying_upload_handlers_on_the_fly:
+ process.
Modifying upload handlers on the fly
------------------------------------
@@ -371,121 +274,3 @@ list::
@csrf_protect
def _upload_file_view(request):
... # Process request
-
-
-Writing custom upload handlers
-------------------------------
-
-.. currentmodule:: django.core.files.uploadhandler
-
-.. class:: FileUploadHandler
-
-All file upload handlers should be subclasses of
-``django.core.files.uploadhandler.FileUploadHandler``. You can define upload
-handlers wherever you wish.
-
-Required methods
-~~~~~~~~~~~~~~~~
-
-Custom file upload handlers **must** define the following methods:
-
-.. method:: FileUploadHandler.receive_data_chunk(raw_data, start)
-
- Receives a "chunk" of data from the file upload.
-
- ``raw_data`` is a byte string containing the uploaded data.
-
- ``start`` is the position in the file where this ``raw_data`` chunk
- begins.
-
- The data you return will get fed into the subsequent upload handlers'
- ``receive_data_chunk`` methods. In this way, one handler can be a
- "filter" for other handlers.
-
- Return ``None`` from ``receive_data_chunk`` to short-circuit remaining
- upload handlers from getting this chunk. This is useful if you're
- storing the uploaded data yourself and don't want future handlers to
- store a copy of the data.
-
- If you raise a ``StopUpload`` or a ``SkipFile`` exception, the upload
- will abort or the file will be completely skipped.
-
-.. method:: FileUploadHandler.file_complete(file_size)
-
- Called when a file has finished uploading.
-
- The handler should return an ``UploadedFile`` object that will be stored
- in ``request.FILES``. Handlers may also return ``None`` to indicate that
- the ``UploadedFile`` object should come from subsequent upload handlers.
-
-Optional methods
-~~~~~~~~~~~~~~~~
-
-Custom upload handlers may also define any of the following optional methods or
-attributes:
-
-.. attribute:: FileUploadHandler.chunk_size
-
- Size, in bytes, of the "chunks" Django should store into memory and feed
- into the handler. That is, this attribute controls the size of chunks
- fed into ``FileUploadHandler.receive_data_chunk``.
-
- For maximum performance the chunk sizes should be divisible by ``4`` and
- should not exceed 2 GB (2\ :sup:`31` bytes) in size. When there are
- multiple chunk sizes provided by multiple handlers, Django will use the
- smallest chunk size defined by any handler.
-
- The default is 64*2\ :sup:`10` bytes, or 64 KB.
-
-.. method:: FileUploadHandler.new_file(field_name, file_name, content_type, content_length, charset, content_type_extra)
-
- Callback signaling that a new file upload is starting. This is called
- before any data has been fed to any upload handlers.
-
- ``field_name`` is a string name of the file ``<input>`` field.
-
- ``file_name`` is the unicode filename that was provided by the browser.
-
- ``content_type`` is the MIME type provided by the browser -- E.g.
- ``'image/jpeg'``.
-
- ``content_length`` is the length of the image given by the browser.
- Sometimes this won't be provided and will be ``None``.
-
- ``charset`` is the character set (i.e. ``utf8``) given by the browser.
- Like ``content_length``, this sometimes won't be provided.
-
- ``content_type_extra`` is extra information about the file from the
- ``content-type`` header. See :attr:`UploadedFile.content_type_extra
- <django.core.files.uploadedfile.UploadedFile.content_type_extra>`.
-
- This method may raise a ``StopFutureHandlers`` exception to prevent
- future handlers from handling this file.
-
- .. versionadded:: 1.7
-
- The ``content_type_extra`` parameter was added.
-
-.. method:: FileUploadHandler.upload_complete()
-
- Callback signaling that the entire upload (all files) has completed.
-
-.. method:: FileUploadHandler.handle_raw_input(input_data, META, content_length, boundary, encoding)
-
- Allows the handler to completely override the parsing of the raw
- HTTP input.
-
- ``input_data`` is a file-like object that supports ``read()``-ing.
-
- ``META`` is the same object as ``request.META``.
-
- ``content_length`` is the length of the data in ``input_data``. Don't
- read more than ``content_length`` bytes from ``input_data``.
-
- ``boundary`` is the MIME boundary for this request.
-
- ``encoding`` is the encoding of the request.
-
- Return ``None`` if you want upload handling to continue, or a tuple of
- ``(POST, FILES)`` if you want to return the new data structures suitable
- for the request directly.
Please sign in to comment.
Something went wrong with that request. Please try again.