Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AsgiWhiteNoise and async WhiteNoiseMiddleware #359

Open
wants to merge 131 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 128 commits
Commits
Show all changes
131 commits
Select commit Hold shift + click to select a range
9a56981
Add ASGI adapter
kmichel Sep 14, 2020
8590e82
add venv to gitignore
Archmonger Feb 7, 2022
24c5319
Merge remote-tracking branch 'upstream/master' into asgi-compat
Archmonger Feb 7, 2022
391bd8e
add aiofile
Archmonger Feb 7, 2022
3e735ec
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 7, 2022
42fbe15
Merge branch 'asgi-compat' of https://github.com/Archmonger/whitenois…
Archmonger Feb 7, 2022
aec9ac1
Revert "add venv to gitignore"
Archmonger Feb 8, 2022
cf4ee09
add ASGI extra
Archmonger Feb 10, 2022
9616e78
absolute imports
Archmonger Feb 10, 2022
35b9003
use SimpleNamespace
Archmonger Feb 10, 2022
8eef8d3
add zero-copy send todo
Archmonger Feb 10, 2022
2f0f685
replace empty equality with boolean operation
Archmonger Feb 10, 2022
f4931e3
minor syntax or performance improvements
Archmonger Feb 10, 2022
3fb5a38
BaseWhiteNoise class
Archmonger Feb 10, 2022
3bb4088
Merge remote-tracking branch 'upstream/master' into asgi-compat
Archmonger Feb 10, 2022
8a0930e
use f-strings
Archmonger Feb 10, 2022
e6fd03b
fix imports
Archmonger Feb 10, 2022
8fc1682
Merge remote-tracking branch 'upstream/master' into asgi-compat
Archmonger Feb 10, 2022
64fc5c7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 10, 2022
83fea65
Revert "minor syntax or performance improvements"
Archmonger Feb 10, 2022
58f3743
Revert "use f-strings"
Archmonger Feb 10, 2022
f79aec7
re-add missing import
Archmonger Feb 10, 2022
e18df38
Remove upper bound
Archmonger Feb 10, 2022
45301b2
fix recceive call
Archmonger Feb 10, 2022
88ccf21
fix some of the tests
Archmonger Feb 11, 2022
998cf5e
temporarily remove broken tests
Archmonger Feb 11, 2022
d611e90
fix py3.10 deprecation warning
Archmonger Feb 11, 2022
54c7148
convert wsgi to asgi
Archmonger Feb 12, 2022
1057a77
remove contrived tests
Archmonger Feb 12, 2022
ea2e84c
AsgiWhiteNoise -> AsyncWhiteNoise
Archmonger Feb 12, 2022
6dcecc0
Revert "convert wsgi to asgi"
Archmonger Feb 12, 2022
c506e96
ASGI v3 static file server
Archmonger Jun 21, 2023
9892bea
Merge remote-tracking branch 'upstream/main' into asgi-compat
Archmonger Jun 21, 2023
e636411
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 21, 2023
41468a1
guarantee_single_callable
Archmonger Jun 22, 2023
08c70fb
customizable block size
Archmonger Jun 22, 2023
240ce22
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 22, 2023
be3796c
clean up comment
Archmonger Jun 22, 2023
b81ab6b
Merge branch 'asgi-compat' of https://github.com/Archmonger/whitenois…
Archmonger Jun 22, 2023
7aeb29a
Add functional middleware
Archmonger Jun 22, 2023
cd0ab1b
add aiofiles to all tests
Archmonger Jun 22, 2023
1265f03
temporarily add aiofiles to tox.ini
Archmonger Jun 22, 2023
f5e9d37
temporarily disable asgi tests
Archmonger Jun 22, 2023
0e42cb8
async -> asgi
Archmonger Jun 22, 2023
279b342
aiofile != aiofiles
Archmonger Jul 20, 2023
890fdd8
fix comment
Archmonger Jul 20, 2023
ba3bed4
properly close Django file responses
Archmonger Jul 21, 2023
7af5b32
async find_file within AsgiWhiteNoise
Archmonger Jul 21, 2023
ee068a7
AsyncSlicedFile
Archmonger Jul 21, 2023
f09b384
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 21, 2023
ca26354
simplify __call__ method
Archmonger Jul 21, 2023
c2c320c
Add WSGI compat docstring
Archmonger Jul 21, 2023
d0e9bfa
Merge branch 'asgi-compat' of https://github.com/Archmonger/whitenois…
Archmonger Jul 21, 2023
33ad918
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 21, 2023
067ab33
aget_response update comment
Archmonger Jul 21, 2023
a15422f
add prototypes and warnings for call/serve
Archmonger Jul 21, 2023
8d01092
update middleware comment
Archmonger Jul 21, 2023
d6bb4d8
fix compatibilty with SecurityMiddleware
Archmonger Jul 21, 2023
2a386cc
fix name
Archmonger Jul 21, 2023
06a874f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 21, 2023
1f9f8b3
refactor WhiteNoiseFileResponse
Archmonger Jul 21, 2023
64c1c27
reduce code duplication for AsyncSlicedFile
Archmonger Jul 21, 2023
f3b00ab
Merge branch 'asgi-compat' of https://github.com/Archmonger/whitenois…
Archmonger Jul 21, 2023
68f19c2
update WhiteNoiseFileResponse docstring
Archmonger Jul 21, 2023
00f064d
customizable block_size
Archmonger Jul 21, 2023
e5f7526
docstring update
Archmonger Jul 21, 2023
9ee0d7d
rename aiofile param
Archmonger Jul 21, 2023
d97a05b
fix file seeking
Archmonger Jul 23, 2023
2dbc8b2
add async pytest for django
Archmonger Jul 23, 2023
de3456e
merge imports
Archmonger Jul 23, 2023
f0f0a66
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 23, 2023
511da12
comment out everything from test_asgi
Archmonger Jul 23, 2023
7fd30ef
handle file closure within iterator
Archmonger Jul 24, 2023
62de430
remove pytest asyncio
Archmonger Jul 24, 2023
bb62496
async file closure
Archmonger Jul 24, 2023
d839cf8
Py 3.8 compatibility
Archmonger Jul 24, 2023
b40ff65
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 24, 2023
522a50b
remove unused import
Archmonger Jul 24, 2023
518a46b
start from scratch for ASGI tests
Archmonger Jul 24, 2023
ed07496
Add async iterator support to old django versions
Archmonger Jul 24, 2023
3bff168
Merge branch 'asgi-compat' of https://github.com/Archmonger/whitenois…
Archmonger Jul 24, 2023
4c98b5e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 24, 2023
4656a53
new base for AsyncSlicedFile
Archmonger Jul 24, 2023
4a77e8b
Py 3.12+ compatibility
Archmonger Jul 24, 2023
dd2d700
Merge branch 'asgi-compat' of https://github.com/Archmonger/whitenois…
Archmonger Jul 24, 2023
ea74977
refactor file closure and docstrings
Archmonger Jul 25, 2023
2bc2a7b
way more performant way of retaining WSGI compatibility
Archmonger Jul 26, 2023
d648ffc
move file closure to AsgiFileServer loop
Archmonger Jul 26, 2023
f6bf0c2
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 26, 2023
367a17d
Support for ASGI on Django<4.2
Archmonger Jul 26, 2023
e3bea39
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 26, 2023
b895b8f
resolve flake8 lint warning
Archmonger Jul 26, 2023
7132626
Remove `USE_ASYNC`. Always use the best available method.
Archmonger Jul 27, 2023
56fe9d1
remove type hints that break visual similarity between sync/async code
Archmonger Jul 27, 2023
17b9dc7
reduce tab depth
Archmonger Jul 27, 2023
2dc620e
add first batch of ASGI tests
Archmonger Jul 27, 2023
6d4baf3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 27, 2023
f8fb573
complete tests for asgi.py
Archmonger Jul 28, 2023
b91dd7e
get_response is always mandatory
Archmonger Jul 28, 2023
5eaac51
Merge branch 'asgi-compat' of https://github.com/Archmonger/whitenois…
Archmonger Jul 28, 2023
292de02
More tests
Archmonger Jul 28, 2023
3117769
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 28, 2023
5de2d63
minor refactoring
Archmonger Jul 28, 2023
6197911
Merge branch 'asgi-compat' of https://github.com/Archmonger/whitenois…
Archmonger Jul 28, 2023
37dfac6
Make aiofiles mandatory because of Django
Archmonger Jul 28, 2023
f20e90d
format setup.cfg
Archmonger Jul 28, 2023
865a68d
WHITENOISE_BLOCK_SIZE docs
Archmonger Jul 28, 2023
d8f564e
rename base to wsgi
Archmonger Jul 28, 2023
4c2ca1d
QuickStart for other ASGI apps
Archmonger Jul 28, 2023
b238278
AsgiWhiteNoise docs
Archmonger Jul 28, 2023
fb118ad
Merge remote-tracking branch 'upstream/main' into asgi-compat
Archmonger Jul 28, 2023
8e8c63b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 28, 2023
bd580a2
minor comment cleanup
Archmonger Jul 28, 2023
371ca90
minor docs wordsmithing
Archmonger Jul 28, 2023
6695140
Remove block size configuration attributes
Archmonger Jul 28, 2023
018f864
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 28, 2023
3a0180a
use dict comphrension for header conversion
Archmonger Jul 28, 2023
29c678c
headers -> wsgi_headers
Archmonger Jul 28, 2023
6da4dfd
serve doesn't need stubs
Archmonger Jul 28, 2023
b1c8ae3
Add ASGI to readme and pkg info
Archmonger Jul 29, 2023
88e8a86
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 29, 2023
1f0fa9f
More readible way of converting ASGI app
Archmonger Aug 2, 2023
5b74486
asgi_app -> user_app
Archmonger Aug 2, 2023
9f77bf7
no need for make_bytes
Archmonger Aug 4, 2023
484ddb4
make middleware sync capable
Archmonger Aug 6, 2023
07b8b3d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 6, 2023
4db8458
reduce LOC changes for serve
Archmonger Aug 6, 2023
503e957
rename to `AsyncSlicedFile`
Archmonger Aug 7, 2023
67f8dee
Update tests/test_asgi.py
Archmonger Mar 23, 2024
1f8c4b2
Update docs/asgi.rst
Archmonger Mar 23, 2024
aa36dce
Update docs/asgi.rst
Archmonger Mar 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.rst
Expand Up @@ -31,7 +31,7 @@ useful on Heroku, OpenShift and other PaaS providers.)
It's designed to work nicely with a CDN for high-traffic sites so you don't have to
sacrifice performance to benefit from simplicity.

WhiteNoise works with any WSGI-compatible app but has some special auto-configuration
WhiteNoise works with any ASGI or WSGI compatible app but has some special auto-configuration
features for Django.

WhiteNoise takes care of best-practices for you, for instance:
Expand Down
76 changes: 76 additions & 0 deletions docs/asgi.rst
@@ -0,0 +1,76 @@
Using WhiteNoise with any ASGI application
==========================================

.. note:: These instructions apply to any ASGI application. However, for Django
applications you would be better off using the :doc:`WhiteNoiseMiddleware
<django>` class which makes integration easier.

To enable WhiteNoise you need to wrap your existing ASGI application in a
WhiteNoise instance and tell it where to find your static files. For example:

.. code-block:: python

from whitenoise import AsgiWhiteNoise

from my_project import MyASGIApp

application = MyWAGIApp()
Archmonger marked this conversation as resolved.
Show resolved Hide resolved
application = AsgiWhiteNoise(application, root="/path/to/static/files")
application.add_files("/path/to/more/static/files", prefix="more-files/")

On initialization, WhiteNoise walks over all the files in the directories that have
been added (descending into sub-directories) and builds a list of available static files.
Any requests which match a static file get served by WhiteNoise, all others are passed
through to the original WSGI application.
Archmonger marked this conversation as resolved.
Show resolved Hide resolved


.. tip:: ``AsgiWhiteNoise`` inherits all interfaces from WSGI ``WhiteNoise`` but adds
support for ASGI applications. See the :doc:`WSGI WhiteNoise documentation <wsgi>` for
more details.


AsgiWhiteNoise API
------------------

``AsgiWhiteNoise`` inherits its interface from WSGI ``WhiteNoise``, however, ``application``
must be an ASGI application.

See the section on WSGI ``WhiteNoise`` :ref:`interface <interface>` for details.


Compression Support
--------------------

See the section on WSGI ``WhiteNoise`` :ref:`compression support <compression>` for details.


Caching Headers
---------------

See the section on WSGI ``WhiteNoise`` :ref:`caching headers <caching>` for details.


Index Files
-----------

See the section on WSGI ``WhiteNoise`` :ref:`index files <index_files>` for details.


Using a Content Distribution Network
------------------------------------

See the instructions for :ref:`using a CDN with Django <cdn>` . The same principles
apply here although obviously the exact method for generating the URLs for your static
files will depend on the libraries you're using.


Redirecting to HTTPS
--------------------

See the section on WSGI ``WhiteNoise`` :ref:`redirecting to HTTPS <https>` for details.


Configuration attributes
------------------------

See the section on WSGI ``WhiteNoise`` :ref:`configuration attributes <configuration>` for details.
2 changes: 1 addition & 1 deletion docs/changelog.rst
Expand Up @@ -180,7 +180,7 @@ Other changes include:
``wsgi.py``.
See the :ref:`documentation <django-middleware>` for more details.

(The :doc:`pure WSGI <base>` integration is still available for non-Django apps.)
(The :doc:`pure WSGI <wsgi>` integration is still available for non-Django apps.)

* The ``whitenoise.django.GzipManifestStaticFilesStorage`` alias has now
been removed. Instead you should use the correct import path:
Expand Down
2 changes: 1 addition & 1 deletion docs/django.rst
Expand Up @@ -2,7 +2,7 @@ Using WhiteNoise with Django
============================

.. note:: To use WhiteNoise with a non-Django application see the
:doc:`generic WSGI documentation <base>`.
:doc:`generic WSGI documentation <wsgi>` or the :doc:`generic ASGI documentation <asgi>`.

This guide walks you through setting up a Django project with WhiteNoise.
In most cases it shouldn't take more than a couple of lines of configuration.
Expand Down
29 changes: 25 additions & 4 deletions docs/index.rst
Expand Up @@ -79,8 +79,28 @@ WhiteNoise instance and tell it where to find your static files. For example:
application = WhiteNoise(application, root="/path/to/static/files")
application.add_files("/path/to/more/static/files", prefix="more-files/")

And that's it, you're ready to go. For more details see the :doc:`full
documentation <base>`.
And that's it, you're ready to go. For more details see the :doc:`full WSGI
documentation <wsgi>`.


QuickStart for other ASGI apps
------------------------------

To enable WhiteNoise you need to wrap your existing ASGI application in a
WhiteNoise instance and tell it where to find your static files. For example:

.. code-block:: python

from whitenoise import AsgiWhiteNoise

from my_project import MyASGIApp

application = MyASGIApp()
application = AsgiWhiteNoise(application, root="/path/to/static/files")
application.add_files("/path/to/more/static/files", prefix="more-files/")

And that's it, you're ready to go. For more details see the :doc:`full ASGI
documentation <asgi>`.


Using WhiteNoise with Flask
Expand All @@ -94,7 +114,7 @@ the standard WSGI protocol it is easy to integrate with WhiteNoise (see the
Compatibility
-------------

WhiteNoise works with any WSGI-compatible application and is tested on Python
WhiteNoise works with any ASGI or WSGI compatible application and is tested on Python
**3.8** – **3.12**, on both Linux and Windows.

Django WhiteNoiseMiddleware is tested with Django versions **3.2** --- **4.2**
Expand Down Expand Up @@ -202,6 +222,7 @@ MIT Licensed

self
django
base
wsgi
asgi
flask
changelog
6 changes: 5 additions & 1 deletion docs/base.rst → docs/wsgi.rst
Expand Up @@ -27,12 +27,14 @@ See the sections on :ref:`compression <compression>` and :ref:`caching <caching>
for further details.


.. _interface:

WhiteNoise API
--------------

.. class:: WhiteNoise(application, root=None, prefix=None, \**kwargs)

:param callable application: Original WSGI application
:param Callable application: Original WSGI application
:param str root: If set, passed to ``add_files`` method
:param str prefix: If set, passed to ``add_files`` method
:param \**kwargs: Sets :ref:`configuration attributes <configuration>` for this instance
Expand Down Expand Up @@ -146,6 +148,8 @@ apply here although obviously the exact method for generating the URLs for your
files will depend on the libraries you're using.


.. _https:

Redirecting to HTTPS
--------------------

Expand Down
5 changes: 4 additions & 1 deletion setup.cfg
@@ -1,7 +1,7 @@
[metadata]
name = whitenoise
version = 6.5.0
description = Radically simplified static file serving for WSGI applications
description = Radically simplified static file serving for ASGI or WSGI applications
long_description = file: README.rst
long_description_content_type = text/x-rst
url = https://github.com/evansd/whitenoise
Expand All @@ -26,6 +26,7 @@ classifiers =
Programming Language :: Python :: 3.11
Programming Language :: Python :: 3.12
Programming Language :: Python :: Implementation :: CPython
Topic :: Internet :: WWW/HTTP :: ASGI :: Middleware
Topic :: Internet :: WWW/HTTP :: WSGI :: Middleware
Typing :: Typed
keywords = Django
Expand All @@ -35,6 +36,8 @@ project_urls =

[options]
packages = find:
install_requires =
aiofiles>=22.1.0
python_requires = >=3.8
include_package_data = True
package_dir =
Expand Down
5 changes: 3 additions & 2 deletions src/whitenoise/__init__.py
@@ -1,5 +1,6 @@
from __future__ import annotations

from .base import WhiteNoise
from .asgi import AsgiWhiteNoise
from .wsgi import WhiteNoise

__all__ = ["WhiteNoise"]
__all__ = ["AsgiWhiteNoise", "WhiteNoise"]
91 changes: 91 additions & 0 deletions src/whitenoise/asgi.py
@@ -0,0 +1,91 @@
from __future__ import annotations

import asyncio

from asgiref.compatibility import guarantee_single_callable

from .string_utils import decode_path_info
from whitenoise.base import BaseWhiteNoise

# This is the same size as wsgiref.FileWrapper
BLOCK_SIZE = 8192


class AsgiWhiteNoise(BaseWhiteNoise):
user_app = None

async def __call__(self, scope, receive, send):
# Ensure ASGI v2 is converted to ASGI v3
if not self.user_app:
self.user_app = guarantee_single_callable(self.application)

# Determine if the request is for a static file
path = decode_path_info(scope["path"])
static_file = None
if scope["type"] == "http":
if self.autorefresh and hasattr(asyncio, "to_thread"):
# Use a thread while searching disk for files on Python 3.9+
static_file = await asyncio.to_thread(self.find_file, path)
elif self.autorefresh:
static_file = self.find_file(path)
else:
static_file = self.files.get(path)

# Serve static file if it exists
if static_file:
await AsgiFileServer(static_file)(scope, receive, send)
return

# Serve the user's ASGI application
await self.user_app(scope, receive, send)


class AsgiFileServer:
"""Simple ASGI application that streams a StaticFile over HTTP in chunks."""

def __init__(self, static_file):
self.static_file = static_file

async def __call__(self, scope, receive, send):
# Convert ASGI headers into WSGI headers. Allows us to reuse all of our WSGI
# header logic inside of aget_response().
wsgi_headers = {
"HTTP_" + key.decode().upper().replace("-", "_"): value.decode()
for key, value in scope["headers"]
}

# Get the WhiteNoise file response
response = await self.static_file.aget_response(scope["method"], wsgi_headers)

# Start a new HTTP response for the file
await send(
{
"type": "http.response.start",
"status": response.status,
"headers": [
# Convert headers back to ASGI spec
(key.lower().encode(), value.encode())
for key, value in response.headers
],
}
)

# Head responses have no body, so we terminate early
if response.file is None:
await send({"type": "http.response.body", "body": b""})
return

# Stream the file response body
async with response.file as async_file:
while True:
chunk = await async_file.read(BLOCK_SIZE)
more_body = bool(chunk)
await send(
{
"type": "http.response.body",
"body": chunk,
"more_body": more_body,
}
)
if not more_body:
break
27 changes: 3 additions & 24 deletions src/whitenoise/base.py
Expand Up @@ -6,18 +6,16 @@
from posixpath import normpath
from typing import Callable
from wsgiref.headers import Headers
from wsgiref.util import FileWrapper

from .media_types import MediaTypes
from .responders import IsDirectoryError
from .responders import MissingFileError
from .responders import Redirect
from .responders import StaticFile
from .string_utils import decode_path_info
from .string_utils import ensure_leading_trailing_slash


class WhiteNoise:
class BaseWhiteNoise:
# Ten years is what nginx sets a max age if you use 'expires max;'
# so we'll follow its lead
FOREVER = 10 * 365 * 24 * 60 * 60
Expand Down Expand Up @@ -71,27 +69,8 @@ def __init__(
if root is not None:
self.add_files(root, prefix)

def __call__(self, environ, start_response):
path = decode_path_info(environ.get("PATH_INFO", ""))
if self.autorefresh:
static_file = self.find_file(path)
else:
static_file = self.files.get(path)
if static_file is None:
return self.application(environ, start_response)
else:
return self.serve(static_file, environ, start_response)

@staticmethod
def serve(static_file, environ, start_response):
response = static_file.get_response(environ["REQUEST_METHOD"], environ)
status_line = f"{response.status} {response.status.phrase}"
start_response(status_line, list(response.headers))
if response.file is not None:
file_wrapper = environ.get("wsgi.file_wrapper", FileWrapper)
return file_wrapper(response.file)
else:
return []
def __call__(self, *args, **kwargs):
raise NotImplementedError("Subclasses must implement `__call__`")

def add_files(self, root, prefix=None):
root = os.path.abspath(root)
Expand Down