Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serving static files with Daphne #87

Closed
orokusaki opened this issue Mar 12, 2016 · 28 comments
Closed

Serving static files with Daphne #87

orokusaki opened this issue Mar 12, 2016 · 28 comments

Comments

@orokusaki
Copy link
Contributor

I noticed during my quick testing of daphne / runworker that there is no automatic support for static files. I know that, in the ideal world, static files are served from a CDN, but over the past few years it's become very common practice to serve static files from your application's front-end server (e.g., using WhiteNoise - also see their IAQ).

With that said, is there a way to serve static in production with Daphne? Or, is that something you'd be interested in merging, if I was to find a reasonable way to add WhiteNoise-like functionality in a fork?

@andrewgodwin
Copy link
Member

Serving large files over Channels is possible but not quite as good as doing it from a local filesystem - remember, every response is serialised onto the wire and back - but Channels does in fact ship with support for staticfiles, which it turns on for Daphne when you're using runserver.

I'm not averse to something like WhiteNoise, especially as I like the "you should be caching anyway" mantra, and you can implement it similarly with a wrapper around the Django http.request ASGI consumer (the equivalent of wrapping the WSGI application). I would definitely be interested in merging this or contributing support out to another project to give them ASGI support.

@jpadilla
Copy link

Here's one way to do so for anyone interested: https://github.com/jacobian/channels-example/blob/master/chat/routing.py#L5-L8

@jacobian
Copy link
Member

@andrewgodwin it seems like perhaps Daphne should be the bit that can serve static files, thus preventing the need to shuttle the file over the wire. Does that sound right? Or is there a reason to implement static file serving as a channel consumer?

@andrewgodwin
Copy link
Member

No, Daphne isn't necessarily co-located with the codebase; static files should either be served in front of it (by e.g. nginx) or behind it (via Django), IMO. Daphne's job is purely shutting data to and from HTTP/Channels, and I want to try and avoid adding any complexity if we can help it.

I did improve the core Handler a couple of days ago so any FileResponse is chunked up much better for transfer, which means the new WhiteNoise Django Middleware version should work well with that.

@exp0nge
Copy link

exp0nge commented Apr 27, 2016

@andrewgodwin How are you setting up WhiteNoise to work with Channels in a production environment (without runserver)?

@andrewgodwin
Copy link
Member

I am not yet running it, I need to get around to that.

@exp0nge
Copy link

exp0nge commented Apr 27, 2016

I mean I tried WhiteNoise with Heroku, and JS/CSS files loaded fine but 404'd on images.

@bastianh
Copy link
Contributor

bastianh commented Jun 13, 2016

I was testing WhiteNoise with dokku (http://dokku.viewdocs.io/dokku/) this weekend and it only worked for every second request for the same file.
With the first request it send the file, but when I reload in chrome or firefox and the browser asked for checking with the if-modified header the result was an empty 200 response instead of a not modified.
When trying with curl there were no errors.

@andrewgodwin
Copy link
Member

Weird, probably an incompatability in WhiteNoise I should look into. What version were you using?

@bastianh
Copy link
Contributor

I was using the current versions from pypi

whitenoise==3.2
asgi-redis==0.13.1
asgiref==0.13.3           # via asgi-redis, channels, daphne
daphne==0.12.1            # via channels
channels==0.14.1

might also have to do with the nginx setup? I will try to find out more later but would be nice if you have any ideas.

you can try by accessing this file: https://ptest.lambda.leebe.de/static/CACHE/css/cc0e6539359b.css

@helderco
Copy link

Has anyone got whitenoise to work? Anything else to do besides the django integration in their manual?

@helderco
Copy link

I thought it wasn't working but I'm actually having the same issue as @bastianh (I'm using docker).

I'm holding out on using django channels to keep it simple, or do you have another suggestion for static files?

@exp0nge
Copy link

exp0nge commented Jun 29, 2016

I ended up using S3+CloudFront, I think that's your best bet if you want to get something working and something that scales well.

@helderco
Copy link

I use S3 on the media files, but I prefer having my static files in the docker container, next to the code that generated them (versions).

@bastianh
Copy link
Contributor

I decided to put nginx into the container and then I decided to use the patched nginx from https://nchan.slact.net/ to directly handle websockets and channels for this project.

@andrewgodwin
Copy link
Member

Found the issue - a bug with empty streaming responses - and it's out now in 0.15.1.

@exp0nge
Copy link

exp0nge commented Jun 29, 2016

What is this fix exactly for?

@andrewgodwin
Copy link
Member

It makes whitenoise work properly with Daphne, and this is what I'm suggesting for static file serving for now on - no need to reinvent the wheel.

@slact
Copy link

slact commented Jul 7, 2016

Hello there, I'm the author of Nchan. I'm just curious how you're using it -- looks like you're serving large-ish files over WS -- and if you're also using the javascript client or rolling your own.

@johnnyflute
Copy link

so the 'python manage.py runsever' can be used to handle static file or not

@exp0nge
Copy link

exp0nge commented Apr 20, 2017

@johnnyflute It's been a while since I used this but maybe Andrew's answer can help:

It makes whitenoise work properly with Daphne, and this is what I'm suggesting for static file serving for now on - no need to reinvent the wheel.

So you need to do some extra leg work.

@andrewgodwin
Copy link
Member

Runserver works properly with static files and has basically since launch. Daphne by itself does not.

@puterleat
Copy link

puterleat commented May 23, 2017

Joining this late, but the problem with Whitenoise is that it won't work for client uploaded media. Taking the case where these are stored locally (or on a mounted volume in dokku) dj_static works quite nicely with standard django (no channels) by wrapping the WSGI application. Could this also work for asgi/daphne?

@andrewgodwin
Copy link
Member

If it's a WSGI wrapper/middleware it won't work. You'll need to either serve those files using a file-serving webserver (nginx, etc), or find a Django-level application that works.

@benwhalley
Copy link

Sorry - I wasn't being clear. I realise it won't work out of the box as a wsgi wrapper, but I suppose my question was, will this general approach still be feasible with Daphne.

My use case is a smallish app where I don't want to use S3 as the storage. I have a lot of user-uploaded assets which are used for processing. For staticfiles I use cloudfront as a CDN, so load on the server isn't an issue. For the mediafiles there's not much opportunity for caching anyway. I've previously used dj_static for both staticfiles and mediafiles which has worked well, but I now want to add channels to my app.

For staticfiles this is OK because, as you note, whitenoise now works with Daphne, so I can switch to that. However for mediafiles this won't work because it builds a manifest of all images at startuptime, not on the fly. I suppose what I'd really like is for something like dj_static to be built into Daphne. I would really rather not have to reconfigure nginx to serve the media files.

@andrewgodwin
Copy link
Member

I'm pretty resistant to building static file serving into Daphne, mainly as I want to try and keep it as simple and maintainable as possible; doing static file serving right requires stream options, a mimetype library, and more, and that's out of scope with the maintenance team we have.

I think what's needed is something on the Django end that correctly serves user media if required, that plugs in well. I'm sorry nothing perfectly fills this gap right now, but most people I know already host static files on a subdomain or CDN URL and so they already have a separate server configured to handle them.

@TheBlusky
Copy link

I had a similar issue, WhiteNoise couldn't work with daphne as it was configure as a WSGI wrapper (with application = DjangoWhiteNoise(application)).

However, you can configure WhiteNoise as a django middleware, adding :

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',
    ....
]

to your settings.py file.

Worked for me with daphne.

@qinyanjuidavid
Copy link

@andrewgodwin I have been trying to setup subdomain on the django channels but no call is been made, ws://xyz.127.0.0.1:8000/ws/chat/2ff6539c-f8b1-4004-aa20-14ad6ba9f2ed/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests