[master] Fixed #18972 -- Refactored chunking algorithm stealing shameles... #921

Closed
wants to merge 3 commits into
from

Conversation

Projects
None yet
3 participants
Contributor

woodm1979 commented Mar 18, 2013

...sly from datagroks code.

@charettes charettes referenced this pull request Mar 18, 2013

Closed

Fixed bug #18972 #782

django/core/servers/basehttp.py
@@ -18,6 +18,7 @@
except ImportError: # Python 2
from urlparse import urljoin
from django.utils.six.moves import socketserver
+from io import BytesIO
@charettes

charettes Mar 18, 2013

Member

Move that import before os.

django/core/servers/basehttp.py
@@ -27,6 +28,11 @@
__all__ = ['WSGIServer', 'WSGIRequestHandler']
+# If data is too large, socket will choke, so write chunks no larger than 32MB
+# at a time.
+# FIXME: Document rationale for 32MB and not some smaller or larger value.
@charettes

charettes Mar 18, 2013

Member

This should be fixed.

@woodm1979

woodm1979 Mar 18, 2013

Contributor

I'm not remotely qualified to define the rationale here. But it was provided by the original author of this code.

tests/builtin_server/tests.py
+def send_big_data_app(environ, start_response):
+ start_response(str('200 OK'), [(str('Content-Type'), str('text/plain'))])
+ # Return a blob of data that slightly exceeds chunk size.
+ return [bytes('x' * (32 * 1024 * 1024 + 5))]
@charettes

charettes Mar 18, 2013

Member

Use MAX_SOCKET_CHUNK_SIZE here.

tests/builtin_server/tests.py
+#
+# Tests for #18972: The logic that performs the math to break data into 32MB
+# chunks was flawed, BUT it didn't actually cause any problems.
+#
@charettes

charettes Mar 18, 2013

Member

Move those comments to the ServerHandlerChunksProperly's doc.

django/core/servers/basehttp.py
+
+# If data is too large, socket will choke, so write chunks no larger than 32MB
+# at a time.
+# FIXME: Document rationale for 32MB and not some smaller or larger value.
@carljm

carljm Mar 19, 2013

Owner

I think we should replace this FIXME with the rationale (fuzzy though it is) from the original ticket that added the 32MB chunk size: https://code.djangoproject.com/ticket/5596#comment:4

Member

charettes commented Mar 20, 2013

Moved all your work with attribution on Trac, will wait for @carljm RFC go there and commit.

@charettes charettes closed this Mar 20, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment