Slow handling of connection incoming buffer #33

Merged
merged 1 commit into from Feb 15, 2013

Conversation

Projects
None yet
3 participants
Contributor

malor commented Feb 14, 2013

A connection incoming buffer is extended using += operator. This leads
to unnecessary memory allocations and significant performance drops when
transferring big amounts of data (i. e. a few megabytes). Using of the array('c')
(an array of bytes) data type enables us to handle incoming data efficiently.

I used this dummy code for testing:
worker: http://xsnippet.org/359371/
client: http://xsnippet.org/359372/
binary data: dd if=/dev/urandom of=data.bin bs=512 count=10000

Use array.array('c') as a type of incomming buffer
A connection incoming buffer is extended using += operator. This leads
to unnecessary memory allocations and significant performance drops when
transferring big amounts of data (i. e. a few megabytes). Using of the array('c')
(an array of bytes) data type enables us to handle incoming data efficiently.
Contributor

eskil commented Feb 15, 2013

This looks good. Thanks for the fix!

Contributor

klange commented Feb 15, 2013

Agreed. This looks useful.

klange added a commit that referenced this pull request Feb 15, 2013

Merge pull request #33 from malor/master
Fix slow handling of connection incoming buffer

@klange klange merged commit b568ce7 into YelpArchive:master Feb 15, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment