Skip to content

Use sendfile() to speed up transfers #152

Closed
@giampaolo

Description

@giampaolo

From g.rodola on January 24, 2011 08:06:14

Many Unix kernels provide a function called sendfile(2), a system call which 
provides a "zero-copy" way of copying data from one file (or socket) descriptor 
to another, which should result in a considerable speed-up when sending a file 
from server to client: http://www.proftpd.org/docs/howto/Sendfile.html This 
kind of function is particularly useful for softwares such as FTP servers and 
it would be good if pyftpdlib can take advantage of it.

= Implementation =

sendfile() is available on different UNIX platforms such as Linux, *BSD and AIX. 
There are a bunch of python wrappers available. Here is one: 
http://pypi.python.org/pypi/py-sendfile/ Medusa also provides one, and also 
provides an example of integration with asynchat: 
http://www.nightmare.com/medusa/ As for python itself, sendfile() has been 
proposed for inclusion in stdlib; here's a patch which is supposed to be added 
in python 3.3: http://bugs.python.org/issue10882 = Windows = 

Windows provides a similar function (TransmitFile): 
http://msdn.microsoft.com/en-us/library/ms740565(v=vs.85).aspx ...and it is 
exposed via pywin32 extension: 
http://sourceforge.net/tracker/index.php?func=detail&aid=1962146&group_id=78018&atid=551956
 = Integration with pyftpdlib = 

pyftpdlib can either implement its own wrapper for sendfile/TransmitFile or 
depend on py-sendfile/pywin32 modules.
The latter choice seems to make more sense.
In this case, changes would consist in modifying DTPHandler class so that it 
uses the new function when one of the two modules is installed, otherwise 
fallback on default transfer method/implementation.
Use of sendfile can be enabled/disabled via a DTPHandler.use_sendfile class 
attribute defaulting to True or False depending on whether the required module 
is installed.
The use of sendfile() will be explicitly avoided/disabled by pyftypdlib itself 
in case of:

- transfers in ASCII mode
- throttled transfers via ThrottledDTPHandler class
- when SSL/TLS is used on the data channel

Original issue: http://code.google.com/p/pyftpdlib/issues/detail?id=152

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions