Async transfers efficiency in xrootd

[esindril]

From Sebastien:

The issue (as I understand it) is that the buffer size for each write is limited to 2M, due to this line in XrdBuffManager::Recalc:

// Make sure the request is within our limits
if (sz <= 0 || sz > maxsz) return 0;

with this definition of maxsz in the XrdBuffManager constructor :
maxsz(1<<(XRD_BUSHIFT+XRD_BUCKETS-1+5)) and these hardcoded values in XrdBuffer.hh :1234

#define XRD_BUCKETS 12
#define XRD_BUSHIFT 10

This gives me a maximum transfer rate of 60 to 70MB/s, limited by the ability of the xrootd server to read from the network (the TCP buffers are full). However, recompiling xrootd with XRD_BUCKETS 16 and using 32MB buffers gave 300MB/s and XRD_BUCKETS 17 with 64MB buffers gave 500MB/s (limit seemed to have been somewhere else then).

From Andy:

OK, I came up with a solution that would work. If the requested size is greater that the default 2M then the allocation comes out of a global pool (i.e. a pool common to all buffer managers). That feature has to be enabled via config but would work (assuming changing a const int in the class to an int does not change the size of the object -- it shouldn't, but I will check it out). The global pool would not be subject to statistical reshaping.

[esindril]

@abh3 is this already pushed?

[esindril]

Ok, found it. This was fixed in 0ba1bfb.