Proxy Cache (memory or disk) not returning proper stat information.

[abh3]

Specifically, for xrootd file locking to work correctly, stat() must fill in the st_dev and st_ino which should be unique to the file in question. These fields are rarely if ever set, especially for files that will be created in the cache (i.e. open+create), leading to locking failure. Below is a question that sparked this analysis.

Hello XRootD folks,

I'm GeorgeV at RAL, I work on Echo and we're seeing a weird issue with our XRootD servers.

Since putting in the XRootD caching proxy, it has been denying all of
our writes via XRootD with messages like this:

170627 17:00:14 53626 XrootdXeq: Output file
dteam:test/nagios_check_xrootd_20170627_170014_ceph-gw1 is already
opened by 15 readers; open denied.

All of our writes at this moment are from the Nagios functional check.
It is however of some concern that the proxy seems to think that every
file is opened by multiple readers, even if it is a newly generated
filename and the file is yet to be created for the first time.

With my limited knowledge of C++ and the XRootD codebase I have tracked
this down to:

xrootd/src/XrdXrootd/XrdXrootdXeq.cc

Line 1306 in 0ecced4

if (!(popt & XROOTDXP_NOLK) && (rc = Locker->Lock(xp, doforce)))

It seems to use this method:

xrootd/src/XrdXrootd/XrdXrootdFileLock1.cc

Line 80 in 1bf3552

int XrdXrootdFileLock1::Lock(XrdXrootdFile *fp, int force)

This didn't use to happen before we put the proxy in place and we don't intend to switch back (as that solved a number of other issues). It's also difficult for us to bypass the proxy for writes due to how we have the two set up.

Has anyone seen this before? Any ideas would be much appreaciated.