support stat() and unlink() in XrdEc

[wyang007]

Is it possible to add support of stat() and unlink() in XrdEc ?

[xrootd-dev]

In a native xrootd cluster? Well, anything is possible because it's simply software. Is it easy? No, not in this case. No individual server knows it's part of an EC group as that is externally layered on the servers. So, a stat(0 or unlink() call would only affect an individual server leaving the stripes on other servers unaffected. Now, yes, to do an EC layout you need a redirector and, yes, unlink is supported across multiple servers but as you and I have found it is unreliable because a server may be down at the time an unlink happens. A true stat call in an EC layout would require that the redirector do some additional work or at least delegate it to a particular server. Doable but the code is simply not there to do it. Alll of this requires some amount of architectural thinking in order to implement something that resembles a central metadata repository but is actually distributed (unless what you want is a central repo with a single point of failure). That is not an easy task but certainly doable. However, that said, today nothing you mentioned exists or if it does exist no one is happy with it. Andy

…

On Wed, 17 Nov 2021, Wei Yang wrote: Is it possible to add support of stat() and unlink() in XrdEc ? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #1559 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1

[simonmichal]

@wyang007 : sorry for the late reply! I pushed recently two enhancements: 673dabf and 080d872, with those two stat will at least return correct file size. Also, now if some stripes are missing since the client will now the correct file size, xrdcp will be able to correctly copy the file out of the storage.

[wyang007]

I think we can close this one as we already made many changes. these requests are no longer relevant.