PSS with disk cache enabled doesn't update files changing in the backend

[apeters1971]

Once a file is cached in xCache, the local copy is never updated even if the inode, size or modification time in the backend has changed. That is very unfortunate. Can this trivial cache eviction policy be added?

[wyang007]

This is known. We had a light discussion of whether we should do this. For experiment data we should not. But for general purpose cache it seems this is a legitimate request. I think we need to understand how costly it is. I am copying this to xcache-l@slac.stanford.edu. It is a mailing list dedicated for Xcache and is open for everyone to subscribe. So far it has zero traffic (and spam). regards,

…

-- Wei Yang | yangw@slac.stanford.edu<mailto:yangw@slac.stanford.edu> | 650-926-3338(O) From: Andreas-Joachim Peters <notifications@github.com<mailto:notifications@github.com>> Reply-To: xrootd/xrootd <reply@reply.github.com<mailto:reply@reply.github.com>> Date: Wednesday, March 14, 2018 at 8:16 AM To: xrootd/xrootd <xrootd@noreply.github.com<mailto:xrootd@noreply.github.com>> Cc: Subscribed <subscribed@noreply.github.com<mailto:subscribed@noreply.github.com>> Subject: [xrootd/xrootd] PSS with disk cache enabled doesn't update files changing in the backend (#668) Once a file is cached in xCache, the local copy is never updated even if the inode, size or modification time in the backend has changed. That is very unfortunate. Can this trivial cache eviction policy added? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#668>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AE9TA-Kl5qC8kdTptoCOaRiVgNe9C33Mks5teTQ1gaJpZM4Sqn8b>.

[xrootd-dev]

On 03/14/18 09:11, Wei Yang wrote: This is known. We had a light discussion of whether we should do this. For experiment data we should not. But for general purpose cache it seems this is a legitimate request. I think we need to understand how costly it is.

Ah, I didn't know this was being discussed :) And yes, once the cache has a file (even if incomplete) it will not even open the remote connection until it needs to get a block outside of what it already has. This is intentional and improves file-open performance and resilience for read-only data. So, clearly we need an option to force checking if a remote file has changed on open ... or even have a plugin that can make this decision based on filename. However, we are opening a can of worms here ... what if the file changes while it is being read? There is no way for the cache to detected that unless the server sends out notifications of all writes it has received. Matevz

…

I am copying this to ***@***.*** It is a mailing list dedicated for Xcache and is open for everyone to subscribe. So far it has zero traffic (and spam). regards, -- Wei Yang | ***@***.******@***.***> | 650-926-3338(O) From: Andreas-Joachim Peters ***@***.******@***.***>> Reply-To: xrootd/xrootd ***@***.******@***.***>> Date: Wednesday, March 14, 2018 at 8:16 AM To: xrootd/xrootd ***@***.******@***.***>> Cc: Subscribed ***@***.******@***.***>> Subject: [xrootd/xrootd] PSS with disk cache enabled doesn't update files changing in the backend (#668) Once a file is cached in xCache, the local copy is never updated even if the inode, size or modification time in the backend has changed. That is very unfortunate. Can this trivial cache eviction policy added? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#668>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AE9TA-Kl5qC8kdTptoCOaRiVgNe9C33Mks5teTQ1gaJpZM4Sqn8b>. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#668 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AD7YjmU691sSE24_E5S4P6d19bZQAuacks5teUEjgaJpZM4Sqn8b>. -------------------------------------------------------------------------------- Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1

[abh3]

Correct. The XrootD Xcache assume read-only file. If you want that kind of flexibility then you need to use XcacheH (though only HTTP access is available). This has to to do with semantics. In the XRootD world, caching of files that change makes no sense. That is not true if the HTTP world. So, if you need that kind of flexibility, use XcacheH.