xrdfs ls lists files multiply

[olifre]

Maybe this is a configuration issue, but I am not sure... Maybe it's also a visualization bug.

Our setup is:

• Backing filesystem is a LustreFS.
• A single xrootd as manager, used only for redirecting.
• 4 data servers actually used for doing the IO.

However, running
xrdfs storm.physik.uni-bonn.de ls -l /lustre/grid/atlas/atlaslocalgroupdisk/
reveals each file four times, likely since each data server sees the same FS.
storm.physik.uni-bonn.de is our manager.

Using xrdfs ls on a single data-server shows the files only once, as expected.

All servers can access the shared filesystem, and I am using the directive:

cms.dfs limit 0 lookup distrib mdhold 0 redirect verify retries 2

Am I missing something, or is this expected behaviour / an inconvenience / a bug?

[olifre]

Additional note: Mounting the FS with xrootdfs, I see each file only once, as expected.

[abh3]

Well, some people could call it a feature, though obviously, for you, it’s a bug. It works out that the client actually asks all the servers in the cluster for the directory contents and then pieces them together. That works just fine for JBOD setups but not for DFS setups (i.e. your Lustre setup) and the client really doesn’t know which kind you have. So, I think you want one of: a) Directing the xrdfs command to a data server not a redirector, b) Adding a comment here that you would like an enhancement and be able to specify whether or not you want a merged listing or not and perhaps a config envar to set this as the default, or c) say that duplicates in the listing should be eliminated. Of course, one would have to define what a duplicate really is as two servers could potentially have identically named files with different attributes. While that would be a mistake it would be one that you could see. Depending on how “duplicate is defined” such information could be potentially masked. Andy From: olifre Sent: Tuesday, July 11, 2017 2:21 PM To: xrootd/xrootd Cc: Subscribed Subject: [xrootd/xrootd] xrdfs ls lists files multiply (#541) Maybe this is a configuration issue, but I am not sure... Maybe it's also a visualization bug. Our setup is: a.. Backing filesystem is a LustreFS. b.. A single xrootd as manager, used only for redirecting. c.. 4 data servers actually used for doing the IO. However, running xrdfs storm.physik.uni-bonn.de ls -l /lustre/grid/atlas/atlaslocalgroupdisk/ reveals each file four times. All servers can access the shared filesystem, and I am using the directive: cms.dfs limit 0 lookup distrib mdhold 0 redirect verify retries 2 Am I missing something, or is this expected behaviour / a bug? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[olifre]

Well, some people could call it a feature, though obviously, for you, it’s a bug.

Thanks for your reply! This at least clarifies it's not a configuration mistake on my end. Then indeed, I guess what I really want is rather an "enhancement".

That works just fine for JBOD setups but not for DFS setups (i.e. your Lustre setup) and the client really doesn’t know which kind you have.

This raises the question: How does xrootdfs know that? It presents the files only once in our configuration, even though it knows all the data servers. So I guess it does some kind of merging already.

Based on that, I would go with the option to let xrdfs ls do what xrootdfs does. This appears to be the "least surprising" solution for the user.
My guess is, without reading the code, that xrootdfs uses some kind of option "c", i.e. some clever merging.
Does that sound sensible?

Cheers, Oliver

[wyang007]

xrootdfs does merge duplicate items. Depend on how you view xrdfs. In some use cases, you may want it to remove the repeated entries. In other cases, one may want it to show those items multiple times - xrdfs is also a tool that can show us how things actually work in the xrootd cluster, at a lower / raw level. I guess the question is whether we want make it optional… regards,

…

-- Wei Yang | yangw@slac.stanford.edu<mailto:yangw@slac.stanford.edu> | 650-926-3338(O) From: olifre <notifications@github.com<mailto:notifications@github.com>> Reply-To: xrootd/xrootd <reply@reply.github.com<mailto:reply@reply.github.com>> Date: Tuesday, July 11, 2017 at 2:50 PM To: xrootd/xrootd <xrootd@noreply.github.com<mailto:xrootd@noreply.github.com>> Cc: Subscribed <subscribed@noreply.github.com<mailto:subscribed@noreply.github.com>> Subject: Re: [xrootd/xrootd] xrdfs ls lists files multiply (#541) Well, some people could call it a feature, though obviously, for you, it’s a bug. Thanks for your reply! This at least clarifies it's not a configuration mistake on my end. Then indeed, I guess what I really want is rather an "enhancement". That works just fine for JBOD setups but not for DFS setups (i.e. your Lustre setup) and the client really doesn’t know which kind you have. This raises the question: How does xrootdfs know that? It presents the files only once in our configuration, even though it knows all the data servers. So I guess it does some kind of merging already. Based on that, I would go with the option to let xrdfs ls do what xrootdfs does. This appears to be the "least surprising" solution for the user. My guess is, without reading the code, that xrootdfs uses some kind of option "c", i.e. some clever merging. Does that sound sensible? Cheers, Oliver — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#541 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AE9TAwSNA0Oj5vJAoSMGIJKcZkKZQFr4ks5sM-47gaJpZM4OU1JX>.

[olifre]

Depend on how you view xrdfs. In some use cases, you may want it to remove the repeated entries. In other cases, one may want it to show those items multiple times

I understand. For me as a user, it was confusing that "ls" shows entries which are actually one and the same file multiple times, so I would at least call it "unexpected" compared to a normal "ls".
Of course, the functionality to see the "inner workings" is crucial for any kind of debugging.

So to summarize: In my humble opinion (and this is only my opinion as a user, not a request), I would suggest the following:

• xrdfs ls could merge duplicate entries by default, since this is closer to the expectation of a user, and thus less surprising.
• A new option for xrdfs ls could turn off the merging.
• For ls -l it might be nice if the data server name / IP is also listed (or, if -l is passed together with -u?). Then the output might be even more helpful, and would feel "correct" even without merging.
• For ls -u I totally understand the necessity to never merge.

As I said, this is only what I would naïvely expect as a user as more natural behaviour of ls. The suggested changes would of course change the existing behaviour, but not remove any functionality.

[abh3]

OK, we have it on the list of enhancements. The options you propose seem good start. Let's see where we go from here. Thanks for bringing this up.

[olifre]

OK, we have it on the list of enhancements. The options you propose seem good start. Let's see where we go from here. Thanks for bringing this up.

Thanks to you for taking my ideas / proposal into consideration!