RFE: "xrd isfileonline" equivalent for "xrdfs"

[jmuf]

"xrd" had a command to query whether a file was disk-resident ("isfileonline"). This seems to have disappeared with the newer "xrdfs" - could we get something like it back (&documented)?

[ljanyst]

It was not implemented because the only thing it did was to do a stat and see if an offline flag was not set. I think it's redundant and we would need to add a lot of commands like this for completeness sake. Do you really think it's needed/useful?

]==> xrdfs castorpublic stat /castor/cern.ch/user/l/ljanyst/user.RichardHawkings.0108173.topmix_Egamma.AOD.v2._00039.pool.root
Path:  /castor/cern.ch/user/l/ljanyst/user.RichardHawkings.0108173.topmix_Egamma.AOD.v2._00039.pool.root
Id:    2418223087666331648
Size:  791773689
Flags: 117 (XBitSet|Other|POSCPending|IsReadable|IsWritable)

BTW. Castor seems to be setting a POSCPending flag for a file that is offline...

[esindril]

In order for the offline flag to be passed on from the server to the client the Dev.uuid should be 0 in file XrdXrootdStat.icc:83. But apparently this field is never set and therefore even if the buf.st_dev is 0, the offline flag is not set for the client to see.

The current behaviour in the Castor XRootD plugin is to pass on any information about the file from the Castor namespace if the file is online and to set all the bits in the st_mode field to 1 if it is offline. Therefore, in the example you showed above you should have also seen the offline flag - but it is not shown since the server does not set is properly.

I can leave the Castor XRootD implementation as is and one can test for the offline flag (when it is correctly passed on) or I can set only this flag when the file is offline.

[ljanyst]

I think we should implement this properly, the way it is now is rather confusing. @abh3 could you please have a look to see if the offline flag can be easily passed from the plugin to the protocol layer so that we could put it in 3.3.6?

Lukasz

[esindril]

Actually looking more closely at it, there is a small trick done in the code by using unions. Part of the union is used to write data to it and then later on the other part of the union is used to retrieve the previously written value by reinterpreting the memory. The problem was in the Castor XRootD plugin which was not setting also the inode to 0 such that the Dev.uuid would be 0.

I fixed this and now you should also get the offline flag in the list of flags. After some reading on the topic, it seems that doing such a trick can sometimes lead to undefined behaviour ... just saying. In my case at least, it works as expected.

Cheers,
Elvin

[ljanyst]

OK, thanks Elvin!

[ljanyst]

Added a generic query interface:
33e37b0

You can query for a given flag:

]=[pon 14/01/27 10:25 CET]=[ljanyst@idefix:~/Projects/xroot/code/master/build]
]==> xrdfs castorpublic stat  /castor/cern.ch/user/l/ljanyst/user.RichardHawkings.0108173.topmix_Egamma.AOD.v2._00039.pool.root -q "IsDir"      
Path:   /castor/cern.ch/user/l/ljanyst/user.RichardHawkings.0108173.topmix_Egamma.AOD.v2._00039.pool.root
Id:     2418223087666331648
Size:   791773689
Flags:  117 (XBitSet|Other|POSCPending|IsReadable|IsWritable)
Query:  IsDir 
[ERROR] Query response negative
]=[pon 14/01/27 10:25 CET]=[ljanyst@idefix:~/Projects/xroot/code/master/build]
]==> echo $?
55

]=[pon 14/01/27 10:26 CET]=[ljanyst@idefix:~/Projects/xroot/code/master/build]
]==> xrdfs castorpublic stat  /castor/cern.ch/user/l/ljanyst/user.RichardHawkings.0108173.topmix_Egamma.AOD.v2._00039.pool.root -q "IsReadable"   
Path:   /castor/cern.ch/user/l/ljanyst/user.RichardHawkings.0108173.topmix_Egamma.AOD.v2._00039.pool.root
Id:     2418223087666331648
Size:   791773689
Flags:  117 (XBitSet|Other|POSCPending|IsReadable|IsWritable)
Query:  IsReadable 
]=[pon 14/01/27 10:26 CET]=[ljanyst@idefix:~/Projects/xroot/code/master/build]
]==> echo $?                                                                                                                                   
0

Query of an or'd combination of flags:

]=[pon 14/01/27 10:25 CET]=[ljanyst@idefix:~/Projects/xroot/code/master/build]
]==> xrdfs castorpublic stat  /castor/cern.ch/user/l/ljanyst/user.RichardHawkings.0108173.topmix_Egamma.AOD.v2._00039.pool.root -q "IsDir|Other|IsReadable"
Path:   /castor/cern.ch/user/l/ljanyst/user.RichardHawkings.0108173.topmix_Egamma.AOD.v2._00039.pool.root
Id:     2418223087666331648
Size:   791773689
Flags:  117 (XBitSet|Other|POSCPending|IsReadable|IsWritable)
Query:  IsDir|Other|IsReadable 
]=[pon 14/01/27 10:25 CET]=[ljanyst@idefix:~/Projects/xroot/code/master/build]
]==> echo $?                                                                                                                                               
0

Query for an and'd combination of flags:

]=[pon 14/01/27 10:25 CET]=[ljanyst@idefix:~/Projects/xroot/code/master/build]
]==> xrdfs castorpublic stat  /castor/cern.ch/user/l/ljanyst/user.RichardHawkings.0108173.topmix_Egamma.AOD.v2._00039.pool.root -q "IsDir&Other"           
Path:   /castor/cern.ch/user/l/ljanyst/user.RichardHawkings.0108173.topmix_Egamma.AOD.v2._00039.pool.root
Id:     2418223087666331648
Size:   791773689
Flags:  117 (XBitSet|Other|POSCPending|IsReadable|IsWritable)
Query:  IsDir&Other 
[ERROR] Query response negative
]=[pon 14/01/27 10:26 CET]=[ljanyst@idefix:~/Projects/xroot/code/master/build]
]==> echo $?                                                                                                                                    
55

]=[pon 14/01/27 10:26 CET]=[ljanyst@idefix:~/Projects/xroot/code/master/build]
]==> xrdfs castorpublic stat  /castor/cern.ch/user/l/ljanyst/user.RichardHawkings.0108173.topmix_Egamma.AOD.v2._00039.pool.root -q "XBitSet&Other"
Path:   /castor/cern.ch/user/l/ljanyst/user.RichardHawkings.0108173.topmix_Egamma.AOD.v2._00039.pool.root
Id:     2418223087666331648
Size:   791773689
Flags:  117 (XBitSet|Other|POSCPending|IsReadable|IsWritable)
Query:  XBitSet&Other 
]=[pon 14/01/27 10:26 CET]=[ljanyst@idefix:~/Projects/xroot/code/master/build]
]==> echo $?                                                                                                                                      
0

Note: you get a different shell code when it fails for other reasons:

]=[pon 14/01/27 10:26 CET]=[ljanyst@idefix:~/Projects/xroot/code/master/build]
]==> xrdfs castorpublic stat  /castor/cern.ch/user/l/ljanyst/user.RichardHawkings -q "IsReadable" 
[ERROR] Server responded with an error: [3011] Unable to stat /castor/cern.ch/user/l/ljanyst/user.RichardHawkings; No such file or directory
]=[pon 14/01/27 10:27 CET]=[ljanyst@idefix:~/Projects/xroot/code/master/build]
]==> echo $?                                                                                     
54