XrdCl: Opaque information during open recovery?

[bbockelm]

Hi,

(Full debug log is here: http://glidemon.web.cern.ch/glidemon/show.php?log=http://vocms095.cern.ch/mon/cms1590/141114_184357_crab3test-2:bbockelm_crab_xrd_multi1/job_out.36.0.txt)

During a file-open recovery, the client keeps the opaque information from dCache (org.dcache.uuid=63425c0f-eefc-4990-83ec-82a4419d4e50) when it returns to the redirector. This later gets passed to other sites within the federation.

If we return to the redirector, other than updating tried, should we keep all opaque info?

Brian

PS - Is there a way to disable automatic recovery? We kinda re-implement it within CMSSW now.

[ljanyst]

Hmm, is there any harm done by keeping it?

For the second question: yes.
https://github.com/xrootd/xrootd/blob/master/src/XrdCl/XrdClFile.hh#L364

[bbockelm]

In this case - yes, there's harm. We look for the org.dcache.uuid key to identify the remote host as dCache. If it's present, we change how we re-build the tried= string if the pool host fails.

I'm starting to wonder - should we have a plugin point so the client can influence redirections (i.e., affect the opaque information or decide to abort the recovery / ignore kXR_redirect)?

Can you update the documentation of that line in XrdClFile.hh to denote which key affects file opens?

[ljanyst]

I could, in principle, reset it, but to what? To nothing? Remember what the CGI was at the redirector and reset it to that?

I think it would be possible to write a plugin to influence redirections/revocery, but I doubt I will have time to do it.

The XrdCl::File keys affect all of the requests.

[bbockelm]

My thought on this was to reset the CGI to URL at the redirector except for the "tried" key.

Don't worry too much about the redirections plugin - it's low priority; would just save me quite a bit of time if we ever wanted detailed management of the recovery (currently, I'd have to disable redirections and recovery, then reimplement them myself).

[abh3]

Is this also involved with Matevz issue of the tried string? If so, we should discuss this. Otherwise, could you explain exactly what is meant by "reset the CGI to URL at the redirector except for the "tried" key" (i.e. give an example)?

[abh3]

Oh, never mind. Is see what you mean. Some system has added cgi information to a URL and now you'd like to get that removed in certain cases. That's not exactly straightforward in that there are times you want this and time you might not want this. That said, if you only want cgi manipulation if the client independently decides to go back to a redirector for error recovery and that the cgi should then be reset to what was originally on the URL plus whatever the client added on its own (as it's the only one that knows if it can be removed); then I don't see a problem with this behavior.

[bbockelm]

Hi Andy,

Yes - I would like to see what you have in the second half of your comment:

"""
CGI manipulation if the client independently decides to go back to a redirector for error recovery and that the CGI should then be reset to what was originally on the URL plus whatever the client added on its own (as it's the only one that knows if it can be removed)
"""

[abh3]

This has been fixed in 4.2. The server is now propagating the opaque information. It also allows the server to test for redirection loops.