-
Notifications
You must be signed in to change notification settings - Fork 149
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
python client hangs forever #836
Comments
Here is debug output of the request:
|
@vkuznet : thanks for reporting this problem. What do you actually mean by 'waits forever'? The default request timeout is 1800 sec, so it should timeout only after 1.5h (have you waited that long?). Could you please run the script with envar XRD_LOGLEVEL=Dump and without the timeout set to 10, please leave it running for 5 minutes or so and give us the output? At the first glance it seems that XRootD client got redirected to llrxrd-redir.in2p3.fr but failed to connect to this host. Maybe you could follow that lead? Thanks, |
"forever" means that I waited 20-30 min and nothing happened. Here is a log with XRD_LOGLEVEL=Dump and waiting 5 minutes:
|
Hi @vkuznet - For the last log snippet, can you confirm that you decreased the timeout value to be below the amount of time you waited? It wasn't clear from your response whether this was so. Brian |
Unrelatedly, one word of warning about the python bindings, in case if it helps in the future -- Never use the "asynchronous" features. There are deadlock issues at the core of the implementation's design (basically, it calls Python from a non-Python thread ... no mechanism to prevent lock inversion or check that the Python interpreter still exists). It will need a reimplementation prior to them being safe to use. |
yes, I removed timeout argument from the call. Here is a code now:
```
#!/usr/bin/env python
from XRootD import client
from XRootD.client.flags import OpenFlags
f=client.File()
redirector='root://cms-xrd-global.cern.ch'
#redirector='root://xrootd-cms.infn.it'
#redirector='root://cmsxrootd.fnal.gov'
fin='%s//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root' % redirector
print(fin)
data=f.open(fin, OpenFlags.READ)
print("data", data)
```
and, I tried different redirectors.
My suspicious it is related to the node iptables settings, the node is on a
Internet2 and may be some settings should be applied to make requests pass
through. But I don't know what xrootd requires (if any).
…On 0, Brian P Bockelman ***@***.***> wrote:
Hi @vkuznet -
For the last log snippet, can you confirm that you decreased the timeout value to be below the amount of time you waited? It wasn't clear from your response whether this was so.
Brian
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#836 (comment)
|
So, in that case, the internal timeout uses the default value of 1800 seconds -- you'll need to wait at least that long. That said, it does look like there are connectivity issues. Notice that it immediately fails IPv6 addresses, and is repeatedly trying:
But always seems to be timing out for that TCP connect. I can confirm locally that the server appears responsive. |
Well, I verified that all of the hosts listed in your script (including
the ones that were commented out) are reachable via WAN. So, unless there
is some strange iptables setting on the host where you trying this out,
connectivity should not be an issue.
Hoever, cms-xrd-global.cern.ch has an IPv6 address and that address will
be tried first if you are IPv6 enabled (that's an RFC requireent). If your
IPv6 setup is broken then there will be a 2 minute delay before the next
IPv6 address is tried (the host has two IPv6 addresses). Then the IPv4
address will be tried. So, if this is the case it will take 4 minutes to
establish a connection. This may be the problem.
Andy
…On Mon, 8 Oct 2018, Valentin Kuznetsov wrote:
yes, I removed timeout argument from the call. Here is a code now:
```
#!/usr/bin/env python
from XRootD import client
from XRootD.client.flags import OpenFlags
f=client.File()
redirector='root://cms-xrd-global.cern.ch'
#redirector='root://xrootd-cms.infn.it'
#redirector='root://cmsxrootd.fnal.gov'
fin='%s//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root' % redirector
print(fin)
data=f.open(fin, OpenFlags.READ)
print("data", data)
```
and, I tried different redirectors.
My suspicious it is related to the node iptables settings, the node is on a
Internet2 and may be some settings should be applied to make requests pass
through. But I don't know what xrootd requires (if any).
On 0, Brian P Bockelman ***@***.***> wrote:
> Hi @vkuznet -
>
> For the last log snippet, can you confirm that you decreased the timeout value to be below the amount of time you waited? It wasn't clear from your response whether this was so.
>
> Brian
>
> --
> You are receiving this because you were mentioned.
> Reply to this email directly or view it on GitHub:
> #836 (comment)
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
#836 (comment)
|
Could you post the name of the host on which you are running this script?
…On Mon, 8 Oct 2018, Valentin Kuznetsov wrote:
"forever" means that I waited 20-30 min and nothing happened. Here is a log with XRD_LOGLEVEL=Dump and waiting 5 minutes:
```
[2018-10-08 08:55:31.084582 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory
[2018-10-08 08:55:31.084849 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory
[2018-10-08 08:55:31.084965 -0400][Info ][Utility ] Env: Importing from shell XRD_NETWORKSTACK=IPAll as NetworkStack
[2018-10-08 08:55:31.084999 -0400][Debug ][PlugInMgr ] Initializing plug-in manager...
[2018-10-08 08:55:31.085011 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs...
[2018-10-08 08:55:31.085019 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d...
[2018-10-08 08:55:31.085036 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory
[2018-10-08 08:55:31.085106 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /home/kuznetv/.xrootd/client.plugins.d...
[2018-10-08 08:55:31.085158 -0400][Dump ][PlugInMgr ] Processing: /home/kuznetv/.xrootd/client.plugins.d/client-plugin.conf
[2018-10-08 08:55:31.085210 -0400][Debug ][PlugInMgr ] Unable to find 'lib' key in the config file /home/kuznetv/.xrootd/client.plugins.d/client-plugin.conf, ignoring this config
[2018-10-08 08:55:31.101915 -0400][Debug ][Poller ] Available pollers: built-in
[2018-10-08 08:55:31.101948 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in
[2018-10-08 08:55:31.101960 -0400][Debug ][Poller ] Creating poller: built-in
[2018-10-08 08:55:31.101973 -0400][Debug ][Poller ] Creating and starting the built-in poller...
[2018-10-08 08:55:31.102147 -0400][Debug ][Poller ] Using 1 poller threads
[2018-10-08 08:55:31.102166 -0400][Debug ][TaskMgr ] Starting the task manager...
[2018-10-08 08:55:31.102205 -0400][Debug ][TaskMgr ] Task manager started
[2018-10-08 08:55:31.102215 -0400][Debug ][JobMgr ] Starting the job manager...
[2018-10-08 08:55:31.102287 -0400][Debug ][JobMgr ] Job manager started, 3 workers
[2018-10-08 08:55:31.102298 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2018-10-08 08:55:31 -0400]
[2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] URL: root://cms-xrd-global.cern.ch//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
[2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Protocol: root
[2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] User Name:
[2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Password:
[2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Host Name: cms-xrd-global.cern.ch
[2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Port: 1094
[2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Path: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
[2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] URL: root://cms-xrd-global.cern.ch//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
[2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Protocol: root
[2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] User Name:
[2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Password:
[2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Host Name: cms-xrd-global.cern.ch
[2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Port: 1094
[2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Path: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
[2018-10-08 08:55:31.102462 -0400][Debug ][File ] ***@***.***://cms-xrd-global.cern.ch:1094//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root] Sending an open command
[2018-10-08 08:55:31.102512 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Sending message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat )
[2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] URL: cms-xrd-global.cern.ch:1094
[2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Protocol: root
[2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] User Name:
[2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Password:
[2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Host Name: cms-xrd-global.cern.ch
[2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Port: 1094
[2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Path:
[2018-10-08 08:55:31.102577 -0400][Debug ][PostMaster ] Creating new channel to: cms-xrd-global.cern.ch:1094 1 stream(s)
[2018-10-08 08:55:31.102605 -0400][Debug ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Stream parameters: Network Stack: IPAll, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800
[2018-10-08 08:55:31.107979 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" to be run at: [2018-10-08 08:55:46 -0400]
[2018-10-08 08:55:31.108022 -0400][Dump ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Sending message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xfd286c60) through substream 0 expecting answer at 0
[2018-10-08 08:55:31.341022 -0400][Debug ][PostMaster ] [cms-xrd-global.cern.ch:1094] Found 4 address(es): [::ffff:188.184.161.66]:1094, [::ffff:188.184.67.41]:1094, [2001:1458:301:69::100:3c]:1094, [2001:1458:201:a4::100:323]:1094
[2018-10-08 08:55:31.341070 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Attempting connection to [2001:1458:201:a4::100:323]:1094
[2018-10-08 08:55:31.341102 -0400][Error ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
[2018-10-08 08:55:31.341120 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Attempting connection to [2001:1458:301:69::100:3c]:1094
[2018-10-08 08:55:31.341136 -0400][Error ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
[2018-10-08 08:55:31.341151 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Attempting connection to [::ffff:188.184.67.41]:1094
[2018-10-08 08:55:31.341196 -0400][Debug ][Poller ] Adding socket 0xfd279890 to the poller
[2018-10-08 08:55:31.452359 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Async connection call returned
[2018-10-08 08:55:31.452420 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Sending out the initial hand shake + kXR_protocol
[2018-10-08 08:55:31.452472 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Wrote a message: (0xe8000950), 44 bytes
[2018-10-08 08:55:31.679451 -0400][Dump ][XRootDTransport ] [msg: 0xe8000950] Expecting 8 bytes of message body
[2018-10-08 08:55:31.679483 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header, size: 8
[2018-10-08 08:55:31.679493 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received a message of 16 bytes
[2018-10-08 08:55:31.679508 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 310)
[2018-10-08 08:55:31.679522 -0400][Dump ][XRootDTransport ] [msg: 0xe8000950] Expecting 8 bytes of message body
[2018-10-08 08:55:31.679529 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header, size: 8
[2018-10-08 08:55:31.679548 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received a message of 16 bytes
[2018-10-08 08:55:31.679558 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] kXR_protocol successful (type: manager [meta], protocol version 310)
[2018-10-08 08:55:31.692590 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Sending out kXR_login request, username: kuznetv, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=python3.6&xrd.info=&xrd.hostname=goofy&xrd.rn=v4.8.4, dual-stack: true, private IPv4: false, private IPv6: true
[2018-10-08 08:55:31.692667 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Wrote a message: (0xe8000b40), 109 bytes
[2018-10-08 08:55:31.803724 -0400][Dump ][XRootDTransport ] [msg: 0xe8000b40] Expecting 16 bytes of message body
[2018-10-08 08:55:31.803757 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header, size: 8
[2018-10-08 08:55:31.803769 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received a message of 24 bytes
[2018-10-08 08:55:31.803783 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Logged in, session: 79ef03008ff50100bb01000033f10300
[2018-10-08 08:55:31.803796 -0400][Debug ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Stream 0 connected.
[2018-10-08 08:55:31.803807 -0400][Debug ][Utility ] Monitor library name not set. No monitoring
[2018-10-08 08:55:31.803848 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Wrote a message: kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xfd286c60), 122 bytes
[2018-10-08 08:55:31.803861 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Successfully sent message: kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xfd286c60).
[2018-10-08 08:55:31.803879 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) has been successfully sent.
[2018-10-08 08:55:31.803889 -0400][Dump ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0.0] All messages consumed, disable uplink
[2018-10-08 08:55:31.966285 -0400][Dump ][XRootDTransport ] [msg: 0xe8000b70] Expecting 25 bytes of message body
[2018-10-08 08:55:31.966318 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header for 0xe8000b70 size: 8
[2018-10-08 08:55:31.966337 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message 0xe8000b70 of 33 bytes
[2018-10-08 08:55:31.966347 -0400][Dump ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Handling received message: 0xe8000b70.
[2018-10-08 08:55:31.966515 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Got kXR_redirect response to message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): llrxrd-redir.in2p3.fr, port 1094
[2018-10-08 08:55:31.966582 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Current server has been assigned as a load-balancer for message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat )
[2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] URL: llrxrd-redir.in2p3.fr:1094/
[2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Protocol: root
[2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] User Name:
[2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Password:
[2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Host Name: llrxrd-redir.in2p3.fr
[2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Port: 1094
[2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Path:
[2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] URL: llrxrd-redir.in2p3.fr:1094
[2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Protocol: root
[2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] User Name:
[2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Password:
[2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Host Name: llrxrd-redir.in2p3.fr
[2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Port: 1094
[2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Path:
[2018-10-08 08:55:31.966725 -0400][Debug ][PostMaster ] Creating new channel to: llrxrd-redir.in2p3.fr:1094 1 stream(s)
[2018-10-08 08:55:31.966755 -0400][Debug ][PostMaster ] [llrxrd-redir.in2p3.fr:1094 #0] Stream parameters: Network Stack: IPAll, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800
[2018-10-08 08:55:31.976328 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" to be run at: [2018-10-08 08:55:46 -0400]
[2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] URL: fake://fake:111//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
[2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Protocol: fake
[2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] User Name:
[2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Password:
[2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Host Name: fake
[2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Port: 111
[2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Path: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
[2018-10-08 08:55:31.976469 -0400][Dump ][PostMaster ] [llrxrd-redir.in2p3.fr:1094 #0] Sending message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xfd286c60) through substream 0 expecting answer at 0
[2018-10-08 08:55:32.102341 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:55:32.102398 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:55:47 -0400]
[2018-10-08 08:55:32.201143 -0400][Debug ][PostMaster ] [llrxrd-redir.in2p3.fr:1094] Found 2 address(es): [::ffff:134.158.132.31]:1094, [2001:660:302c:23:134:158:132:31]:1094
[2018-10-08 08:55:32.201226 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [2001:660:302c:23:134:158:132:31]:1094
[2018-10-08 08:55:32.201263 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
[2018-10-08 08:55:32.201281 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [::ffff:134.158.132.31]:1094
[2018-10-08 08:55:32.201336 -0400][Debug ][Poller ] Adding socket 0xe00014d0 to the poller
[2018-10-08 08:55:46.103428 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:55:46.103500 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:56:01 -0400]
[2018-10-08 08:55:46.103508 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:55:46.103516 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:56:01 -0400]
[2018-10-08 08:55:46.981402 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 15 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:55:46.981512 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 15 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:55:47.103645 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:55:47.103718 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:56:02 -0400]
[2018-10-08 08:56:01.104516 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:56:01.104604 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:56:16 -0400]
[2018-10-08 08:56:01.104617 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:56:01.104628 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:56:16 -0400]
[2018-10-08 08:56:01.996635 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 30 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:56:01.996711 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 30 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:56:02.104693 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:56:02.104750 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:56:17 -0400]
[2018-10-08 08:56:16.105797 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:56:16.105891 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:56:31 -0400]
[2018-10-08 08:56:16.105904 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:56:16.105914 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:56:31 -0400]
[2018-10-08 08:56:17.011781 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 46 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:56:17.011858 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 46 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:56:17.105982 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:56:17.106042 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:56:32 -0400]
[2018-10-08 08:56:31.107045 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:56:31.107140 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:56:46 -0400]
[2018-10-08 08:56:31.107153 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:56:31.107164 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:56:46 -0400]
[2018-10-08 08:56:32.026935 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 61 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:56:32.027014 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 61 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:56:32.107269 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:56:32.107362 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:56:47 -0400]
[2018-10-08 08:56:46.108312 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:56:46.108439 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:57:01 -0400]
[2018-10-08 08:56:46.108458 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:56:46.108470 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:57:01 -0400]
[2018-10-08 08:56:47.042094 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 76 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:56:47.042165 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 76 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:56:47.108567 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:56:47.108627 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:57:02 -0400]
[2018-10-08 08:57:01.109656 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:57:01.109743 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:57:16 -0400]
[2018-10-08 08:57:01.109756 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:57:01.109767 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:57:16 -0400]
[2018-10-08 08:57:02.057230 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 91 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:57:02.057307 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 91 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:57:02.109845 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:57:02.109903 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:57:17 -0400]
[2018-10-08 08:57:16.115139 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:57:16.115233 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:57:31 -0400]
[2018-10-08 08:57:16.115245 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:57:16.115256 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:57:31 -0400]
[2018-10-08 08:57:17.072395 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 106 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:57:17.072481 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 106 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:57:17.115348 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:57:17.115395 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:57:32 -0400]
[2018-10-08 08:57:31.116720 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:57:31.116802 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:57:46 -0400]
[2018-10-08 08:57:31.116811 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:57:31.116834 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:57:46 -0400]
[2018-10-08 08:57:32.087475 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 121 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:57:32.087539 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 121 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:57:32.116916 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:57:32.116956 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:57:47 -0400]
[2018-10-08 08:57:39.497598 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Async connection call returned
[2018-10-08 08:57:39.497687 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to connect: Connection timed out
[2018-10-08 08:57:39.497697 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Closing the socket
[2018-10-08 08:57:39.497709 -0400][Debug ][Poller ] <x><--><x> Removing socket from the poller
[2018-10-08 08:57:39.497747 -0400][Error ][PostMaster ] [llrxrd-redir.in2p3.fr:1094 #0] elapsed = 127, pConnectionWindow = 120 seconds.
[2018-10-08 08:57:39.501587 -0400][Debug ][PostMaster ] [llrxrd-redir.in2p3.fr:1094] Found 2 address(es): [::ffff:134.158.132.31]:1094, [2001:660:302c:23:134:158:132:31]:1094
[2018-10-08 08:57:39.501646 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [2001:660:302c:23:134:158:132:31]:1094
[2018-10-08 08:57:39.501678 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
[2018-10-08 08:57:39.501697 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [::ffff:134.158.132.31]:1094
[2018-10-08 08:57:39.501755 -0400][Debug ][Poller ] Adding socket 0xe00014d0 to the poller
[2018-10-08 08:57:46.118105 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:57:46.118199 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:58:01 -0400]
[2018-10-08 08:57:46.118211 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:57:46.118222 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:58:01 -0400]
[2018-10-08 08:57:47.118299 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:57:47.118352 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:58:02 -0400]
[2018-10-08 08:57:47.509869 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 136 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:57:47.509946 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 136 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:58:01.119354 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:58:01.119473 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:58:16 -0400]
[2018-10-08 08:58:01.119487 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:58:01.119497 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:58:16 -0400]
[2018-10-08 08:58:02.119594 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:58:02.119717 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:58:17 -0400]
[2018-10-08 08:58:02.525057 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 151 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:58:02.525145 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 151 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:58:16.120719 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:58:16.120811 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:58:31 -0400]
[2018-10-08 08:58:16.120824 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:58:16.120835 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:58:31 -0400]
[2018-10-08 08:58:17.120914 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:58:17.120977 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:58:32 -0400]
[2018-10-08 08:58:17.540216 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 166 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:58:17.540283 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 166 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:58:31.122100 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:58:31.122187 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:58:46 -0400]
[2018-10-08 08:58:31.122197 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:58:31.122205 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:58:46 -0400]
[2018-10-08 08:58:32.122292 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:58:32.122382 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:58:47 -0400]
[2018-10-08 08:58:32.555377 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 181 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:58:32.555462 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 181 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:58:46.123574 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:58:46.123664 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:59:01 -0400]
[2018-10-08 08:58:46.123677 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:58:46.123687 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:59:01 -0400]
[2018-10-08 08:58:47.123798 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:58:47.123860 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:59:02 -0400]
[2018-10-08 08:58:47.570573 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 196 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:58:47.570646 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 196 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:59:01.124987 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:59:01.125077 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:59:16 -0400]
[2018-10-08 08:59:01.125090 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:59:01.125101 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:59:16 -0400]
[2018-10-08 08:59:02.125225 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:59:02.125314 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:59:17 -0400]
[2018-10-08 08:59:02.585785 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 211 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:59:02.585852 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 211 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:59:16.126660 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:59:16.126759 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:59:31 -0400]
[2018-10-08 08:59:16.126772 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:59:16.126783 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:59:31 -0400]
[2018-10-08 08:59:17.126893 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:59:17.126950 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:59:32 -0400]
[2018-10-08 08:59:17.601007 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 226 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:59:17.601078 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 226 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:59:31.128343 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:59:31.128470 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:59:46 -0400]
[2018-10-08 08:59:31.128484 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:59:31.128494 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:59:46 -0400]
[2018-10-08 08:59:32.128616 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:59:32.128669 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:59:47 -0400]
[2018-10-08 08:59:32.616224 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 241 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:59:32.616291 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 241 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 08:59:46.130087 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 08:59:46.130179 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:00:01 -0400]
[2018-10-08 08:59:46.130192 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 08:59:46.130221 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:00:01 -0400]
[2018-10-08 08:59:46.729528 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Async connection call returned
[2018-10-08 08:59:46.729610 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to connect: Connection timed out
[2018-10-08 08:59:46.729621 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Closing the socket
[2018-10-08 08:59:46.729633 -0400][Debug ][Poller ] <x><--><x> Removing socket from the poller
[2018-10-08 08:59:46.729673 -0400][Error ][PostMaster ] [llrxrd-redir.in2p3.fr:1094 #0] elapsed = 127, pConnectionWindow = 120 seconds.
[2018-10-08 08:59:46.733009 -0400][Debug ][PostMaster ] [llrxrd-redir.in2p3.fr:1094] Found 2 address(es): [::ffff:134.158.132.31]:1094, [2001:660:302c:23:134:158:132:31]:1094
[2018-10-08 08:59:46.733048 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [2001:660:302c:23:134:158:132:31]:1094
[2018-10-08 08:59:46.733071 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
[2018-10-08 08:59:46.733084 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [::ffff:134.158.132.31]:1094
[2018-10-08 08:59:46.733123 -0400][Debug ][Poller ] Adding socket 0xe00014d0 to the poller
[2018-10-08 08:59:47.130334 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 08:59:47.130388 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:00:02 -0400]
[2018-10-08 08:59:47.733476 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 256 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 08:59:47.733507 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 256 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 09:00:01.131836 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 09:00:01.131924 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:00:16 -0400]
[2018-10-08 09:00:01.131936 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 09:00:01.131946 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:00:16 -0400]
[2018-10-08 09:00:02.132024 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 09:00:02.132077 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:00:17 -0400]
[2018-10-08 09:00:02.748601 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 271 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 09:00:02.748673 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 271 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 09:00:16.137566 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 09:00:16.137660 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:00:31 -0400]
[2018-10-08 09:00:16.137674 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 09:00:16.137685 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:00:31 -0400]
[2018-10-08 09:00:17.137789 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 09:00:17.137882 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:00:32 -0400]
[2018-10-08 09:00:17.763766 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 286 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 09:00:17.763831 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 286 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 09:00:31.139111 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 09:00:31.139210 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:00:46 -0400]
[2018-10-08 09:00:31.139222 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 09:00:31.139232 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:00:46 -0400]
[2018-10-08 09:00:32.139331 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 09:00:32.139430 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:00:47 -0400]
[2018-10-08 09:00:32.778911 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 301 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 09:00:32.778983 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 301 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 09:00:46.140339 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 09:00:46.140482 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:01:01 -0400]
[2018-10-08 09:00:46.140498 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 09:00:46.140510 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:01:01 -0400]
[2018-10-08 09:00:47.140624 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 09:00:47.140676 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:01:02 -0400]
[2018-10-08 09:00:47.794052 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 316 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 09:00:47.794115 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 316 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 09:01:01.141884 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 09:01:01.141979 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:01:16 -0400]
[2018-10-08 09:01:01.141993 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 09:01:01.142004 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:01:16 -0400]
[2018-10-08 09:01:02.142070 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 09:01:02.142113 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:01:17 -0400]
[2018-10-08 09:01:02.809137 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 331 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 09:01:02.809211 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 331 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
[2018-10-08 09:01:16.143427 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 09:01:16.143523 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:01:31 -0400]
[2018-10-08 09:01:16.143536 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
[2018-10-08 09:01:16.143547 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:01:31 -0400]
[2018-10-08 09:01:17.143658 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 09:01:17.143711 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:01:32 -0400]
[2018-10-08 09:01:17.824328 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 346 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 09:01:17.824447 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 346 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
```
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
#836 (comment)
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|
goofy.geop.uc.edu
…On 0, xrootd-dev ***@***.***> wrote:
Could you post the name of the host on which you are running this script?
On Mon, 8 Oct 2018, Valentin Kuznetsov wrote:
> "forever" means that I waited 20-30 min and nothing happened. Here is a log with XRD_LOGLEVEL=Dump and waiting 5 minutes:
> ```
> [2018-10-08 08:55:31.084582 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory
> [2018-10-08 08:55:31.084849 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory
> [2018-10-08 08:55:31.084965 -0400][Info ][Utility ] Env: Importing from shell XRD_NETWORKSTACK=IPAll as NetworkStack
> [2018-10-08 08:55:31.084999 -0400][Debug ][PlugInMgr ] Initializing plug-in manager...
> [2018-10-08 08:55:31.085011 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs...
> [2018-10-08 08:55:31.085019 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d...
> [2018-10-08 08:55:31.085036 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory
> [2018-10-08 08:55:31.085106 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /home/kuznetv/.xrootd/client.plugins.d...
> [2018-10-08 08:55:31.085158 -0400][Dump ][PlugInMgr ] Processing: /home/kuznetv/.xrootd/client.plugins.d/client-plugin.conf
> [2018-10-08 08:55:31.085210 -0400][Debug ][PlugInMgr ] Unable to find 'lib' key in the config file /home/kuznetv/.xrootd/client.plugins.d/client-plugin.conf, ignoring this config
> [2018-10-08 08:55:31.101915 -0400][Debug ][Poller ] Available pollers: built-in
> [2018-10-08 08:55:31.101948 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in
> [2018-10-08 08:55:31.101960 -0400][Debug ][Poller ] Creating poller: built-in
> [2018-10-08 08:55:31.101973 -0400][Debug ][Poller ] Creating and starting the built-in poller...
> [2018-10-08 08:55:31.102147 -0400][Debug ][Poller ] Using 1 poller threads
> [2018-10-08 08:55:31.102166 -0400][Debug ][TaskMgr ] Starting the task manager...
> [2018-10-08 08:55:31.102205 -0400][Debug ][TaskMgr ] Task manager started
> [2018-10-08 08:55:31.102215 -0400][Debug ][JobMgr ] Starting the job manager...
> [2018-10-08 08:55:31.102287 -0400][Debug ][JobMgr ] Job manager started, 3 workers
> [2018-10-08 08:55:31.102298 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2018-10-08 08:55:31 -0400]
> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] URL: root://cms-xrd-global.cern.ch//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Protocol: root
> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] User Name:
> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Password:
> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Host Name: cms-xrd-global.cern.ch
> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Port: 1094
> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Path: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] URL: root://cms-xrd-global.cern.ch//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Protocol: root
> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] User Name:
> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Password:
> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Host Name: cms-xrd-global.cern.ch
> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Port: 1094
> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Path: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
> [2018-10-08 08:55:31.102462 -0400][Debug ][File ] ***@***.***://cms-xrd-global.cern.ch:1094//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root] Sending an open command
> [2018-10-08 08:55:31.102512 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Sending message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat )
> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] URL: cms-xrd-global.cern.ch:1094
> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Protocol: root
> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] User Name:
> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Password:
> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Host Name: cms-xrd-global.cern.ch
> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Port: 1094
> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Path:
> [2018-10-08 08:55:31.102577 -0400][Debug ][PostMaster ] Creating new channel to: cms-xrd-global.cern.ch:1094 1 stream(s)
> [2018-10-08 08:55:31.102605 -0400][Debug ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Stream parameters: Network Stack: IPAll, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800
> [2018-10-08 08:55:31.107979 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" to be run at: [2018-10-08 08:55:46 -0400]
> [2018-10-08 08:55:31.108022 -0400][Dump ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Sending message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xfd286c60) through substream 0 expecting answer at 0
> [2018-10-08 08:55:31.341022 -0400][Debug ][PostMaster ] [cms-xrd-global.cern.ch:1094] Found 4 address(es): [::ffff:188.184.161.66]:1094, [::ffff:188.184.67.41]:1094, [2001:1458:301:69::100:3c]:1094, [2001:1458:201:a4::100:323]:1094
> [2018-10-08 08:55:31.341070 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Attempting connection to [2001:1458:201:a4::100:323]:1094
> [2018-10-08 08:55:31.341102 -0400][Error ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
> [2018-10-08 08:55:31.341120 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Attempting connection to [2001:1458:301:69::100:3c]:1094
> [2018-10-08 08:55:31.341136 -0400][Error ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
> [2018-10-08 08:55:31.341151 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Attempting connection to [::ffff:188.184.67.41]:1094
> [2018-10-08 08:55:31.341196 -0400][Debug ][Poller ] Adding socket 0xfd279890 to the poller
> [2018-10-08 08:55:31.452359 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Async connection call returned
> [2018-10-08 08:55:31.452420 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Sending out the initial hand shake + kXR_protocol
> [2018-10-08 08:55:31.452472 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Wrote a message: (0xe8000950), 44 bytes
> [2018-10-08 08:55:31.679451 -0400][Dump ][XRootDTransport ] [msg: 0xe8000950] Expecting 8 bytes of message body
> [2018-10-08 08:55:31.679483 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header, size: 8
> [2018-10-08 08:55:31.679493 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received a message of 16 bytes
> [2018-10-08 08:55:31.679508 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 310)
> [2018-10-08 08:55:31.679522 -0400][Dump ][XRootDTransport ] [msg: 0xe8000950] Expecting 8 bytes of message body
> [2018-10-08 08:55:31.679529 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header, size: 8
> [2018-10-08 08:55:31.679548 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received a message of 16 bytes
> [2018-10-08 08:55:31.679558 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] kXR_protocol successful (type: manager [meta], protocol version 310)
> [2018-10-08 08:55:31.692590 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Sending out kXR_login request, username: kuznetv, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=python3.6&xrd.info=&xrd.hostname=goofy&xrd.rn=v4.8.4, dual-stack: true, private IPv4: false, private IPv6: true
> [2018-10-08 08:55:31.692667 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Wrote a message: (0xe8000b40), 109 bytes
> [2018-10-08 08:55:31.803724 -0400][Dump ][XRootDTransport ] [msg: 0xe8000b40] Expecting 16 bytes of message body
> [2018-10-08 08:55:31.803757 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header, size: 8
> [2018-10-08 08:55:31.803769 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received a message of 24 bytes
> [2018-10-08 08:55:31.803783 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Logged in, session: 79ef03008ff50100bb01000033f10300
> [2018-10-08 08:55:31.803796 -0400][Debug ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Stream 0 connected.
> [2018-10-08 08:55:31.803807 -0400][Debug ][Utility ] Monitor library name not set. No monitoring
> [2018-10-08 08:55:31.803848 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Wrote a message: kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xfd286c60), 122 bytes
> [2018-10-08 08:55:31.803861 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Successfully sent message: kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xfd286c60).
> [2018-10-08 08:55:31.803879 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) has been successfully sent.
> [2018-10-08 08:55:31.803889 -0400][Dump ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0.0] All messages consumed, disable uplink
> [2018-10-08 08:55:31.966285 -0400][Dump ][XRootDTransport ] [msg: 0xe8000b70] Expecting 25 bytes of message body
> [2018-10-08 08:55:31.966318 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header for 0xe8000b70 size: 8
> [2018-10-08 08:55:31.966337 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message 0xe8000b70 of 33 bytes
> [2018-10-08 08:55:31.966347 -0400][Dump ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Handling received message: 0xe8000b70.
> [2018-10-08 08:55:31.966515 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Got kXR_redirect response to message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): llrxrd-redir.in2p3.fr, port 1094
> [2018-10-08 08:55:31.966582 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Current server has been assigned as a load-balancer for message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat )
> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] URL: llrxrd-redir.in2p3.fr:1094/
> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Protocol: root
> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] User Name:
> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Password:
> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Host Name: llrxrd-redir.in2p3.fr
> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Port: 1094
> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Path:
> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] URL: llrxrd-redir.in2p3.fr:1094
> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Protocol: root
> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] User Name:
> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Password:
> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Host Name: llrxrd-redir.in2p3.fr
> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Port: 1094
> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Path:
> [2018-10-08 08:55:31.966725 -0400][Debug ][PostMaster ] Creating new channel to: llrxrd-redir.in2p3.fr:1094 1 stream(s)
> [2018-10-08 08:55:31.966755 -0400][Debug ][PostMaster ] [llrxrd-redir.in2p3.fr:1094 #0] Stream parameters: Network Stack: IPAll, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800
> [2018-10-08 08:55:31.976328 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" to be run at: [2018-10-08 08:55:46 -0400]
> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] URL: fake://fake:111//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Protocol: fake
> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] User Name:
> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Password:
> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Host Name: fake
> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Port: 111
> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Path: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
> [2018-10-08 08:55:31.976469 -0400][Dump ][PostMaster ] [llrxrd-redir.in2p3.fr:1094 #0] Sending message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xfd286c60) through substream 0 expecting answer at 0
> [2018-10-08 08:55:32.102341 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:55:32.102398 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:55:47 -0400]
> [2018-10-08 08:55:32.201143 -0400][Debug ][PostMaster ] [llrxrd-redir.in2p3.fr:1094] Found 2 address(es): [::ffff:134.158.132.31]:1094, [2001:660:302c:23:134:158:132:31]:1094
> [2018-10-08 08:55:32.201226 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [2001:660:302c:23:134:158:132:31]:1094
> [2018-10-08 08:55:32.201263 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
> [2018-10-08 08:55:32.201281 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [::ffff:134.158.132.31]:1094
> [2018-10-08 08:55:32.201336 -0400][Debug ][Poller ] Adding socket 0xe00014d0 to the poller
> [2018-10-08 08:55:46.103428 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:55:46.103500 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:56:01 -0400]
> [2018-10-08 08:55:46.103508 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:55:46.103516 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:56:01 -0400]
> [2018-10-08 08:55:46.981402 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 15 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:55:46.981512 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 15 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:55:47.103645 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:55:47.103718 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:56:02 -0400]
> [2018-10-08 08:56:01.104516 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:56:01.104604 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:56:16 -0400]
> [2018-10-08 08:56:01.104617 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:56:01.104628 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:56:16 -0400]
> [2018-10-08 08:56:01.996635 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 30 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:56:01.996711 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 30 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:56:02.104693 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:56:02.104750 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:56:17 -0400]
> [2018-10-08 08:56:16.105797 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:56:16.105891 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:56:31 -0400]
> [2018-10-08 08:56:16.105904 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:56:16.105914 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:56:31 -0400]
> [2018-10-08 08:56:17.011781 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 46 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:56:17.011858 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 46 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:56:17.105982 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:56:17.106042 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:56:32 -0400]
> [2018-10-08 08:56:31.107045 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:56:31.107140 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:56:46 -0400]
> [2018-10-08 08:56:31.107153 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:56:31.107164 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:56:46 -0400]
> [2018-10-08 08:56:32.026935 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 61 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:56:32.027014 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 61 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:56:32.107269 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:56:32.107362 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:56:47 -0400]
> [2018-10-08 08:56:46.108312 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:56:46.108439 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:57:01 -0400]
> [2018-10-08 08:56:46.108458 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:56:46.108470 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:57:01 -0400]
> [2018-10-08 08:56:47.042094 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 76 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:56:47.042165 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 76 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:56:47.108567 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:56:47.108627 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:57:02 -0400]
> [2018-10-08 08:57:01.109656 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:57:01.109743 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:57:16 -0400]
> [2018-10-08 08:57:01.109756 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:57:01.109767 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:57:16 -0400]
> [2018-10-08 08:57:02.057230 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 91 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:57:02.057307 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 91 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:57:02.109845 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:57:02.109903 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:57:17 -0400]
> [2018-10-08 08:57:16.115139 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:57:16.115233 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:57:31 -0400]
> [2018-10-08 08:57:16.115245 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:57:16.115256 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:57:31 -0400]
> [2018-10-08 08:57:17.072395 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 106 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:57:17.072481 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 106 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:57:17.115348 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:57:17.115395 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:57:32 -0400]
> [2018-10-08 08:57:31.116720 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:57:31.116802 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:57:46 -0400]
> [2018-10-08 08:57:31.116811 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:57:31.116834 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:57:46 -0400]
> [2018-10-08 08:57:32.087475 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 121 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:57:32.087539 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 121 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:57:32.116916 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:57:32.116956 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:57:47 -0400]
> [2018-10-08 08:57:39.497598 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Async connection call returned
> [2018-10-08 08:57:39.497687 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to connect: Connection timed out
> [2018-10-08 08:57:39.497697 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Closing the socket
> [2018-10-08 08:57:39.497709 -0400][Debug ][Poller ] <x><--><x> Removing socket from the poller
> [2018-10-08 08:57:39.497747 -0400][Error ][PostMaster ] [llrxrd-redir.in2p3.fr:1094 #0] elapsed = 127, pConnectionWindow = 120 seconds.
> [2018-10-08 08:57:39.501587 -0400][Debug ][PostMaster ] [llrxrd-redir.in2p3.fr:1094] Found 2 address(es): [::ffff:134.158.132.31]:1094, [2001:660:302c:23:134:158:132:31]:1094
> [2018-10-08 08:57:39.501646 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [2001:660:302c:23:134:158:132:31]:1094
> [2018-10-08 08:57:39.501678 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
> [2018-10-08 08:57:39.501697 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [::ffff:134.158.132.31]:1094
> [2018-10-08 08:57:39.501755 -0400][Debug ][Poller ] Adding socket 0xe00014d0 to the poller
> [2018-10-08 08:57:46.118105 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:57:46.118199 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:58:01 -0400]
> [2018-10-08 08:57:46.118211 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:57:46.118222 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:58:01 -0400]
> [2018-10-08 08:57:47.118299 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:57:47.118352 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:58:02 -0400]
> [2018-10-08 08:57:47.509869 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 136 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:57:47.509946 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 136 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:58:01.119354 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:58:01.119473 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:58:16 -0400]
> [2018-10-08 08:58:01.119487 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:58:01.119497 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:58:16 -0400]
> [2018-10-08 08:58:02.119594 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:58:02.119717 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:58:17 -0400]
> [2018-10-08 08:58:02.525057 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 151 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:58:02.525145 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 151 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:58:16.120719 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:58:16.120811 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:58:31 -0400]
> [2018-10-08 08:58:16.120824 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:58:16.120835 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:58:31 -0400]
> [2018-10-08 08:58:17.120914 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:58:17.120977 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:58:32 -0400]
> [2018-10-08 08:58:17.540216 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 166 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:58:17.540283 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 166 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:58:31.122100 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:58:31.122187 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:58:46 -0400]
> [2018-10-08 08:58:31.122197 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:58:31.122205 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:58:46 -0400]
> [2018-10-08 08:58:32.122292 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:58:32.122382 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:58:47 -0400]
> [2018-10-08 08:58:32.555377 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 181 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:58:32.555462 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 181 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:58:46.123574 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:58:46.123664 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:59:01 -0400]
> [2018-10-08 08:58:46.123677 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:58:46.123687 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:59:01 -0400]
> [2018-10-08 08:58:47.123798 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:58:47.123860 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:59:02 -0400]
> [2018-10-08 08:58:47.570573 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 196 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:58:47.570646 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 196 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:59:01.124987 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:59:01.125077 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:59:16 -0400]
> [2018-10-08 08:59:01.125090 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:59:01.125101 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:59:16 -0400]
> [2018-10-08 08:59:02.125225 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:59:02.125314 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:59:17 -0400]
> [2018-10-08 08:59:02.585785 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 211 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:59:02.585852 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 211 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:59:16.126660 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:59:16.126759 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:59:31 -0400]
> [2018-10-08 08:59:16.126772 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:59:16.126783 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:59:31 -0400]
> [2018-10-08 08:59:17.126893 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:59:17.126950 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:59:32 -0400]
> [2018-10-08 08:59:17.601007 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 226 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:59:17.601078 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 226 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:59:31.128343 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:59:31.128470 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:59:46 -0400]
> [2018-10-08 08:59:31.128484 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:59:31.128494 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:59:46 -0400]
> [2018-10-08 08:59:32.128616 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:59:32.128669 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:59:47 -0400]
> [2018-10-08 08:59:32.616224 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 241 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:59:32.616291 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 241 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 08:59:46.130087 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 08:59:46.130179 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:00:01 -0400]
> [2018-10-08 08:59:46.130192 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 08:59:46.130221 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:00:01 -0400]
> [2018-10-08 08:59:46.729528 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Async connection call returned
> [2018-10-08 08:59:46.729610 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to connect: Connection timed out
> [2018-10-08 08:59:46.729621 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Closing the socket
> [2018-10-08 08:59:46.729633 -0400][Debug ][Poller ] <x><--><x> Removing socket from the poller
> [2018-10-08 08:59:46.729673 -0400][Error ][PostMaster ] [llrxrd-redir.in2p3.fr:1094 #0] elapsed = 127, pConnectionWindow = 120 seconds.
> [2018-10-08 08:59:46.733009 -0400][Debug ][PostMaster ] [llrxrd-redir.in2p3.fr:1094] Found 2 address(es): [::ffff:134.158.132.31]:1094, [2001:660:302c:23:134:158:132:31]:1094
> [2018-10-08 08:59:46.733048 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [2001:660:302c:23:134:158:132:31]:1094
> [2018-10-08 08:59:46.733071 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
> [2018-10-08 08:59:46.733084 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [::ffff:134.158.132.31]:1094
> [2018-10-08 08:59:46.733123 -0400][Debug ][Poller ] Adding socket 0xe00014d0 to the poller
> [2018-10-08 08:59:47.130334 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 08:59:47.130388 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:00:02 -0400]
> [2018-10-08 08:59:47.733476 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 256 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 08:59:47.733507 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 256 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 09:00:01.131836 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 09:00:01.131924 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:00:16 -0400]
> [2018-10-08 09:00:01.131936 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 09:00:01.131946 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:00:16 -0400]
> [2018-10-08 09:00:02.132024 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 09:00:02.132077 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:00:17 -0400]
> [2018-10-08 09:00:02.748601 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 271 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 09:00:02.748673 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 271 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 09:00:16.137566 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 09:00:16.137660 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:00:31 -0400]
> [2018-10-08 09:00:16.137674 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 09:00:16.137685 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:00:31 -0400]
> [2018-10-08 09:00:17.137789 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 09:00:17.137882 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:00:32 -0400]
> [2018-10-08 09:00:17.763766 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 286 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 09:00:17.763831 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 286 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 09:00:31.139111 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 09:00:31.139210 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:00:46 -0400]
> [2018-10-08 09:00:31.139222 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 09:00:31.139232 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:00:46 -0400]
> [2018-10-08 09:00:32.139331 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 09:00:32.139430 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:00:47 -0400]
> [2018-10-08 09:00:32.778911 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 301 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 09:00:32.778983 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 301 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 09:00:46.140339 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 09:00:46.140482 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:01:01 -0400]
> [2018-10-08 09:00:46.140498 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 09:00:46.140510 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:01:01 -0400]
> [2018-10-08 09:00:47.140624 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 09:00:47.140676 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:01:02 -0400]
> [2018-10-08 09:00:47.794052 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 316 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 09:00:47.794115 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 316 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 09:01:01.141884 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 09:01:01.141979 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:01:16 -0400]
> [2018-10-08 09:01:01.141993 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 09:01:01.142004 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:01:16 -0400]
> [2018-10-08 09:01:02.142070 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 09:01:02.142113 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:01:17 -0400]
> [2018-10-08 09:01:02.809137 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 331 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 09:01:02.809211 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 331 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
> [2018-10-08 09:01:16.143427 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
> [2018-10-08 09:01:16.143523 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:01:31 -0400]
> [2018-10-08 09:01:16.143536 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
> [2018-10-08 09:01:16.143547 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:01:31 -0400]
> [2018-10-08 09:01:17.143658 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
> [2018-10-08 09:01:17.143711 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:01:32 -0400]
> [2018-10-08 09:01:17.824328 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 346 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
> [2018-10-08 09:01:17.824447 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 346 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>
> ```
>
> --
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly or view it on GitHub:
> #836 (comment)
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-DEV list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#836 (comment)
|
OK, so your node is not IPv6 capable. I will have to ask our client
developer why he's trying to use IPv6 addresses. Normallyy, DNS shouldn't
return them but I suppose we should check that we actually can use IPv6
regardless of the DNS returns. In the mean time your solution is to set
the following envar:
XRD_NETWORKSTACK IPv4
Let us know if that works for you.
Andy
…On Mon, 8 Oct 2018, Valentin Kuznetsov wrote:
goofy.geop.uc.edu
On 0, xrootd-dev ***@***.***> wrote:
> Could you post the name of the host on which you are running this script?
>
> On Mon, 8 Oct 2018, Valentin Kuznetsov wrote:
>
>> "forever" means that I waited 20-30 min and nothing happened. Here is a log with XRD_LOGLEVEL=Dump and waiting 5 minutes:
>> ```
>> [2018-10-08 08:55:31.084582 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory
>> [2018-10-08 08:55:31.084849 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory
>> [2018-10-08 08:55:31.084965 -0400][Info ][Utility ] Env: Importing from shell XRD_NETWORKSTACK=IPAll as NetworkStack
>> [2018-10-08 08:55:31.084999 -0400][Debug ][PlugInMgr ] Initializing plug-in manager...
>> [2018-10-08 08:55:31.085011 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs...
>> [2018-10-08 08:55:31.085019 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d...
>> [2018-10-08 08:55:31.085036 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory
>> [2018-10-08 08:55:31.085106 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /home/kuznetv/.xrootd/client.plugins.d...
>> [2018-10-08 08:55:31.085158 -0400][Dump ][PlugInMgr ] Processing: /home/kuznetv/.xrootd/client.plugins.d/client-plugin.conf
>> [2018-10-08 08:55:31.085210 -0400][Debug ][PlugInMgr ] Unable to find 'lib' key in the config file /home/kuznetv/.xrootd/client.plugins.d/client-plugin.conf, ignoring this config
>> [2018-10-08 08:55:31.101915 -0400][Debug ][Poller ] Available pollers: built-in
>> [2018-10-08 08:55:31.101948 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in
>> [2018-10-08 08:55:31.101960 -0400][Debug ][Poller ] Creating poller: built-in
>> [2018-10-08 08:55:31.101973 -0400][Debug ][Poller ] Creating and starting the built-in poller...
>> [2018-10-08 08:55:31.102147 -0400][Debug ][Poller ] Using 1 poller threads
>> [2018-10-08 08:55:31.102166 -0400][Debug ][TaskMgr ] Starting the task manager...
>> [2018-10-08 08:55:31.102205 -0400][Debug ][TaskMgr ] Task manager started
>> [2018-10-08 08:55:31.102215 -0400][Debug ][JobMgr ] Starting the job manager...
>> [2018-10-08 08:55:31.102287 -0400][Debug ][JobMgr ] Job manager started, 3 workers
>> [2018-10-08 08:55:31.102298 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2018-10-08 08:55:31 -0400]
>> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] URL: root://cms-xrd-global.cern.ch//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
>> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Protocol: root
>> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] User Name:
>> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Password:
>> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Host Name: cms-xrd-global.cern.ch
>> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Port: 1094
>> [2018-10-08 08:55:31.102380 -0400][Dump ][Utility ] Path: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
>> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] URL: root://cms-xrd-global.cern.ch//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
>> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Protocol: root
>> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] User Name:
>> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Password:
>> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Host Name: cms-xrd-global.cern.ch
>> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Port: 1094
>> [2018-10-08 08:55:31.102436 -0400][Dump ][Utility ] Path: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
>> [2018-10-08 08:55:31.102462 -0400][Debug ][File ] ***@***.***://cms-xrd-global.cern.ch:1094//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root] Sending an open command
>> [2018-10-08 08:55:31.102512 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Sending message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat )
>> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] URL: cms-xrd-global.cern.ch:1094
>> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Protocol: root
>> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] User Name:
>> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Password:
>> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Host Name: cms-xrd-global.cern.ch
>> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Port: 1094
>> [2018-10-08 08:55:31.102550 -0400][Dump ][Utility ] Path:
>> [2018-10-08 08:55:31.102577 -0400][Debug ][PostMaster ] Creating new channel to: cms-xrd-global.cern.ch:1094 1 stream(s)
>> [2018-10-08 08:55:31.102605 -0400][Debug ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Stream parameters: Network Stack: IPAll, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800
>> [2018-10-08 08:55:31.107979 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" to be run at: [2018-10-08 08:55:46 -0400]
>> [2018-10-08 08:55:31.108022 -0400][Dump ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Sending message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xfd286c60) through substream 0 expecting answer at 0
>> [2018-10-08 08:55:31.341022 -0400][Debug ][PostMaster ] [cms-xrd-global.cern.ch:1094] Found 4 address(es): [::ffff:188.184.161.66]:1094, [::ffff:188.184.67.41]:1094, [2001:1458:301:69::100:3c]:1094, [2001:1458:201:a4::100:323]:1094
>> [2018-10-08 08:55:31.341070 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Attempting connection to [2001:1458:201:a4::100:323]:1094
>> [2018-10-08 08:55:31.341102 -0400][Error ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
>> [2018-10-08 08:55:31.341120 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Attempting connection to [2001:1458:301:69::100:3c]:1094
>> [2018-10-08 08:55:31.341136 -0400][Error ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
>> [2018-10-08 08:55:31.341151 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Attempting connection to [::ffff:188.184.67.41]:1094
>> [2018-10-08 08:55:31.341196 -0400][Debug ][Poller ] Adding socket 0xfd279890 to the poller
>> [2018-10-08 08:55:31.452359 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Async connection call returned
>> [2018-10-08 08:55:31.452420 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Sending out the initial hand shake + kXR_protocol
>> [2018-10-08 08:55:31.452472 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Wrote a message: (0xe8000950), 44 bytes
>> [2018-10-08 08:55:31.679451 -0400][Dump ][XRootDTransport ] [msg: 0xe8000950] Expecting 8 bytes of message body
>> [2018-10-08 08:55:31.679483 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header, size: 8
>> [2018-10-08 08:55:31.679493 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received a message of 16 bytes
>> [2018-10-08 08:55:31.679508 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 310)
>> [2018-10-08 08:55:31.679522 -0400][Dump ][XRootDTransport ] [msg: 0xe8000950] Expecting 8 bytes of message body
>> [2018-10-08 08:55:31.679529 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header, size: 8
>> [2018-10-08 08:55:31.679548 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received a message of 16 bytes
>> [2018-10-08 08:55:31.679558 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] kXR_protocol successful (type: manager [meta], protocol version 310)
>> [2018-10-08 08:55:31.692590 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Sending out kXR_login request, username: kuznetv, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=python3.6&xrd.info=&xrd.hostname=goofy&xrd.rn=v4.8.4, dual-stack: true, private IPv4: false, private IPv6: true
>> [2018-10-08 08:55:31.692667 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Wrote a message: (0xe8000b40), 109 bytes
>> [2018-10-08 08:55:31.803724 -0400][Dump ][XRootDTransport ] [msg: 0xe8000b40] Expecting 16 bytes of message body
>> [2018-10-08 08:55:31.803757 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header, size: 8
>> [2018-10-08 08:55:31.803769 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received a message of 24 bytes
>> [2018-10-08 08:55:31.803783 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Logged in, session: 79ef03008ff50100bb01000033f10300
>> [2018-10-08 08:55:31.803796 -0400][Debug ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Stream 0 connected.
>> [2018-10-08 08:55:31.803807 -0400][Debug ][Utility ] Monitor library name not set. No monitoring
>> [2018-10-08 08:55:31.803848 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Wrote a message: kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xfd286c60), 122 bytes
>> [2018-10-08 08:55:31.803861 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Successfully sent message: kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xfd286c60).
>> [2018-10-08 08:55:31.803879 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) has been successfully sent.
>> [2018-10-08 08:55:31.803889 -0400][Dump ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0.0] All messages consumed, disable uplink
>> [2018-10-08 08:55:31.966285 -0400][Dump ][XRootDTransport ] [msg: 0xe8000b70] Expecting 25 bytes of message body
>> [2018-10-08 08:55:31.966318 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header for 0xe8000b70 size: 8
>> [2018-10-08 08:55:31.966337 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message 0xe8000b70 of 33 bytes
>> [2018-10-08 08:55:31.966347 -0400][Dump ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Handling received message: 0xe8000b70.
>> [2018-10-08 08:55:31.966515 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Got kXR_redirect response to message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): llrxrd-redir.in2p3.fr, port 1094
>> [2018-10-08 08:55:31.966582 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Current server has been assigned as a load-balancer for message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat )
>> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] URL: llrxrd-redir.in2p3.fr:1094/
>> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Protocol: root
>> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] User Name:
>> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Password:
>> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Host Name: llrxrd-redir.in2p3.fr
>> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Port: 1094
>> [2018-10-08 08:55:31.966630 -0400][Dump ][Utility ] Path:
>> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] URL: llrxrd-redir.in2p3.fr:1094
>> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Protocol: root
>> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] User Name:
>> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Password:
>> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Host Name: llrxrd-redir.in2p3.fr
>> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Port: 1094
>> [2018-10-08 08:55:31.966697 -0400][Dump ][Utility ] Path:
>> [2018-10-08 08:55:31.966725 -0400][Debug ][PostMaster ] Creating new channel to: llrxrd-redir.in2p3.fr:1094 1 stream(s)
>> [2018-10-08 08:55:31.966755 -0400][Debug ][PostMaster ] [llrxrd-redir.in2p3.fr:1094 #0] Stream parameters: Network Stack: IPAll, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800
>> [2018-10-08 08:55:31.976328 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" to be run at: [2018-10-08 08:55:46 -0400]
>> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] URL: fake://fake:111//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
>> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Protocol: fake
>> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] User Name:
>> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Password:
>> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Host Name: fake
>> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Port: 111
>> [2018-10-08 08:55:31.976394 -0400][Dump ][Utility ] Path: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
>> [2018-10-08 08:55:31.976469 -0400][Dump ][PostMaster ] [llrxrd-redir.in2p3.fr:1094 #0] Sending message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xfd286c60) through substream 0 expecting answer at 0
>> [2018-10-08 08:55:32.102341 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:55:32.102398 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:55:47 -0400]
>> [2018-10-08 08:55:32.201143 -0400][Debug ][PostMaster ] [llrxrd-redir.in2p3.fr:1094] Found 2 address(es): [::ffff:134.158.132.31]:1094, [2001:660:302c:23:134:158:132:31]:1094
>> [2018-10-08 08:55:32.201226 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [2001:660:302c:23:134:158:132:31]:1094
>> [2018-10-08 08:55:32.201263 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
>> [2018-10-08 08:55:32.201281 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [::ffff:134.158.132.31]:1094
>> [2018-10-08 08:55:32.201336 -0400][Debug ][Poller ] Adding socket 0xe00014d0 to the poller
>> [2018-10-08 08:55:46.103428 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:55:46.103500 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:56:01 -0400]
>> [2018-10-08 08:55:46.103508 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:55:46.103516 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:56:01 -0400]
>> [2018-10-08 08:55:46.981402 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 15 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:55:46.981512 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 15 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:55:47.103645 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:55:47.103718 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:56:02 -0400]
>> [2018-10-08 08:56:01.104516 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:56:01.104604 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:56:16 -0400]
>> [2018-10-08 08:56:01.104617 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:56:01.104628 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:56:16 -0400]
>> [2018-10-08 08:56:01.996635 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 30 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:56:01.996711 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 30 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:56:02.104693 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:56:02.104750 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:56:17 -0400]
>> [2018-10-08 08:56:16.105797 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:56:16.105891 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:56:31 -0400]
>> [2018-10-08 08:56:16.105904 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:56:16.105914 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:56:31 -0400]
>> [2018-10-08 08:56:17.011781 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 46 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:56:17.011858 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 46 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:56:17.105982 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:56:17.106042 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:56:32 -0400]
>> [2018-10-08 08:56:31.107045 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:56:31.107140 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:56:46 -0400]
>> [2018-10-08 08:56:31.107153 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:56:31.107164 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:56:46 -0400]
>> [2018-10-08 08:56:32.026935 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 61 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:56:32.027014 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 61 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:56:32.107269 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:56:32.107362 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:56:47 -0400]
>> [2018-10-08 08:56:46.108312 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:56:46.108439 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:57:01 -0400]
>> [2018-10-08 08:56:46.108458 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:56:46.108470 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:57:01 -0400]
>> [2018-10-08 08:56:47.042094 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 76 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:56:47.042165 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 76 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:56:47.108567 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:56:47.108627 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:57:02 -0400]
>> [2018-10-08 08:57:01.109656 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:57:01.109743 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:57:16 -0400]
>> [2018-10-08 08:57:01.109756 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:57:01.109767 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:57:16 -0400]
>> [2018-10-08 08:57:02.057230 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 91 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:57:02.057307 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 91 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:57:02.109845 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:57:02.109903 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:57:17 -0400]
>> [2018-10-08 08:57:16.115139 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:57:16.115233 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:57:31 -0400]
>> [2018-10-08 08:57:16.115245 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:57:16.115256 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:57:31 -0400]
>> [2018-10-08 08:57:17.072395 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 106 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:57:17.072481 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 106 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:57:17.115348 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:57:17.115395 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:57:32 -0400]
>> [2018-10-08 08:57:31.116720 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:57:31.116802 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:57:46 -0400]
>> [2018-10-08 08:57:31.116811 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:57:31.116834 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:57:46 -0400]
>> [2018-10-08 08:57:32.087475 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 121 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:57:32.087539 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 121 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:57:32.116916 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:57:32.116956 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:57:47 -0400]
>> [2018-10-08 08:57:39.497598 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Async connection call returned
>> [2018-10-08 08:57:39.497687 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to connect: Connection timed out
>> [2018-10-08 08:57:39.497697 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Closing the socket
>> [2018-10-08 08:57:39.497709 -0400][Debug ][Poller ] <x><--><x> Removing socket from the poller
>> [2018-10-08 08:57:39.497747 -0400][Error ][PostMaster ] [llrxrd-redir.in2p3.fr:1094 #0] elapsed = 127, pConnectionWindow = 120 seconds.
>> [2018-10-08 08:57:39.501587 -0400][Debug ][PostMaster ] [llrxrd-redir.in2p3.fr:1094] Found 2 address(es): [::ffff:134.158.132.31]:1094, [2001:660:302c:23:134:158:132:31]:1094
>> [2018-10-08 08:57:39.501646 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [2001:660:302c:23:134:158:132:31]:1094
>> [2018-10-08 08:57:39.501678 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
>> [2018-10-08 08:57:39.501697 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [::ffff:134.158.132.31]:1094
>> [2018-10-08 08:57:39.501755 -0400][Debug ][Poller ] Adding socket 0xe00014d0 to the poller
>> [2018-10-08 08:57:46.118105 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:57:46.118199 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:58:01 -0400]
>> [2018-10-08 08:57:46.118211 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:57:46.118222 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:58:01 -0400]
>> [2018-10-08 08:57:47.118299 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:57:47.118352 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:58:02 -0400]
>> [2018-10-08 08:57:47.509869 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 136 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:57:47.509946 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 136 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:58:01.119354 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:58:01.119473 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:58:16 -0400]
>> [2018-10-08 08:58:01.119487 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:58:01.119497 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:58:16 -0400]
>> [2018-10-08 08:58:02.119594 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:58:02.119717 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:58:17 -0400]
>> [2018-10-08 08:58:02.525057 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 151 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:58:02.525145 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 151 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:58:16.120719 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:58:16.120811 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:58:31 -0400]
>> [2018-10-08 08:58:16.120824 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:58:16.120835 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:58:31 -0400]
>> [2018-10-08 08:58:17.120914 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:58:17.120977 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:58:32 -0400]
>> [2018-10-08 08:58:17.540216 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 166 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:58:17.540283 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 166 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:58:31.122100 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:58:31.122187 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:58:46 -0400]
>> [2018-10-08 08:58:31.122197 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:58:31.122205 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:58:46 -0400]
>> [2018-10-08 08:58:32.122292 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:58:32.122382 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:58:47 -0400]
>> [2018-10-08 08:58:32.555377 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 181 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:58:32.555462 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 181 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:58:46.123574 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:58:46.123664 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:59:01 -0400]
>> [2018-10-08 08:58:46.123677 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:58:46.123687 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:59:01 -0400]
>> [2018-10-08 08:58:47.123798 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:58:47.123860 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:59:02 -0400]
>> [2018-10-08 08:58:47.570573 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 196 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:58:47.570646 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 196 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:59:01.124987 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:59:01.125077 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:59:16 -0400]
>> [2018-10-08 08:59:01.125090 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:59:01.125101 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:59:16 -0400]
>> [2018-10-08 08:59:02.125225 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:59:02.125314 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:59:17 -0400]
>> [2018-10-08 08:59:02.585785 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 211 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:59:02.585852 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 211 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:59:16.126660 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:59:16.126759 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:59:31 -0400]
>> [2018-10-08 08:59:16.126772 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:59:16.126783 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:59:31 -0400]
>> [2018-10-08 08:59:17.126893 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:59:17.126950 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:59:32 -0400]
>> [2018-10-08 08:59:17.601007 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 226 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:59:17.601078 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 226 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:59:31.128343 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:59:31.128470 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 08:59:46 -0400]
>> [2018-10-08 08:59:31.128484 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:59:31.128494 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 08:59:46 -0400]
>> [2018-10-08 08:59:32.128616 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:59:32.128669 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 08:59:47 -0400]
>> [2018-10-08 08:59:32.616224 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 241 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:59:32.616291 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 241 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 08:59:46.130087 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 08:59:46.130179 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:00:01 -0400]
>> [2018-10-08 08:59:46.130192 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 08:59:46.130221 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:00:01 -0400]
>> [2018-10-08 08:59:46.729528 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Async connection call returned
>> [2018-10-08 08:59:46.729610 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to connect: Connection timed out
>> [2018-10-08 08:59:46.729621 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Closing the socket
>> [2018-10-08 08:59:46.729633 -0400][Debug ][Poller ] <x><--><x> Removing socket from the poller
>> [2018-10-08 08:59:46.729673 -0400][Error ][PostMaster ] [llrxrd-redir.in2p3.fr:1094 #0] elapsed = 127, pConnectionWindow = 120 seconds.
>> [2018-10-08 08:59:46.733009 -0400][Debug ][PostMaster ] [llrxrd-redir.in2p3.fr:1094] Found 2 address(es): [::ffff:134.158.132.31]:1094, [2001:660:302c:23:134:158:132:31]:1094
>> [2018-10-08 08:59:46.733048 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [2001:660:302c:23:134:158:132:31]:1094
>> [2018-10-08 08:59:46.733071 -0400][Error ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Unable to initiate the connection: [ERROR] Socket error: Network is unreachable
>> [2018-10-08 08:59:46.733084 -0400][Debug ][AsyncSock ] [llrxrd-redir.in2p3.fr:1094 #0.0] Attempting connection to [::ffff:134.158.132.31]:1094
>> [2018-10-08 08:59:46.733123 -0400][Debug ][Poller ] Adding socket 0xe00014d0 to the poller
>> [2018-10-08 08:59:47.130334 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 08:59:47.130388 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:00:02 -0400]
>> [2018-10-08 08:59:47.733476 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 256 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 08:59:47.733507 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 256 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 09:00:01.131836 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 09:00:01.131924 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:00:16 -0400]
>> [2018-10-08 09:00:01.131936 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 09:00:01.131946 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:00:16 -0400]
>> [2018-10-08 09:00:02.132024 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 09:00:02.132077 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:00:17 -0400]
>> [2018-10-08 09:00:02.748601 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 271 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 09:00:02.748673 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 271 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 09:00:16.137566 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 09:00:16.137660 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:00:31 -0400]
>> [2018-10-08 09:00:16.137674 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 09:00:16.137685 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:00:31 -0400]
>> [2018-10-08 09:00:17.137789 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 09:00:17.137882 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:00:32 -0400]
>> [2018-10-08 09:00:17.763766 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 286 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 09:00:17.763831 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 286 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 09:00:31.139111 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 09:00:31.139210 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:00:46 -0400]
>> [2018-10-08 09:00:31.139222 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 09:00:31.139232 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:00:46 -0400]
>> [2018-10-08 09:00:32.139331 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 09:00:32.139430 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:00:47 -0400]
>> [2018-10-08 09:00:32.778911 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 301 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 09:00:32.778983 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 301 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 09:00:46.140339 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 09:00:46.140482 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:01:01 -0400]
>> [2018-10-08 09:00:46.140498 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 09:00:46.140510 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:01:01 -0400]
>> [2018-10-08 09:00:47.140624 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 09:00:47.140676 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:01:02 -0400]
>> [2018-10-08 09:00:47.794052 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 316 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 09:00:47.794115 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 316 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 09:01:01.141884 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 09:01:01.141979 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:01:16 -0400]
>> [2018-10-08 09:01:01.141993 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 09:01:01.142004 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:01:16 -0400]
>> [2018-10-08 09:01:02.142070 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 09:01:02.142113 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:01:17 -0400]
>> [2018-10-08 09:01:02.809137 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 331 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 09:01:02.809211 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 331 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>> [2018-10-08 09:01:16.143427 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
>> [2018-10-08 09:01:16.143523 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 09:01:31 -0400]
>> [2018-10-08 09:01:16.143536 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094"
>> [2018-10-08 09:01:16.143547 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: llrxrd-redir.in2p3.fr:1094" at [2018-10-08 09:01:31 -0400]
>> [2018-10-08 09:01:17.143658 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
>> [2018-10-08 09:01:17.143711 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 09:01:32 -0400]
>> [2018-10-08 09:01:17.824328 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 346 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
>> [2018-10-08 09:01:17.824447 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 346 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 08:55:31 -0400
>>
>> ```
>>
>> --
>> You are receiving this because you are subscribed to this thread.
>> Reply to this email directly or view it on GitHub:
>> #836 (comment)
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-DEV list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>
>
> --
> You are receiving this because you were mentioned.
> Reply to this email directly or view it on GitHub:
> #836 (comment)
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#836 (comment)
|
Here is new round, my environment
code:
and output (I waited for code to finish by itself without any timeouts):
and the result is the same
|
Hi Valentin, So, it looks like things are behaving to spec here. Can you confirm that the client host is unable to connect to Brian |
Well, no it's quite a bit different (though the ssame result). This time
you did get to cms-xrd-global.cern.ch which redirected you to
xrootd.ba.infn.it. Now, that machine should have redirected you to
cms02.lcg.cscs.ch but instead it never responded. So, the client tried
again but this time it timed out (apparently that xrootd was totally dead
at the time). This was tried several times (btw the IP address was
correct and it was IPv4). So, we go back to cms-xrd-global.cern.ch and ask
for some other host and we get redirected to
xrootd-cms-redir-01.cr.cnaf.infn.it We try to connect using an IPv4
address (it also has an IPv6 one). However, that connection timed out as
well. Note that this redirector should have redirected you to
cms02.lcg.cscs.ch if it were responding. So, back to the cms redirector to
ask for something other than the two failing hosts. This time you are sent
to xrootd-redic.pi.infn.it which should have reditected you to
se-xrd01.jinr-t1.ru that would then redirect you to se-xrd01.jinr-t1.ru ut
that didn't happen. Unfortunately, there were no more options within the
overall timeout and it was deemed that you weren't making any progress
after numerous attempts so you got an operation expired.
Mind you, all of this worked fine for me so it may have been a bad day for
Italy at the time but surely the Italian hosts just were not responding
13:17:56 but were at 20:26.
I suppose you can try again to see if this was simply a transient failure.
Andy
…On Mon, 8 Oct 2018, Valentin Kuznetsov wrote:
Here is new round, my environment
```
env | egrep "^Xrd|^XRD"
XRD_NETWORKSTACK=IPv4
XrdSecGSIUSERPROXY=/data/vk/proxy
XRD_LOGLEVEL=Dump
XrdSecDEBUG=2
```
code:
```
from XRootD import client
from XRootD.client.flags import OpenFlags
f=client.File()
redirector='root://cms-xrd-global.cern.ch'
#redirector='root://xrootd-cms.infn.it'
#redirector='root://cmsxrootd.fnal.gov'
fin='%s//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root' % redirector
print(fin)
data=f.open(fin, OpenFlags.READ)
print("data", data)
```
and output (I waited for code to finish by itself without any timeouts):
```
[2018-10-08 12:47:41.993259 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory
[2018-10-08 12:47:41.993546 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory
[2018-10-08 12:47:41.993669 -0400][Info ][Utility ] Env: Importing from shell XRD_NETWORKSTACK=IPv4 as NetworkStack
[2018-10-08 12:47:41.993704 -0400][Debug ][PlugInMgr ] Initializing plug-in manager...
[2018-10-08 12:47:41.993715 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs...
[2018-10-08 12:47:41.993723 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d...
[2018-10-08 12:47:41.993740 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory
[2018-10-08 12:47:41.993809 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /home/kuznetv/.xrootd/client.plugins.d...
[2018-10-08 12:47:41.993863 -0400][Dump ][PlugInMgr ] Processing: /home/kuznetv/.xrootd/client.plugins.d/client-plugin.conf
[2018-10-08 12:47:41.993914 -0400][Debug ][PlugInMgr ] Unable to find 'lib' key in the config file /home/kuznetv/.xrootd/client.plugins.d/client-plugin.conf, ignoring this config
[2018-10-08 12:47:42.010680 -0400][Debug ][Poller ] Available pollers: built-in
[2018-10-08 12:47:42.010712 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in
[2018-10-08 12:47:42.010724 -0400][Debug ][Poller ] Creating poller: built-in
[2018-10-08 12:47:42.010738 -0400][Debug ][Poller ] Creating and starting the built-in poller...
[2018-10-08 12:47:42.010862 -0400][Debug ][Poller ] Using 1 poller threads
[2018-10-08 12:47:42.010882 -0400][Debug ][TaskMgr ] Starting the task manager...
[2018-10-08 12:47:42.010920 -0400][Debug ][TaskMgr ] Task manager started
[2018-10-08 12:47:42.010930 -0400][Debug ][JobMgr ] Starting the job manager...
[2018-10-08 12:47:42.011006 -0400][Debug ][JobMgr ] Job manager started, 3 workers
[2018-10-08 12:47:42.011017 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2018-10-08 12:47:42 -0400]
[2018-10-08 12:47:42.011101 -0400][Dump ][Utility ] URL: root://cms-xrd-global.cern.ch//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
[2018-10-08 12:47:42.011101 -0400][Dump ][Utility ] Protocol: root
[2018-10-08 12:47:42.011101 -0400][Dump ][Utility ] User Name:
[2018-10-08 12:47:42.011101 -0400][Dump ][Utility ] Password:
[2018-10-08 12:47:42.011101 -0400][Dump ][Utility ] Host Name: cms-xrd-global.cern.ch
[2018-10-08 12:47:42.011101 -0400][Dump ][Utility ] Port: 1094
[2018-10-08 12:47:42.011101 -0400][Dump ][Utility ] Path: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
[2018-10-08 12:47:42.011145 -0400][Dump ][Utility ] URL: root://cms-xrd-global.cern.ch//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
[2018-10-08 12:47:42.011145 -0400][Dump ][Utility ] Protocol: root
[2018-10-08 12:47:42.011145 -0400][Dump ][Utility ] User Name:
[2018-10-08 12:47:42.011145 -0400][Dump ][Utility ] Password:
[2018-10-08 12:47:42.011145 -0400][Dump ][Utility ] Host Name: cms-xrd-global.cern.ch
[2018-10-08 12:47:42.011145 -0400][Dump ][Utility ] Port: 1094
[2018-10-08 12:47:42.011145 -0400][Dump ][Utility ] Path: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
[2018-10-08 12:47:42.011164 -0400][Debug ][File ] ***@***.***://cms-xrd-global.cern.ch:1094//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root] Sending an open command
[2018-10-08 12:47:42.011218 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Sending message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat )
[2018-10-08 12:47:42.011256 -0400][Dump ][Utility ] URL: cms-xrd-global.cern.ch:1094
[2018-10-08 12:47:42.011256 -0400][Dump ][Utility ] Protocol: root
[2018-10-08 12:47:42.011256 -0400][Dump ][Utility ] User Name:
[2018-10-08 12:47:42.011256 -0400][Dump ][Utility ] Password:
[2018-10-08 12:47:42.011256 -0400][Dump ][Utility ] Host Name: cms-xrd-global.cern.ch
[2018-10-08 12:47:42.011256 -0400][Dump ][Utility ] Port: 1094
[2018-10-08 12:47:42.011256 -0400][Dump ][Utility ] Path:
[2018-10-08 12:47:42.011283 -0400][Debug ][PostMaster ] Creating new channel to: cms-xrd-global.cern.ch:1094 1 stream(s)
[2018-10-08 12:47:42.011312 -0400][Debug ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Stream parameters: Network Stack: IPv4, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800
[2018-10-08 12:47:42.017062 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" to be run at: [2018-10-08 12:47:57 -0400]
[2018-10-08 12:47:42.017097 -0400][Dump ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Sending message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xd2ac1c60) through substream 0 expecting answer at 0
[2018-10-08 12:47:42.133024 -0400][Debug ][PostMaster ] [cms-xrd-global.cern.ch:1094] Found 2 address(es): [::ffff:188.184.161.66]:1094, [::ffff:188.184.67.41]:1094
[2018-10-08 12:47:42.133068 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Attempting connection to [::ffff:188.184.67.41]:1094
[2018-10-08 12:47:42.133117 -0400][Debug ][Poller ] Adding socket 0xd2ab4890 to the poller
[2018-10-08 12:47:42.246117 -0400][Debug ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Async connection call returned
[2018-10-08 12:47:42.246161 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Sending out the initial hand shake + kXR_protocol
[2018-10-08 12:47:42.246206 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Wrote a message: (0xbc000950), 44 bytes
[2018-10-08 12:47:42.359179 -0400][Dump ][XRootDTransport ] [msg: 0xbc000950] Expecting 8 bytes of message body
[2018-10-08 12:47:42.359213 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header, size: 8
[2018-10-08 12:47:42.359224 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received a message of 16 bytes
[2018-10-08 12:47:42.359238 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 310)
[2018-10-08 12:47:42.359251 -0400][Dump ][XRootDTransport ] [msg: 0xbc000950] Expecting 8 bytes of message body
[2018-10-08 12:47:42.359259 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header, size: 8
[2018-10-08 12:47:42.359267 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received a message of 16 bytes
[2018-10-08 12:47:42.359276 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] kXR_protocol successful (type: manager [meta], protocol version 310)
[2018-10-08 12:47:42.371342 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Sending out kXR_login request, username: kuznetv, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=python3.6&xrd.info=&xrd.hostname=goofy&xrd.rn=v4.8.4, dual-stack: true, private IPv4: false, private IPv6: true
[2018-10-08 12:47:42.371426 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Wrote a message: (0xbc000b40), 109 bytes
[2018-10-08 12:47:42.484227 -0400][Dump ][XRootDTransport ] [msg: 0xbc000b40] Expecting 16 bytes of message body
[2018-10-08 12:47:42.484263 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header, size: 8
[2018-10-08 12:47:42.484276 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received a message of 24 bytes
[2018-10-08 12:47:42.484292 -0400][Debug ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Logged in, session: ac1a04008ff5010059010000671c0400
[2018-10-08 12:47:42.484306 -0400][Debug ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Stream 0 connected.
[2018-10-08 12:47:42.484317 -0400][Debug ][Utility ] Monitor library name not set. No monitoring
[2018-10-08 12:47:42.484361 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Wrote a message: kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xd2ac1c60), 122 bytes
[2018-10-08 12:47:42.484381 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Successfully sent message: kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xd2ac1c60).
[2018-10-08 12:47:42.484397 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) has been successfully sent.
[2018-10-08 12:47:42.484423 -0400][Dump ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0.0] All messages consumed, disable uplink
[2018-10-08 12:47:42.598624 -0400][Dump ][XRootDTransport ] [msg: 0xbc000b70] Expecting 21 bytes of message body
[2018-10-08 12:47:42.598660 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message header for 0xbc000b70 size: 8
[2018-10-08 12:47:42.598680 -0400][Dump ][AsyncSock ] [cms-xrd-global.cern.ch:1094 #0.0] Received message 0xbc000b70 of 29 bytes
[2018-10-08 12:47:42.598691 -0400][Dump ][PostMaster ] [cms-xrd-global.cern.ch:1094 #0] Handling received message: 0xbc000b70.
[2018-10-08 12:47:42.598772 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Got kXR_redirect response to message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): xrootd.ba.infn.it, port 1094
[2018-10-08 12:47:42.598834 -0400][Dump ][XRootD ] [cms-xrd-global.cern.ch:1094] Current server has been assigned as a load-balancer for message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat )
[2018-10-08 12:47:42.598880 -0400][Dump ][Utility ] URL: xrootd.ba.infn.it:1094/
[2018-10-08 12:47:42.598880 -0400][Dump ][Utility ] Protocol: root
[2018-10-08 12:47:42.598880 -0400][Dump ][Utility ] User Name:
[2018-10-08 12:47:42.598880 -0400][Dump ][Utility ] Password:
[2018-10-08 12:47:42.598880 -0400][Dump ][Utility ] Host Name: xrootd.ba.infn.it
[2018-10-08 12:47:42.598880 -0400][Dump ][Utility ] Port: 1094
[2018-10-08 12:47:42.598880 -0400][Dump ][Utility ] Path:
[2018-10-08 12:47:42.598930 -0400][Dump ][Utility ] URL: xrootd.ba.infn.it:1094
[2018-10-08 12:47:42.598930 -0400][Dump ][Utility ] Protocol: root
[2018-10-08 12:47:42.598930 -0400][Dump ][Utility ] User Name:
[2018-10-08 12:47:42.598930 -0400][Dump ][Utility ] Password:
[2018-10-08 12:47:42.598930 -0400][Dump ][Utility ] Host Name: xrootd.ba.infn.it
[2018-10-08 12:47:42.598930 -0400][Dump ][Utility ] Port: 1094
[2018-10-08 12:47:42.598930 -0400][Dump ][Utility ] Path:
[2018-10-08 12:47:42.598971 -0400][Debug ][PostMaster ] Creating new channel to: xrootd.ba.infn.it:1094 1 stream(s)
[2018-10-08 12:47:42.599001 -0400][Debug ][PostMaster ] [xrootd.ba.infn.it:1094 #0] Stream parameters: Network Stack: IPv4, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800
[2018-10-08 12:47:42.605065 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: xrootd.ba.infn.it:1094" to be run at: [2018-10-08 12:47:57 -0400]
[2018-10-08 12:47:42.605135 -0400][Dump ][Utility ] URL: fake://fake:111//store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
[2018-10-08 12:47:42.605135 -0400][Dump ][Utility ] Protocol: fake
[2018-10-08 12:47:42.605135 -0400][Dump ][Utility ] User Name:
[2018-10-08 12:47:42.605135 -0400][Dump ][Utility ] Password:
[2018-10-08 12:47:42.605135 -0400][Dump ][Utility ] Host Name: fake
[2018-10-08 12:47:42.605135 -0400][Dump ][Utility ] Port: 111
[2018-10-08 12:47:42.605135 -0400][Dump ][Utility ] Path: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root
[2018-10-08 12:47:42.605198 -0400][Dump ][PostMaster ] [xrootd.ba.infn.it:1094 #0] Sending message kXR_open (file: /store/data/Run2018C/Tau/NANOAOD/14Sep2018_ver3-v1/60000/6FA4CC7C-8982-DE4C-BEED-C90413312B35.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) (0xd2ac1c60) through substream 0 expecting answer at 0
[2018-10-08 12:47:42.759078 -0400][Debug ][PostMaster ] [xrootd.ba.infn.it:1094] Found 1 address(es): [::ffff:90.147.66.75]:1094
[2018-10-08 12:47:42.759128 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Attempting connection to [::ffff:90.147.66.75]:1094
[2018-10-08 12:47:42.759178 -0400][Debug ][Poller ] Adding socket 0xb4001460 to the poller
[2018-10-08 12:47:43.011019 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:47:43.011070 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:47:58 -0400]
[2018-10-08 12:47:57.011967 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:47:57.012056 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:48:12 -0400]
[2018-10-08 12:47:57.012068 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:47:57.012079 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:48:12 -0400]
[2018-10-08 12:47:57.614605 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 15 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:47:57.614660 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 15 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:47:58.012120 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:47:58.012156 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:48:13 -0400]
[2018-10-08 12:48:12.013097 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:48:12.013183 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:48:27 -0400]
[2018-10-08 12:48:12.013194 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:48:12.013205 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:48:27 -0400]
[2018-10-08 12:48:12.629737 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 30 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:48:12.629789 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 30 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:48:13.013295 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:48:13.013350 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:48:28 -0400]
[2018-10-08 12:48:27.014384 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:48:27.014530 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:48:42 -0400]
[2018-10-08 12:48:27.014543 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:48:27.014554 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:48:42 -0400]
[2018-10-08 12:48:27.644483 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 45 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:48:27.644527 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 45 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:48:28.014647 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:48:28.014746 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:48:43 -0400]
[2018-10-08 12:48:42.015807 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:48:42.015883 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:48:57 -0400]
[2018-10-08 12:48:42.015891 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:48:42.015899 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:48:57 -0400]
[2018-10-08 12:48:42.659626 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 60 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:48:42.659708 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 60 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:48:43.015986 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:48:43.016044 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:48:58 -0400]
[2018-10-08 12:48:57.017119 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:48:57.017211 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:49:12 -0400]
[2018-10-08 12:48:57.017224 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:48:57.017236 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:49:12 -0400]
[2018-10-08 12:48:57.674693 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 75 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:48:57.674747 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 75 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:48:58.017312 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:48:58.017363 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:49:13 -0400]
[2018-10-08 12:49:12.018232 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:49:12.018331 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:49:27 -0400]
[2018-10-08 12:49:12.018344 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:49:12.018356 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:49:27 -0400]
[2018-10-08 12:49:12.689837 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 90 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:49:12.689902 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 90 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:49:13.018495 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:49:13.018595 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:49:28 -0400]
[2018-10-08 12:49:27.019848 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:49:27.019945 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:49:42 -0400]
[2018-10-08 12:49:27.019958 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:49:27.019970 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:49:42 -0400]
[2018-10-08 12:49:27.705001 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 105 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:49:27.705068 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 105 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:49:28.020071 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:49:28.020125 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:49:43 -0400]
[2018-10-08 12:49:42.021364 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:49:42.021480 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:49:57 -0400]
[2018-10-08 12:49:42.021493 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:49:42.021503 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:49:57 -0400]
[2018-10-08 12:49:42.720162 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 120 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:49:42.720212 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 120 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:49:43.021625 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:49:43.021704 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:49:58 -0400]
[2018-10-08 12:49:49.993490 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Async connection call returned
[2018-10-08 12:49:49.993578 -0400][Error ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Unable to connect: Connection timed out
[2018-10-08 12:49:49.993590 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Closing the socket
[2018-10-08 12:49:49.993603 -0400][Debug ][Poller ] <x><--><x> Removing socket from the poller
[2018-10-08 12:49:49.993661 -0400][Error ][PostMaster ] [xrootd.ba.infn.it:1094 #0] elapsed = 127, pConnectionWindow = 120 seconds.
[2018-10-08 12:49:49.995541 -0400][Debug ][PostMaster ] [xrootd.ba.infn.it:1094] Found 1 address(es): [::ffff:90.147.66.75]:1094
[2018-10-08 12:49:49.995605 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Attempting connection to [::ffff:90.147.66.75]:1094
[2018-10-08 12:49:49.995671 -0400][Debug ][Poller ] Adding socket 0xb4001460 to the poller
[2018-10-08 12:49:57.022913 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:49:57.023004 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:50:12 -0400]
[2018-10-08 12:49:57.023017 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:49:57.023028 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:50:12 -0400]
[2018-10-08 12:49:58.003744 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 136 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:49:58.003826 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 136 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:49:58.023141 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:49:58.023227 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:50:13 -0400]
[2018-10-08 12:50:12.024475 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:50:12.024569 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:50:27 -0400]
[2018-10-08 12:50:12.024583 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:50:12.024594 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:50:27 -0400]
[2018-10-08 12:50:13.018932 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 151 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:50:13.019020 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 151 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:50:13.024696 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:50:13.024761 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:50:28 -0400]
[2018-10-08 12:50:27.026052 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:50:27.026151 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:50:42 -0400]
[2018-10-08 12:50:27.026163 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:50:27.026174 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:50:42 -0400]
[2018-10-08 12:50:28.026284 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:50:28.026374 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:50:43 -0400]
[2018-10-08 12:50:28.034057 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 166 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:50:28.034131 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 166 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:50:42.027617 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:50:42.027708 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:50:57 -0400]
[2018-10-08 12:50:42.027720 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:50:42.027730 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:50:57 -0400]
[2018-10-08 12:50:43.027825 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:50:43.027873 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:50:58 -0400]
[2018-10-08 12:50:43.049071 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 181 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:50:43.049143 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 181 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:50:57.028918 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:50:57.029006 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:51:12 -0400]
[2018-10-08 12:50:57.029018 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:50:57.029029 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:51:12 -0400]
[2018-10-08 12:50:58.029146 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:50:58.029249 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:51:13 -0400]
[2018-10-08 12:50:58.064174 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 196 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:50:58.064249 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 196 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:51:12.030535 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:51:12.030628 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:51:27 -0400]
[2018-10-08 12:51:12.030640 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:51:12.030652 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:51:27 -0400]
[2018-10-08 12:51:13.030764 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:51:13.030859 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:51:28 -0400]
[2018-10-08 12:51:13.079332 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 211 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:51:13.079586 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 211 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:51:27.032149 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:51:27.032240 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:51:42 -0400]
[2018-10-08 12:51:27.032252 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:51:27.032281 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:51:42 -0400]
[2018-10-08 12:51:28.032391 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:51:28.032489 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:51:43 -0400]
[2018-10-08 12:51:28.094590 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 226 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:51:28.094661 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 226 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:51:42.033770 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:51:42.033856 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:51:57 -0400]
[2018-10-08 12:51:42.033867 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:51:42.033877 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:51:57 -0400]
[2018-10-08 12:51:43.033938 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:51:43.033959 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:51:58 -0400]
[2018-10-08 12:51:43.109674 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 241 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:51:43.109755 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 241 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:51:57.035289 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:51:57.035377 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:52:12 -0400]
[2018-10-08 12:51:57.035389 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:51:57.035400 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:52:12 -0400]
[2018-10-08 12:51:57.225543 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Async connection call returned
[2018-10-08 12:51:57.225616 -0400][Error ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Unable to connect: Connection timed out
[2018-10-08 12:51:57.225628 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Closing the socket
[2018-10-08 12:51:57.225641 -0400][Debug ][Poller ] <x><--><x> Removing socket from the poller
[2018-10-08 12:51:57.225678 -0400][Error ][PostMaster ] [xrootd.ba.infn.it:1094 #0] elapsed = 128, pConnectionWindow = 120 seconds.
[2018-10-08 12:51:57.227519 -0400][Debug ][PostMaster ] [xrootd.ba.infn.it:1094] Found 1 address(es): [::ffff:90.147.66.75]:1094
[2018-10-08 12:51:57.227582 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Attempting connection to [::ffff:90.147.66.75]:1094
[2018-10-08 12:51:57.227649 -0400][Debug ][Poller ] Adding socket 0xb4001460 to the poller
[2018-10-08 12:51:58.035522 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:51:58.035569 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:52:13 -0400]
[2018-10-08 12:51:58.228749 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 256 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:51:58.228800 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 256 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:52:12.036940 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:52:12.037031 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:52:27 -0400]
[2018-10-08 12:52:12.037044 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:52:12.037055 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:52:27 -0400]
[2018-10-08 12:52:13.037157 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:52:13.037209 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:52:28 -0400]
[2018-10-08 12:52:13.243916 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 271 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:52:13.243996 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 271 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:52:27.038534 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:52:27.038633 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:52:42 -0400]
[2018-10-08 12:52:27.038646 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:52:27.038657 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:52:42 -0400]
[2018-10-08 12:52:28.038767 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:52:28.038819 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:52:43 -0400]
[2018-10-08 12:52:28.259064 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 286 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:52:28.259133 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 286 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:52:42.040216 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:52:42.040319 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:52:57 -0400]
[2018-10-08 12:52:42.040331 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:52:42.040341 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:52:57 -0400]
[2018-10-08 12:52:43.040456 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:52:43.040509 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:52:58 -0400]
[2018-10-08 12:52:43.274216 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 301 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:52:43.274269 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 301 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:52:57.041904 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:52:57.041996 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:53:12 -0400]
[2018-10-08 12:52:57.042009 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:52:57.042020 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:53:12 -0400]
[2018-10-08 12:52:58.042151 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:52:58.042206 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:53:13 -0400]
[2018-10-08 12:52:58.289335 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 316 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:52:58.289399 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 316 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:53:12.043608 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:53:12.043695 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:53:27 -0400]
[2018-10-08 12:53:12.043708 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:53:12.043719 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:53:27 -0400]
[2018-10-08 12:53:13.043841 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:53:13.043939 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:53:28 -0400]
[2018-10-08 12:53:13.304509 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 331 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:53:13.304576 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 331 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:53:27.045326 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:53:27.045456 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:53:42 -0400]
[2018-10-08 12:53:27.045474 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:53:27.045486 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:53:42 -0400]
[2018-10-08 12:53:28.045599 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:53:28.045667 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:53:43 -0400]
[2018-10-08 12:53:28.319684 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 346 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:53:28.319752 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 346 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:53:42.046802 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:53:42.046895 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:53:57 -0400]
[2018-10-08 12:53:42.046908 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:53:42.046919 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:53:57 -0400]
[2018-10-08 12:53:43.047037 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:53:43.047128 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:53:58 -0400]
[2018-10-08 12:53:43.334856 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 361 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:53:43.334926 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 361 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:53:57.048162 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:53:57.048266 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:54:12 -0400]
[2018-10-08 12:53:57.048279 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:53:57.048290 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:54:12 -0400]
[2018-10-08 12:53:58.048445 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:53:58.048541 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:54:13 -0400]
[2018-10-08 12:53:58.350003 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 376 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:53:58.350083 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 376 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:54:04.457504 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Async connection call returned
[2018-10-08 12:54:04.457590 -0400][Error ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Unable to connect: Connection timed out
[2018-10-08 12:54:04.457602 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Closing the socket
[2018-10-08 12:54:04.457615 -0400][Debug ][Poller ] <x><--><x> Removing socket from the poller
[2018-10-08 12:54:04.457654 -0400][Error ][PostMaster ] [xrootd.ba.infn.it:1094 #0] elapsed = 127, pConnectionWindow = 120 seconds.
[2018-10-08 12:54:04.886260 -0400][Debug ][PostMaster ] [xrootd.ba.infn.it:1094] Found 1 address(es): [::ffff:90.147.66.75]:1094
[2018-10-08 12:54:04.886288 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Attempting connection to [::ffff:90.147.66.75]:1094
[2018-10-08 12:54:04.886322 -0400][Debug ][Poller ] Adding socket 0xb4001460 to the poller
[2018-10-08 12:54:12.049841 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:54:12.049930 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:54:27 -0400]
[2018-10-08 12:54:12.049943 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:54:12.049954 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:54:27 -0400]
[2018-10-08 12:54:13.050059 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:54:13.050114 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:54:28 -0400]
[2018-10-08 12:54:13.895380 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 391 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:54:13.895440 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 391 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:54:27.051468 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:54:27.051554 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:54:42 -0400]
[2018-10-08 12:54:27.051566 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:54:27.051576 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:54:42 -0400]
[2018-10-08 12:54:28.051721 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:54:28.051814 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:54:43 -0400]
[2018-10-08 12:54:28.910556 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 406 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:54:28.910625 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 406 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:54:42.053136 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:54:42.053234 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:54:57 -0400]
[2018-10-08 12:54:42.053247 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:54:42.053258 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:54:57 -0400]
[2018-10-08 12:54:43.053357 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:54:43.053470 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:54:58 -0400]
[2018-10-08 12:54:43.925750 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 421 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:54:43.925817 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 421 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:54:57.054644 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:54:57.054736 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:55:12 -0400]
[2018-10-08 12:54:57.054749 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:54:57.054760 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:55:12 -0400]
[2018-10-08 12:54:58.054843 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:54:58.054909 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:55:13 -0400]
[2018-10-08 12:54:58.940958 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 436 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:54:58.941032 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 436 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:55:12.055939 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:55:12.056033 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:55:27 -0400]
[2018-10-08 12:55:12.056046 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:55:12.056057 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:55:27 -0400]
[2018-10-08 12:55:13.056131 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:55:13.056192 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:55:28 -0400]
[2018-10-08 12:55:13.956170 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 451 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:55:13.956276 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 451 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:55:27.057112 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:55:27.057201 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:55:42 -0400]
[2018-10-08 12:55:27.057213 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:55:27.057223 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:55:42 -0400]
[2018-10-08 12:55:28.057331 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:55:28.057438 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:55:43 -0400]
[2018-10-08 12:55:28.971462 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 466 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:55:28.971546 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 466 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:55:42.058766 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:55:42.058865 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:55:57 -0400]
[2018-10-08 12:55:42.058877 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:55:42.058889 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:55:57 -0400]
[2018-10-08 12:55:43.059003 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:55:43.059075 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:55:58 -0400]
[2018-10-08 12:55:43.986672 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 481 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:55:43.986742 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 481 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:55:57.060459 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:55:57.060550 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:56:12 -0400]
[2018-10-08 12:55:57.060560 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:55:57.060568 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:56:12 -0400]
[2018-10-08 12:55:58.060675 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:55:58.060727 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:56:13 -0400]
[2018-10-08 12:55:59.001891 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 497 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:55:59.001949 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 497 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:56:12.062131 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:56:12.062222 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:56:27 -0400]
[2018-10-08 12:56:12.062264 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:56:12.062276 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:56:27 -0400]
[2018-10-08 12:56:12.201502 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Async connection call returned
[2018-10-08 12:56:12.201570 -0400][Error ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Unable to connect: Connection timed out
[2018-10-08 12:56:12.201590 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Closing the socket
[2018-10-08 12:56:12.201603 -0400][Debug ][Poller ] <x><--><x> Removing socket from the poller
[2018-10-08 12:56:12.201638 -0400][Error ][PostMaster ] [xrootd.ba.infn.it:1094 #0] elapsed = 128, pConnectionWindow = 120 seconds.
[2018-10-08 12:56:12.203812 -0400][Debug ][PostMaster ] [xrootd.ba.infn.it:1094] Found 1 address(es): [::ffff:90.147.66.75]:1094
[2018-10-08 12:56:12.203864 -0400][Debug ][AsyncSock ] [xrootd.ba.infn.it:1094 #0.0] Attempting connection to [::ffff:90.147.66.75]:1094
[2018-10-08 12:56:12.203920 -0400][Debug ][Poller ] Adding socket 0xb4001460 to the poller
[2018-10-08 12:56:13.062384 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:56:13.062473 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:56:28 -0400]
[2018-10-08 12:56:14.206013 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 512 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:56:14.206101 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 512 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:56:27.063799 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:56:27.063890 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:56:42 -0400]
[2018-10-08 12:56:27.063902 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:56:27.063914 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:56:42 -0400]
[2018-10-08 12:56:28.064010 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:56:28.064061 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:56:43 -0400]
[2018-10-08 12:56:29.221075 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 527 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:56:29.221143 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 527 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:56:42.065292 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:56:42.065383 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:56:57 -0400]
[2018-10-08 12:56:42.065395 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:56:42.065427 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:56:57 -0400]
[2018-10-08 12:56:43.065553 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:56:43.065653 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:56:58 -0400]
[2018-10-08 12:56:44.236161 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 542 seconds, TTL: 1200, allocated SIDs: 0, open files: 0
[2018-10-08 12:56:44.236230 -0400][Dump ][XRootDTransport ] [cms-xrd-global.cern.ch:1094 #0.0] Stream inactive since 542 seconds, stream timeout: 60, allocated SIDs: 0, wait barrier: 2018-10-08 12:47:42 -0400
[2018-10-08 12:56:57.066918 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: cms-xrd-global.cern.ch:1094"
[2018-10-08 12:56:57.067033 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: cms-xrd-global.cern.ch:1094" at [2018-10-08 12:57:12 -0400]
[2018-10-08 12:56:57.067046 -0400][Dump ][TaskMgr ] Running task: "TickGeneratorTask for: xrootd.ba.infn.it:1094"
[2018-10-08 12:56:57.067058 -0400][Dump ][TaskMgr ] Will rerun task "TickGeneratorTask for: xrootd.ba.infn.it:1094" at [2018-10-08 12:57:12 -0400]
[2018-10-08 12:56:58.067164 -0400][Dump ][TaskMgr ] Running task: "FileTimer task"
[2018-10-08 12:56:58.067251 -0400][Dump ][TaskMgr ] Will rerun task "FileTimer task" at [2018-10-08 12:57:13 -0400]
[2018-10-08
|
Brian,
here is output from lxplus:
```
telnet xrootd-redic.pi.infn.it 1094
Trying 193.205.76.83...
Connected to xrootd-redic.pi.infn.it (193.205.76.83).
Escape character is '^]'.
```
so I immediately get the connection. And here is output from goofy node:
```
./telnet xrootd-redic.pi.infn.it 1094
Trying 193.205.76.83...
```
and telnet commands sits there without showing a connection.
And, it also (partially) answer Andy's reply. I tried almost at the same time
from lxplus and goofy the telnet command so I doubt (even though you still may
be right) that Italian host was down.
Is there any way to specify a chain of hosts to connect such that we can avoid
possibly unstable Italian hosts?
…On 0, Brian P Bockelman ***@***.***> wrote:
Hi Valentin,
So, it looks like things are behaving to spec here. Can you confirm that the client host is unable to connect to `xrootd-redic.pi.infn.it:1094`?
Brian
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#836 (comment)
|
@vkuznet - I think you're getting errors on all the available redirectors in Europe. You might just need to ask UC to fix the network connectivity of that host -- I bet there will be many other issues with other sites, eventually leading you to have to maintain a long blacklist. |
I agree with Brian. Since you can connect via lxplus and I can connect via
my SLAC machine but you cannot then the likely problem is a) something is
wrong with your outgoing IP tables, b) soeone setup a firewall rule that
prevents you from getting to those hosts, or c) the network you are using
wound up routing you using an nonsymetric path (that was a notrious
problem when paths to LHCONE and Intenet 2 were present -- though my
understanding is that they fixed that problem). You can try a traceroute
command to the hosts you can't connect to to see where the routing stops.
Andy
…On Mon, 8 Oct 2018, Brian P Bockelman wrote:
@vkuznet - I think you're getting errors on all the available redirectors in Europe. You might just need to ask UC to fix the network connectivity of that host -- I bet there will be many other issues with other sites, eventually leading you to have to maintain a long blacklist.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#836 (comment)
|
Ah, I went through the problem posts and didn't see what version you are
using. If it's 4.8.5-RCx we seem to have uncovered a "hang" issue simiar
to yours (this was done by another tester). So, what are you using?
…On Mon, 8 Oct 2018, Brian P Bockelman wrote:
@vkuznet - I think you're getting errors on all the available redirectors in Europe. You might just need to ask UC to fix the network connectivity of that host -- I bet there will be many other issues with other sites, eventually leading you to have to maintain a long blacklist.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#836 (comment)
|
Andy, Brain
thanks for help and suggestion. I contacted local admin.
Meanwhile, the traceroute does not show the hops so it is internal networking
problem (I should try it first though). And, I'm using 4.8.4 version.
…On 0, Andrew Hanushevsky ***@***.***> wrote:
I agree with Brian. Since you can connect via lxplus and I can connect via
my SLAC machine but you cannot then the likely problem is a) something is
wrong with your outgoing IP tables, b) soeone setup a firewall rule that
prevents you from getting to those hosts, or c) the network you are using
wound up routing you using an nonsymetric path (that was a notrious
problem when paths to LHCONE and Intenet 2 were present -- though my
understanding is that they fixed that problem). You can try a traceroute
command to the hosts you can't connect to to see where the routing stops.
Andy
On Mon, 8 Oct 2018, Brian P Bockelman wrote:
> @vkuznet - I think you're getting errors on all the available redirectors in Europe. You might just need to ask UC to fix the network connectivity of that host -- I bet there will be many other issues with other sites, eventually leading you to have to maintain a long blacklist.
>
> --
> You are receiving this because you commented.
> Reply to this email directly or view it on GitHub:
> #836 (comment)
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#836 (comment)
|
Hi,
I installed pyxrootd on one of the node and I'm experience strange behavior. Basically I execute this commands:
The problem is that if I not specify timeout option
f.open
waits forever. With provided timeout I get this error:The same works on other nodes, e.g. lxplus and I can access file via CERN or FNAL xrootd redirectors just fine over there.
I cannot find any flag which may provide more information about what cause the hang.
Could you please provide any hint how to debug this issue? May be it is node configuration, e.g. iptables, issue which I'm not aware or something similar.
Thanks,
Valentin.
The text was updated successfully, but these errors were encountered: