Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FTS transfers in rucio-dev container fail #4525

Closed
maany opened this issue Apr 8, 2021 · 0 comments · Fixed by #4526
Closed

FTS transfers in rucio-dev container fail #4525

maany opened this issue Apr 8, 2021 · 0 comments · Fixed by #4526
Assignees
@maany
Labels
Docker & Kubernetes
Milestone
1.25.3

Comments

@maany
Copy link
Member

maany commented Apr 8, 2021 •
edited

Motivation

Running rucio-conveyer-submitter fails. The failure occurs due to authentication timeout with fts container as seen below:

2021-04-01 12:57:57,873 root    807     DEBUG   conveyor-submitter[0/1] : Start to register transfer state
2021-04-01 12:57:57,878 root    807     DEBUG   conveyor-submitter[0/1] : Finished to register transfer state
2021-04-01 12:57:57,878 root    807     INFO    conveyor-submitter[0/1] : PREPARING REQUEST 8fc34a41702a459ea082df3b8b1e4cf3 DID test:file2 TO SUBMITTING STATE PREVIOUS None FROM [('XRD1', 'root://xrd1:1094//rucio/test/f3/14/file2', '1a305d2edbba4d33b6d6b551504932a1', 0)] TO ['root://xrd3:1096//rucio/test/f3/14/file2'] USING https://fts:8446 
2021-04-01 12:57:57,878 root    807     DEBUG   conveyor-submitter[0/1] : Start to prepare transfer
2021-04-01 12:57:57,884 root    807     DEBUG   conveyor-submitter[0/1] : Finished to prepare transfer
2021-04-01 12:57:57,884 root    807     INFO    conveyor-submitter[0/1] : About to submit job to https://fts:8446 with timeout None
FTS3.py ::: ('/opt/rucio/etc/usercertkey.pem', '/opt/rucio/etc/usercertkey.pem')
2021-04-01 12:57:57,886 urllib3.connectionpool  807     DEBUG   Starting new HTTPS connection (1): fts:8446
2021-04-01 12:57:58,860 urllib3.connectionpool  807     DEBUG   https://fts:8446 "POST /jobs HTTP/1.1" 419 228
2021-04-01 12:57:58,862 root    807     WARNING Failed to submit transfer to https://fts:8446, error: <html>
 <head>
  <title>419 Authentication Timeout</title>
 </head>
 <body>
  <h1>419 Authentication Timeout</h1>
  The authentication has expired<br /><br />
No delegation found for &quot;/CN=Rucio User&quot;
 </body>
</html>
2021-04-01 12:57:58,862 root    807     ERROR   conveyor-submitter[0/1] : Wrong answer returned by the transfer tool.
Details: No transfer id returned by https://fts:8446

This happens because fts-rest-delegate -vf -s https://fts:8446 -H 9999 fails in

https://github.com/rucio/rucio/blob/master/tools/docker_activate_rses.sh#L61

root@d0419870471e rucio]# fts-rest-delegate -vf -s https://fts:8446 -H 9999
  File "/usr/bin/fts-rest-delegate", line 12
    except Exception, e:
                    ^
SyntaxError: invalid syntax

which happens because fts-rest-cli is using python2 syntax

https://github.com/cern-fts/fts-rest/blob/develop/src/cli/fts-rest-delegate#L12

However, the default python in the rucio-dev container is python3

https://github.com/rucio/containers/blob/master/dev/Dockerfile#L96

Modification

In the Dockerfile for rucio-dev container, we need to install python-setuptools, python-pip and requests package before switching default python to python3

https://github.com/rucio/containers/blob/master/dev/Dockerfile#L96

This sets up the minimum requirements for running fts-rest-cli. I will create an issue for this in rucio/containers (rucio/containers#128)

In the docker_activate_rses.sh, we need to invoke fts-rest-delegate as follows:

/usr/bin/python2.7 /usr/bin/fts-rest-delegate -vf -s https://fts:8446 -H 9999
@maany maany self-assigned this Apr 8, 2021
@maany maany mentioned this issue Apr 8, 2021
Closed
maany added a commit to maany/rucio that referenced this issue Apr 8, 2021
@maany
Docker & Kubernetes: use python2 to run fts-rest-delegate rucio#4525
562e2ec
@maany maany mentioned this issue Apr 8, 2021
Merged
bari12 pushed a commit that referenced this issue Apr 13, 2021
@maany
Docker & Kubernetes: use python2 to run fts-rest-delegate #4525 (#4526
20d8cd6 
)
bari12 pushed a commit that referenced this issue Apr 13, 2021
@maany @bari12
Docker & Kubernetes: use python2 to run fts-rest-delegate #4525 (#4526
daf9bb7 
)
@bari12 bari12 added this to the 1.25.3 milestone Apr 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@maany maany

Labels
Docker & Kubernetes
Projects
None yet
Milestone
1.25.3
Development

Successfully merging a pull request may close this issue.

Docker & Kubernetes: use python2 to run fts-rest-delegate #4525 maany/rucio
2 participants
@bari12 @maany