Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send User-Agent as the monitor info #2154

Merged
merged 1 commit into from Jan 12, 2024
Merged

Conversation

bbockelm
Copy link
Contributor

After the initial login to the xrootd bridge (and after parsing the first request's headers), perform a kXR_set for the monitoring info. The User-Agent header is sent as the info.

This is only done for the first request of a connection. It is assumed that all subsequent connections belong to the same user agent.

Copy link
Member

@abh3 abh3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me, Cedric do you have issues with this?

@abh3 abh3 requested a review from ccaffy January 11, 2024 09:37
Copy link
Contributor

@ccaffy ccaffy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me too! Thanks @bbockelm !

@abh3 abh3 merged commit e78cf48 into xrootd:devel Jan 12, 2024
13 checks passed
@amadio amadio added this to the 5.6.5 milestone Jan 19, 2024
@amadio
Copy link
Member

amadio commented Jan 23, 2024

Looks like this breaks HTTP transfers. I will need to revert and make another release.

Here are excerpts from the logs without this change:

240123 15:19:11 1088645 http_Protocol: getDataOneShot BuffAvailable: 1048576 maxread: 1048576
240123 15:19:11 1088645 http_Protocol: getDataOneShot sslavail: 1048576
240123 15:19:11 1088645 http_Protocol: read 139 of 1048576 bytes
240123 15:19:11 1088645 http_Protocol:  rc:32 got hdr line: PUT /0-size-file-dest HTTP/1.1

240123 15:19:11 1088645 http_Protocol:  Parsing first line: PUT /0-size-file-dest HTTP/1.1

240123 15:19:11 1088645 http_Protocol:  rc:27 got hdr line: Host: gentoo.cern.ch:8080

240123 15:19:11 1088645 http_Protocol:  rc:24 got hdr line: User-Agent: curl/8.5.0

240123 15:19:11 1088645 http_Protocol:  rc:13 got hdr line: Accept: */*

240123 15:19:11 1088645 http_Protocol:  rc:22 got hdr line: Expect: 100-continue

240123 15:19:11 1088645 http_Protocol:  rc:19 got hdr line: Content-Length: 0

240123 15:19:11 1088645 http_Protocol:  rc:2 got hdr line: 

240123 15:19:11 1088645 http_Protocol:  rc:2 detected header end.
240123 15:19:11 1088645 XrootdBridge: unknown.5:37@gentoo login as nobody
240123 15:19:11 1088645 unknown.5:37@gentoo http_Protocol:  Process. lp:0x7fc060007bc0 reqstate: 0
240123 15:19:11 1088645 unknown.5:37@gentoo Xrootd_Protocol: 0000 Bridge req=3010 dlen=18 blen=18
240123 15:19:11 1088645 unknown.5:37@gentoo http_Protocol: Process is exiting rc:1
240123 15:19:11 1088645 unknown.5:37@gentoo Xrootd_Protocol: 0000 open odm /0-size-file-dest
240123 15:19:11 1088645 unknown.5:37@gentoo Xrootd_Response: 0000 sending 4 data bytes
240123 15:19:11 1088645 http_Req:  XrdHttpReq::Data! final=False
240123 15:19:11 1088645 unknown.5:37@gentoo http_Req: PostProcessHTTPReq req: 4 reqstate: 0
240123 15:19:11 1088645 unknown.5:37@gentoo http_Req: fhandle:0:0:0:0
240123 15:19:11 1088645 unknown.5:37@gentoo http_Protocol: Sending resp: 100 header len:67
240123 15:19:11 1088645 http_Protocol: Sending 67 bytes
240123 15:19:11 1088645 Xrd_Poll: Poller 0 enabled unknown.5:37@gentoo
240123 15:19:14 154173 unknown.5:37@gentoo Xrd_Poll: Link 37 terminating: hangup
240123 15:19:14 154193 Xrd_Sched: running unknown.5:37@gentoo inq=0
240123 15:19:14 154193 http_Protocol:  Cleanup
240123 15:19:14 154193 http_Protocol:  Reset
240123 15:19:14 154193 http_Req:  XrdHttpReq request ended.
240123 15:19:14 154193 XrootdXeq: unknown.5:37@gentoo disc 0:00:03
240123 15:19:14 154193 unknown.5:37@gentoo Xrootd_File: serializing access w Ref=0 /0-size-file-dest
240123 15:19:14 154193 unknown.5:37@gentoo Xrootd_File: closing w /0-size-file-dest
240123 15:19:14 154193 unknown.5:37@gentoo Xrd_Poll: Poller 0 removing FD 37
240123 15:19:14 154193 unknown.5:37@gentoo Xrd_Poll: FD 37 detached from poller 0; num=0

and then with this change:

240123 15:19:53 1117520 http_Protocol: getDataOneShot BuffAvailable: 1048576 maxread: 1048576
240123 15:19:53 1117520 http_Protocol: getDataOneShot sslavail: 1048576
240123 15:19:53 1117520 http_Protocol: read 139 of 1048576 bytes
240123 15:19:53 1117520 http_Protocol:  rc:32 got hdr line: PUT /0-size-file-dest HTTP/1.1

240123 15:19:53 1117520 http_Protocol:  Parsing first line: PUT /0-size-file-dest HTTP/1.1

240123 15:19:53 1117520 http_Protocol:  rc:27 got hdr line: Host: gentoo.cern.ch:8080

240123 15:19:53 1117520 http_Protocol:  rc:24 got hdr line: User-Agent: curl/8.5.0

240123 15:19:53 1117520 http_Protocol:  rc:13 got hdr line: Accept: */*

240123 15:19:53 1117520 http_Protocol:  rc:22 got hdr line: Expect: 100-continue

240123 15:19:53 1117520 http_Protocol:  rc:19 got hdr line: Content-Length: 0

240123 15:19:53 1117520 http_Protocol:  rc:2 got hdr line: 

240123 15:19:53 1117520 http_Protocol:  rc:2 detected header end.
240123 15:19:53 1117520 XrootdBridge: unknown.1:35@gentoo login as nobody
240123 15:19:53 1117520 unknown.1:35@gentoo http_Protocol:  Process. lp:0x7fe2e40077b0 reqstate: 0
240123 15:19:53 1117520 unknown.1:35@gentoo http_Protocol: Setting monitor info curl/8.5.0
240123 15:19:53 1117520 unknown.1:35@gentoo Xrootd_Protocol: 0000 Bridge req=3018 dlen=23 blen=23
240123 15:19:53 1117520 unknown.1:35@gentoo Xrootd_Protocol: 0000 set monitor info curl/8.5.0
240123 15:19:53 1117520 unknown.1:35@gentoo Xrootd_Response: 0000 sending 4 data bytes
240123 15:19:53 1117520 http_Req:  XrdHttpReq::Data! final=False
240123 15:19:53 1117520 unknown.1:35@gentoo http_Req: PostProcessHTTPReq req: 4 reqstate: 0
240123 15:19:53 1117520 unknown.1:35@gentoo http_Req: fhandle:0:0:0:0
240123 15:19:53 1117520 unknown.1:35@gentoo http_Protocol: Sending resp: 100 header len:67
240123 15:19:53 1117520 http_Protocol: Sending 67 bytes
240123 15:19:53 1117520 unknown.1:35@gentoo http_Protocol:  Process. lp:(nil) reqstate: 0
240123 15:19:53 1117520 unknown.1:35@gentoo Xrootd_Protocol: 0000 Bridge req=3003 dlen=0 blen=0
240123 15:19:53 1117520 unknown.1:35@gentoo http_Protocol: Process is exiting rc:1
240123 15:19:53 1117520 unknown.1:35@gentoo Xrootd_Response: 0000 sending err 3004: close does not refer to an open file
240123 15:19:53 1117520 http_Req:  XrdHttpReq::Error
240123 15:19:53 1117520 unknown.1:35@gentoo http_Req: PostProcessHTTPReq req: 4 reqstate: 0
240123 15:19:53 1117520 unknown.1:35@gentoo http_Req: PostProcessHTTPReq mapping Xrd error [3004] to status code [500]
240123 15:19:53 1117520 http_Req:  XrdHttpReq request ended.
240123 15:19:53 1117520 http_Protocol:  Cleanup
240123 15:19:53 1117520 http_Protocol:  Reset
240123 15:19:53 1117520 http_Req:  XrdHttpReq request ended.
240123 15:19:53 1117520 XrootdXeq: unknown.1:35@gentoo disc 0:00:00 (send failure)
240123 15:19:53 1117520 unknown.1:35@gentoo Xrd_Poll: Poller 0 removing FD 35
240123 15:19:53 1117520 unknown.1:35@gentoo Xrd_Poll: FD 35 detached from poller 0; num=0
240123 15:20:15 1117536 Xrd_Inet: Accepted connection on port 8080 from 37@gentoo.cern.ch
240123 15:20:15 1118528 Xrd_Sched: running main accept inq=0
240123 15:20:15 1117536 anon.0:37@gentoo http_Protocol: received dlen: 16
240123 15:20:15 1117536 anon.0:37@gentoo http_Protocol: received dump: 22 03 01 02 00 01 00 01 -04 03 03 75 -128 -30 -42 00 

@bbockelm Did you test locally the change when proposing the pull request?

@abh3
Copy link
Member

abh3 commented Jan 25, 2024

@bbockelm Could you look at this again and see why this is happening?

@amadio
Copy link
Member

amadio commented Jan 25, 2024

@abh3 Cedric already created a pull request with a fix, which I confirmed and merged into devel.

@abh3
Copy link
Member

abh3 commented Jan 25, 2024 via email

@ccaffy
Copy link
Contributor

ccaffy commented Jan 25, 2024

Hi @abh3,

Brian used the Bridge to set the user-agent for the transfer and did not handle that case in the callback that is ran once the bridge executed the command. This prevented the file open to happen. I've added that handling (very simple handling) in the PostProcessHttpReq() member function so it does not mess-up the rest of the operations :)

@amadio
Copy link
Member

amadio commented Jan 25, 2024

The problem was introduced by this pull request, and after I found out it broke http transfers, Cedric looked at it and fixed it. Today I'm going to make another release (5.6.6) including the fix.

@abh3
Copy link
Member

abh3 commented Jan 25, 2024 via email

@amadio
Copy link
Member

amadio commented Jan 25, 2024

My plan is to expand as much as possible our test suite to cover the most common operations such that we catch problems with patches when the pull request is opened. In #2134, the problem was caught by the new CI, but not the old. I am working on adding tests with http(s) transfers and a few authentication methods for 5.7.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants