Cannot get connection information when using TCP Fast Open

[irl]

I did this

Using the easy interface and the CURLOPT_TCP_FASTOPEN option, fetch a webpage. Then look for CURLINFO_LOCAL_PORT.

#include <stdio.h>
#include <curl/curl.h>

int main(void)
{
  CURL *curl;
  CURLcode res;

  curl = curl_easy_init();
  if(curl) {
    curl_easy_setopt(curl, CURLOPT_URL, "http://pathspider.net/");
    curl_easy_setopt(curl, CURLOPT_TCP_FASTOPEN, 1);
    res = curl_easy_perform(curl);

    if(CURLE_OK == res) {
      long *ct;
      res = curl_easy_getinfo(curl, CURLINFO_LOCAL_PORT, &ct);

      if((CURLE_OK == res) && ct == NULL) {
        printf("The operating was successful but we got nothing!\n");
      } else {
        printf("We used local port: %d\n", *ct);
      }
    }

    curl_easy_cleanup(curl);
  }
  return 0;
}

Instead of getting a port, I got a NULL, which is suboptimal for my use case.

I expected the following

A source port.

curl/libcurl version

curl 7.52.1 (x86_64-pc-linux-gnu) libcurl/7.52.1 OpenSSL/1.0.2k zlib/1.2.8 libidn2/0.16 libpsl/0.17.0 (+libidn2/0.16) libssh2/1.7.0 nghttp2/1.18.1 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL

operating system

Debian testing (curl 7.52.1-3)

[jay]

That looks wrong, it doesn't write a pointer it writes a long to the address you pass it

--- a	Thu Mar 16 17:02:35 2017
+++ b	Thu Mar 16 17:08:38 2017
@@ -13,13 +13,10 @@
     res = curl_easy_perform(curl);
 
     if(CURLE_OK == res) {
-      long *ct;
+      long ct;
       res = curl_easy_getinfo(curl, CURLINFO_LOCAL_PORT, &ct);
-
-      if((CURLE_OK == res) && ct == NULL) {
-        printf("The operating was successful but we got nothing!\n");
-      } else {
-        printf("We used local port: %d\n", *ct);
+      if(CURLE_OK == res) {
+        printf("We used local port: %d\n", ct);
       }
     }
 

[bagder]

(I took this as an opportunity and added this as an example to the man page in 49f7b13)

[jay]

Despite the example he still has a point, it looks like the local port isn't saved when fastopen is used, see https://github.com/curl/curl/blob/curl-7_53_1/lib/connect.c#L674

[bagder]

hm, I wonder why that was made. I can't remember any discussions around that. Surely that would all work fine even with TFO ?

[ghedo]

So, I don't quite remember the reasoning behind this, but one possible explanation is that when Curl_updateconninfo() is called the connection may not actually be established yet. With TFO on Linux this would only happen after the first sendto().

[ghedo]

That said, there might be a better way of avoiding that problem.

[jay]

AFAICT it's only called after the connection is made... or at least what I interpret that to be, I'm not familiar with TFO. It's only called these two places:
https://github.com/curl/curl/blob/curl-7_53_1/lib/connect.c#L808
https://github.com/curl/curl/blob/curl-7_53_1/lib/url.c#L6843

[bagder]

And even if it was called before the connection is made, all the function calls within Curl_updateconninfo check return codes and take precautions so they should fail gracefully...

[ghedo]

@jay define "connection is made" :) The TCP connection is initiated on the wire when curl starts sending the HTTP request (or whatever). That is when sendto() is called. Before then the socket is not connected.

@bagder doesn't look graceful to me, given the error = SOCKERRNO, curl will not proceed https://github.com/curl/curl/blob/curl-7_53_1/lib/connect.c#L823 (edit: nvm, missed a return there).

[ghedo]

It does print a bunch of failf() though, which is not exactly nice during normal (for TFOl operation...

[ghedo]

And even if they do fail nicely, that still leaves the problem of getinfo returning nothing. Should Curl_updateconninfo() be called again after the initial send for TFO and that condition removed?

[irl]

Sorry for the example, I was attempting to replicate the problem I was finding in pycurl in C.

I attempted the following in sendf.c but this didn't work:

...
#ifdef MSG_FASTOPEN /* Linux */
  if(conn->bits.tcp_fastopen) {
    bytes_written = sendto(sockfd, mem, len, MSG_FASTOPEN,
                           conn->ip_addr->ai_addr, conn->ip_addr->ai_addrlen);
    conn->bits.tcp_fastopen = FALSE;
    Curl_persistconninfo(conn);
  }
  else
#endif
...

I guess this is probably not how these functions work.

Edit: maybe this should have been Curl_updateconninfo() instead of Curl_persistconninfo()?
Edit: I guess it's not that easy, something about transport not being connected.

[ghedo]

@irl hmm, that could be caused by the fact that a non-blocking socket is used. Another experiment would be to call Curl_updateconninfo() after the first successful send. Basically just before https://github.com/curl/curl/blob/master/lib/sendf.c#L373 but only if bytes_written is greater than 0.

I can't test right now, I'll look into this later.

[irl]

Yep, non-blocking socket issue. Adding Curl_updateconninfo() after the first successful send works in the case that TCP fast open failed and a non-TFO TCP connection was used, but doesn't work in the case that TFO was successful and only one send is ever performed (because the whole GET request fits on the SYN).

I'm playing with setting a needs_update bit and adding Curl_updateconninfo() after both the first successful write and the first successful read.

[irl]

Pull request #1335 should solve this, just doing a little more testing.

Edit: nope, needs more work. It's the right idea, and if you drop the clearing of the flags it works, but I think the test conditions in the if statements are wrong.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[irl]

This is still an issue, a PR is waiting for review in #1847.