intermittent CURLE_SSL_CONNECT_ERROR (35) triggered by SEC_E_BUFFER_TOO_SMALL and SEC_E_MESSAGE_ALTERED in schannel for https on Windows 8.1 and below

[plujon]

curl-7.70, compiled with nmake mode=dll
Windows: 7 and 8.1, but not 10

I have observed on multiple Windows machines that https requests performed using curl_easy_perform will fail on occasion (on the order of 1/100 tries). The underlying error is a return value of SEC_E_BUFFER_TOO_SMALL (0x80090321) or SEC_E_MESSAGE_ALTERED (0x8009030) from InitializeSecurityContext.

I have been able to reproduce the intermittent failures on Windows 7 and Windows 8.1 machines. I have not been able to reproduce the issue on Windows 10. The problem reproduced when connecting to Ubuntu 16.04 LTS, Ubuntu 18.04 LTS, and Ubuntu 20.04 LTS servers.

I think the bug is in Microsoft code. But perhaps a note should be added to users of libcurl that schannel is unreliable on older Windows.

/* runme 999 https://hi.eewe.us/hi */
#include <curl/curl.h>
#include <stdio.h>

int main(int argc, char *argv[])
{
  if (argc < 1) return 1;
  if (argc < 2) { printf("%s <url>\n", argv[0]); return 1; }
  int count = 1;
  const char *url = NULL;
  for (int i = 1; i < argc; ++i) {
    int x = strtoul(argv[i], 0, 0);
    if (x)
      count = x;
    else
      url = argv[i];
  }
  if (!url) return 1;
  for (int i = 0; i < count; ++i) {
    CURL *curl = curl_easy_init();
    if (!curl) return 1;
    char errbuf[CURL_ERROR_SIZE];
    errbuf[0] = '\0';
    if (curl_easy_setopt(curl, CURLOPT_URL, url)) return 1;
    if (curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errbuf)) return 1;
    printf("get %s\n", url);
    CURLcode res = curl_easy_perform(curl);
    if (CURLE_OK != res) {
      printf("failed after %d attempts\nlibcurl: (%d) %s\n",
              i, res, errbuf);
      return 2;
    }
    curl_easy_cleanup(curl);
  }
  return 0;
}

[plujon]

See Waffle/waffle#128 (comment) and steffengy/schannel-rs#8 .

[bagder]

I think the bug is in Microsoft code. But perhaps a note should be added to users of libcurl that schannel is unreliable on older Windows.

Any suggestion as where (in which documentation) we should add such a mention? Also, I would like to have it more specific than just "is unreliable on older Windows". In general we always recommend user to go with recent versions of everything rather than older, and if you get stuck on older Windows versions using schannel you're already not following that advice...

[plujon]

Probably winbuild/BUILD.WINDOWS.txt .

Perhaps there are other documents on how to build libcurl and why one would choose openssl, mbedtls, nghttp2, zlib, sspi, or not, and I never stumbled upon them. When reading the above document, I wondered why anybody would choose OpenSSL on Windows if the platform already had native support (in something called "sppi", whatever that means), thus simplifying the life of the developer. But now I know at least one reason.

[jay]

This strikes me as a corner case and I'm not convinced that we should note it.

[mback2k]

We could think about handling SEC_E_BUFFER_TOO_SMALL with a retry (if we can figure out which buffer was too small), but I guess SEC_E_MESSAGE_ALTERED should remain a hard failure for obvious reasons.

[plujon]

Users of libcurl on Windows who use schannel (the default) may want to use:

CURLcode res = curl_easy_perform(curl);

if (CURLE_SSL_CONNECT_ERROR /* 35 */ == res) {
  /* might be spurious, https://github.com/curl/curl/issues/5488 */
  res = curl_easy_perform(curl);
}

if (CURLE_OK != res) { . . . }

After adding the above, the test code above succeeds 1000/1000 times for me on Windows 8.1.

The very wary coder might want to use CURLOPT_ERRORBUFFER and search the error buffer for SEC_E_BUFFER_TOO_SMALL or SEC_E_MESSAGE_ALTERED.

[bagder]

I prefer we find and fix the problem in libcurl.

[jay]

I don't think there is a problem in libcurl. Schannel could certainly have a problem. I plan to put this in known issues unless anyone is going to work on it.