HTTP request bodies randomly gets cut off in Windows

[null8626]

Describe the bug
HTTP request bodies done with dpp::cluster::request randomly gets cut off in Windows.

To Reproduce
I am currently using the latest version of D++ obtained from dl.dpp.dev, version 10.1.0.

#include <dpp/dpp.h>

#include <semaphore>
#include <iostream>
#include <cstdlib>
#include <utility>

static std::binary_semaphore g_sem{0};

int main() {
  const auto discord_token{std::getenv("BOT_TOKEN")};

  if (discord_token == nullptr) {
    std::cerr << "error: missing BOT_TOKEN environment variable" << std::endl;
    return 1;
  }

  dpp::cluster bot{discord_token};

  bot.start(dpp::start_type::st_return);

  dpp::http_headers headers{};

  headers.insert(std::pair("Authorization", "top.gg API token"));
  headers.insert(std::pair("Connection", "close"));
  headers.insert(std::pair("Content-Type", "application/json"));
  headers.insert(std::pair("User-Agent", "topgg (https://github.com/top-gg-community/cpp-sdk) D++"));

  bot.request("https://top.gg/api/bots/264811613708746752", dpp::http_method::m_get, [](const auto& response) {
    std::cout << "===BODY START===\n" << response.body << "\n===BODY END===\nSIZE: " << response.body.size() << std::endl;
    g_sem.release();
  }, "", "application/json", headers);

  g_sem.acquire();

  return 0;
}

Expected behavior
The HTTP request should've retrieved the whole body. This request sometimes only gives less body than expected, and the difference between requests is usually random.

For example, when I ran the code above, I got:

• 2,041 bytes
• 3,746 bytes (the entire body as expected)
• 3,746 bytes (the entire body as expected)
• 3,410 bytes

Screenshots
First iteration:

Second and third iteration:

No re-compilations were done between iterations, I simply reran the executable.

System Details:

• OS: Windows 11, compiled with Microsoft Visual C++ under Visual Studio 2022
• Discord Client: None -- I was using dpp::cluster purely to test dpp::cluster::request.

Additional context
I was testing the Top.gg C++ SDK and needed dpp::cluster::request to send requests to the Top.gg API.

[null8626]

Fixed it by removing the Connection: close header.

Though this was also partly D++'s fault, as D++ also writes its own Connection header, which causes duplicate Connection headers.

[Jaskowicz1]

Reopening as we should fix this.

[braindigitalis]

connection headers from the user should be discarded. It is not something the user should try to force direct control over and would break http_client
There are probably other headers we should block user setting too, such as content-length (this is calculated by the lib).

[braindigitalis]

these are the headers set by httpsclient.cpp which should not be allowed in user headers

"Host: " + this->hostname + "\r\n"
"pragma: no-cache\r\n"
"Connection: keep-alive\r\n"
"Content-Length: " + // ....