Sizes in body when using client.post()

[ghost]

Hello,

When I do
curl -d "q=%3A&version=2.2&start=0&rows=10&indent=on" http://192.168.0.202:8080/solr/lg_fr_alpha4/select/?

the ouput XML stream is fine. When I use wireshark I can see in the middle of the XML stream, some size values once in a while:

<int name="warranty">-1</int>
<int 

2000
name="in_stock_format">0

breaking the XML. I guess that's part of the HTTP Post protocol. When I use:

req << header("Content-Length", content_length);
req << header("Content-Type", content_type);
req << body(solrQuery);

http::client client;
http::client::response response;

response = client.post(req);

string myString = body(response);

The "2000" size values appears also in "myString". What is the right way to use http::client or response such that this size values disappear from myString ??? Thank you.

[ghost]

I use cpp-netlib-0.9.3 and boost_1_49_0

[divyekapoor]

#69 - Hasn't been fixed yet I guess.

[deanberris]

This is a bug -- and Divye is right this hasn't been fixed yet.

Would anybody else be willing to submit a pull request for this? I can certainly take a look at this but I'll be busy with large-churn efforts in the meantime for other parts of the library.

[jbandela]

This bug appears in the async implementation in 0.9.4. It occurs because the body handler is not handling chunked encoding and the chunk sizes are being incorporated into the text of the body

[jbandela]

As mentioned above, the body handler does not handle chunked encoding. Here is a quick fix. Use the function below instead of body() to obtain the body. For example

std::string body = chunked_body(response);

std::string chunked_body(boost::network::http::client::response& response){
std::string body;
std::string partial_parsed = boost::network::http::body(response);
auto transfer_encoding_range = boost::network::http::headers(response)["Transfer-Encoding"];
if (!empty(transfer_encoding_range) &&boost::iequals(boost::begin(transfer_encoding_range)->second, "chunked")) {
auto begin = partial_parsed.begin();
std::string crlf = "\r\n";
for(auto iter = std::search(begin,partial_parsed.end(),crlf.begin(),crlf.end());iter != partial_parsed.end();iter = std::search(begin,partial_parsed.end(),crlf.begin(),crlf.end())){
std::string line(begin,iter);
if(line.empty()) break;
std::stringstream stream(line);
int len;
stream >> std::hex >> len;
iter += 2;
if(!len) break;
if(len <= partial_parsed.end() - iter){
body.insert(body.end(),iter,iter + len);
iter += len;
}
begin = iter;
}

}
else{
    std::swap(body,partial_parsed);
}

return body;

}

[lijieyu]

Hi jbandela,

I found a small issue in chunked_body() . I added "iter +=2" as below, it works fine. Hope it helps.

if(!len) break;
if(len <= partial_parsed.end() - iter){
body.insert(body.end(),iter,iter + len);
iter += len;
}
iter += 2; // added this line, it works fine
begin = iter;
}

[deanberris]

I just remembered this sitting on my inbox. Can you make this part of the actual parsing implementation for the body in 0.9-devel? I'm looking to release 0.9.5 soon and would love to have this fixed in that version.

[skystrife]

I've added my attempt at integrating jbandela's function into the codebase as a pull request. Not entirely sure is it's in the correct place, but it appears to be fixing the issue for me.