Sizes in body when using client.post() #110

Closed
BobYoplait opened this Issue Apr 25, 2012 · 8 comments

Projects

None yet

6 participants

@BobYoplait

Hello,

When I do
curl -d "q=%3A&version=2.2&start=0&rows=10&indent=on" http://192.168.0.202:8080/solr/lg_fr_alpha4/select/?

the ouput XML stream is fine. When I use wireshark I can see in the middle of the XML stream, some size values once in a while:

<int name="warranty">-1</int>
<int 

2000
name="in_stock_format">0

breaking the XML. I guess that's part of the HTTP Post protocol. When I use:

req << header("Content-Length", content_length);
req << header("Content-Type", content_type);
req << body(solrQuery);

http::client client;
http::client::response response;

response = client.post(req);

string myString = body(response);

The "2000" size values appears also in "myString". What is the right way to use http::client or response such that this size values disappear from myString ??? Thank you.

@BobYoplait

I use cpp-netlib-0.9.3 and boost_1_49_0

@divyekapoor

#69 - Hasn't been fixed yet I guess.

@deanberris deanberris was assigned May 10, 2012
@deanberris
Member

This is a bug -- and Divye is right this hasn't been fixed yet.

Would anybody else be willing to submit a pull request for this? I can certainly take a look at this but I'll be busy with large-churn efforts in the meantime for other parts of the library.

@jbandela

This bug appears in the async implementation in 0.9.4. It occurs because the body handler is not handling chunked encoding and the chunk sizes are being incorporated into the text of the body

@jbandela

As mentioned above, the body handler does not handle chunked encoding. Here is a quick fix. Use the function below instead of body() to obtain the body. For example

std::string body = chunked_body(response);

std::string chunked_body(boost::network::http::client::response& response){
std::string body;
std::string partial_parsed = boost::network::http::body(response);
auto transfer_encoding_range = boost::network::http::headers(response)["Transfer-Encoding"];
if (!empty(transfer_encoding_range) &&boost::iequals(boost::begin(transfer_encoding_range)->second, "chunked")) {
auto begin = partial_parsed.begin();
std::string crlf = "\r\n";
for(auto iter = std::search(begin,partial_parsed.end(),crlf.begin(),crlf.end());iter != partial_parsed.end();iter = std::search(begin,partial_parsed.end(),crlf.begin(),crlf.end())){
std::string line(begin,iter);
if(line.empty()) break;
std::stringstream stream(line);
int len;
stream >> std::hex >> len;
iter += 2;
if(!len) break;
if(len <= partial_parsed.end() - iter){
body.insert(body.end(),iter,iter + len);
iter += len;
}
begin = iter;
}

}
else{
    std::swap(body,partial_parsed);
}

return body;

}

@lijieyu
lijieyu commented Dec 28, 2012

Hi jbandela,

I found a small issue in chunked_body() . I added "iter +=2" as below, it works fine. Hope it helps.

if(!len) break;
if(len <= partial_parsed.end() - iter){
body.insert(body.end(),iter,iter + len);
iter += len;
}
iter += 2; // added this line, it works fine
begin = iter;
}

@deanberris
Member

I just remembered this sitting on my inbox. Can you make this part of the actual parsing implementation for the body in 0.9-devel? I'm looking to release 0.9.5 soon and would love to have this fixed in that version.

@skystrife

I've added my attempt at integrating jbandela's function into the codebase as a pull request. Not entirely sure is it's in the correct place, but it appears to be fixing the issue for me.

@deanberris deanberris pushed a commit that referenced this issue Feb 25, 2013
@skystrife skystrife Fix issues #110 and #168. bd90c39
@deanberris deanberris pushed a commit that referenced this issue Feb 28, 2013
@skystrife skystrife Fix issue #110 and issue #168. c6c1607
@deanberris deanberris closed this Jan 13, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment