New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sizes in body when using client.post() #110

Closed
BobYoplait opened this Issue Apr 25, 2012 · 8 comments

Comments

Projects
None yet
6 participants
@BobYoplait

BobYoplait commented Apr 25, 2012

Hello,

When I do
curl -d "q=%3A&version=2.2&start=0&rows=10&indent=on" http://192.168.0.202:8080/solr/lg_fr_alpha4/select/?

the ouput XML stream is fine. When I use wireshark I can see in the middle of the XML stream, some size values once in a while:

<int name="warranty">-1</int>
<int 

2000
name="in_stock_format">0

breaking the XML. I guess that's part of the HTTP Post protocol. When I use:

req << header("Content-Length", content_length);
req << header("Content-Type", content_type);
req << body(solrQuery);

http::client client;
http::client::response response;

response = client.post(req);

string myString = body(response);

The "2000" size values appears also in "myString". What is the right way to use http::client or response such that this size values disappear from myString ??? Thank you.

@BobYoplait

This comment has been minimized.

Show comment
Hide comment
@BobYoplait

BobYoplait Apr 25, 2012

I use cpp-netlib-0.9.3 and boost_1_49_0

BobYoplait commented Apr 25, 2012

I use cpp-netlib-0.9.3 and boost_1_49_0

@divyekapoor

This comment has been minimized.

Show comment
Hide comment
@divyekapoor

divyekapoor Apr 28, 2012

#69 - Hasn't been fixed yet I guess.

divyekapoor commented Apr 28, 2012

#69 - Hasn't been fixed yet I guess.

@ghost ghost assigned deanberris May 10, 2012

@deanberris

This comment has been minimized.

Show comment
Hide comment
@deanberris

deanberris May 10, 2012

Member

This is a bug -- and Divye is right this hasn't been fixed yet.

Would anybody else be willing to submit a pull request for this? I can certainly take a look at this but I'll be busy with large-churn efforts in the meantime for other parts of the library.

Member

deanberris commented May 10, 2012

This is a bug -- and Divye is right this hasn't been fixed yet.

Would anybody else be willing to submit a pull request for this? I can certainly take a look at this but I'll be busy with large-churn efforts in the meantime for other parts of the library.

@jbandela

This comment has been minimized.

Show comment
Hide comment
@jbandela

jbandela May 20, 2012

This bug appears in the async implementation in 0.9.4. It occurs because the body handler is not handling chunked encoding and the chunk sizes are being incorporated into the text of the body

jbandela commented May 20, 2012

This bug appears in the async implementation in 0.9.4. It occurs because the body handler is not handling chunked encoding and the chunk sizes are being incorporated into the text of the body

@jbandela

This comment has been minimized.

Show comment
Hide comment
@jbandela

jbandela May 21, 2012

As mentioned above, the body handler does not handle chunked encoding. Here is a quick fix. Use the function below instead of body() to obtain the body. For example

std::string body = chunked_body(response);

std::string chunked_body(boost::network::http::client::response& response){
std::string body;
std::string partial_parsed = boost::network::http::body(response);
auto transfer_encoding_range = boost::network::http::headers(response)["Transfer-Encoding"];
if (!empty(transfer_encoding_range) &&boost::iequals(boost::begin(transfer_encoding_range)->second, "chunked")) {
auto begin = partial_parsed.begin();
std::string crlf = "\r\n";
for(auto iter = std::search(begin,partial_parsed.end(),crlf.begin(),crlf.end());iter != partial_parsed.end();iter = std::search(begin,partial_parsed.end(),crlf.begin(),crlf.end())){
std::string line(begin,iter);
if(line.empty()) break;
std::stringstream stream(line);
int len;
stream >> std::hex >> len;
iter += 2;
if(!len) break;
if(len <= partial_parsed.end() - iter){
body.insert(body.end(),iter,iter + len);
iter += len;
}
begin = iter;
}

}
else{
    std::swap(body,partial_parsed);
}

return body;

}

jbandela commented May 21, 2012

As mentioned above, the body handler does not handle chunked encoding. Here is a quick fix. Use the function below instead of body() to obtain the body. For example

std::string body = chunked_body(response);

std::string chunked_body(boost::network::http::client::response& response){
std::string body;
std::string partial_parsed = boost::network::http::body(response);
auto transfer_encoding_range = boost::network::http::headers(response)["Transfer-Encoding"];
if (!empty(transfer_encoding_range) &&boost::iequals(boost::begin(transfer_encoding_range)->second, "chunked")) {
auto begin = partial_parsed.begin();
std::string crlf = "\r\n";
for(auto iter = std::search(begin,partial_parsed.end(),crlf.begin(),crlf.end());iter != partial_parsed.end();iter = std::search(begin,partial_parsed.end(),crlf.begin(),crlf.end())){
std::string line(begin,iter);
if(line.empty()) break;
std::stringstream stream(line);
int len;
stream >> std::hex >> len;
iter += 2;
if(!len) break;
if(len <= partial_parsed.end() - iter){
body.insert(body.end(),iter,iter + len);
iter += len;
}
begin = iter;
}

}
else{
    std::swap(body,partial_parsed);
}

return body;

}

@lijieyu

This comment has been minimized.

Show comment
Hide comment
@lijieyu

lijieyu Dec 28, 2012

Hi jbandela,

I found a small issue in chunked_body() . I added "iter +=2" as below, it works fine. Hope it helps.

if(!len) break;
if(len <= partial_parsed.end() - iter){
body.insert(body.end(),iter,iter + len);
iter += len;
}
iter += 2; // added this line, it works fine
begin = iter;
}

lijieyu commented Dec 28, 2012

Hi jbandela,

I found a small issue in chunked_body() . I added "iter +=2" as below, it works fine. Hope it helps.

if(!len) break;
if(len <= partial_parsed.end() - iter){
body.insert(body.end(),iter,iter + len);
iter += len;
}
iter += 2; // added this line, it works fine
begin = iter;
}

@deanberris

This comment has been minimized.

Show comment
Hide comment
@deanberris

deanberris Jan 29, 2013

Member

I just remembered this sitting on my inbox. Can you make this part of the actual parsing implementation for the body in 0.9-devel? I'm looking to release 0.9.5 soon and would love to have this fixed in that version.

Member

deanberris commented Jan 29, 2013

I just remembered this sitting on my inbox. Can you make this part of the actual parsing implementation for the body in 0.9-devel? I'm looking to release 0.9.5 soon and would love to have this fixed in that version.

@skystrife

This comment has been minimized.

Show comment
Hide comment
@skystrife

skystrife Feb 24, 2013

I've added my attempt at integrating jbandela's function into the codebase as a pull request. Not entirely sure is it's in the correct place, but it appears to be fixing the issue for me.

skystrife commented Feb 24, 2013

I've added my attempt at integrating jbandela's function into the codebase as a pull request. Not entirely sure is it's in the correct place, but it appears to be fixing the issue for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment