-
Notifications
You must be signed in to change notification settings - Fork 321
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle early responses #473
Conversation
9965eb7
to
8529ffa
Compare
.rubocop.yml
Outdated
## Lint ######################################################################## | ||
|
||
Lint/HandleExceptions: | ||
Enabled: false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we return nil
or something in that rescue
, Rubocop will be satisfied.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since #send_request
is a "command" method, I feel a bit weird explicitly returning a value. I always considered adding a comment to be enough when swallowing errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with you, but I'm slightly concerned about disabling this particular cop project-wide. It could catch a real bug someday. Perhaps disabling it just the once with an inline comment is an agreeable compromise?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, updated.
cf2aeaa
to
eda5b52
Compare
lib/http/request/writer.rb
Outdated
@@ -100,6 +111,8 @@ def write(data) | |||
break unless data.bytesize > length | |||
data = data.byteslice(length..-1) | |||
end | |||
rescue Errno::EPIPE | |||
raise # re-raise Errno::EPIPE |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO this code is pretty self-explanatory and doesn't need comment.
lib/http/request/writer.rb
Outdated
write(chunk) | ||
end | ||
rescue Errno::EPIPE # rubocop:disable Lint/HandleExceptions | ||
# server doesn't need any more data |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO it worth to explicitly add nil
at the end of rescue
block and remove rubocop disable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh. It was discussed previously... Mmm... I still think we should just return nil
from the rescue
. Also body can be shortened:
def send_request
each_chunk { |chunk| write chunk }
rescue Errno::EPIPE
# no more data
nil
end
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On one hand I think that Lint/HandleExceptions
is pretty useless and enforces developer to write more than needed. On the other hand I prefer to see:
begin
# ...
rescue
nil
end
than
begin
# ...
rescue
end
IMO it's more readable that way. Easier to understand that nil
is result in case of error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know that one might argue that we don't require explicit nil in case of return if something
, but it's a bit different - it's one-liner. And reading it is pretty straight-forward to me. And when we explicitly return from non-one-liner code we add nil
to return:
if something
# ...
return nil # <-- explicit `nil` here looks better than bare `return`
end
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, updated.
eda5b52
to
8f9081c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Please rebase on top of master though.
HTTP servers typically wait until the whole request body has been received before sending the response. However, some servers might send out a response early, before they've read the whole request body. For example, they might detect the request headers were invalid, and they want to tell that to the client immediately. For example, sending a large request to http://google.com will return an early `400 Bad Request` response. In this case HTTP.rb will raise an Errno::EPIPE exception, because it tried to write to the socket but it couldn't because the server closed its end. However, the server still returned a valid response, so it's not really erroneous behaviour (e.g. curl doesn't print any errors in the same situations). # Before HTTP.get("http://google.com", body: "a" * 1*1024*1024) #~> Errno::EPIPE # After HTTP.get("http://google.com", body: "a" * 1*1024*1024) #=> #<HTTP::Response 400 Bad Request ...> This commit ignores Errno::EPIPE exceptions when writing request data to the socket.
We want to stop writing after server has closed its end of the pipe (as no more data will be accepted) and proceed with reading the response.
8f9081c
to
1c3d6a0
Compare
@ixti Done. |
Thanks! |
Update ruby-http to 4.4.1. ## 4.4.1 (2020-03-29) * Backport [#590](httprb/http#590) Fix parser failing on some edge cases. ([@ixti]) ## 4.4.0 (2020-03-25) * Backport [#587](httprb/http#587) Fix redirections when server responds with multiple Location headers. ([@ixti]) * Backport [#599](httprb/http#599) Allow passing HTTP::FormData::{Multipart,UrlEncoded} object directly. ([@ixti]) ## 4.3.0 (2020-01-09) * Backport [#581](httprb/http#581) Add Ruby-2.7 compatibility. ([@ixti], [@janko]) ## 4.2.0 (2019-10-22) * Backport [#489](httprb/http#489) Fix HTTP parser. ([@ixti], [@fxposter]) ## 4.1.1 (2019-03-12) * Add `HTTP::Headers::ACCEPT_ENCODING` constant. ([@ixti]) ## 4.1.0 (2019-03-11) * [#533](httprb/http#533) Add URI normalizer feature that allows to swap default URI normalizer. ([@mamoonraja]) ## 4.0.5 (2019-02-15) * Backport [#532](httprb/http#532) from master. Fix pipes support in request bodies. ([@ixti]) ## 4.0.4 (2019-02-12) * Backport [#506](httprb/http#506) from master. Skip auto-deflate when there is no body. ([@Bonias]) ## 4.0.3 (2019-01-18) * Fix missing URL in response wrapped by auto inflate. ([@ixti]) * Provide `HTTP::Request#inspect` method for debugging purposes. ([@ixti]) ## 4.0.2 (2019-01-15) * [#506](httprb/http#506) Fix instrumentation feature. ([@paul]) ## 4.0.1 (2019-01-14) * [#515](httprb/http#515) Fix `#build_request` and `#request` to respect default options. ([@RickCSong]) ## 4.0.0 (2018-10-15) * [#482](httprb/http#482) [#499](httprb/http#499) Introduce new features injection API with 2 new feaures: instrumentation (compatible with ActiveSupport::Notification) and logging. ([@paul]) * [#473](httprb/http#473) Handle early responses. ([@janko-m]) * [#468](httprb/http#468) Rewind `HTTP::Request::Body#source` once `#each` is complete. ([@ixti]) * [#467](httprb/http#467) Drop Ruby 2.2 support. ([@ixti]) * [#436](httprb/http#436) Raise ConnectionError when writing to socket fails. ([@janko-m]) * [#438](httprb/http#438) Expose `HTTP::Request::Body#source`. ([@janko-m]) * [#446](httprb/http#446) Simplify setting a timeout. ([@mikegee]) * [#451](httprb/http#451) Reduce memory usage when reading response body. ([@janko-m]) * [#458](httprb/http#458) Extract HTTP::Client#build_request method. ([@tycoon]) * [#462](httprb/http#462) Fix HTTP::Request#headline to allow two leading slashes in path. ([@scarfacedeb]) * [#454](httprb/http#454) [#464](httprb/http#464) [#384](httprb/http#384) Fix #readpartial not respecting max length argument. ([@janko-m], [@marshall-lee])
307: Improve error handling, rescue EPIPE and Net::* errors r=brunoocasali a=brunoocasali # Pull Request This PR fixes a bad DX when comes to errors, if a user tries to upload a big dataset, they will face some errors, one of them was related to the `Errno::EPIPE` error, which is really hard to figure out. The idea of this PR is to handle the errors and improve the user experience. ## What does this PR do? Fixes #305 ## PR checklist Please check if your PR fulfills the following requirements: - [x] Does this PR fix an existing issue? - [x] Have you read the contributing guidelines? - [x] Have you made sure that the title is accurate and descriptive of the changes? Thank you so much for contributing to Meilisearch! - [ ] Try something with other lib, which address the pipes error better like: httprb/http#473 Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
HTTP servers typically wait until the whole request body has been received before sending the response. However, some servers might send out a response early, before they've read the whole request body. For example, they might detect the request headers were invalid, and they want to tell that to the client immediately.
For example, sending a large request to http://google.com will return an early
400 Bad Request
response. In this case http.rb will raise anErrno::EPIPE
exception (wrapped in aHTTP::ConnectionError
), because it tried to write to the socket but it couldn't because the server closed its end. However, the server still returned a valid response, so it's not really erroneous behaviour (e.g. curl doesn't print any errors in the same situations).This commit changes
Request::Writer
to stop writing when anErrno::EPIPE
exception occurred.