Skip to content
This repository has been archived by the owner on Dec 7, 2018. It is now read-only.

Pipelining Fix/Refactoring #73

Merged
merged 18 commits into from Jul 26, 2013
Merged

Conversation

adstage-david
Copy link

Did some refactoring and made it so the request parser can handle pipelining. This closes #69 and #55 and makes #56 obsolete.

All of the specs pass, and I've run most of the examples and they all seem to work as well as they do on master (Rackup handler examples seem to be broken on master?)

Cleanup Changes:

  • Reel::Connection delegates all of its work to Reel::Request::Parser (for socket reads) and Reel::Response::Writer (for socket writes).
  • All reads/writes still go through the connection object, which makes sure the connection is in the proper state for the action and raises a Reel::Connection::StateError if not. No other objects know the parser/writer exist.
  • Reel::Request and Reel::Websocket no longer require a direct link to the request parser - they are initialized with a Reel::RequestInfo object that includes the parsed request headers, url, method, http_version.
  • Reel::Response::Writer knows how to render proper HTTP responses, but will send the socket to Response#render if that method exists (for the StreamResponse subclass) (there was a TODO about extracting this logic).

API Changes:

  • #read can now only be called on the Reel::Request, not connection, since http_parser may accidentally consume bytes of multiple bodies, each Reel::Request keeps an internal buffer of bytes parsed but not yet pulled with #read. This buffer is read from before calling @connection#readpartial
  • added #<< and #write to Reel::Request so you can call both #read and #write on the request itself.
  • Removed #close from Reel::Request - since this should probably be called from the Reel::Connection.

Parser Details:

The Reel::Request::Parser maintains an internal buffer of incoming requests still reading (if body hasn't been read in yet), one currently awaiting response, and a queue of requests to be responded to after the current request is handled. The underlying http parser calls the callbacks on this class when bytes buffered to it add to headers or body.

Reel::Request::Parser has #current_request, which will return the current request being read/responded to, or block and read from the raw socket until a full set of headers has been discovered and the body is ready to read.

@@ -40,21 +42,32 @@ def detach
end

# Reset the current request state
def reset_request(state = :header)
def reset_request(state = :standard)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:standard?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wasn't sure what to call it, it's not in charge of knowing whether it's in "headers" any more, just knowing that the connection still has incoming requests and isn't a websocket. Maybe :ready? The only values it can have are :websocket, :closed, or ____. What do we call the blank?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:ready is fine

@coveralls
Copy link

Coverage Status

Coverage increased (+0%) when pulling 607a696 on adstage-david:pipeline_fix into 341b23f on celluloid:master.

@parser.readpartial(size)
end

def outstanding_request
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor bikeshedding, but perhaps this should be #current_request

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Added this a few days after the first one. I'll make them line up.

@coveralls
Copy link

Coverage Status

Coverage increased (+0%) when pulling 8f0c5b8 on adstage-david:pipeline_fix into 341b23f on celluloid:master.

@adstage-david
Copy link
Author

Thanks for the feedback - everything you mentioned should be fixed now.

@coveralls
Copy link

Coverage Status

Coverage increased (+0%) when pulling 8f0c5b8 on adstage-david:pipeline_fix into 341b23f on celluloid:master.

@tarcieri
Copy link
Member

Looks good now, thanks! 👍

tarcieri added a commit that referenced this pull request Jul 26, 2013
@tarcieri tarcieri merged commit 512aa65 into celluloid:master Jul 26, 2013

if block_given?
# Callback from the http_parser will be calling add_body directly
@on_body = Proc.new(&block)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't need wrapping in another proc does it? @on_body = block should suffice. Given that you're switching on if block_given? you could drop the &block parameter declaration, and make this line @on_body = Proc.new (which inherently consumes the block argument).

tarcieri added a commit that referenced this pull request Jul 26, 2013
@tarcieri
Copy link
Member

Looking over this again, I'm really not a fan of the callback-driven body API. I think that Reel should support a Body object that supports a #read/#readpartial-style API.

I already landed this and will be keeping it around, but I'll probably try to make that change today.

@tarcieri
Copy link
Member

Okay, I see that API was my fault ;)

end
end
it 'streams body properly with #body &block' do
with_socket_pair(8) do |client, connection|
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why "8"? If you're going to bother testing a buffer size, use 1, or preferably 1 and >1

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no good reason, I was just playing with different numbers to make sure they all worked as expected and I left off on 8. It definitely makes sense to have at least one case that's a byte at a time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah after looking at how the tests are written this is probably ok. I'd expect the edge cases to crop up at 1 though

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pipelining Hangs Reel Server
5 participants