While it was technically interesting and fun to tunnel arbitrary protocols over a semi-compliant Rack interface, nobody actually does it (and anybody who does can look in our git history). This was from back in 2009 when this was one of the few servers that could handle chunked uploads,were one of the few users of chunked uploads, nowadays everyone does it! (or do they? :) A newer version of exec_cgi.rb still lives on in the repository of yet another horribly-named server, but there's no point in bloating the installation footprint of somewhat popular server such as unicorn.
There may be code in a before_fork hook which should run only once, document an example using a guard variable since it may not be immediately obvious to all users. Inspired-by: Bráulio Bhavamitra <firstname.lastname@example.org> http://bogomips.org/unicorn-public/m/20141004015707.GA1951@dcvr.yhbt.net.html
The PrivateTmp feature of systemd breaks the usage of /tmp for the shared Unix domain socket between nginx and unicorn, so discourage the use of /tmp in that case. While we're at it, use consistent paths for everything and use an obviously intended-for-user-customization "/path/to" prefix instead of "/tmp" ML-Ref: CAKLVLx_t+9zWMhquMWDfStrxS7xrNoGmN0ZDsjSCUE=VxU+oyQ@mail.gmail.com Reported-by: David Wilkins <email@example.com>
In my testing, only dropped clients over Unix domain sockets or loopback TCP were detected with this option. Since many nginx+unicorn combinations run on the same host, this is not a problem. Furthermore, tcp_nodelay:true appears to work over loopback, so remove the requirement for tcp_nodelay:false.
This patch checks incoming connections and avoids calling the application if the connection has been closed. It works by sending the beginning of the HTTP response before calling the application to see if the socket can successfully be written to. By enabling this feature users can avoid wasting application rendering time only to find the connection is closed when attempting to write, and throwing out the result. When a client disconnects while being queued or processed, Nginx will log HTTP response 499 but the application will log a 200. Enabling this feature will minimize the time window during which the problem can arise. The feature is disabled by default and can be enabled by adding 'check_client_connection true' to the unicorn config. [ew: After testing this change, Tom Burns wrote: So we just finished the US Black Friday / Cyber Monday weekend running unicorn forked with the last version of the patch I had sent you. It worked splendidly and helped us handle huge flash sales without increased response time over the weekend. Whereas in previous flash traffic scenarios we would see the number of HTTP 499 responses grow past the number of real HTTP 200 responses, over the weekend we saw no growth in 499s during flash sales. Unexpectedly the patch also helped us ward off a DoS attack where the attackers were disconnecting immediately after making a request. ref: <CAK4qKG3rkfVYLyeqEqQyuNEh_nZ8yw0X_cwTxJfJ+TOU+y8Ffirstname.lastname@example.org> ] Signed-off-by: Eric Wong <email@example.com>
This adds a little more flexibility to the nginx config, especially as protocols (e.g. SPDY) become more prevalent. Suggested-by: Eike Herzbach <firstname.lastname@example.org>
Ruby 2.0.0dev is the future and includes a CoW-friendly GC, so we shall encourage folks to give Ruby 2.0.0dev a spin.
This was broken since v3.3.1 since nginx relies on a closed socket (and not Content-Length/Transfer-Encoding) to detect a response completion. We have to close the client socket before invoking GC to ensure the client sees the response in a timely manner.  - commit b72a86f
I've tested with nginx 1.0.0 and confirmed "proxy_buffering off;" can cause Unicorn to block on a slow client reading a large response. While there's a potential (client-visible) performance improvement with Rails 3.1 streaming responses, it can also hurt the server with slow clients. Rainbows! with (ThreadSpawn or ThreadPool) is probably the best way to do streaming responses efficiently from all angles (from a server, client and programmer time perspective).
There's no need to use listen unless you use non-default port or can enable "deferred" or "httpready" (which you usually want).
Sending the same signal faster than the receiver can process means signals can get lost.
This makes the user (sysadmin in this case) more aware if the upgrade fails or doesn't work as intended. This change could be more useful for Rainbows! with its long-running responses.
Thanks to Michael Guterl for informing us of the issues and testing the monkey patch.
From RFC 2616, section 14.20: > Comparison of expectation values is case-insensitive for > unquoted tokens (including the 100-continue token), and is > case-sensitive for quoted-string expectation-extensions.
Now that upstream curl supports this functionality, there's no reason to duplicate it here as an example.
This change gives applications full control to deny clients from uploading unwanted message bodies. This also paves the way for doing things like upload progress notification within applications in a Rack::Lint-compatible manner. Since we don't support HTTP keepalive, so we have more freedom here by being able to close TCP connections and deny clients the ability to write to us (and thus wasting our bandwidth). While I could've left this feature off by default indefinitely for maximum backwards compatibility (for arguably broken applications), Unicorn is not and has never been about supporting the lowest common denominator.
Then hopefully soon we'll be able to get rid of this script...
This includes an example of tunneling the git protocol inside a TE:chunked HTTP request. The example is unfortunately contrived in that it relies on the custom examples/cat-chunk-proxy.rb script in the client. My initial wish was to have a generic tool like curl(1) operate like this: cat > ~/bin/cat-chunk-proxy.sh <<EOF #!/bin/sh exec curl -sfNT- http://$1:$2/ EOF chmod +x ~/bin/cat-chunk-proxy.sh GIT_PROXY_COMMAND=cat-chunk-proxy.sh git clone git://0:8080/foo Unfortunately, curl will attempt a blocking read on stdin before reading the TCP socket; causing the git-clone consumer to starve. This does not appear to be a problem with the new server code for handling chunked requests.
This adds support for handling POST/PUT request bodies sent with chunked transfer encodings ("Transfer-Encoding: chunked"). Attention has been paid to ensure that a client cannot OOM us by sending an extremely large chunk. This implementation is pure Ruby as the Ragel-based implementation in rfuzz didn't offer a streaming interface. It should be reasonably close to RFC-compliant but please test it in an attempt to break it. The more interesting part is the ability to stream data to the hosted Rack application as it is being transferred to the server. This can be done regardless if the input is chunked or not, enabling the streaming of POST/PUT bodies can allow the hosted Rack application to process input as it receives it. See examples/echo.ru for an example echo server over HTTP. Enabling streaming also allows Rack applications to support upload progress monitoring previously supported by Mongrel handlers. Since Rack specifies that the input needs to be rewindable, this input is written to a temporary file (a la tee(1)) as it is streamed to the application the first time. Subsequent rewinded reads will read from the temporary file instead of the socket. Streaming input to the application is disabled by default since applications may not necessarily read the entire input body before returning. Since this is a completely new feature we've never seen in any Ruby HTTP application server before, we're taking the safe route by leaving it disabled by default. Enabling this can only be done globally by changing the Unicorn HttpRequest::DEFAULTS hash: Unicorn::HttpRequest::DEFAULTS["unicorn.stream_input"] = true Similarly, a Rack application can check if streaming input is enabled by checking the value of the "unicorn.stream_input" key in the environment hashed passed to it. All of this code has only been lightly tested and test coverage is lacking at the moment.  - http://tools.ietf.org/html/rfc2616#section-3.6.1