troubles with cowboy 2.0.0-pre.9 performance

[ghost]

Hi, Loïc, sorry to bother you with synthetic benchmarks, but could you maybe look into one here the-benchmarker/web-frameworks#58

It's basically just testing how fast it can make requests like get /, get /user/:id, and post /user. Actually the benchmark itself doesn't really matter since I was using wrk instead of its client written in crystal. And my problem is, testing with wrk cowboy 2.0.0-pre.9 in this benchmark seems to be slower than cowboy 1.1. I'm using the same ranch options for both, and the code in the handlers is also almost identical.

The latest results for cowboys are here the-benchmarker/web-frameworks#58 (comment)

Is it maybe a bit too early to test cowboy 2? Or am I just doing something wrong there? Or maybe it's because I'm testing it on my laptop instead of a real server? Then maybe there are some additional options to get more performance out of cowboy on a smaller machine?

my fork of the benchmark for

• cowboy 1.1 https://github.com/idi-ot/which_is_the_fastest/tree/master/elixir/cowboy1
• cowboy 2.0-pre https://github.com/idi-ot/which_is_the_fastest/tree/master/elixir/cowboy2

[essen]

It's true that I did not perform benchmarks yet for Cowboy 2.0. I'm mostly focusing on the API right now.

I expect the performance of HTTP/1.1 to be lower than in Cowboy 1.0 due to the architectural changes. On the other hand, those changes were made to have a common interface with HTTP/2, which is The Future™ and will perform much better than HTTP/1.1 ever could. So it's a trade off.

Still, there are definitely optimizations to be made, and I didn't do them yet, so Cowboy 2.0 will perform better later on. That'll have to happen after 2.0 is released though.

[ghost]

OK, great, thank you for the clarification. I think I'll close the issue then and maybe return to it after cowboy 2.0 is released.

[essen]

By the way, if you really want to get performance out of Cowboy 2.0, and/or use a separate framework on top of it, then you probably want to implement a stream handler rather than a normal handler. A benchmark using those replying immediately would perform probably even better than 1.0.

[ghost]

I am fairly new to cowboy, so pardon my ignorance (I will probably have to get a bit more familiar with the code base), but do you refer to cowboy_req:stream_reply from the streaming loop section here https://ninenines.eu/docs/en/cowboy/2.0/guide/loop_handlers/ or to the yet unfinished chapter here https://ninenines.eu/docs/en/cowboy/2.0/guide/streams/? Would such a handler work with http/1.1?

[essen]

Basically replacing cowboy_stream_h with your own. It works with all HTTP versions.

[ghost]

Do I need to indicate to cowboy on startup that the handler is going to be a stream handler?

[ghost]

Here is what I understand such a handler would look like in elixir

defmodule MyCowboy.Handler do
  @behaviour :cowboy_stream

  def init(_stream_id, %{method: method, path: path} = req, _opts) do
    {[handle(method, split_path(path))], []}
  end

  def data(_stream_id, _is_fin, _data, state), do: {[], state}

  def info(_stream_id, _info, state), do: {[], state}

  def terminate(_stream_id, _reason, _state), do: :ok

  def early_error(_stream_id, _reason, _partial_req, resp, _opts), do: resp


  defp handle("GET", []), do: {:response, 200, %{}, ""}
  defp handle("GET", ["user", id]), do: {:response, 200, %{}, id}
  defp handle("POST", ["user"]), do: {:response, 200, %{}, ""}

  defp split_path(path) do
    segments = :binary.split(path, "/", [:global])
    for segment <- segments, segment != "", do: segment
  end
end

Is it at least remotely correct?

[essen]

No. https://ninenines.eu/docs/en/cowboy/2.0/manual/cowboy_stream/

There's a stream_handlers option that allows you to define the chain of handlers. See https://ninenines.eu/docs/en/cowboy/2.0/manual/cowboy_http/ and https://ninenines.eu/docs/en/cowboy/2.0/manual/cowboy_http2/ for details.

This is a low level interface that can be used for many different things from automatic compression, access logs, to having your own framework on top of low level Cowboy.

[ghost]

With MyCowboy.Handler as above, curl "http://localhost:3000/user/1" just hangs. I thought just returning {:response, 200, %{}, id} from handle/2would make cowboy send out the response. Do I need to add something else?

[essen]

Well you have to set the content-length header if you send a body.

[essen]

And also if you don't probably.

[ghost]

Thank you, working well now. I really like this interface.

[ghost]

Oh, still not working actually. I just receive

HTTP/1.1 200 OK
content-length:

and nothing else.

Here's my current version of the handler

defmodule MyCowboy.StreamHandler do
  @behaviour :cowboy_stream

  def init(_stream_id, %{method: method, path: path}, _opts) do
    {[handle(method, split_path(path))], []}
  end

  def data(_stream_id, _is_fin, _data, state), do: {[], state}

  def info(_stream_id, _info, state), do: {[], state}

  def terminate(_stream_id, _reason, _state), do: :ok

  def early_error(_stream_id, _reason, _partial_req, resp, _opts), do: resp


  defp handle("GET", []) do
    {:response, 200, %{"content-length" => 0}, ""}
  end
  defp handle("GET", ["user", id]) do
    {:response, 200, %{"content-length" => :erlang.size(id)}, id}
  end
  defp handle("POST", ["user"]) do
    {:response, 200, %{"content-length" => 0}, ""}
  end

  defp split_path(path) do
    segments = :binary.split(path, "/", [:global])
    for segment <- segments, segment != "", do: segment
  end
end

[essen]

The header value is an iodata (iolist or binary).

[ghost]

I've replaced the handle/2 functions with

defp handle("GET", []) do
  {:response, 200, %{"content-length" => "0"}, ""}
end
defp handle("GET", ["user", id]) do
  {:response, 200, %{"content-length" => "#{:erlang.size(id)}"}, id}
end
defp handle("POST", ["user"]) do
  {:response, 200, %{"content-length" => "0"}, ""}
end

and run wrk -t30 -c40 -d60s http://localhost:3000 and got

# cowboy2
Running 1m test @ http://localhost:3000
  30 threads and 40 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.20ms    1.98ms 125.64ms   95.54%
    Req/Sec     1.02k   216.99     2.01k    70.91%
  1826968 requests in 1.00m, 66.21MB read
Requests/sec:  30422.13
Transfer/sec:      1.10MB

so it seems to be on par with cowboy 1.1, which got

# cowboy1
> wrk -t30 -c40 -d60s http://localhost:3000
Running 1m test @ http://localhost:3000
  30 threads and 40 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.45ms    3.65ms 128.25ms   96.48%
    Req/Sec     1.01k   218.63     2.09k    72.75%
  1802528 requests in 1.00m, 156.43MB read
Requests/sec:  30003.77
Transfer/sec:      2.60MB

[OvermindDL1]

Requests/sec: 30422.13
Transfer/sec: 1.10MB

cowboy2 has more requests/sec but less transfer/sec? Is wrk using http2 mode? How does it compare with http1 mode if it is?

[ghost]

Fewer headers, no Server: Cowboy and date headers. I don't think wrk supports http/2

[essen]

Yeah those headers are set by cowboy_req, and you're short-circuiting that whole thing. Stream handlers (or the code you write on top) are responsible for all headers, the low level part of Cowboy only deals with the connection and transfer-encoding headers in HTTP/1.1, and the pseudo-headers in HTTP/2.

[OvermindDL1]

Ah wow, half the data just from removing the usual headers, interesting. Though empty bodies so eh. ^.^

Any chance of documenting those headers somewhere, either on or linked from https://ninenines.eu/docs/en/cowboy/2.0/manual/cowboy_stream/ too?

[essen]

Yep, should probably mentioned, please open a ticket.