Add Tornado HTTP benchmark

[pitrou]

BPO	19236
Nosy	@brettcannon, @birkenfeld, @gpshead, @pitrou, @scoder
Files	tornado_bench.patch tornado_bench2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-10-24.19:59:18.202>
created_at = <Date 2013-10-12.19:15:31.600>
labels = ['type-feature', 'performance']
title = 'Add Tornado HTTP benchmark'
updated_at = <Date 2013-10-24.19:59:18.201>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2013-10-24.19:59:18.201>
actor = 'python-dev'
assignee = 'none'
closed = True
closed_date = <Date 2013-10-24.19:59:18.202>
closer = 'python-dev'
components = ['Benchmarks']
creation = <Date 2013-10-12.19:15:31.600>
creator = 'pitrou'
dependencies = []
files = ['32067', '32263']
hgrepos = []
issue_num = 19236
keywords = ['patch']
message_count = 8.0
messages = ['199613', '199614', '199615', '199616', '199662', '199716', '200620', '201173']
nosy_count = 6.0
nosy_names = ['brett.cannon', 'georg.brandl', 'gregory.p.smith', 'pitrou', 'scoder', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue19236'
versions = ['3rd party']

[pitrou]

Tornado 3.1.1 is compatible with Python 2.6, 2.7, 3.2 and upwards, which makes it a good candidate for a networking benchmark.

Here is a patch adding a HTTP (server + client) benchmark. A trivial HTTP server is spawned which writes a sizable body in response to requests. HTTP clients then connect with a certain amount of concurrency and fetch the HTTP server's resource.

(note: the patch omits the Tornado lib itself for readability)

[gpshead]

makes sense to me.

[scoder]

+1. Does it give reasonably predictable timings?

[pitrou]

+1. Does it give reasonably predictable timings?

Yes. There is a small warmup phase at the beginning (the first few
rounds are skipped). The benchmark eats 100% CPU on one core, mostly
userspace (at least under Linux).

[scoder]

Another thing: you are only using one thread for both the client and the server, and the app writes the entire content in one go (during one call to get()). Wouldn't it be more interesting to make the app's get() method asynchronous as well, so that each chunk actually gets passed separately? Otherwise, the fact that it's writing chunks wouldn't really contribute all that much to the benchmark.

Or maybe split it up into two benchmarks, one that exercises the asynchronous client (as your current code does) and one that shifts more work to the server side? Not sure if it's worth it, but might be worth trying. I think it would cover two different use cases that way.

[pitrou]

Wouldn't it be more interesting to make the app's get() method
asynchronous as well, so that each chunk actually gets passed
separately?

That's a good point. I'll try to look into it.

Or maybe split it up into two benchmarks, one that exercises the
asynchronous client (as your current code does) and one that shifts more
work to the server side? Not sure if it's worth it, but might be worth
trying. I think it would cover two different use cases that way.

You still need a client to exercise the server, and a server to exercise the client, so I'm not sure how to separate them (short of using an external utility, which would add dependencies).

[pitrou]

Attached patch makes the get() function asynchronous.

[python-dev]

New changeset 9a1136898539 by Antoine Pitrou in branch 'default':
Close bpo-19236: add a Tornado-based HTTP benchmark
http://hg.python.org/benchmarks/rev/9a1136898539