Add benchmarks

README.md

@@ -79,6 +79,8 @@ Here are some caveats to keep in mind and fancier things you can do:
 - `SynchronizedAdapter` and `SynchronizedSession` accept an optional parameter called `num_threads`, which gives the maximum number of threads the adapter will use. If not specified, the adapter will use one thread per request.
 - `finish_all()` accepts an optional parameter called `timeout`, which gives the maximum amount of time (in seconds) that the adapter will wait for a thread to finish.
 
+See [`benchmark/`](benchmark/) for notes on performance.
+
 # License
 
 Copyright (C) 2019 Aleksejs Popovs, NCC Group

benchmark/README.md

@@ -0,0 +1,18 @@
+Requests-Racer seems to perform much better when your requests have a POST body.
+
+Here are some benchmarks, collected using `benchmark.py` in this directory running on a laptop on a wireless connection in Seattle, against a server in Boston.
+
+Values in the table are the spread (i.e., max value - min value) of the times when the requests were processed by the server. Values are given in milliseconds and are the median of 5 runs.
+
+| Number of simultaneous requests |  GET | POST 128 bytes | POST 1024 bytes | POST 2048 bytes | POST 4096 bytes |
+| ------------------------------- | ---: | -------------: | --------------: | --------------: | --------------: |
+| 1 | 0 | 0 | 0 | 0 | 0 |
+| 2 | 0 | 0 | 0 | 0 | 0 |
+| 4 | 1 | 2 | 2 | 2 | 2 |
+| 8 | 21 | 4 | 5 | 4 | 4 |
+| 16 | 45 | 8 | 8 | 8 | 9 |
+| 32 | 85 | 16 | 19 | 15 | 13 |
+| 64 | 172 | 30 | 33 | 29 | 30 |
+| 128 | 387 | 62 | 63 | 66 | 53 |
+| 256 | 974 | 120 | 123 | 119 | 128 |
+| 512 | 2098 | 264 | 249 | 238 | 253 |

benchmark/benchmark.py

@@ -0,0 +1,47 @@
+import sys
+import time
+
+from requests_racer import SynchronizedSession
+
+# the script is just "<?= microtime(true) ?>"
+URL = 'http://example.com/time_raw.php'
+TRIES = 5
+
+def median(l):
+	n = len(l)
+	ls = sorted(l)
+	if n % 2 == 1:
+		return ls[n // 2]
+	return (ls[n // 2 - 1] + ls[n // 2]) / 2
+
+def main():
+	session = SynchronizedSession()
+
+	for i in range(10):
+		n = 2**i
+
+		spreads = []
+
+		for payload_size in [0, 128, 1024, 2048, 4096]:
+			spreads_here = []
+
+			for j in range(TRIES):
+				if payload_size == 0:
+					requests = [session.get(URL) for _ in range(n)]
+				else:
+					requests = [session.post(URL, data='a'*payload_size) for _ in range(n)]
+
+				session.finish_all()
+
+				request_times = [float(r.text) for r in requests]
+				spread = max(request_times) - min(request_times)
+
+				spreads_here.append(spread)
+
+			spreads.append(median(spreads_here))
+
+		spreads_ms = [round(t * 1000) for t in spreads]
+		print(n, *spreads_ms, sep=' | ')
+
+if __name__ == '__main__':
+	main()