Skip to content

Latest commit

 

History

History
244 lines (185 loc) · 6.21 KB

postgres-opaque.md

File metadata and controls

244 lines (185 loc) · 6.21 KB
Raw

Hydra GCP Performance Benchmarks (token strategy=opaque, database=postgres)

In this document you will find benchmark results for different endpoints of ORY Hydra. All benchmarks are executed using rakyll/hey. Please note that these benchmarks run against the postgres storage adapter.

Please note performance may greatly differs between deployments (e.g. request latency, database configuration) and tweaking individual things may greatly improve performance. This is also not indicative of long-term performance as database sizes grow. Take these results with a very large grain of salt.

All benchmarks run 10,000 requests in total, with 100 concurrent requests. All benchmarks run on a n1-highcpu-4 Compute Engine VM (4 vCPUs, 3.6GB memory) in the us-east1-b zone on Google Compute Engine. To provide enough entropy for random number generation, haveged was installed and setup on the test machine. Where applicable, the n1-standard-2 (2 vCPUs, 7.5 GB memory) machine type is used for the PostgreSQL instance using Google Cloud SQL running in the same zone.

BCrypt

ORY Hydra uses BCrypt to obfuscate secrets of OAuth 2.0 Clients. When using flows such as the OAuth 2.0 Client Credentials Grant, ORY Hydra validates the client credentials using BCrypt which causes (by design) CPU load. CPU load and performance depend on the BCrypt cost which can be set using the environment variable BCRYPT_COST. For these benchmarks, we have set BCRYPT_COST=8.

OAuth 2.0

This section contains various benchmarks against OAuth 2.0 endpoints

Token Introspection


Summary:
  Total:	2.8980 secs
  Slowest:	0.6674 secs
  Fastest:	0.0017 secs
  Average:	0.0265 secs
  Requests/sec:	3450.6454
  
  Total data:	1550000 bytes
  Size/request:	155 bytes

Response time histogram:
  0.002 [1]	|
  0.068 [9266]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.135 [425]	|■■
  0.201 [175]	|■
  0.268 [74]	|
  0.335 [48]	|
  0.401 [9]	|
  0.468 [0]	|
  0.534 [1]	|
  0.601 [0]	|
  0.667 [1]	|


Latency distribution:
  10% in 0.0049 secs
  25% in 0.0078 secs
  50% in 0.0142 secs
  75% in 0.0271 secs
  90% in 0.0529 secs
  95% in 0.0930 secs
  99% in 0.2233 secs

Details (average, fastest, slowest):
  DNS+dialup:	0.0000 secs, 0.0017 secs, 0.6674 secs
  DNS-lookup:	0.0000 secs, 0.0000 secs, 0.0082 secs
  req write:	0.0000 secs, 0.0000 secs, 0.0072 secs
  resp wait:	0.0263 secs, 0.0016 secs, 0.6674 secs
  resp read:	0.0000 secs, 0.0000 secs, 0.0016 secs

Status code distribution:
  [200]	10000 responses

Client Credentials Grant

This endpoint uses BCrypt.


Summary:
  Total:	62.3955 secs
  Slowest:	9.3477 secs
  Fastest:	0.0248 secs
  Average:	0.5780 secs
  Requests/sec:	160.2681
  
  Total data:	1569843 bytes
  Size/request:	157 bytes

Response time histogram:
  0.025 [1]	|
  0.957 [8704]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  1.889 [713]	|■■■
  2.822 [237]	|■
  3.754 [161]	|■
  4.686 [116]	|■
  5.619 [41]	|
  6.551 [17]	|
  7.483 [2]	|
  8.415 [5]	|
  9.348 [2]	|


Latency distribution:
  10% in 0.1244 secs
  25% in 0.1851 secs
  50% in 0.3165 secs
  75% in 0.5864 secs
  90% in 1.1919 secs
  95% in 2.1605 secs
  99% in 4.3705 secs

Details (average, fastest, slowest):
  DNS+dialup:	0.0000 secs, 0.0248 secs, 9.3477 secs
  DNS-lookup:	0.0000 secs, 0.0000 secs, 0.0144 secs
  req write:	0.0000 secs, 0.0000 secs, 0.0145 secs
  resp wait:	0.5779 secs, 0.0247 secs, 9.3475 secs
  resp read:	0.0001 secs, 0.0000 secs, 0.0118 secs

Status code distribution:
  [200]	9999 responses

Error distribution:
  [1]	Post http://localhost:9000/oauth2/token: EOF

OAuth 2.0 Client Management

Creating OAuth 2.0 Clients

This endpoint uses BCrypt and generates IDs and secrets by reading from which negatively impacts performance. Performance will be better if IDs and secrets are set in the request as opposed to generated by Hydra GCP.

This test is currently disabled due to issues with /dev/urandom being inaccessible in the CI.

Listing OAuth 2.0 Clients


Summary:
  Total:	1.6794 secs
  Slowest:	0.3340 secs
  Fastest:	0.0010 secs
  Average:	0.0151 secs
  Requests/sec:	5954.4652
  
  Total data:	4150000 bytes
  Size/request:	415 bytes

Response time histogram:
  0.001 [1]	|
  0.034 [9093]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.068 [509]	|■■
  0.101 [177]	|■
  0.134 [90]	|
  0.168 [48]	|
  0.201 [29]	|
  0.234 [14]	|
  0.267 [13]	|
  0.301 [18]	|
  0.334 [8]	|


Latency distribution:
  10% in 0.0020 secs
  25% in 0.0031 secs
  50% in 0.0064 secs
  75% in 0.0145 secs
  90% in 0.0318 secs
  95% in 0.0561 secs
  99% in 0.1515 secs

Details (average, fastest, slowest):
  DNS+dialup:	0.0000 secs, 0.0010 secs, 0.3340 secs
  DNS-lookup:	0.0000 secs, 0.0000 secs, 0.0064 secs
  req write:	0.0000 secs, 0.0000 secs, 0.0065 secs
  resp wait:	0.0150 secs, 0.0009 secs, 0.3340 secs
  resp read:	0.0001 secs, 0.0000 secs, 0.0061 secs

Status code distribution:
  [200]	10000 responses

Fetching a specific OAuth 2.0 Client


Summary:
  Total:	1.5138 secs
  Slowest:	0.3426 secs
  Fastest:	0.0009 secs
  Average:	0.0134 secs
  Requests/sec:	6605.9601
  
  Total data:	4130000 bytes
  Size/request:	413 bytes

Response time histogram:
  0.001 [1]	|
  0.035 [9293]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.069 [395]	|■■
  0.103 [122]	|■
  0.138 [82]	|
  0.172 [38]	|
  0.206 [27]	|
  0.240 [22]	|
  0.274 [4]	|
  0.308 [13]	|
  0.343 [3]	|


Latency distribution:
  10% in 0.0018 secs
  25% in 0.0029 secs
  50% in 0.0059 secs
  75% in 0.0130 secs
  90% in 0.0274 secs
  95% in 0.0469 secs
  99% in 0.1449 secs

Details (average, fastest, slowest):
  DNS+dialup:	0.0000 secs, 0.0009 secs, 0.3426 secs
  DNS-lookup:	0.0000 secs, 0.0000 secs, 0.0104 secs
  req write:	0.0000 secs, 0.0000 secs, 0.0074 secs
  resp wait:	0.0132 secs, 0.0008 secs, 0.3426 secs
  resp read:	0.0001 secs, 0.0000 secs, 0.0090 secs

Status code distribution:
  [200]	10000 responses