Skip to content
Benchmarks for Bref running on AWS Lambda
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
http-application
php-bench
php-function
symfony
.gitignore
Makefile
README.md
benchmark-coldstarts.sh
benchmark-phpbench.sh
benchmark-warm.sh
template.yaml

README.md

Benchmarks for Bref running on AWS Lambda.

Scenarios

  • PHP function: a simple PHP function, see php-function/index.php
  • HTTP application: a simple PHP script that returns Hello world, see http-application/index.php

Average (warm) execution time

Average execution time for a lambda that doesn't do anything.

Number of samples: 100-200

128 512 1024 2048
PHP function 130ms 32ms 20ms 17ms
HTTP application 13ms 5ms 3ms 2ms
Symfony 56ms 12ms 6ms 5ms

For comparison on a 512M Digital Ocean droplet we get 1ms for "HTTP application" and 6ms for Symfony.

CPU performance

The more RAM, the more CPU power is allocated to the lambda. This is clearly visible when running PHP's official bench.php script.

128 512 1024 2048
bench.php 6s 1.8s 0.8s 0.4s

For comparison bench.php runs in 1.3s on a 512M Digital Ocean droplet, in 0.8s on a 2.8Ghz i7 and in 0.6s on a 3.2Ghz i5.

Cold starts

Number of samples: 20

128 512 1024 2048
PHP function 600ms 280ms 225ms 215ms
HTTP application 500ms 270ms 270ms 215ms
Symfony 5.6s 1.7s 850ms 550ms

Notes:

  • Symfony's cold start is high at the moment because we do not pre-generate the cache before deploying (it is generated on the first hit). This is an optimization that is planned.

Reproducing

You will need to install dependencies of Bref. Then:

  • clone the repository
  • create a S3 bucket in us-east-2 and change the bucket name in Makefile (replace it with yours)
  • make deploy
  • make bench-coldstarts
  • make bench-warm

CloudWatch dashboard URL to use to check out the metrics in one place.

You can’t perform that action at this time.