Rails JSON Serializer Benchmarks

This repo contains a simplified Ruby on Rails app that implements various JSON serialization libraries. The application also contains a simple suite of tools to run benchmarks via Apache Bench (ab).

Set up the app

Once you've downloaded the repo, you should be able to install all of the dependencies and populate a local database by running the following command:

bin/setup

Note

For the most reliable benchmarks, this app should be run in production mode. The production environment has been configured to use the same database as development.

Run the production application on your local machine with the following command

bundle exec rails server -e production

Benchmark

The benchmark will test implementations of following serialization libraries:

• ActiveModelSerializers (Current solution)
• Alba
• Blueprinter
• JBuilder
• Panko

Each controller endpoint has been defined to be as similar as possible, in terms of the data that is loaded and rendered in the request body.

By default, the benchmark script will run 1,000 queries with a concurrency of 2.

The class Benchmark::AB acts as a wrapper around the AB command-line tool, effectively calling:

ab -c2 -n1000 http://127.0.0.1:3000/benchmark/[solution name]/results.json

Benchmark results

The results below were gathered from running the benchmark script (bin/benchmark) on my local development machine, while running the Rails application in production mode.

Environment spec

Rails version: 6.1.4.1
Ruby version: ruby 2.7.4p191
MySQL version: 5.7.36

Machine spec

MacBook Pro (16-inch, 2019)
2.4 GHz 8-Core Intel Core i9
64 GB 2667 MHz DDR4

Abbreviated results

Time taken to execute 1,000 requests

Test:   alba                            11.189 seconds
Test:   panko                           11.677 seconds
Test:   blueprinter                     12.146 seconds
Test:   j_builder                       16.978 seconds
Test:   active_model_serializers        21.035 seconds

Full results

Full results from each benchmark can be found in the benchmarks directory.

• ./benchmarks/alba.txt
• ./benchmarks/panko.txt
• ./benchmarks/blueprinter.txt
• ./benchmarks/j_builder.txt
• ./benchmarks/active_model_serializers.txt

Results with OJ

As an optional extra benchmark, I added the OJ gem and set this as the default JSON library. This is disabled by default, but can be activated by setting the ENV variable OJ when running the application server. Applying OJ is clearly beneficial, but also yields results in a different order.

Test:   panko                           9.579 seconds
Test:   alba                            11.034 seconds
Test:   blueprinter                     12.104 seconds
Test:   j_builder                       13.994 seconds
Test:   active_model_serializers        17.874 seconds

Results with ActionController::API

As an optional extra benchmark, I added a feature flag to use ActionController::API as a base class, instead of ActionController::Base. This is disabled by default, but can be activated by setting the ENV variable API_MODE when running the application server. Changing the base class for ApplicationController provided a fairly unremarkable enhancement compared to ActionController::Base.

Test:   alba                            11.118 seconds
Test:   panko                           11.757 seconds
Test:   blueprinter                     11.946 seconds
Test:   j_builder                       17.020 seconds
Test:   active_model_serializers        20.867 seconds