Approximate history:
- Originally created as
graphql-bench
by Hasura - Modified by Prisma to use
vegeta
to generate load instead ofwrk2
and to change the environmental setup - Modified by Benjie (PostGraphile) to add greater automation, to make sure
each environment is torn down before the next one is set up, to fix
PostGraphile setup, and to add
postgraphile@next
testing
I (@benjie) ran these benchmarks on a Digital Ocean compute-optimised droplet, specifically the 8GB RAM, 4 vCPU, 50GB SSD Droplet that works out at $0.119/hr.
Assuming you want to do the same, I've included as many instructions as I can remember below. (The commands are from memory or screenshots so may not be exact - please validate as you go through.)
Really, if you're serious about benchmarking, you should run the client
(vegeta
) on completely different hardware to the server (Prisma /
PostGraphile); and they should connect to a similar/identical database that's
hosted on yet another server. This would prevent the various softwares from
interfering from each other. Sadly I did not have the time to accomplish this.
- Visit https://cloud.digitalocean.com/droplets/new
- Select "One-click apps"
- Select "Docker 17.12.0~ce on 16.04"
- Select Optimized Droplet with 8GB of RAM
- Add you SSH key
- Create
- SSH into the box
ssh root@IP.ADDRESS.HERE
- Run
docker run hello-world
- you should receive "Hello from Docker"
First, update the box and reboot. Then:
The benchmarking tool.
- SSH into the box
ssh root@IP.ADDRESS.HERE
mkdir vegeta && cd vegeta
curl -LO https://github.com/tsenart/vegeta/releases/download/v6.3.0/vegeta-v6.3.0-linux-amd64.tar.gz
tar xf vegeta*
cp vegeta /usr/local/bin
Required to set up Prisma locally.
- SSH into the box
ssh root@IP.ADDRESS.HERE
mkdir node && cd node
curl -LO https://nodejs.org/dist/v10.1.0/node-v10.1.0-linux-x64.tar.xz
tar xf node-v*
ln -s node-v10.1.0-linux-x64 latest
- Add
PATH="$PATH:/root/node/latest/bin"
to your~/.bashrc
- Log out then SSH into the box
ssh root@IP.ADDRESS.HERE
node --version
- should give 10.1.0
Benchmarking consumes a lot of file descriptors! The values below worked for me, but you might want to raise them higher still...
- edit
/etc/security/limits.conf
;
add "root soft nofile 16192" add "* soft nofile 16192" - edit
/etc/pam.d/common-session
and add "session required pam_limits.so" - reboot server
shutdown -r now
Required to load the chinook
dataset into the PostGraphile database.
sudo apt update && sudo apt install postgresql-client
- SSH into the box
ssh root@IP.ADDRESS.HERE
git clone https://github.com/benjie/graphql-bench-prisma.git
- change into the
prisma
foldergraphql-bench-prisma/testcandidates/prisma
- run the following (line by line)
npm install -g prisma
mkdir project
cd project
prisma init
# answers: (new database, MySQL)
cp ../clisetup/datamodel.prisma .
cp ../clisetup/docker-compose.yml .
docker-compose up -d
prisma deploy
prisma import --data ../import/chinook.zip
echo "VERIFY THE FOLLOWING"
cat ../queries/byArtistId.json | curl -X POST -d @- -H 'Content-Type: application/json' -i http://localhost:4466
You should receive a sensible result from the curl request - if you do - well done!
These instructions were based on the following instructions given to me by @do4gr, Prisma's engineer responsible for benchmarking:
The files are probably not in the correct folders for the script to work, sorry. I just set that one up by hand I think. This is what you need to do
Install the Prisma CLInpm install -g prisma
Setup a new projectprisma init
Change the datamodel to the one in the datamodel.graphql file
Runprisma deploy
and choose local
Then start the docker container
then runprisma import --data chinook.zip
to import the data
then you can verify in the databrowser that there is data (edited)You then have a prisma instance running locally with that data. and in the bench.json can target it with
{ “dir”: “prisma”, “url”: “http://localhost:4466” }
- SSH into the box
ssh root@IP.ADDRESS.HERE
cd graphql-bench-prisma/vegeta
- [OPTIONAL] edit bench.json with the configuration you wish to run
./bench.sh
- The results are written to the
results
folder under each project's directory
I recommend copying the resulting files to your local computer, which you can do easily with rsync:
rsync -avz root@IP.ADDRESS.HERE:graphql-bench-prisma path/to/destination/
Once done, you can change into the testcandidates
and run node ./parser.js
(you may need to tweak the dataFilter
at the top of the file if you've ran
more than one benchmark) - this will write a data.json
file into the
visualizer/src
folder.
You can then change into visualizer
and run yarn && yarn start
- this will
spawn a React server rendering the data.