A clone of the web app, Steam, and its page for the Stardew Valley game.
GET /api/features/:gameId
- returns all features for a specific game
POST /api/features/
- adds more features to the database
PUT /api/features/:gameId
- updates a specific games features
DELETE /api/features/:gameId
- removes the features for a specific game
- MySQL
- CouchDB
I chose MySQL as the database for this service. I want to become more familiar and comfortable with using it, since it is a popular database. The syntax that MySQL uses is also similar to some other relational databases, such as PostgreSQL, so becoming more familiar and confident in it would be beneficial for learning how to use other databases as well.
Deployed mysql on one EC2 instance, deployed service on a separate EC2 instance.
I had trouble seeding the database instance, but the fix was manipulating the security group to allow access to the instance. The database would only seed around 6.7 million datasets, instead of the full 10 million, and it took over 3 hours. Due to lack of time and the need to move on, I could not go back to optimize the seeding script. I decided to work with what had already been seeded.
I had trouble connecting the service instance to the database instance, but once again it was just a security group issue.
This test was done via loader.io: clients per second with a duration of 15s, endpoint: 5000000.
DBMS | Route | RPS | LATENCY | ERROR RATE |
---|---|---|---|---|
MySQL | GET | 1 | 133 ms | 0.0% |
MySQL | GET | 10 | 131 ms | 0.0% |
MySQL | GET | 100 | 127 ms | 0.0% |
MySQL | GET | 1000 | 3550 ms | 0.0% |
MySQL | POST | 1 | ------- | 100% |
I decided to see if gzipping the bundle.js file would make a sizable difference. I only tested the GET request because the database was too full and wouldn't accept any POST requests, nor does this service use it. Here are the results:
DBMS | Route | RPS | LATENCY | ERROR RATE |
---|---|---|---|---|
MySQL | GET | 1 | 132 ms | 0.0% |
MySQL | GET | 10 | 132 ms | 0.0% |
MySQL | GET | 100 | 126 ms | 0.0% |
MySQL | GET | 1000 | 3402 ms | 0.0% |
MySQL | GET | 5000 | 8012 ms | 57.4% |
MySQL | GET | 10000 | 9210 ms | 82.3% |
Unfortunately, I hadn't tested the 5000 or 10,000 rps ranges in my initial stress tests, so I cannot compare those values. The latency for 1000 rps did decrease a bit.
I decided to try redis caching to further optimize the requests. Here are the results:
DBMS | Route | RPS | LATENCY | ERROR RATE |
---|---|---|---|---|
MySQL | GET | 1 | 131 ms | 0.0% |
MySQL | GET | 10 | 132 ms | 0.0% |
MySQL | GET | 100 | 125 ms | 0.0% |
MySQL | GET | 1000 | 3392 ms | 0.0% |
MySQL | GET | 5000 | 7966 ms | 55.3% |
MySQL | GET | 10000 | 8891 ms | 76.0% |
Both the latency and the error rate dropped significantly.