Repository navigation
Performance
How fast Databucket serves a load test - many virtual users reserving, changing and releasing records at the same time - on a bucket with 1 million records. The tests are in the databucket-performance-tests project (Gatling). Each measurement has a description with its conclusions, and every run its Gatling report with charts and the times of every step - all reports.
-
13 ms (95th percentile) to find a record by rules and reserve it, on 1 million records, with 100 virtual users - when the searched properties are indexed and the reservation doesn't count the free records (
count=false). - About 220 reservations per second at most on one laptop - each one reserve, change, and change and release (about 670 requests per second).
- No record handed out twice and no failed request, under any load.
Everything on one laptop: Databucket and PostgreSQL 18 in Docker (the docker compose of the repository, default settings) and Gatling. A server with Databucket and PostgreSQL on their own machines would be faster; this wasn't measured. Measured on Databucket 3.16-3.18; later versions didn't change these parts.
The data: a bucket of 1,000,000 users (a JSON object each: type, country, origin, status, group, age...), 20 % of them already reserved.
100 virtual users, each signed in with its own account, started within 30 s, then 2 minutes of full load, with a pause of 0.5-2 s between the steps:
- 40 % create a user, change it and release it,
- 30 % find a premium customer from Poland by rules (3 % of the records), reserve it, change it and release it,
- 30 % find the users created earlier and delete one.
| No indexes | Indexed properties | + reserving without counting | |
|---|---|---|---|
| Median | 930 ms | 14 ms | 9 ms |
| 95th percentile | 4,853 ms | 50 ms | 12 ms |
| Throughput | 40 req/s | 114 req/s | 116 req/s |
| Failed requests | 0 | 0 | 0 |
Without indexes every search reads the whole bucket, and every step waits for the server: about 6,200 requests in a run. With indexes, about 17,500 - the throughput is then set by the pauses of the virtual users, not by the server. The reservation still counted all free matching records for its answer (available), which took most of its time; without counting (count=false) it is about 10 times faster (p95 155 → 13 ms).
The same load for 10 minutes instead of 2, on Databucket 3.18.1 with the indexes set in the settings of the bucket: p50 10 ms, p95 15 ms, p99 19 ms, 126 req/s, 0 failed of 79,915 requests - steady from the 2nd minute to the end.
Descriptions: indexes, reserving without counting, 10 minutes. Gatling reports: no indexes, indexed properties, + reserving without counting, 10 minutes.
Only the reservation, without any pauses: every virtual user finds a free premium customer from Poland and reserves it, changes it, then changes and releases it - and starts again as soon as it gets the answer. Levels of 10, 50, 100 and 200 users, 2 minutes each. Two orders of the reservation: sort=id (the free record with the lowest id - all users go for the same first free records and skip the ones others are taking) and sort=random (records spread over the whole bucket).
| sort | Users | Reservations/s | Requests/s | p50 | p95 |
|---|---|---|---|---|---|
| id | 10 | 180 | 541 | 17 ms | 22 ms |
| id | 50 | 223 | 670 | 69 ms | 108 ms |
| id | 100 | 209 | 627 | 139 ms | 278 ms |
| id | 200 | 201 | 600 | 286 ms | 598 ms |
| random | 10 | 61 | 184 | 22 ms | 125 ms |
| random | 50 | 76 | 229 | 145 ms | 495 ms |
| random | 100 | 80 | 240 | 355 ms | 677 ms |
| random | 200 | 78 | 233 | 796 ms | 1,181 ms |
- The maximum is about 220 reservations per second, reached at about 50 users. More users don't get more records, only longer answers: at 200 users the 95th percentile is 0.6 s.
-
sort=randomcosts about 3 times more: every reservation puts the matching free records in a random order. - No reservation failed and no record was given to two users, also at 200 users without pauses: a reservation skips the records another reservation is taking at the same moment (
FOR UPDATE SKIP LOCKED), without waiting for it. - For comparison: the 100 users of test 1, with their pauses, need about 11 reservations per second - about 5 % of this capacity.
Description: capacity. Gatling reports of every level: all reports, e.g. sort=id, 50 users.
- Index the properties your rules search by - up to 5 per bucket, in the settings of the bucket (see Buckets). On a big bucket this makes the biggest difference.
-
Don't count when you don't need the number:
count=falsefor/reserveand for/get(see Get started with API). The clients don't count when they reserve one record or read a list. -
Keep
sort=id(the default) unless the test needs randomly chosen records. - Without pauses, about 50 parallel users per Databucket server like this one is the most that pays off; with realistic pauses, hundreds.
- Databucket handles at most 30 requests at the same time by default (the setting
databucket.max-concurrent-requests, or the environment variableDATABUCKET_MAX_CONCURRENT_REQUESTS; it is also the number of database connections); a bigger server may take more. Not measured. - Check how many free records match your rules before the run -
/reservewithlimit=0- and use the ready-made Gatling project as a start.
Wiki
Work in the project
- Navigation
- Overview
- Searching
- Filtering
- Create/modify/remove data
- Data reservation
- Data details
- Data history
- Tasks execution
- Data export
- Data import
- Data generation
Project settings
Management
API