Skip to content

Enabled GRAPH.RO_QUERY command support. Updated Readme accordingly. #15

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 2, 2021
Merged

Enabled GRAPH.RO_QUERY command support. Updated Readme accordingly. #15

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
187 changes: 116 additions & 71 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,84 +22,129 @@ make
## Usage of redisgraph-benchmark-go

```
$ ./redisgraph-benchmark-go --help
Usage of ./redisgraph-benchmark-go:
-a string
Password for Redis Auth.
-c uint
number of clients. (default 50)
-debug int
Client debug level.
-graph-key string
graph key. (default "graph")
-h string
Server hostname. (default "127.0.0.1")
-l Loop. Run the tests forever.
-n uint
Total number of requests (default 1000000)
-p int
Server port. (default 6379)
-rps int
Max rps. If 0 no limit is applied and the DB is stressed up to maximum.
$ $ ./redisgraph-benchmark-go --help
Usage of ./redisgraph-benchmark-go:
-a string
Password for Redis Auth.
-c uint
number of clients. (default 50)
-continue-on-error
Continue benchmark in case of error replies.
-debug int
Client debug level.
-enable-exporter-rps
Push results to redistimeseries exporter in real-time. Time granularity is set via the -reporting-period parameter.
-exporter-rts-auth string
RedisTimeSeries Password for Redis Auth.
-exporter-rts-host string
RedisTimeSeries hostname. (default "127.0.0.1")
-exporter-rts-port int
RedisTimeSeries port. (default 6379)
-exporter-run-name string
Run name. (default "perf-run")
-graph-key string
graph key. (default "graph")
-h string
Server hostname. (default "127.0.0.1")
-json-out-file string
Name of json output file to output benchmark results. If not set, will not print to json. (default "benchmark-results.json")
-n uint
Total number of requests (default 1000000)
-p int
Server port. (default 6379)
-query value
Specify a RedisGraph query to send in quotes. Each command that you specify is run with its ratio. For example: -query="CREATE (n)" -query-ratio=1
-query-ratio value
The query ratio vs other queries used in the same benchmark. Each command that you specify is run with its ratio. For example: -query="CREATE (n)" -query-ratio=0.5 -query="MATCH (n) RETURN n" -query-ratio=0.5
-query-ro value
Specify a RedisGraph read-only query to send in quotes. You can run multiple commands (both read/write) on the same benchmark. Each command that you specify is run with its ratio. For example: -query="CREATE (n)" -query-ratio=0.5 -query-ro="MATCH (n) RETURN n" -query-ratio=0.5
-random-int-max int
__rand_int__ upper value limit. __rand_int__ distribution is uniform Random (default 1000000)
-random-int-min int
__rand_int__ lower value limit. __rand_int__ distribution is uniform Random (default 1)
-random-seed int
Random seed to use. (default 12345)
-reporting-period duration
Period to report stats. (default 10s)
-rps int
Max rps. If 0 no limit is applied and the DB is stressed up to maximum.
-v Output version and exit
```

## Sample output - 1M commands
## Sample output - 100K write commands

```
$ redisgraph-benchmark-go -graph-key graph "CREATE (u:User)"
Debug level: 0.
Total clients: 50. Commands per client: 20000 Total commands: 1000000
Test time Total Commands Total Errors Command Rate Client p50 with RTT(ms) Graph Internal p50 with RTT(ms)
53s [100.0%] 1000000 0 [0.0%] 8002.89 2.097 0.000
################# RUNTIME STATS #################
Total Duration 53.001 Seconds
Total Commands issued 1000000
Total Errors 0 ( 0.000 %)
Throughput summary: 18868 requests per second
Overall Client Latency summary (msec):
p50 p95 p99
2.097 5.347 9.063
################## GRAPH STATS ##################
Total Empty resultsets 1000000 ( 100.000 %)
Total Nodes created 1000000
Total Nodes deleted 0
Total Labels added 0
Total Properties set 0
Total Relationships created 0
Total Relationships deleted 0
Overall RedisGraph Internal Execution time Latency summary (msec):
p50 p95 p99
0.000 0.000 0.000
$ redisgraph-benchmark-go -n 100000 -graph-key graph -query "CREATE (u:User)"
2021/07/12 11:44:13 redisgraph-benchmark-go (git_sha1:)
2021/07/12 11:44:13 RTS export disabled.
2021/07/12 11:44:13 Debug level: 0.
2021/07/12 11:44:13 Using random seed: 12345.
2021/07/12 11:44:13 Total clients: 50. Commands per client: 2000 Total commands: 100000
2021/07/12 11:44:13 Trying to extract RedisGraph version info
2021/07/12 11:44:13 Detected RedisGraph version 999999

Test time Total Commands Total Errors Command Rate Client p50 with RTT(ms) Graph Internal Time p50 (ms)
10s [100.0%] 100000 0 [0.0%] 9997.46 2.698 (2.698) 2.589 (2.589)
################# RUNTIME STATS #################
Total Duration 10.004 Seconds
Total Commands issued 100000
Total Errors 0 ( 0.000 %)
Throughput summary: 9996 requests per second
## Overall RedisGraph resultset stats table
| QUERY | NODES CREATED | NODES DELETED | LABELS ADDED | PROPERTIES SET | RELATIONSHIPS CREATED | RELATIONSHIPS DELETED |
|-----------------|---------------|---------------|--------------|----------------|------------------------|------------------------|
| CREATE (u:User) | 100000 | 0 | 0 | 0 | 0 | 0 |
| Total | 100000 | 0 | 0 | 0 | 0 | 0 |
## Overall RedisGraph Internal Execution Time summary table
| QUERY | INTERNAL AVG LATENCY(MS) | INTERNAL P50 LATENCY(MS) | INTERNAL P95 LATENCY(MS) | INTERNAL P99 LATENCY(MS) |
|-----------------|----------------------------|--------------------------|--------------------------|--------------------------|
| CREATE (u:User) | 2.599 | 2.589 | 2.912 | 3.648 |
| Total | 2.599 | 2.589 | 2.912 | 3.648 |
## Overall Client Latency summary table
| QUERY | OPS/SEC | TOTAL CALLS | TOTAL ERRORS | AVG LATENCY(MS) | P50 LATENCY(MS) | P95 LATENCY(MS) | P99 LATENCY(MS) |
|-----------------|---------|-------------|--------------|------------------|-----------------|-----------------|-----------------|
| CREATE (u:User) | 9996 | 100000 | 0 | 2.745 | 2.698 | 3.048 | 4.007 |
| Total | 9996 | 100000 | 0 | 2.745 | 2.698 | 3.048 | 4.007 |
2021/07/12 11:44:23 Saving JSON results file to benchmark-results.json
```


## Sample output - running in loop mode ( Ctrl+c to stop )
## Sample output - running mixed read and writes benchmark

```
$ redisgraph-benchmark-go -l -graph-key graph "CREATE (:Rider {name:'A'})-[:rides]->(:Team {name:'Z'})"
Debug level: 0.
Running in loop until you hit Ctrl+C
Test time Total Commands Total Errors Command Rate Client p50 with RTT(ms) Graph Internal p50 with RTT(ms)
^C 11s [----%] 136649 0 [0.0%] 7854.48 3.667 0.000
received Ctrl-c - shutting down

################# RUNTIME STATS #################
Total Duration 11.516 Seconds
Total Commands issued 140704
Total Errors 0 ( 0.000 %)
Throughput summary: 12217 requests per second
Overall Client Latency summary (msec):
p50 p95 p99
3.751 6.887 8.623
################## GRAPH STATS ##################
Total Empty resultsets 140705 ( 100.000 %)
Total Nodes created 281410
Total Nodes deleted 0
Total Labels added 0
Total Properties set 281410
Total Relationships created 140705
Total Relationships deleted 0
Overall RedisGraph Internal Execution time Latency summary (msec):
p50 p95 p99
0.000 0.000 0.000
$ redisgraph-benchmark-go -n 100000 -graph-key graph -query "CREATE (u:User)" -query-ratio 0.5 -query-ro "MATCH (n) return COUNT(n)" -query-ratio 0.5
2021/07/12 11:45:38 redisgraph-benchmark-go (git_sha1:)
2021/07/12 11:45:38 RTS export disabled.
2021/07/12 11:45:38 Debug level: 0.
2021/07/12 11:45:38 Using random seed: 12345.
2021/07/12 11:45:38 Total clients: 50. Commands per client: 2000 Total commands: 100000
2021/07/12 11:45:38 Trying to extract RedisGraph version info
2021/07/12 11:45:38 Detected RedisGraph version 999999

Test time Total Commands Total Errors Command Rate Client p50 with RTT(ms) Graph Internal Time p50 (ms)
10s [100.0%] 100000 0 [0.0%] 9996.09 1.179 (1.179) 0.155 (0.155)
################# RUNTIME STATS #################
Total Duration 10.004 Seconds
Total Commands issued 100000
Total Errors 0 ( 0.000 %)
Throughput summary: 9996 requests per second
## Overall RedisGraph resultset stats table
| QUERY | NODES CREATED | NODES DELETED | LABELS ADDED | PROPERTIES SET | RELATIONSHIPS CREATED | RELATIONSHIPS DELETED |
|---------------------------|---------------|---------------|--------------|----------------|------------------------|------------------------|
| CREATE (u:User) | 49921 | 0 | 0 | 0 | 0 | 0 |
| MATCH (n) return COUNT(n) | 0 | 0 | 0 | 0 | 0 | 0 |
| Total | 49921 | 0 | 0 | 0 | 0 | 0 |
## Overall RedisGraph Internal Execution Time summary table
| QUERY | INTERNAL AVG LATENCY(MS) | INTERNAL P50 LATENCY(MS) | INTERNAL P95 LATENCY(MS) | INTERNAL P99 LATENCY(MS) |
|---------------------------|----------------------------|--------------------------|--------------------------|--------------------------|
| CREATE (u:User) | 3.825 | 3.913 | 4.639 | 5.249 |
| MATCH (n) return COUNT(n) | 0.050 | 0.048 | 0.067 | 0.100 |
| Total | 1.935 | 0.155 | 4.442 | 4.929 |
## Overall Client Latency summary table
| QUERY | OPS/SEC | TOTAL CALLS | TOTAL ERRORS | AVG LATENCY(MS) | P50 LATENCY(MS) | P95 LATENCY(MS) | P99 LATENCY(MS) |
|---------------------------|---------|-------------|--------------|------------------|-----------------|-----------------|-----------------|
| CREATE (u:User) | 4990 | 49921 | 0 | 4.041 | 4.061 | 4.843 | 5.930 |
| MATCH (n) return COUNT(n) | 5006 | 50079 | 0 | 0.236 | 0.178 | 0.442 | 1.201 |
| Total | 9996 | 100000 | 0 | 2.135 | 1.179 | 4.611 | 5.287 |
2021/07/12 11:45:48 Saving JSON results file to benchmark-results.json
```
1 change: 1 addition & 0 deletions globals.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ var clientSide_AllQueries_InstantLatencies *hdrhistogram.Histogram
var serverSide_AllQueries_GraphInternalTime_InstantLatencies *hdrhistogram.Histogram

var benchmarkQueries arrayStringParameters
var benchmarkQueriesRO arrayStringParameters
var benchmarkQueryRates arrayStringParameters

const Inf = rate.Limit(math.MaxFloat64)
Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.13

require (
github.com/HdrHistogram/hdrhistogram-go v1.0.1
github.com/RedisGraph/redisgraph-go v1.0.1-0.20210122150500-aa0feaa960ce
github.com/RedisGraph/redisgraph-go v1.0.1-0.20210524170722-ddcecfd1bef5
github.com/RedisTimeSeries/redistimeseries-go v1.4.4
github.com/gomodule/redigo v2.0.0+incompatible
github.com/google/go-cmp v0.5.4 // indirect
Expand Down
12 changes: 6 additions & 6 deletions multi-query.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,11 @@ func sample(cdf []float32) int {
return bucket
}

func prepareCommandsDistribution(cmds []string, cmdRates []float64) (int, []float32) {
func prepareCommandsDistribution(queries arrayStringParameters, cmds []string, cmdRates []float64) (int, []float32) {
var totalDifferentCommands = len(cmds)
var totalRateSum = 0.0
var err error
for i, rawCmdString := range benchmarkQueries {
for i, rawCmdString := range queries {
cmds[i] = rawCmdString
if i >= len(benchmarkQueryRates) {
cmdRates[i] = 1
Expand All @@ -38,11 +38,11 @@ func prepareCommandsDistribution(cmds []string, cmdRates []float64) (int, []floa
log.Fatalf("Total ratio should be 1.0 ( currently is %f )", totalRateSum)
}
// probability density function
if len(benchmarkQueryRates) > 0 && (len(benchmarkQueryRates) != len(benchmarkQueries)) {
log.Fatalf("When specifiying -query-rate parameter, you need to have the same number of -query and -query-rate parameters. Number of time -query ( %d ) != Number of times -query-params ( %d )", len(benchmarkQueries), len(benchmarkQueryRates))
if len(benchmarkQueryRates) > 0 && (len(benchmarkQueryRates) != (len(benchmarkQueries) + len(benchmarkQueriesRO))) {
log.Fatalf("When specifiying -query-rate parameter, you need to have the same number of -query/-query-ro and -query-rate parameters. Number of time -query ( %d ) != Number of times -query-params ( %d )", len(benchmarkQueries), (len(benchmarkQueryRates) + len(benchmarkQueriesRO)))
}
pdf := make([]float32, len(benchmarkQueries))
cdf := make([]float32, len(benchmarkQueries))
pdf := make([]float32, len(queries))
cdf := make([]float32, len(queries))
for i := 0; i < len(cmdRates); i++ {
pdf[i] = float32(cmdRates[i])
cdf[i] = 0
Expand Down
24 changes: 18 additions & 6 deletions redisgraph-bechmark-go.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,9 @@ func main() {
randomIntMin := flag.Int64("random-int-min", 1, "__rand_int__ lower value limit. __rand_int__ distribution is uniform Random")
randomIntMax := flag.Int64("random-int-max", 1000000, "__rand_int__ upper value limit. __rand_int__ distribution is uniform Random")
graphKey := flag.String("graph-key", "graph", "graph key.")
flag.Var(&benchmarkQueries, "query", "Specify a RedisGraph query to send in quotes. Each command that you specify is run with its ratio. For example: -query=\"CREATE (n)\" -query-ratio=2")
flag.Var(&benchmarkQueryRates, "query-ratio", "The query ratio vs other queries used in the same benchmark. Each command that you specify is run with its ratio. For example: -query=\"CREATE (n)\" -query-ratio=10 -query=\"MATCH (n) RETURN n\" -query-ratio=1")
flag.Var(&benchmarkQueries, "query", "Specify a RedisGraph query to send in quotes. Each command that you specify is run with its ratio. For example: -query=\"CREATE (n)\" -query-ratio=1")
flag.Var(&benchmarkQueriesRO, "query-ro", "Specify a RedisGraph read-only query to send in quotes. You can run multiple commands (both read/write) on the same benchmark. Each command that you specify is run with its ratio. For example: -query=\"CREATE (n)\" -query-ratio=0.5 -query-ro=\"MATCH (n) RETURN n\" -query-ratio=0.5")
flag.Var(&benchmarkQueryRates, "query-ratio", "The query ratio vs other queries used in the same benchmark. Each command that you specify is run with its ratio. For example: -query=\"CREATE (n)\" -query-ratio=0.5 -query=\"MATCH (n) RETURN n\" -query-ratio=0.5")
jsonOutputFile := flag.String("json-out-file", "benchmark-results.json", "Name of json output file to output benchmark results. If not set, will not print to json.")
cliUpdateTick := flag.Duration("reporting-period", time.Second*10, "Period to report stats.")
// data sink
Expand Down Expand Up @@ -98,9 +99,20 @@ func main() {
} else {
log.Printf("Running in loop until you hit Ctrl+C\n")
}
queries := make([]string, len(benchmarkQueries))
cmdRates := make([]float64, len(benchmarkQueries))
totalDifferentCommands, cdf := prepareCommandsDistribution(queries, cmdRates)
queries := make([]string, len(benchmarkQueries)+len(benchmarkQueriesRO))
queryIsReadOnly := make([]bool, len(benchmarkQueries)+len(benchmarkQueriesRO))
cmdRates := make([]float64, len(benchmarkQueries)+len(benchmarkQueriesRO))
readAndWriteQueries := append(benchmarkQueries, benchmarkQueriesRO...)

for i := 0; i < len(queries); i++ {
queryIsReadOnly[i] = false
// read-only queries are located after the read/write ones in queries
// so we start on len(benchmarkQueries) to tag them
if i >= len(benchmarkQueries) {
queryIsReadOnly[i] = true
}
}
totalDifferentCommands, cdf := prepareCommandsDistribution(readAndWriteQueries, queries, cmdRates)

createRequiredGlobalStructs(totalDifferentCommands)

Expand Down Expand Up @@ -144,7 +156,7 @@ func main() {
if uint64(client_id) == (*clients - uint64(1)) {
clientTotalCmds = samplesPerClientRemainder + samplesPerClient
}
go ingestionRoutine(&rgs[client_id], *continueOnError, queries, cdf, *randomIntMin, randLimit, clientTotalCmds, *loop, *debug, &wg, useRateLimiter, rateLimiter, graphDatapointsChann)
go ingestionRoutine(&rgs[client_id], *continueOnError, queries, queryIsReadOnly, cdf, *randomIntMin, randLimit, clientTotalCmds, *loop, *debug, &wg, useRateLimiter, rateLimiter, graphDatapointsChann)
}

// enter the update loopupdateCLIupdateCLI
Expand Down
12 changes: 8 additions & 4 deletions workers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,15 @@ import (
"time"
)

func ingestionRoutine(rg *redisgraph.Graph, continueOnError bool, cmdS []string, commandsCDF []float32, randomIntPadding, randomIntMax int64, number_samples uint64, loop bool, debug_level int, wg *sync.WaitGroup, useLimiter bool, rateLimiter *rate.Limiter, statsChannel chan GraphQueryDatapoint) {
func ingestionRoutine(rg *redisgraph.Graph, continueOnError bool, cmdS []string, commandIsRO []bool, commandsCDF []float32, randomIntPadding, randomIntMax int64, number_samples uint64, loop bool, debug_level int, wg *sync.WaitGroup, useLimiter bool, rateLimiter *rate.Limiter, statsChannel chan GraphQueryDatapoint) {
defer wg.Done()
for i := 0; uint64(i) < number_samples || loop; i++ {
cmdPos := sample(commandsCDF)
sendCmdLogic(rg, cmdS[cmdPos], randomIntPadding, randomIntMax, cmdPos, continueOnError, debug_level, useLimiter, rateLimiter, statsChannel)
sendCmdLogic(rg, cmdS[cmdPos], commandIsRO[cmdPos], randomIntPadding, randomIntMax, cmdPos, continueOnError, debug_level, useLimiter, rateLimiter, statsChannel)
}
}

func sendCmdLogic(rg *redisgraph.Graph, query string, randomIntPadding, randomIntMax int64, cmdPos int, continueOnError bool, debug_level int, useRateLimiter bool, rateLimiter *rate.Limiter, statsChannel chan GraphQueryDatapoint) {
func sendCmdLogic(rg *redisgraph.Graph, query string, readOnly bool, randomIntPadding, randomIntMax int64, cmdPos int, continueOnError bool, debug_level int, useRateLimiter bool, rateLimiter *rate.Limiter, statsChannel chan GraphQueryDatapoint) {
if useRateLimiter {
r := rateLimiter.ReserveN(time.Now(), int(1))
time.Sleep(r.Delay())
Expand All @@ -28,7 +28,11 @@ func sendCmdLogic(rg *redisgraph.Graph, query string, randomIntPadding, randomIn
var queryResult *redisgraph.QueryResult
processedQuery := processQuery(query, randomIntPadding, randomIntMax)
startT := time.Now()
queryResult, err = rg.Query(processedQuery)
if readOnly {
queryResult, err = rg.ROQuery(processedQuery)
} else {
queryResult, err = rg.Query(processedQuery)
}
endT := time.Now()

duration := endT.Sub(startT)
Expand Down