Skip to content

Commit

Permalink
Merge pull request #17 from grafana/feat/13-tls-refactor
Browse files Browse the repository at this point in the history
  • Loading branch information
@oleiade
oleiade committed Nov 2, 2023
2 parents 6f71940 + 9bb7ed0 commit b5c72ea
Show file tree
Hide file tree
Showing 12 changed files with 1,208 additions and 375 deletions.
232 changes: 118 additions & 114 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@
This is a Redis client library for [k6](https://github.com/grafana/k6),
implemented as an extension using the [xk6](https://github.com/grafana/xk6) system.

| :exclamation: This extension is going under heavy changes and is about to make it to k6's core. USE AT YOUR OWN RISK! |
| --------------------------------------------------------------------------------------------------------------------- |
| :exclamation: This extension is an experimental project, and breaking changes are possible. USE AT YOUR OWN RISK! |
|-------------------------------------------------------------------------------------------------------------------|

## Build

To build a `k6` binary with this extension, first ensure you have the prerequisites:
This extension is available as an [experimental k6 module](https://k6.io/docs/javascript-api/k6-experimental/redis/) since k6 v0.40.0, so you don't need to build it with xk6 yourself, and can use it with the main k6 binary. Note that your script must import `k6/experimental/redis` instead of `k6/x/redis` if you're using the module bundled in k6.

However, if you prefer to build it from source using xk6, first ensure you have the prerequisites:

- [Go toolchain](https://go101.org/article/go-toolchain.html)
- Git
Expand Down Expand Up @@ -39,21 +41,19 @@ with Redis in a seemingly synchronous manner.
For instance, if you were to depend on values stored in Redis to perform HTTP calls, those HTTP calls should be made in the context of the Redis promise chain:

```javascript
// Instantiate a new redis client
const redisClient = new redis.Client({
addrs: __ENV.REDIS_ADDRS.split(",") || new Array("localhost:6379"), // in the form of "host:port", separated by commas
password: __ENV.REDIS_PASSWORD || "",
})
// Instantiate a new Redis client using a URL.
// The connection will be established on the first command call.
const client = new redis.Client('redis://localhost:6379');

export default function() {
// Once the SRANDMEMBER operation is succesfull,
// it resolves the promise and returns the random
// set member value to the caller of the resolve callback.
//
// The next promise performs the synchronous HTTP call, and
// returns a promise to the next operation, which uses the
// returns a promise to the next operation, which uses the
// passed URL value to store some data in redis.
redisClient.srandmember('client_ids')
client.srandmember('client_ids')
.then((randomID) => {
const url = `https://my.url/${randomID}`
const res = http.get(url)
Expand All @@ -63,134 +63,138 @@ export default function() {
// return a promise resolving to the URL
return url
})
.then((url) => redisClient.hincrby('k6_crocodile_fetched', url, 1))
.then((url) => client.hincrby('k6_crocodile_fetched', url, 1));
}
```

## Example test scripts
You can see more complete examples in the [/examples](/examples) directory.


### Single-node client

In this example we demonstrate two scenarios: one load testing a redis instance, another using redis as an external data store used throughout the test itself.
As shown in the above example, the simplest way to create a new `Client` instance that connects to a single Redis server is by passing a URL string. It must be in the format:

```
redis[s]://[[username][:password]@][host][:port][/db-number]
```

A client can also be instantiated using an object, for more flexibility:
```javascript
import { check } from "k6";
import http from "k6/http";
import redis from "k6/x/redis";
import exec from "k6/execution";
import { textSummary } from "https://jslib.k6.io/k6-summary/0.0.1/index.js";

export const options = {
scenarios: {
redisPerformance: {
executor: "shared-iterations",
vus: 10,
iterations: 200,
exec: "measureRedisPerformance",
},
usingRedisData: {
executor: "shared-iterations",
vus: 10,
iterations: 200,
exec: "measureUsingRedisData",
},
const client = new redis.Client({
socket: {
host: 'localhost',
port: 6379,
},
};
username: 'someusername',
password: 'somepassword',
});
```


### Cluster client

// Instantiate a new redis client
const redisClient = new redis.Client({
addrs: __ENV.REDIS_ADDRS.split(",") || new Array("localhost:6379"), // in the form of "host:port", separated by commas
password: __ENV.REDIS_PASSWORD || "",
You can connect to a cluster of Redis servers by using the `cluster` property, and passing 2 or more node URLs:
```javascript
const client = new redis.Client({
cluster: {
// Cluster options
maxRedirects: 3,
readOnly: true,
routeByLatency: true,
routeRandomly: true,
nodes: ['redis://host1:6379', 'redis://host2:6379']
}
});
```

// Prepare an array of crocodile ids for later use
// in the context of the measureUsingRedisData function.
const crocodileIDs = new Array(0, 1, 2, 3, 4, 5, 6, 7, 8, 9);

export function measureRedisPerformance() {
// VUs are executed in a parallel fashion,
// thus, to ensure that parallel VUs are not
// modifying the same key at the same time,
// we use keys indexed by the VU id.
const key = `foo-${exec.vu.idInTest}`;

redisClient
.set(`foo-${exec.vu.idInTest}`, 1)
.then(() => redisClient.get(`foo-${exec.vu.idInTest}`))
.then((value) => redisClient.incrBy(`foo-${exec.vu.idInTest}`, value))
.then((_) => redisClient.del(`foo-${exec.vu.idInTest}`))
.then((_) => redisClient.exists(`foo-${exec.vu.idInTest}`))
.then((exists) => {
if (exists !== 0) {
throw new Error("foo should have been deleted");
Or the same as above, but using node objects:
```javascript
const client = new redis.Client({
cluster: {
nodes: [
{
socket: {
host: 'host1',
port: 6379,
}
},
{
socket: {
host: 'host2',
port: 6379,
}
}
});
}
]
}
});
```

export function setup() {
redisClient.sadd("crocodile_ids", ...crocodileIDs);
}

export function measureUsingRedisData() {
// Pick a random crocodile id from the dedicated redis set,
// we have filled in setup().
redisClient
.srandmember("crocodile_ids")
.then((randomID) => {
const url = `https://test-api.k6.io/public/crocodiles/${randomID}`;
const res = http.get(url);

check(res, {
"status is 200": (r) => r.status === 200,
"content-type is application/json": (r) =>
r.headers["content-type"] === "application/json",
});

return url;
})
.then((url) => redisClient.hincrby("k6_crocodile_fetched", url, 1));
}
### Sentinel (failover) client

export function teardown() {
redisClient.del("crocodile_ids");
}
A [Redis Sentinel](https://redis.io/docs/management/sentinel/) provides high availability features, as an alternative to a Redis cluster.

export function handleSummary(data) {
redisClient
.hgetall("k6_crocodile_fetched")
.then((fetched) => Object.assign(data, { k6_crocodile_fetched: fetched }))
.then((data) =>
redisClient.set(`k6_report_${Date.now()}`, JSON.stringify(data))
)
.then(() => redisClient.del("k6_crocodile_fetched"));

return {
stdout: textSummary(data, { indent: " ", enableColors: true }),
};
}
You can connect to a sentinel instance by setting additional options in the object passed to the `Client` constructor:
```javascript
const client = new redis.Client({
username: 'someusername',
password: 'somepassword',
socket: {
host: 'localhost',
port: 6379,
},
// Sentinel options
masterName: 'masterhost',
sentinelUsername: 'sentineluser',
sentinelPassword: 'sentinelpass',
});
```

Result output:

```shell
$ ./k6 run test.js
### TLS

A TLS connection can be established in a couple of ways.

If the server has a certificate signed by a public Certificate Authority, you can use the `rediss` URL scheme:
```javascript
const client = new redis.Client('rediss://example.com');
```

Otherwise, you can supply your own self-signed certificate in PEM format using the `socket.tls` object:
```javascript
const client = new redis.Client({
socket: {
host: 'localhost',
port: 6379,
tls: {
ca: [open('ca.crt')],
}
},
});
```

/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
Note that for self-signed certificates, [k6's `insecureSkipTLSVerify` option](https://k6.io/docs/using-k6/k6-options/reference/#insecure-skip-tls-verify) must be enabled (set to `true`).

execution: local
script: test.js
output: -

scenarios: (100.00%) 1 scenario, 10 max VUs, 1m30s max duration (incl. graceful stop):
* default: 10 looping VUs for 1m0s (gracefulStop: 30s)
### TLS client authentication (mTLS)

You can also enable mTLS by setting two additional properties in the `socket.tls` object:

running (1m00.1s), 00/10 VUs, 4954 complete and 0 interrupted iterations
default ✓ [======================================] 10 VUs 1m0s
```javascript
const client = new redis.Client({
socket: {
host: 'localhost',
port: 6379,
tls: {
ca: [open('ca.crt')],
cert: open('client.crt'), // client certificate
key: open('client.key'), // client private key
}
},
});
```


## API

xk6-redis exposes a subset of Redis' [commands](https://redis.io/commands) the core team judged relevant in the context of k6 scripts.
Expand Down Expand Up @@ -228,7 +232,7 @@ xk6-redis exposes a subset of Redis' [commands](https://redis.io/commands) the c
| **LSET** | `lset(key: string, index: number, element: string)` | Sets the list element at `index` to `element`. | On **success**, the promise **resolves** with `"OK"`. If the list does not exist, or the index is out of bounds, the promise is **rejected** with an error. |
| **LREM** | `lrem(key: string, count: number, value: string) => Promise<number>` | Removes the first `count` occurrences of `value` from the list stored at `key`. If `count` is positive, elements are removed from the beginning of the list. If `count` is negative, elements are removed from the end of the list. If `count` is zero, all elements matching `value` are removed. | On **success**, the promise **resolves** with the number of removed elements. If the list does not exist, the promise is **rejected** with an error. |
| **LLEN** | `llen(key: string) => Promise<number>` | Returns the length of the list stored at `key`. If `key` does not exist, it is interpreted as an empty list and 0 is returned. | On **success**, the promise **resolves** with the length of the list at `key`. If the list does not exist, the promise is **rejected** with an error. |

### Hash field operations

| Redis Command | Module function signature | Description | Returns |
Expand Down
10 changes: 5 additions & 5 deletions examples/loadtest.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ export const options = {
},
};

// Instantiate a new redis client
const redisClient = new redis.Client({
addrs: __ENV.REDIS_ADDRS.split(",") || new Array("localhost:6379"), // in the form of "host:port", separated by commas
password: __ENV.REDIS_PASSWORD || "",
});
// Instantiate a new Redis client using a URL
const redisClient = new redis.Client(
// URL in the form of redis[s]://[[username][:password]@][host][:port][/db-number
__ENV.REDIS_URL || "redis://localhost:6379",
);

// Prepare an array of crocodile ids for later use
// in the context of the measureUsingRedisData function.
Expand Down
8 changes: 8 additions & 0 deletions examples/tls/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# How to run a k6 test against a Redis test server with TLS

1. Move in the docker folder `cd docker`
2. Run `sh gen-test-certs.sh` to generate custom TLS certificates that the docker container will use.
3. Run `docker-compose up` to start the Redis server with TLS enabled.
4. Connect to it with `redis-cli --tls --cert ./tests/tls/redis.crt --key ./tests/tls/redis.key --cacert ./tests/tls/ca.crt` and run `AUTH tjkbZ8jrwz3pGiku` to authenticate, and verify that the redis server is properly set up.
5. Build the k6 binary with `xk6 build --with github.com/k6io/xk6-redis=.`
5. Run `./k6 run loadtest-tls.js` to run the k6 load test with TLS enabled.
26 changes: 26 additions & 0 deletions examples/tls/docker/docker-compose.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
version: "3.3"

services:
redis:
image: docker.io/bitnami/redis:7.0.8
user: root
restart: always
environment:
- ALLOW_EMPTY_PASSWORD=false
- REDIS_PASSWORD=tjkbZ8jrwz3pGiku
- REDIS_DISABLE_COMMANDS=FLUSHDB,FLUSHALL
- REDIS_EXTRA_FLAGS=--loglevel verbose --tls-auth-clients optional
- REDIS_TLS_ENABLED=yes
- REDIS_TLS_PORT=6379
- REDIS_TLS_CERT_FILE=/tls/redis.crt
- REDIS_TLS_KEY_FILE=/tls/redis.key
- REDIS_TLS_CA_FILE=/tls/ca.crt
ports:
- "6379:6379"
volumes:
- redis_data:/bitnami/redis/data
- ./tests/tls:/tls

volumes:
redis_data:
driver: local

0 comments on commit b5c72ea

Please sign in to comment.