Scale out using nats

[gedw99]

Docs says to use k8.

I personally think that nats would be much easier and provide the system to be reactive .

Some things NATS would make easy are:

file config is automatically updated on all nodes , so that each node has it

jobs allocated in real time so that nodes are able to reactively get work to do .

Raw csv output also synced back from all nodes in real time so you can see histogram etc in real time , but also batched .

Anyone interested ?

[rogerwelin]

Hi @gedw99 and thanks for the suggestion. I have very limited working experience with Nats. If you have a suggested workflow (for example the one described above) I would be very interested in a PR with updated examples in the README

[gedw99]

i think its only slight changes...

https://github.com/rogerwelin/cassowary#distributed-load-testing:

K8 is just the envelope for the cloud infrastructure and nothing to do with if we use NATS of not. Good news.

We can embed the NATS server. Just a bool flag at startup. Makes things easy for people to not have to run another server.

• if the "embedded nats" flag is false, the cli just needs the url: "localhost:4422".
• nats jetstream can run as InMemory mode, so on K8( there is no volume management needed ).

We can store results over time inside NATS the NATS KV and OBJ store.

• works with embedded or non embedded mode.

JOBS

First, the easiest way is to add a NATS file slurp mode. Keep the existing Http one:

./cassowary run -u http://localhost:8000 -c 1 -f urlpath.txt

• client pushes urlpath.txt and args to nats handler, which then creates a nats queue for each line in the URL list.
• all other clients are sent a job off the queue, do the work and submit those back to the NATS KV.

Starting Load Test with 100 requests using 10 concurrent users

./cassowary run -u http://localhost:8000 -c 10 -n 100 -f urlpath.txt

• same as above, with the NATS job handler ensuring concurrency and affinity.

Prom Metrics

We can store all cassowary metrics as Prom style in NATS too: https://github.com/kmpm/promnats.go

Historgram / boxplots

• same as above except the outputs go back to NATS KV.
• each client can display the plots in real time by subscribing to the nats KV. Or a simple web GUI that shows them in real time.
• at the end of the job run a worker makes the plots, and stores them in NATS OBJ store, so that you can bring up a job later.

SO basically we are adding the scale out and persistence. You can export out easily too. Some people want to put the outputs in git etc.

Just opening this up for comments.

[gedw99]

will close as seems no one into the idea.