As a Concourse dev, it would be nice to know the impact that a particular release has on the overall performance of Concourse. @jama22 pointed out that since a lot of the runtime backlog revolves around stability, this would help us understand whether the changes made in a particular release improve stability or have unforeseen consequences that impact resiliency or efficiency.
Here are some of the benchmarking metrics that we would like to measure against some predefined workload:
Worker:
CPU load
Memory usage
Disk utilization
System load average
Min, max and average number of containers
Min, max and average number of volumes
Network I/O
Standard deviation of the above values among all workers (esp. containers and volumes)
Web:
HTTP response duration
Number of DB connections
Time taken to schedule a build
This issue is to discuss and track work related to building out a benchmarking tool or process.
The text was updated successfully, but these errors were encountered:
We plan to use concourse/concourse/drills to create a performance environment. We can use https://linux.die.net/man/1/stress to simulate high load for CPU, memory and disk I/O.
For containers, volumes and network I/O we need to design a pipeline with several heavy get steps.
I'd also add Brendan Gregg's book, Systems Performance: Enterprise and the Cloud to a reading list. It's very good at breaking down the way different components and systems can manifest performance problems.
As a Concourse dev, it would be nice to know the impact that a particular release has on the overall performance of Concourse. @jama22 pointed out that since a lot of the runtime backlog revolves around stability, this would help us understand whether the changes made in a particular release improve stability or have unforeseen consequences that impact resiliency or efficiency.
Here are some of the benchmarking metrics that we would like to measure against some predefined workload:
Worker:
Web:
This issue is to discuss and track work related to building out a benchmarking tool or process.
The text was updated successfully, but these errors were encountered: