Benchmarking REST APIs?

[ishepherd]

Hi, new user here just want your opinion:

Is Benchmark.Net a sensible choice for performance testing of a REST API?

Most examples I see use it for tiny microbenchmarks. It looks quite suitable for benchmarking any kind of code, I am just wondering if I will hit any particular impedances with blackbox testing of an API.

One example: Can it run the tests in parallel? For code microbenchmarks this would be unhelpful and probably unnecessary. But for API calls that take several seconds each, and many iterations to do, it might be necessary and with your statistical guards in place it might be reasonable to do.

[ishepherd]

Useful past issues:

#659 (comment)
#1314

[adamsitnik]

Hi @ishepherd

BenchmarkDotNet was designed for very accurate micro-benchmarking. All the metrics that we report are specific to the process which is executing the benchmark. The architecture does not have a concept of multiple processes like client and server. So we can tell you have long it takes to send a web request, but we can't report how much memory the server has allocated to handle it.

Most probably https://github.com/aspnet/Benchmarks/ is what you are looking for. It has a client-server concept and can track metrics for both. And it was designed to run web server benchmarks.

Since there is nothing actionable on our side here, I am closing the issue.

Thanks,
Adam