Load test for gluu-client-auth and gluu-pep Kong plugins

[altexy]

No description provided.

[altexy]

We need to design load test to get an idea about performance of our plugins.

Also such test is required for possible performance optimizations.

My idea is to test with minimal possible environment like we do in existing test suite.
For example:

• PostgreSQL
• Kong 0.14
• our plugins
• mock oxd-server

Obviously we cannot test on local developer hardware.
To get realistic numbers we need a VMs with dedicated CPU resources.
For example on Digital Ocean (DO) we need to use Optimized droptes instead of Standard.

By my experience for serious load test we will need:

• powerful DB VM
• powerful Kong VM
• a number of backend VMs, usually it is a bottleneck
• VM for oxd-mock

We cannot use our existing oxd-mock approach, under heavy load it will not work.
We will need to implement different mock server which will accept some percentage of tokens, 95% for example.

[nynymike]

I like the idea. It's a good start.

[altexy]

YuriyZ October 24, 2018 5:11 PM
in 3.1.3.1 performance tests was ~100 req/s. In 4.0 we dropped one layer of serialization, so it should be a bit faster. But I would not expect significant increase because at the end oxd calls oxauth which without tunning is around 100 req/s, and with tunning ~120-140 req/s.

So the real bottleneck is the oxAuth.
IMO it isn’t make sense to test GG, because with Kong we may talk about thousands RPS.

[yuriyz]

It can be good idea to mock oxd and oxauth calls. So you can performance GG code and find out all possible bottlenecks and race conditions inside GG. It will keep GG prepared for oxauth 4.0 and oxd 4.0, Hopefully with couchbase persistence we will get better numbers.

[altexy]

@nynymike @willow9886 we have 2 choices for perf. testing:

1. Use a mock environment (as suggested by @yuriyz above)
2. Test with real Gluu deployment (GG + oxd + GS)

First is perfect for GG perf. tuning, but would not provide the real perf. that the customer may see.
I have a feeling that bottleneck will not be in GG.

Second is much difficult to implement as a full automated test, but would show the real picture.

IMO both approaches make sense, need your advice about priorities.

[nynymike]

Definitely proceed with #1. Performance testing oxd should be a separate exercise. And Gluu Server we already test.