Skip to content
This repository has been archived by the owner on Apr 29, 2022. It is now read-only.

rvansa/keycloak-benchmark

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Keycloak Benchmark

This is a stress test simulating users accessing fictive application that authorizes against one or more Keycloak servers, and admin users who change users' permissions etc.. There's no actual application and no container; the behaviour of particular keycloak-adapter is simulated within the test itself. For backchannel operations (such as logout notification) there's simple webserver (called AppServer) accepting requests from the server. ' Therefore SUT (system under test) covers only the Keycloak servers themselves, shared (PostgreSQL) database and the AppServer.

The test is driven by Gatling and also produces its report with response times statistics. To observe internal behaviour of the server, use of Java Flight Recorder as a low-impact profiler is recommended.

Running the benchmark

After building the project the target/dist/ directory will contain scripts, configuration files and test JARs for this test. You will directly use only script bin/run.sh which orchestrates the benchmark.

The benchmark is configurable using file passed as first argument to this script. The properties file must contain at least these variables:

  • SERVERS = array with server addresses
  • DRIVERS = array with driver addresses
  • KEYCLOAK_DIST = location of keycloak server distribution

Then, you can configure the database using variables DB_ADDRESS, DB_NAME, DB_USER and DB_PASSWORD. Note that PostgreSQL database must be set up separately (this script does not set up DB).

Additional parameters configurable through the properties file are:

  • LOADER_ARGS = any parameters passed to loader
  • DRIVER_ARGS = any parameters passed to drivers
  • RSH = remote shell command (default is ssh)
  • RCP = remote copy command (default is scp)
  • DC_DIR = directory for the domain controller
  • LOG_DIR = directory for the logs
  • SERVER_PORT = TCP port of server (defaults to 8080)
  • APP_ADDRESS = AppServer address
  • APP_PORT = AppServer port

There is an example configuration test/dist/bin/example-properties.sh.

The test will follow these steps:

  • Copy necessary files (including Keycloak server distribution) onto server and driver (load generator) machines
  • Start domain controller on local machine (where the script is executed)
  • Start host controllers and Keycloak servers in clustered mode (using adapted HA-profile)
  • Remove existing realm benchmark-realm from the server and create new one, with configurable number of users.
  • Starts AppServer
  • Run the actual Gatling test on all driver machines
  • Retrieve simulation.log files from driver machines
  • Create report from all drivers
  • Kill all servers

Upon failure, servers are not killed. When the test is successfully finished, the Gatling report will be found in /tmp/report.

Scenarios

Note that the Keycloak realm's user (username, credentials) is a different concept than the User below; here the user means a flow of operations on client/in application. Also, there are active and inactive Keycloak users; inactive users just occupy DB while active users is the set of credentials used in the User scenario. By executing user we mean a Keycloak user that is used in running User scenario instance.

To confuse things a bit more, Gatling reports 'active users' as well; there it means the number of currently running scenario instances (User or Admin).

  • User: each user executes this scenario:
    • get login screen
    • type password incorrectly 0 - n times (constant probability of mistyping it on each attempt)
    • type password correctly
    • application authorizes against keycloak using OAuth
    • application refreshes token 0 - n times (constant probability of repeating this)
    • user logs out (with certain probability)
  • Admin: each admin executes one of these operations:
    • add a new user, and set its password (that's second request)
    • remove existing user
    • list users whose username starts with string consisting of two random letters

There are pauses in between the User operations where appropriate, simulating User's think-time or application interaction not related to Keycloak. The actual pause is randomized with gaussian distribution and std.dev. equal to 1/10 of the mean value.

In order to not contend on this active users, each driver uses distinct subset of them, both for User scenario and Admin operations. Each active user is used by only single instance of User scenario; on the other hand, Admin operation can modify (=remove) both executing and non-executing (but active) user. When admin creates new user, it becomes an active user.

Every second, Gatling creates configurable number of User and Admin scenario instances (test.usersPerSecond or test.adminsPerSecond). The executing user becomes active user after the scenario instance is finished (either in a regular way or by an unexpected response from server).

Below is a table of options for the test (all these are to be found in org.jboss.perf.Options, those not listed below are injected automatically):

Property Default value Description
test.rampUp 30 Period of load ramp-up, in seconds (going from 10% of users/admins per second to full value).
test.duration 60 Full stress test duration, in seconds.
test.rampDown 30 Time for executing users to die out, in seconds.
test.usersPerSecond 100 Number of user scenarios started each second.
test.adminsPerSecond 2 Number of admin scenarios started each second.
test.activeUsers 50 Active users used for the test.
test.totalUsers 100 All users in the database.
test.usernameLength 16 (Fixed) length of username, ASCII lower case characters.
test.passwordLength 16 (Fixed) password length, ASCII lower case characters.
test.loginFailureProbability 0.2 Probability of mistyping the password (value between 0 and 1).
test.refreshTokenProbability 0.5 Probability of asking for a token refresh (value between 0 and 1).
test.logoutProbability 0.8 Probability of executing of logout at the end of the scenario.
test.userResponsePeriod 1 Time to e.g. fill in login screen, in seconds.
test.refreshTokenPeriod 3 Time between token refresh requests, in seconds.
test.addRemoveUserProbability 0.2 Probability of admin adding new user, or removing existing one. These are equal to keep balanced number of active users throughout the test.
test.userRoles 150 Number of distinct roles in the realm
test.userRolesPerUser 5 Number of roles one user has
test.fullReload false Applicable only to loader; reload whole realm (as opposed to only the active users)

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published