This is an Apache JMeter based test suite for two popular openEHR servers: EHRServer and EHRbase. The purpose of this project is to provide tools for benchmarking those servers in order to assess their throughput and performance. Each server is tested for creating EHRs, fetching EHRs, adding compositions and fetching compositions. Tests are prepared in such a way that they allow simulating many users (threads) in order to see how both servers handle large network traffic.
The tests are performed using ePrescription (FHIR) template available in CKM. The template and its instances can be found in ./templates
.
Instances of this template used in the tests were created using openEHR-OPT project. Because EHRbase accepts JSONs and EHRServer accepts both XMLs and JSONs (the latter are translated to XMLs internally, so XMLs should be better), one can invoke following commands to create instances:
./opt.sh ingen path_to_opt path_to_instance_dir 1 json composition
./opt.sh ingen path_to_opt path_to_instance_dir 1 xml composition
ingen
(at least 52a4676645692fb1dd6129439ccb21102f758dc7
version which was used) has a bug which results in inproper ePrescription instances (an item with id at0113
is doubled whereas it should have 0..1 occurences). Doubled elements were removed manually from the generated instances.
openEHR REST API documentation can be found here.
Currently, scripts are prepared only for Windows. Linux scripts will be added in the next release. MATLAB and Python scripts which will allow results processing will also be added.
- Apache JMeter is required (version 5.5 was used during development)
- Apache JMeter
bin
directory has to be set in PATH environment variable - EHRServer and EHRbase have to be configured and run according to their documentation
- Add the template to EHRbase (see EHRbase prerequisites)
- Add the template to EHRserver (see EHRbase prerequisites)
- Before running the test please modify
variables.bat
script which is responsible for setting required environment variables
Execute run_ehrbase_tests.bat
for EHRbase tests or run_ehrserver_tests.bat
for EHRServer tests. Each script calls variables.bat
script which sets necessary environemnt variables and then runs test scripts for 1, 2, 5, 10, 20, 30, 40, 50 threads in a loop. Results generated by the EHRbase/EHRserver tests are saved in ./jmeter-tests/ehrbase/results/
and ./jmeter-tests/ehrserver/results
, respectively. Tests for different number of threads are saved in separate directories. Every time a test for a certain number of threads is run, the previous directory and it's contents for this number of threads is deleted. Below you can find some specific info about the tests. Each tests lasts %OPENEHR_TEST_DURATION%
number of seconds.
EHRServer has some problems with POSTing compositions by more than one thread. Because of that, creating compositions is always performed by one thread.
EHRbase tests can be found in ./jmeter-tests/ehrbase
. For each *.jmx
file there is a corresponding *.bat
file which allows to run the test from command line (suggested).
Project home: link
A template has to be added to the server. It has to be done using POST /definitions/template/adl1.4 endpoint. You can add a template using ./jmeter-tests/ehrbase/ehrbase_post_template.bat
script which has the ePrescription (FHIR)
template hardcoded in the requests body.
- Test:
./jmeter-tests/ehrbase/ehrbase_put_ehr.jmx
- Script:
./jmeter-tests/ehrbase/run_put_ehr.bat
- Test:
./jmeter/ehrbase/erhbase_get_ehr.jmx
- Script:
./jmeter-tests/ehrbase/run_get_ehr.bat
- Test:
./jmeter/ehrbase/ehrbase_post_composition.jmx
- Script:
./jmeter-tests/ehrbase/run_post_composition.bat
- Test:
jmeter/ehrbase/ehrbase_post_contribution.jmx
. - Script:
./jmeter-tests/ehrbase/run_post_contribution.bat
- Test:
./jmeter/ehrbase/ehrbase_get_composition.jmx
- Script:
./jmeter-tests/ehrbase/run_get_composition.bat
- Test:
jmeter/ehrbase/ehrbase_get_contribution.jmx
. - Script:
./jmeter-tests/ehrbase/run_get_contribution.bat
Each time run_ehrserver_tests.bat
is run, ./jmeter-tests/ehrserver/ehrserver_post_auth.jmx
is run in order to retrieve a token necessary for authentication. The token is saved to ./jmeter-tests/ehrserver/data/token.txt
and set as an environment variable used by all other scripts.
A template has to be added manually using EHRServer's admin console. Log in, then go to Templates
and upload an *.opt
file.
- Test:
./jmeter-tests/ehrserver/ehrserver_post_ehrs.jmx
- Script:
./jmeter-tests/ehrserver/run_post_ehrs.bat
In case of errors force 1 thread.
- Test:
./jmeter/ehrserver/erhserver_get_ehrs.jmx
- Script:
./jmeter-tests/ehrserver/run_get_ehrs.bat
- Test:
./jmeter/ehrserver/ehrserver_post_compositions.jmx
- Script:
./jmeter-tests/ehrserver/run_post_compositions.bat
In case of errors force 1 thread.
- Test:
./jmeter/ehrserver/ehrserver_get_compositions.jmx
- Script:
./jmeter-tests/ehrserver/run_get_compositions.bat
- creating EHR with ehr_id created automatically: POST /ehr
- creating EHR with ehr_id provided: PUT /ehr/{ehr_id}
- fetching EHR: GET /ehr/{ehr_id}
- creating composition: POST /ehr/{ehr_id}/composition
- creating contribution: POST /ehr/{ehr_id}/contribution
- fetching composition: GET /ehr/{ehr_id}/composition/{uid_based_id}
- fetching contribution GET /ehr/{ehr_id}/contribution/{contribution_uid}