Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmark informations from white paper #434

Merged
merged 1 commit into from Jan 6, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 6 additions & 1 deletion src/content/1.7/scale/benchmark/_index.md
Expand Up @@ -17,7 +17,12 @@ How to setup the benchmark of your PrestaShop shop
- <a href="https://getcomposer.org/download/composer">composer</a>
- The prestashop version you want to bench

## Prepare your benchmark
## Dockerized installation of benchmark

To get an automatically pre-populated shop, you can use the following github repository:
**[Prestashop performance project](https://github.com/PrestaShop/performance-project)**

## Manually prepare your benchmark

### Prepare your dataset

Expand Down
168 changes: 165 additions & 3 deletions src/content/1.7/scale/benchmark/back-office.md
Expand Up @@ -19,14 +19,176 @@ In order to benchmark the performances of the backoffice of your shop, you will

## What is Gatling?
**[Gatling](https://gatling.io/)** is a stress tool. Development is currently focusing on HTTP support. We won’t explain here the fundamentals of Gatling, but for more details I suggest you to have a look at **[the Gatling project](https://github.com/gatling/gatling)**.

## Automatic run of Gatling on dockerized installation (recommended)

### Prerequisite

- git
- docker

### Clone the repository

To follow these instructions, you will use the following github repository:
**[Prestashop performance project](https://github.com/PrestaShop/performance-project)**

```
git clone git@github.com:PrestaShop/performance-project.git
```

### Shop filled with fixtures

Benchmarks require a pre-populated shop. But this shop content has to be the same at each new bench in order to get reproducible results. Prestashop ShopCreator project will help to generate fixtures, used during shop installation to populate the benched shop.

#### Generate fixtures

cd performance-project
cd shop

`README.md` contains instructions to get a working shop filled with fixtures, using “Prestashop ShopCreator” project

git clone git@github.com:PrestaShop/prestashop-shop-creator.git

ShopCreator requires a config file to know which fixtures it has to generate. shop-creator-config.yml is the one used in this study. You can customize it to fit your own needs. Copy it in the ShopCreator /config/ directory.

cp shop-creator-config.yml prestashop-shop-creator/app/config/

First we need to install required ShopCreator dependencies

composer install

Once your config is ready, run fixture generation in ShopCreator directory

php app/console.php

Fixtures are created in /generated_data/ directory. It contains xml files with data for each Prestashop entity, and images for products, categories…

#### Install a shop with generated fixtures

To use these generated fixtures, you need to copy the directory content into the installation directory of Prestashop before installing the shop

cp -R generated_data/* [prestashop]/install/fixtures/fashion/

Make sure you’re not in debug mode! In config/defines.inc.php you should have:

define('_PS_MODE_DEV_', false);

Once this is done, simply install and run your shop. If you want to use our pre-configured Dockerfile to do this, just keep reading.

#### Install and run the shop with docker

The directory /prestashop-performance/ contains required Dockerfile and scripts to install and run a shop with generated fixtures

cd ..

##### Build the docker image

Copy the previously generated fixtures in the corresponding directory.

cp -R prestashop-shop-creator/generated_data/* prestashop-performance/fixtures/

Then you can build the docker image. This one will embed generated fixtures since the Dockerfile copy them in the image.

cd prestashop-performance
docker build -t prestashop-bench .

Your image is ready to use.

##### Run the shop

Run the shop with the following command:

```
docker run \
-e FIXTURE_FOLDER=fixtures\
-e PS_DOMAIN=localhost:8080\
-e PS_FOLDER_ADMIN=admin1234\
-e PS_ERASE_DB=1 \
-e PS_CANONICAL_REDIRECT_DISABLE=1 \
-e DB_SERVER=host.docker.internal\
-e DB_PORT=3306\
-e DB_NAME=prestashop_bench \
-e DB_USER=prestashop_bench \
-e DB_PASSWD=prestashop_bench \
-p 8080:80 \
prestashop-bench
```

Installation can take several minutes, depending on the size of fixtures you set, and of course depending on the machine you run it on.

Once it is done, you should access this shop from http://localhost:8080 on your browser.

##### Hosting customizations

The docker image above is used for this study purposes. It is deployed on several stacks detailed in Results.
Docker usage makes it easy to customize prestashop installation, or hosting configuration like php tuning, or mysql tuning.

### Run Gatling scenarios

cd performance-project
cd benchmark/gatling

{{% notice note %}}
*NOTICE* : Gatling scenarios will create orders on the tested shop. You should not use it on production environment
{{% /notice %}}

#### Build Gatling image

Official gatling image needs to be lightly customized. More precisely we need to pass some java arguments to the scenarios.

docker build -t prestashop-bench-gatling .

#### Run Gatling

Provided Gatling run command accepts arguments such as user count, customer count, test duration.

```
docker run -it --rm \
-v $dir/results:/opt/gatling/results \
-e JAVA_OPTS="-DusersCount=500
-DcustomersCount=20
-DadminsCount=0
-DrampDurationInSeconds=900
-DhttpBaseUrlFO=http://shop2.stack1.prestashop.net
-DhttpBaseUrlBO=http://shop2.stack1.prestashop.net/ps-admin
-DadminUser=admin@prestashop.com
-DadminPassword=prestashop" \
--add-host=sandbox.prestashop.com:192.168.0.4 \
prestashop/performance-gatling \
-s LoadSimulation
```

In the example above:
* -v $dir/results:/opt/gatling/results
mount result/ directory to get gatling results stored in it
* -DrampDurationInSeconds=900
the gatling simulation will add user count during this period (in seconds)
* -DusersCount=500
User count to run FrontOfficeCrawl scenario (crawling product/category pages)
* -DcustomersCount=500
User count to run FrontOfficeCart scenario (placing order)
* -DadminsCount=500
User count to run BackOfficeCrawl scenario
* -s LoadSimulation
The name of the gatling simulation in the code (in the directory user-files/simulations/)

#### Gatling run script

A more scriptable approach could be using the provided `run.sh` script. This script aims to properly rename result directory to a more readable one, and process these results to extract some useful data such as the needed “order-per-hour” KPI.
You could then write your own `batch.sh` to run multiple gatling benchmarks.

## Manual run of Gatling

### Gatling installation

Download Gatling from **[here](https://gatling.io/download/)**, and in the same way have a look at **[the Gatling quickstart page](https://gatling.io/docs/current/quickstart/)**.

Once unzipping the folder it will look like as shown below:


![Gatling](https://i.imgur.com/devRwHF.png)

## Your Gatling is ready to be run
### Your Gatling is ready to be run
Now let's test if our Gatling works well, so you can launch a sample test included natively in the project with the CLI:

```
Expand All @@ -43,7 +205,7 @@ In my example I run "[1] basic.BasicExempleSimulation"
##### Well done! Our Gatling installation is ready!
After few minutes you can consult the detailed report generated automatically into "results" folder.

## Insert and setup your script
### Insert and setup your script

Download the script "[parcoursbackoffice.scala](../parcoursbackoffice.scala)" and put it under "/gatling/user-files/simulations".

Expand All @@ -58,7 +220,7 @@ Open with your editor the parcoursbackoffice.scala file and setup your script:
![Gatling](https://i.imgur.com/zRMVSiW.png)


## Disable the token
### Disable the token

To avoid handling the multiple generated token on each back-office page, you can just disable it by following the steps below:

Expand Down
16 changes: 12 additions & 4 deletions src/content/1.7/scale/benchmark/front-office.md
Expand Up @@ -9,11 +9,19 @@ aliases:
How to benchmark your PrestaShop Shop (Front-office)
==================

## Benchmark methodology
## Automatically benchmark with `Gatling` (recommended)

Follow instruction on **[Back-Office benchmark page](https://devdocs.prestashop.com/1.7/scale/benchmark/back-office/)** to get a pre-populated shop and to run Gatling scenarios on it.

**[Prestashop performance project](https://github.com/PrestaShop/performance-project)** on Github includes Front-Office scenarios you can edit to get your own scenarios running.

## Benchmark with `Siege` tool

### Benchmark methodology
In order to benchmark the performances of your shop, you will use the siege testing tool.
Try to always use the latest available version <a href="http://download.joedog.org/siege/siege-latest.tar.gz">HERE</a>.

## Setup siege configuration
### Setup siege configuration

Create a txt file ```url.txt``` with various urls from your shop: (prepend with the domain of your shop)

Expand All @@ -34,7 +42,7 @@ http://localhost:8080/recherche?controller=search&s=sweater
http://localhost:8080/2-accueil
```

## Run the siege benchmark
### Run the siege benchmark

Then run a siege benchmark using this file:
```text
Expand Down Expand Up @@ -85,7 +93,7 @@ Longest transaction: 0.57
Shortest transaction: 0.02
```

## Interpret the results
### Interpret the results

In the siege result output, here is the useful results:

Expand Down