New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add functional tests suite #8633

Closed
wants to merge 0 commits into
base: develop
from

Conversation

Projects
None yet
7 participants
@mbadrani
Contributor

mbadrani commented Dec 22, 2017

Questions Answers
Branch? develop
Description? My PR aim to provide the Functional tests campaign
Type? new feature
Category? TE
BC breaks? no
Deprecations? no
Fixed ticket? no
How to test? Please follow the readme file to be able to execute the test suit.

Important guidelines


This change is Reviewable

@prestonBot

This comment has been minimized.

Show comment
Hide comment
@prestonBot

prestonBot Dec 22, 2017

Collaborator

Hello mbadrani!

This is your first pull request on the PrestaShop project. Thank you, and welcome to this Open Source community!

Collaborator

prestonBot commented Dec 22, 2017

Hello mbadrani!

This is your first pull request on the PrestaShop project. Thank you, and welcome to this Open Source community!

@Quetzacoalt91

This comment has been minimized.

Show comment
Hide comment
@Quetzacoalt91

Quetzacoalt91 Dec 22, 2017

Member

Be careful, if you want to run your tests in Travis, the .travis.yml at the root of the repository must be updated. :)

Member

Quetzacoalt91 commented Dec 22, 2017

Be careful, if you want to run your tests in Travis, the .travis.yml at the root of the repository must be updated. :)

@eternoendless eternoendless changed the title from Add Functional tests suit to Add functional tests suite Dec 22, 2017

@eternoendless

This comment has been minimized.

Show comment
Hide comment
@eternoendless

eternoendless Dec 26, 2017

Member

Test fail when launching them on a fresh shop.

@Quetzacoalt91 I think we should merge tests first (after we have made sure they work locally) and then activate them later in Travis in another PR.

Member

eternoendless commented Dec 26, 2017

Test fail when launching them on a fresh shop.

@Quetzacoalt91 I think we should merge tests first (after we have made sure they work locally) and then activate them later in Travis in another PR.

@Quetzacoalt91

This comment has been minimized.

Show comment
Hide comment
@Quetzacoalt91

Quetzacoalt91 Dec 27, 2017

Member

@eternoendless I usually try to make the tests running on Travis at the same time, to prove we can rely on them.

Member

Quetzacoalt91 commented Dec 27, 2017

@eternoendless I usually try to make the tests running on Travis at the same time, to prove we can rely on them.

@eternoendless

This comment has been minimized.

Show comment
Hide comment
@eternoendless

eternoendless Jan 10, 2018

Member

I usually try to make the tests running on Travis at the same time, to prove we can rely on them.

OK let's do it then

Member

eternoendless commented Jan 10, 2018

I usually try to make the tests running on Travis at the same time, to prove we can rely on them.

OK let's do it then

@Quetzacoalt91

This comment has been minimized.

Show comment
Hide comment
@Quetzacoalt91

Quetzacoalt91 Jan 10, 2018

Member

Yep, @fatmaBouchekoua said there are some tests "high" to run, instead of the whole suite.
If these tests pass, we will be able to merge that PR.

Member

Quetzacoalt91 commented Jan 10, 2018

Yep, @fatmaBouchekoua said there are some tests "high" to run, instead of the whole suite.
If these tests pass, we will be able to merge that PR.

@fatmaBouchekoua

This comment has been minimized.

Show comment
Hide comment
@fatmaBouchekoua

fatmaBouchekoua Jan 10, 2018

Contributor

Hello @Quetzacoalt91 ,
you can retest after merging this PR: mbadrani#21

Contributor

fatmaBouchekoua commented Jan 10, 2018

Hello @Quetzacoalt91 ,
you can retest after merging this PR: mbadrani#21

@Quetzacoalt91

This comment has been minimized.

Show comment
Hide comment
@Quetzacoalt91

Quetzacoalt91 Jan 17, 2018

Member

I made a new try on my side, here are the result of the high test suite:

[...] All tests OK
      Print invoice
        ✓ should click on "DOCUMENTS" subtab
        ✓ should download the invoice document
        1) should check the "invoice file name"            <-- I saw the download was made 2 times
        2) should check that the "invoice customer" is "John Doe"
        3) should check  the "invoice basic price"  
        4) should check that the "invoice product information" is : "Blouse - Size : S- Color : White"
      Print delivery invoice
        ✓ should download the delivery invoice document
        5) should check the "delivery invoice file name"
        6) should check that the "delivery invoice customer" is : Johan DOE
        7) should check that the "delivery invoice product information" is : Blouse - Size : S- Color : White
        8) should check that the "delivery invoice product carrier" is : My carrier"

  Create order in FO
    Open the browser and connect to the FO
      ✓ should open the browser
      Create Standard Product
        ✓ should open browser
        Check the product creation
          ✓ should open browser
          Create a pack of products
            ✓ should open browser
            Check the product in the catalog
              ✓ should open browser
              ✓ should open browser
              ✓ should open browser
              Create product with combination
                Check the product in the catalog
                  Create virtual Product
                    ✓ should open browser
                    Check the product in the catalog
                      ✓ should open browser
                      ✓ should sign in FO
                      ✓ should log in successfully in BO
                      ✓ should log in successfully in BO
                      ✓ should log in successfully in BO
                      ✓ should log in successfully in BO
                      ✓ should log in successfully in BO
                      ✓ should log in successfully in BO
                      ✓ should log in successfully in BO
                      9) should log in successfully in BO
                      10) should log in successfully in BO
                      11) should log in successfully in BO
                      12) should log in successfully in BO

Then my computer died, running out of memory. Too many instances of Google Chrome were opened. The browser was requested 9 times, which seems unexpected.

Also, when you try to close the "welcome" modal, selenium does not wait for the new page to be loaded, it directly set the step in error.

Member

Quetzacoalt91 commented Jan 17, 2018

I made a new try on my side, here are the result of the high test suite:

[...] All tests OK
      Print invoice
        ✓ should click on "DOCUMENTS" subtab
        ✓ should download the invoice document
        1) should check the "invoice file name"            <-- I saw the download was made 2 times
        2) should check that the "invoice customer" is "John Doe"
        3) should check  the "invoice basic price"  
        4) should check that the "invoice product information" is : "Blouse - Size : S- Color : White"
      Print delivery invoice
        ✓ should download the delivery invoice document
        5) should check the "delivery invoice file name"
        6) should check that the "delivery invoice customer" is : Johan DOE
        7) should check that the "delivery invoice product information" is : Blouse - Size : S- Color : White
        8) should check that the "delivery invoice product carrier" is : My carrier"

  Create order in FO
    Open the browser and connect to the FO
      ✓ should open the browser
      Create Standard Product
        ✓ should open browser
        Check the product creation
          ✓ should open browser
          Create a pack of products
            ✓ should open browser
            Check the product in the catalog
              ✓ should open browser
              ✓ should open browser
              ✓ should open browser
              Create product with combination
                Check the product in the catalog
                  Create virtual Product
                    ✓ should open browser
                    Check the product in the catalog
                      ✓ should open browser
                      ✓ should sign in FO
                      ✓ should log in successfully in BO
                      ✓ should log in successfully in BO
                      ✓ should log in successfully in BO
                      ✓ should log in successfully in BO
                      ✓ should log in successfully in BO
                      ✓ should log in successfully in BO
                      ✓ should log in successfully in BO
                      9) should log in successfully in BO
                      10) should log in successfully in BO
                      11) should log in successfully in BO
                      12) should log in successfully in BO

Then my computer died, running out of memory. Too many instances of Google Chrome were opened. The browser was requested 9 times, which seems unexpected.

Also, when you try to close the "welcome" modal, selenium does not wait for the new page to be loaded, it directly set the step in error.

@fatmaBouchekoua

This comment has been minimized.

Show comment
Hide comment
@fatmaBouchekoua

fatmaBouchekoua Jan 18, 2018

Contributor

Hello @Quetzacoalt91 ,
you have to add the --DIR parameter to the command line which refers to the downloads directory on your machine and definitely, it must be ended by a / .
Can you retest please, and let me know about results?

Contributor

fatmaBouchekoua commented Jan 18, 2018

Hello @Quetzacoalt91 ,
you have to add the --DIR parameter to the command line which refers to the downloads directory on your machine and definitely, it must be ended by a / .
Can you retest please, and let me know about results?

@Quetzacoalt91

This comment has been minimized.

Show comment
Hide comment
@Quetzacoalt91

Quetzacoalt91 Feb 5, 2018

Member

Just tried the autoupgrade scenario, the installation was actually executed 3 times.

/tmp/node_modules/mocha/bin/mocha /tmp/test/campaigns/high/13_installation --URL=prestashop-web --SELENIUM=selenium-chrome --DIR=/home/seluser/Downloads/ --DB_SERVER=db --DB_USER=root --DB_PASSWD=admin --RCTARGET=/tmp --RCLINK=http://localhost/prestashop_1.7.3.0-RC1.zip
wait-for-it.sh: waiting 200 seconds for prestashop:80
wait-for-it.sh: prestashop:80 is available after 93 seconds


  The shop installation
    Open the browser and connect installation interface
      ✓ should open the browser
      ✓ should go to install page 
    Step 1 : Choosing language
      ✓ should choose "English" language
      ✓ should click on "Next" button
    Step 2 : Agreeing license agreements
      ✓ should click on "I agree to the above terms and conditions " button
      ✓ should click on "Next" button
    Step 3 : Checking system compatibility
      ✓ should check the test compatibility green box
      ✓ should click on "Next" button
    Step 4 : Inserting the shop information
      ✓ should set the "Shop name" input
      ✓ should set the "Country" input
      ✓ should set the "First name" input
      ✓ should set the "Last name" input
      ✓ should set the "E-mail address" input
      ✓ should set the "Shop password" input
      ✓ should set the "Re-type to confirm" input
      ✓ should click on "Next" button
    Step 5 : Setting the BD configuration
      ✓ should set the "Database server address" input
      ✓ should set the "Database name" input
      ✓ should set the "Database login" input
      ✓ should set the "Database password" input
      ✓ should click on "Test your database connection now!" button
      ✓ should check for the connection and click on "Attempt to create the database automatically" button
      ✓ should check that the Database is created
      ✓ should click on "Next" button
    Step 6 : Checking installation
      ✓ should create file parameter
      ✓ should create database
      ✓ should create default shop
      ✓ should create database table
      ✓ should create shop information
      ✓ should create demonstration data
      ✓ should create install module
      ✓ should create addons modules
      ✓ should create install theme
      ✓ should finish installation
    Step 7 : Checking that installation finished
      ✓ should check that the installation is finished!
    Login to the Front Office
      ✓ should sign in FO

  The shop installation
    ✓ should open the browser
    ✓ should go to install page 
    Step 1 : Choosing language
      ✓ should choose "English" language
      ✓ should click on "Next" button
    Step 2 : Agreeing license agreements
      ✓ should click on "I agree to the above terms and conditions " button
      ✓ should click on "Next" button
    Step 3 : Checking system compatibility
      ✓ should check the test compatibility green box
      ✓ should click on "Next" button
    Step 4 : Inserting the shop information
      ✓ should set the "Shop name" input
      ✓ should set the "Country" input
      ✓ should set the "First name" input
      ✓ should set the "Last name" input
      ✓ should set the "E-mail address" input
      ✓ should set the "Shop password" input
      ✓ should set the "Re-type to confirm" input
      ✓ should click on "Next" button
    Step 5 : Setting the BD configuration
      ✓ should set the "Database server address" input
      ✓ should set the "Database name" input
      ✓ should set the "Database login" input
      ✓ should set the "Database password" input
      ✓ should click on "Test your database connection now!" button
      ✓ should check for the connection and click on "Attempt to create the database automatically" button
      ✓ should check that the Database is created
      ✓ should click on "Next" button
    Step 6 : Checking installation
      ✓ should create file parameter
      ✓ should create database
      ✓ should create default shop
      ✓ should create database table
      ✓ should create shop information
      ✓ should create demonstration data
      ✓ should create install module
      ✓ should create addons modules
      ✓ should create install theme
      ✓ should finish installation
    Step 7 : Checking that installation finished
      ✓ should check that the installation is finished!
    Login to the Front Office
      ✓ should sign in FO

  The shop installation
    Open the browser and download the RC
      ✓ should open the browser
exec error: Error: Command failed:  mv /tmpadmin /tmpadmin-dev
mv: cannot stat '/tmpadmin': No such file or directory

exec error: Error: Command failed:  mv /tmpinstall /tmpinstall-dev
mv: cannot stat '/tmpinstall': No such file or directory

      ✓ should rename folder "admin" to "admin-dev" and "install" to "install-dev"
      ✓ should download the RC (154344ms)
      ✓ should go to the last stable version URL
    Installation of the last stable version of prestashop
      Step 1 : Choosing language
        1) should choose "English" language
        2) should click on "Next" button
      Step 2 : Agreeing license agreements
        3) should click on "I agree to the above terms and conditions " button
        4) should click on "Next" button
      Step 3 : Checking system compatibility
        5) should check the test compatibility green box
        6) should click on "Next" button
      Step 4 : Inserting the shop information
        7) should set the "Shop name" input
        8) should set the "Country" input
        9) should set the "First name" input
        10) should set the "Last name" input
        11) should set the "E-mail address" input
        12) should set the "Shop password" input
        13) should set the "Re-type to confirm" input
        14) should click on "Next" button
      Step 5 : Setting the BD configuration
        15) should set the "Database server address" input

Member

Quetzacoalt91 commented Feb 5, 2018

Just tried the autoupgrade scenario, the installation was actually executed 3 times.

/tmp/node_modules/mocha/bin/mocha /tmp/test/campaigns/high/13_installation --URL=prestashop-web --SELENIUM=selenium-chrome --DIR=/home/seluser/Downloads/ --DB_SERVER=db --DB_USER=root --DB_PASSWD=admin --RCTARGET=/tmp --RCLINK=http://localhost/prestashop_1.7.3.0-RC1.zip
wait-for-it.sh: waiting 200 seconds for prestashop:80
wait-for-it.sh: prestashop:80 is available after 93 seconds


  The shop installation
    Open the browser and connect installation interface
      ✓ should open the browser
      ✓ should go to install page 
    Step 1 : Choosing language
      ✓ should choose "English" language
      ✓ should click on "Next" button
    Step 2 : Agreeing license agreements
      ✓ should click on "I agree to the above terms and conditions " button
      ✓ should click on "Next" button
    Step 3 : Checking system compatibility
      ✓ should check the test compatibility green box
      ✓ should click on "Next" button
    Step 4 : Inserting the shop information
      ✓ should set the "Shop name" input
      ✓ should set the "Country" input
      ✓ should set the "First name" input
      ✓ should set the "Last name" input
      ✓ should set the "E-mail address" input
      ✓ should set the "Shop password" input
      ✓ should set the "Re-type to confirm" input
      ✓ should click on "Next" button
    Step 5 : Setting the BD configuration
      ✓ should set the "Database server address" input
      ✓ should set the "Database name" input
      ✓ should set the "Database login" input
      ✓ should set the "Database password" input
      ✓ should click on "Test your database connection now!" button
      ✓ should check for the connection and click on "Attempt to create the database automatically" button
      ✓ should check that the Database is created
      ✓ should click on "Next" button
    Step 6 : Checking installation
      ✓ should create file parameter
      ✓ should create database
      ✓ should create default shop
      ✓ should create database table
      ✓ should create shop information
      ✓ should create demonstration data
      ✓ should create install module
      ✓ should create addons modules
      ✓ should create install theme
      ✓ should finish installation
    Step 7 : Checking that installation finished
      ✓ should check that the installation is finished!
    Login to the Front Office
      ✓ should sign in FO

  The shop installation
    ✓ should open the browser
    ✓ should go to install page 
    Step 1 : Choosing language
      ✓ should choose "English" language
      ✓ should click on "Next" button
    Step 2 : Agreeing license agreements
      ✓ should click on "I agree to the above terms and conditions " button
      ✓ should click on "Next" button
    Step 3 : Checking system compatibility
      ✓ should check the test compatibility green box
      ✓ should click on "Next" button
    Step 4 : Inserting the shop information
      ✓ should set the "Shop name" input
      ✓ should set the "Country" input
      ✓ should set the "First name" input
      ✓ should set the "Last name" input
      ✓ should set the "E-mail address" input
      ✓ should set the "Shop password" input
      ✓ should set the "Re-type to confirm" input
      ✓ should click on "Next" button
    Step 5 : Setting the BD configuration
      ✓ should set the "Database server address" input
      ✓ should set the "Database name" input
      ✓ should set the "Database login" input
      ✓ should set the "Database password" input
      ✓ should click on "Test your database connection now!" button
      ✓ should check for the connection and click on "Attempt to create the database automatically" button
      ✓ should check that the Database is created
      ✓ should click on "Next" button
    Step 6 : Checking installation
      ✓ should create file parameter
      ✓ should create database
      ✓ should create default shop
      ✓ should create database table
      ✓ should create shop information
      ✓ should create demonstration data
      ✓ should create install module
      ✓ should create addons modules
      ✓ should create install theme
      ✓ should finish installation
    Step 7 : Checking that installation finished
      ✓ should check that the installation is finished!
    Login to the Front Office
      ✓ should sign in FO

  The shop installation
    Open the browser and download the RC
      ✓ should open the browser
exec error: Error: Command failed:  mv /tmpadmin /tmpadmin-dev
mv: cannot stat '/tmpadmin': No such file or directory

exec error: Error: Command failed:  mv /tmpinstall /tmpinstall-dev
mv: cannot stat '/tmpinstall': No such file or directory

      ✓ should rename folder "admin" to "admin-dev" and "install" to "install-dev"
      ✓ should download the RC (154344ms)
      ✓ should go to the last stable version URL
    Installation of the last stable version of prestashop
      Step 1 : Choosing language
        1) should choose "English" language
        2) should click on "Next" button
      Step 2 : Agreeing license agreements
        3) should click on "I agree to the above terms and conditions " button
        4) should click on "Next" button
      Step 3 : Checking system compatibility
        5) should check the test compatibility green box
        6) should click on "Next" button
      Step 4 : Inserting the shop information
        7) should set the "Shop name" input
        8) should set the "Country" input
        9) should set the "First name" input
        10) should set the "Last name" input
        11) should set the "E-mail address" input
        12) should set the "Shop password" input
        13) should set the "Re-type to confirm" input
        14) should click on "Next" button
      Step 5 : Setting the BD configuration
        15) should set the "Database server address" input

@fatmaBouchekoua

This comment has been minimized.

Show comment
Hide comment
@fatmaBouchekoua

fatmaBouchekoua Feb 6, 2018

Contributor

Hello @Quetzacoalt91 ,
If you want to test only the auto upgrade you have to run only the script 3_autoupgrade.js
/tmp/node_modules/mocha/bin/mocha /tmp/test/campaigns/high/13_installation/3_autoupgrade.js --URL=prestashop-web --SELENIUM=selenium-chrome --DIR=/home/seluser/Downloads/ --DB_SERVER=db --DB_USER=root --DB_PASSWD=admin --RCTARGET=/tmp --RCLINK=http://localhost/prestashop_1.7.3.0-RC1.zip

In the 13_installation folder there are 3 installs:

  1. installation with a language equal to the country
  2. installation with a language different to the country
  3. installation of the last stable version from where we are going to upgrade
Contributor

fatmaBouchekoua commented Feb 6, 2018

Hello @Quetzacoalt91 ,
If you want to test only the auto upgrade you have to run only the script 3_autoupgrade.js
/tmp/node_modules/mocha/bin/mocha /tmp/test/campaigns/high/13_installation/3_autoupgrade.js --URL=prestashop-web --SELENIUM=selenium-chrome --DIR=/home/seluser/Downloads/ --DB_SERVER=db --DB_USER=root --DB_PASSWD=admin --RCTARGET=/tmp --RCLINK=http://localhost/prestashop_1.7.3.0-RC1.zip

In the 13_installation folder there are 3 installs:

  1. installation with a language equal to the country
  2. installation with a language different to the country
  3. installation of the last stable version from where we are going to upgrade
@toutantic

This comment has been minimized.

Show comment
Hide comment
@toutantic

toutantic Feb 7, 2018

Member

Running the tests suite should be simple and straightforward.
We need to update the README to look like this

Requirements

  • Install node, java, chrome, mysql...
  • Install Prestashop:
    here is an example of a command line to do it automaticaly:
    php install/index_cli.php --language=fr --country=fr --domain=release.prestoshop.com --db_server=$db_server --db_user=$db_user --db_name=$db_name --db_password=$db_password --firstname=John --lastname=doe --email=demo@prestoshop.com --password=prestodemo --newsletter=0 --db_create=1
  • install dependencies
cd tests/E2E
npm install

Launch tests
npm test

This command should a simple and fast (less than 5min) test suite.
=> developer trying this out should have a good experience

Launch more tests
Give more option to launch more advance tests, like installation and upgrade.

Member

toutantic commented Feb 7, 2018

Running the tests suite should be simple and straightforward.
We need to update the README to look like this

Requirements

  • Install node, java, chrome, mysql...
  • Install Prestashop:
    here is an example of a command line to do it automaticaly:
    php install/index_cli.php --language=fr --country=fr --domain=release.prestoshop.com --db_server=$db_server --db_user=$db_user --db_name=$db_name --db_password=$db_password --firstname=John --lastname=doe --email=demo@prestoshop.com --password=prestodemo --newsletter=0 --db_create=1
  • install dependencies
cd tests/E2E
npm install

Launch tests
npm test

This command should a simple and fast (less than 5min) test suite.
=> developer trying this out should have a good experience

Launch more tests
Give more option to launch more advance tests, like installation and upgrade.

@toutantic

This comment has been minimized.

Show comment
Hide comment
@toutantic

toutantic Feb 7, 2018

Member

Step 2 will be to give the instructions to run it with docker.

The instructions should be only this single line:
docker-compose up

Member

toutantic commented Feb 7, 2018

Step 2 will be to give the instructions to run it with docker.

The instructions should be only this single line:
docker-compose up

@kpodemski

This comment has been minimized.

Show comment
Hide comment
@kpodemski

kpodemski Feb 14, 2018

Contributor

Love it, PR of the year 👍

Contributor

kpodemski commented Feb 14, 2018

Love it, PR of the year 👍

@mbadrani mbadrani closed this Feb 16, 2018

@eternoendless

This comment has been minimized.

Show comment
Hide comment
@eternoendless

eternoendless Feb 16, 2018

Member

Reopened here after rebase: #8768

Member

eternoendless commented Feb 16, 2018

Reopened here after rebase: #8768

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment