Merge pull request #13 from alphagov/selenisation-of-tests

Selenisation of tests

.travis.yml

@@ -3,9 +3,13 @@ language: python
 python:
 - '3.4'
 install:
-- pip install -r requirements_for_test.txt
+- pip install -r requirements.txt
 script:
 - "./scripts/run_tests.sh preview"
+before_script:
+  - "export DISPLAY=:99.0"
+  - "sh -e /etc/init.d/xvfb start"
+  - sleep 5
 notifications:
   slack:
     rooms:

README.md

@@ -6,30 +6,47 @@ Functional tests for Notification applications
 # Running the tests
 ## On a local dev machine
 
+There are two types of test in this repo. Those using selenium webdriver which test user registration, sending of sms notifications and sending of email notifications.
+
+There is an ordering dependency in these tests. The registration test must run before any of the other tests as the user account created is used for all later browser based tests. Each test run will first register a user account using the configured FUNCTIONAL_TEST_EMAIL. The email account will have random characters added so that we do not have uniqueness issues with the email address of registered user.
+
+In addition there are tests of the python client for the notifications api. For the moment, the client tests require an existing user, service and both email and sms templates. The setup of this account is a bit long winded but the client tests have not yet been adapted to do their own setup.
+
+Therefore to run all of the tests in this suite locally you will need to seed a user using the notify. See the config below for FUNCTIONAL_TEST_EMAIL, FUNCTIONAL_TEST_PASSWORD, TEMPLATE_ID, SERVICE_ID.
+
+
 - Create a local environment.sh file in the root directory of the project /notifications-functional-tests/environment.sh
+
 This file is included in the .gitignore to prevent the file from being accidentally committed
 - Make sure all the apps on running locally.
 
 
 Contents of the environment.sh file
 
-```shell    
-    export TWILIO_ACCOUNT_SID=****
-    export TWILIO_AUTH_TOKEN=****
-    export TWILIO_TEST_NUMBER="+447*********"
-    export FUNCTIONAL_TEST_EMAIL=user_name@***.gov.uk
-    export FUNCTIONAL_TEST_PASSWORD=****
-    export NOTIFY_ADMIN_URL=http://localhost:6012
-    export TEMPLATE_ID=***
-    export SERVICE_ID=****
+```shell
+    export ENVIRONMENT=preview
+    export preview_TWILIO_ACCOUNT_SID=****
+    export preview_TWILIO_AUTH_TOKEN=****
+    export preview_TWILIO_TEST_NUMBER="+447*********"
+    export preview_FUNCTIONAL_TEST_EMAIL=[use the notify tests preview email account]
+    export preview_FUNCTIONAL_TEST_PASSWORD=*****
+    export preview_NOTIFY_ADMIN_URL=http://localhost:6012
+    export preview_SMS_TEMPLATE_ID=***
+    export preview_EMAIL_TEMPLATE_ID=***
+    export preview_SERVICE_ID=****
 ```
 
-`TWILIO_ACCOUNT_SID` = account sid for used to read the sms
-`TWILIO_AUTH_TOKEN` = auth token for used to make the api calls to read the sms
-`TWILIO_TEST_NUMBER` = this can be your mobile phone, tests will read the sms from this number and delete the messages when after the test run
-`FUNCTIONAL_TEST_EMAIL` = email to use for the tests, this will need to exist for the sign-in flow to work
-`FUNCTIONAL_TEST_PASSWORD` = password for the functional test user created, this will need to exist for the sign-in flow to work
-`NOTIFY_ADMIN_URL`  = url of the environment 
+`preview_TWILIO_ACCOUNT_SID` = account sid for used to read the sms #
+`preview_TWILIO_AUTH_TOKEN` = auth token for used to make the api calls to read the sms
+`preview_TWILIO_TEST_NUMBER` = for the moment use the preview twilio number - ask for help
+`preview_FUNCTIONAL_TEST_EMAIL` = email to use for the tests, this will need to exist for the sign-in flow to work
+`preview_FUNCTIONAL_TEST_PASSWORD` = password for the functional test user created, this will need to exist for the sign-in flow to work
+`preview_SMS_TEMPLATE_ID` = the id of a test sms template you need to have created for test user and service
+`preview_EMAIL_TEMPLATE_ID` = the id of a test email template you need to have created for test user and service
+`NOTIFY_ADMIN_URL`  = url of the environment
+
+
+- Note: to run locally create a test user account on your local apps using the email and password that matches the values for FUNCTIONAL_TEST_EMAIL and FUNCTIONAL_TEST_PASSWORD in environment.sh
 
 
 Running the tests
@@ -41,30 +58,30 @@ Running the tests
 ## Tests running on Travis
 
 The [notifications-api](https://github.com/alphagov/notifications-api) and [notifications-admin](https://github.com/alphagov/notifications-admin) builds
-will trigger the [notifications-functional-test](https://github.com/alphagov/notifications-functional-tests) build, 
+will trigger the [notifications-functional-test](https://github.com/alphagov/notifications-functional-tests) build,
 as scripted in the [trigger-dependent-build.sh](https://github.com/alphagov/notifications-admin/blob/master/scripts/trigger-dependent-build.sh).
 
 When Travis kicks off the functional tests it will use the encrypted environment variables in the [.travis.yml](https://github.com/alphagov/notifications-functional-tests/blob/master/.travis.yml).
+
+Note on travis environment variables are prefixed 'preview'
+
 To create/update these variable run the following travis commands, replacing the *** with the values.
 ```shell
-    travis encrypt TWILIO_ACCOUNT_SID=*** --add
-    travis encrypt TWILIO_AUTH_TOKEN=*** --add
-    travis encrypt TWILIO_TEST_NUMBER=*** --add
-    travis encrypt FUNCTIONAL_TEST_EMAIL=*** --add
-    travis encrypt FUNCTIONAL_TEST_PASSWORD=*** --add
-    travis encrypt NOTIFY_ADMIN_URL=*** --add
-    travis encrypt TEMPLATE_ID=*** --add
-    travis encrypt SERVICE_ID=*** --add
+    travis encrypt preview_TWILIO_ACCOUNT_SID=*** --add
+    travis encrypt preview_TWILIO_AUTH_TOKEN=*** --add
+    travis encrypt preview_TWILIO_TEST_NUMBER=*** --add
+    travis encrypt preview_FUNCTIONAL_TEST_EMAIL=*** --add
+    travis encrypt preview_FUNCTIONAL_TEST_PASSWORD=*** --add
+    travis encrypt preview_NOTIFY_ADMIN_URL=*** --add
+    travis encrypt preview_SMS_TEMPLATE_ID=*** --add
+    travis encrypt preview_EMAIL_TEMPLATE_ID=*** --add
+    travis encrypt preview_SERVICE_ID=*** --add
 ```
 
-## About CSRF tokens
-Each functional test journey will need to use the Requests.session to retain the notify_admin_session for the requests. 
-The CSRF token is needed for each post request, this is obtained using BeautifulSoup to get the token from the hidden field in the form on the previous page. 
-A convenience method exists to get the token.
-
 ## What we want to test here and what we do not want to test here
-We do not want to test contents of the page, but it is useful to check the page title after a GET request to ensure the flow is correct.
-These test are not intended to be used for load testing, however, some performance tolerances could be added. 
+We do not want to test contents of the page beyond a simple check that would prove we are on the page we expect to be for example check the page title or a heading in the page.
+
+These test are not intended to be used for load testing, however, some performance tolerances could be added.
 Currently the test will fail if the sms is not delivered in a minute, which is too long to wait but it is unclear what a valid wait time should be.
 Testing headers is possible and a good idea to add to these tests.
 

config.py

@@ -8,7 +8,8 @@ class Config(object):
     TWILIO_TEST_NUMBER = os.environ[os.environ['ENVIRONMENT'] + '_TWILIO_TEST_NUMBER']
     TWILIO_ACCOUNT_SID = os.environ[os.environ['ENVIRONMENT'] + '_TWILIO_ACCOUNT_SID']
     TWILIO_AUTH_TOKEN = os.environ[os.environ['ENVIRONMENT'] + '_TWILIO_AUTH_TOKEN']
-    FUNCTIONAL_TEST_EMAIL = os.environ[os.environ['ENVIRONMENT'] + '_FUNCTIONAL_TEST_EMAIL']
+    FUNCTIONAL_TEST_NAME = os.environ['ENVIRONMENT'] + '_Functional Test_'
+    FUNCTIONAL_TEST_EMAIL = os.environ[os.environ['ENVIRONMENT']+'_FUNCTIONAL_TEST_EMAIL']
     FUNCTIONAL_TEST_PASSWORD = os.environ[os.environ['ENVIRONMENT'] + '_FUNCTIONAL_TEST_PASSWORD']
     FUNCTIONAL_EMAIL_TEMPLATE_ID = os.environ[os.environ['ENVIRONMENT'] + '_EMAIL_TEMPLATE_ID']
     FUNCTIONAL_SMS_TEMPLATE_ID = os.environ[os.environ['ENVIRONMENT'] + '_SMS_TEMPLATE_ID']
@@ -18,3 +19,4 @@ class Config(object):
     REGISTRATION_EMAIL_LABEL = 'registration'
     EMAIL_TRIES = 10
     EMAIL_DELAY = 5
+    FUNCTIONAL_TEST_SERVICE_NAME = os.environ['ENVIRONMENT'] + '_Functional Test Service_'

requirements.txt

@@ -1,6 +1,11 @@
-Flask>=0.8
+Flask>=0.10.1
 Flask-Cache==0.13.1
 twilio>=3.3.6
 beautifulsoup4>=4.4.0
+pep8==1.5.7
+pytest==2.8.1
+requests==2.9.1
+retry==0.9.1
+selenium==2.53.1
 
 git+https://github.com/alphagov/notifications-python-client.git@0.3.1#egg=notifications-python-client==0.4.1

scripts/run_tests.sh

@@ -32,16 +32,19 @@ display_result $? 1 "Code style check"
 if [ "$#" -eq 1 ]; then
 	export ENVIRONMENT=$1
 	if [ $1 = "live" ]; then
-		py.test -v tests/test_signin_flow.py tests/test_csv_upload_flow.py tests/test_python_client_flow.py
+		# py.test -v
+    echo 'doing nothing yet'
 	elif [ $1 = "preview" ]; then
-		py.test -v
+		py.test -v -x tests/test_registration_flow.py tests/test_send_sms_from_csv.py tests/test_send_email_from_csv.py tests/test_python_client_flow.py
 	else
 		echo -e "Invalid environment '$1' argument."
 		exit 3
 	fi
 else
-	py.test -v
+    # Note registration *must* run before any other tests as it registers the user for use
+    # in later tests
+	py.test -v -x tests/test_registration_flow.py tests/test_send_sms_from_csv.py tests/test_send_email_from_csv.py tests/test_python_client_flow.py
 fi
-#
+
 display_result $? 3 "Unit tests"
 

tests/conftest.py

@@ -0,0 +1,45 @@
+import uuid
+import pytest
+
+from selenium import webdriver
+
+from config import Config
+
+uuid_for_test_run = str(uuid.uuid1())
+
+
+def _generate_unique_email(email, uuid_):
+    parts = email.split('@')
+    return "{}+{}@{}".format(parts[0], uuid_, parts[1])
+
+functional_test_name = Config.FUNCTIONAL_TEST_NAME + uuid_for_test_run
+functional_test_email = _generate_unique_email(Config.FUNCTIONAL_TEST_EMAIL, uuid_for_test_run)
+functional_test_service_name = Config.FUNCTIONAL_TEST_SERVICE_NAME + uuid_for_test_run
+functional_test_password = Config.FUNCTIONAL_TEST_PASSWORD
+functional_test_mobile = Config.TWILIO_TEST_NUMBER
+
+
+@pytest.fixture(scope="session")
+def test_profile():
+    return {'name': functional_test_name,
+            'email': functional_test_email,
+            'service_name': functional_test_service_name,
+            'password': functional_test_password,
+            'mobile': functional_test_mobile}
+
+
+@pytest.fixture(scope="module")
+def driver(request):
+    driver = webdriver.Firefox()
+
+    def clear_up():
+        driver.delete_all_cookies()
+        driver.close()
+
+    request.addfinalizer(clear_up)
+    return driver
+
+
+@pytest.fixture(scope="session")
+def base_url():
+    return Config.NOTIFY_ADMIN_URL

tests/pages/__init__.py

@@ -0,0 +1 @@
+from .pages import *

tests/pages/element.py

@@ -0,0 +1,62 @@
+from selenium.webdriver.support.ui import WebDriverWait
+
+from tests.pages.locators import (
+    CommonPageLocators,
+    TwoFactorPageLocators,
+    SignUpPageLocators,
+    AddServicePageLocators,
+    EditTemplatePageLocators,
+    UploadCsvLocators
+)
+
+
+class BasePageElement(object):
+
+    def __set__(self, obj, value):
+        driver = obj.driver
+        WebDriverWait(driver, 100).until(
+            lambda driver: driver.find_element_by_name(self.locator))
+        driver.find_element_by_name(self.locator).send_keys(value)
+
+    def __get__(self, obj, owner):
+        driver = obj.driver
+        WebDriverWait(driver, 100).until(
+            lambda driver: driver.find_element_by_name(self.locator))
+        element = driver.find_element_by_name(self.locator)
+        return element.get_attribute("value")
+
+
+class ServiceInputElement(BasePageElement):
+    locator = AddServicePageLocators.SERVICE_INPUT[1]
+
+
+class EmailInputElement(BasePageElement):
+    locator = CommonPageLocators.EMAIL_INPUT[1]
+
+
+class PasswordInputElement(BasePageElement):
+    locator = CommonPageLocators.PASSWORD_INPUT[1]
+
+
+class SmsInputElement(BasePageElement):
+    locator = TwoFactorPageLocators.SMS_INPUT[1]
+
+
+class NameInputElement(BasePageElement):
+    locator = CommonPageLocators.NAME_INPUT[1]
+
+
+class MobileInputElement(BasePageElement):
+    locator = SignUpPageLocators.MOBILE_INPUT[1]
+
+
+class TemplateContentElement(BasePageElement):
+    locator = EditTemplatePageLocators.TEMPLATE_CONTENT_INPUT[1]
+
+
+class FileInputElement(BasePageElement):
+    locator = UploadCsvLocators.FILE_INPUT[1]
+
+
+class SubjectInputElement(BasePageElement):
+    locator = EditTemplatePageLocators.TEMPLATE_SUBJECT_INPUT[1]

tests/pages/locators.py

@@ -0,0 +1,51 @@
+from selenium.webdriver.common.by import By
+
+
+class CommonPageLocators(object):
+    NAME_INPUT = (By.NAME, 'name')
+    EMAIL_INPUT = (By.NAME, 'email_address')
+    PASSWORD_INPUT = (By.NAME, 'password')
+    CONTINUE_BUTTON = (By.CLASS_NAME, 'button')
+
+
+class MainPageLocators(object):
+    SETUP_ACCOUNT_BUTTON = (By.CLASS_NAME, 'button')
+
+
+class SignUpPageLocators(object):
+    MOBILE_INPUT = (By.NAME, 'mobile_number')
+
+
+class TwoFactorPageLocators(object):
+    SMS_INPUT = (By.NAME, 'sms_code')
+
+
+class AddServicePageLocators(object):
+    SERVICE_INPUT = (By.NAME, 'name')
+    ADD_SERVICE_BUTTON = (By.CLASS_NAME, 'button')
+
+
+class DashboardPageLocators(object):
+    H2 = (By.CLASS_NAME, 'navigation-service-name')
+    SMS_TEMPLATES_LINK = (By.LINK_TEXT, 'Text message templates')
+    EMAIL_TEMPLATES_LINK = (By.LINK_TEXT, 'Email templates')
+
+
+class NavigationLocators(object):
+    SIGN_OUT_LINK = (By.LINK_TEXT, 'Sign out')
+
+
+class TemplatePageLocators(object):
+    SEND_FROM_CSV_LINK = (By.LINK_TEXT, 'Send from a CSV file')
+    NEW_TEMPLATE_LINK = (By.LINK_TEXT, 'Add a new template')
+
+
+class EditTemplatePageLocators(object):
+    TEMPLATE_SUBJECT_INPUT = (By.NAME, 'subject')
+    TEMPLATE_CONTENT_INPUT = (By.NAME, 'template_content')
+    SAVE_BUTTON = (By.CLASS_NAME, 'button')
+
+
+class UploadCsvLocators(object):
+    FILE_INPUT = (By.ID, 'file')
+    SEND_BUTTON = (By.CLASS_NAME, 'button')