Skip to content
This repository has been archived by the owner on Jan 6, 2023. It is now read-only.

Initial commit adding integration tests for the webextension. #1332

Merged
merged 12 commits into from Sep 10, 2018
62 changes: 37 additions & 25 deletions .travis.yml
@@ -1,32 +1,44 @@
language: node_js
addons:
firefox: latest
node_js:
- "8"
before_install:
- export DISPLAY=:99.0
- sh -e /etc/init.d/xvfb start
- sleep 3
- npm i -g npm@6
script:
- npm run test
env:
global:
- FIREFOX_BINARY: firefox
_aliases:
- &test
before_install:
- sh -e /etc/init.d/xvfb start
- sleep 10
- &unit-test
env: DISPLAY=:99.0 FIREFOX_BINARY=firefox
addons:
firefox: latest
<<: *test
- &integration-test
env: DISPLAY=:99.0 MOZ_HEADLESS=1 UI_TEST_LOGGING=info GECKODRIVER_VERSION=0.21.0
addons:
firefox: latest-nightly
before_script:
- wget -O /tmp/geckodriver.tar.gz https://github.com/mozilla/geckodriver/releases/download/v${GECKODRIVER_VERSION}/geckodriver-v${GECKODRIVER_VERSION}-linux64.tar.gz
- mkdir $HOME/geckodriver && tar xvf /tmp/geckodriver.tar.gz -C $HOME/geckodriver
- export PATH=$HOME/geckodriver:$PATH
- firefox --version
- geckodriver --version
- npm run build
<<: *test

matrix:
jobs:
include:
- script: npm run test
env:
global:
- FIREFOX_BINARY: firefox
before_install:
- export DISPLAY=:99.0
- sh -e /etc/init.d/xvfb start
- sleep 3
- script: npm run build
- stage:
node_js: 8
<<: *unit-test
script: npm run test:karma
- stage:
node_js: 8
<<: *integration-test
script: npm run test:ui
- stage:
node_js: 8
script: npm run build
env: BUILD_NOTES
- script: npm run lint
- stage:
node_js: 8
script: npm run lint
env: LINT
before_script: npm run build
before_install:
13 changes: 10 additions & 3 deletions package.json
Expand Up @@ -46,9 +46,11 @@
"uuid": "^3.2.1"
},
"devDependencies": {
"babel-cli": "^6.26.0",
"babel-core": "^6.26.0",
"babel-loader": "^7.1.2",
"babel-preset-env": "^1.6.1",
"babel-polyfill": "^6.26.0",
"babel-preset-env": "^1.7.0",
"babel-preset-react": "^6.24.1",
"babel-preset-stage-2": "^6.24.1",
"chai": "^4.1.0",
Expand All @@ -69,20 +71,23 @@
"karma-firefox-launcher": "1.0.1",
"karma-mocha": "1.3.0",
"karma-mocha-reporter": "2.2.4",
"mocha": "^3.2.0",
"logform": "^1.9.1",
"mocha": "^3.5.3",
"node-sass": "^4.7.2",
"npm-run-all": "^4.0.2",
"pontoon-to-webext": "^1.0.2",
"prettier": "^1.4.4",
"rimraf": "^2.6.1",
"sass-loader": "^6.0.6",
"selenium-webdriver": "^4.0.0-alpha.1",
"sinon": "4.0.1",
"sinon-chrome": "2.2.1",
"stylelint": "^8.4.0",
"stylelint-config-recommended": "^2.0.1",
"web-ext": "^1.9.1",
"webpack": "^3.10.0",
"webpack-sources": "1.0.1"
"webpack-sources": "1.0.1",
"winston": "^3.0.0"
},
"homepage": "https://github.com/mozilla/notes#readme",
"license": "MPL-2.0",
Expand All @@ -106,6 +111,8 @@
"start-deved": "npm run webpack && web-ext run -s build --firefox=firefoxdeveloperedition & npm run webpack:watch",
"test": "npm-run-all test:*",
"test:karma": "webpack --config webpack.test-unit.js && NODE_ENV=test karma start",
"test:ui": "test/integration/setup-webext.sh && npm run ui-tests-build && mocha ./test/dist/integration --retries 1 --no-deprecation --reporter=list --compilers js:babel-core/register --require babel-polyfill",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jrbenny35 I get the following

 Error: The geckodriver executable could not be found on the current PATH. Please download the latest version from https://github.com/mozilla/geckodriver/releases/ and ensure it can be found on your PATH.

Could we get something like https://github.com/vladikoff/node-geckodriver into the process so it "just works" when I run test:ui ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is assuming the user is running on Nightly as it seems that plugin keeps up with the most recent releases without a way to install an older version. I'd rather point the user to the geckodriver install and they can install whichever version they need according to their Firefox version.

We could do a docker image, or something like that maybe.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm , I think geckodriver 0.21.0 is backwards compatible, our other Selenium infrastructure didn't have issues running in Stable or Nightly. Especially last couple of versions, don't think they have too many changes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jrbenny35 any ideas about:

FIREFOX_BINARY=/Applications/FirefoxNightly.app/Contents/MacOS/firefox npm run test:ui

> notes@4.3.0-dev test:ui /Users/vladikoff/mozilla/notes
> test/integration/setup-webext.sh && npm run ui-tests-build && mocha ./test/dist/integration --retries 1 --no-deprecation --reporter=list --compilers js:babel-core/register --require babel-polyfill

Webextension built and moved to root dir.

> notes@4.3.0-dev ui-tests-build /Users/vladikoff/mozilla/notes
> babel ./test/integration --quiet --no-babelrc --presets=env -d ./test/dist/integration


  1) The Firefox Notes web extension "before each" hook for "should have a default note named correctly"

  0 passing (3s)
  1 failing

  1) The Firefox Notes web extension "before each" hook for "should have a default note named correctly":
     WebDriverError: Could not install add-on: /var/folders/cv/w3cp7lkj6bq70z90s51jllsc0000gn/T/addon-ee6daf6a-c8ae-4451-9bdc-2893432b63f5.xpi: ERROR_SIGNEDSTATE_REQUIRED: The addon must be signed and isn't.
      at Object.throwDecodedError (node_modules/selenium-webdriver/lib/error.js:550:15)
      at parseHttpResponse (node_modules/selenium-webdriver/lib/http.js:542:13)
      at Executor.execute (node_modules/selenium-webdriver/lib/http.js:468:26)
      at <anonymous>
      at process._tickCallback (internal/process/next_tick.js:188:7)


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not starting Nightly for me locally

Copy link
Contributor Author

@jrbenny35 jrbenny35 Aug 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm , I think geckodriver 0.21.0 is backwards compatible, our other Selenium infrastructure didn't have issues running in Stable or Nightly. Especially last couple of versions, don't think they have too many changes

They kind of work but if someone is using 57, it wouldn't work.

For that error, what is your selenium version?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I installed the stuff package.json. i.e. "selenium-webdriver": "^4.0.0-alpha.1",

do we need some kind of a selenium launcher or do i need to manually run it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's interesting as I have the preference set to ignore signatures.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look right though: WebDriverError: Could not install add-on: /var/folders/cv/w3cp7lkj6bq70z90s51jllsc0000gn/T/addon-ee6daf6a-c8ae-4451-9bdc-2893432b63f5.xpi, the xpi should be named firefox_notes.xpi.

"ui-tests-build": "babel ./test/integration --quiet --no-babelrc --presets=env -d ./test/dist/integration",
"webpack": "webpack --config webpack.config.js",
"webpack:watch": "webpack --config webpack.config.js --watch"
}
Expand Down
96 changes: 96 additions & 0 deletions test/integration/page_objects/base_page.js
@@ -0,0 +1,96 @@
const {By, until, Key} = require('selenium-webdriver');
const { createLogger, transports } = require('winston');
const { format } = require('logform');
const logger = createLogger({
level: process.env.UI_TEST_LOGGING || 'silent',
format: format.combine(
format.colorize(),
format.align(),
format.printf(info => `${info.level}: ${info.message}`),
),
transports: [
new transports.Console(),
]
});

export default class BasePage {

constructor(driver, timeout, root) {
this.timeout = (timeout) ? timeout : 10000;
this.driver = driver;
this.by = By;
this.key = Key;
this.logger = logger;
this.root = root;
this.until = until;
this.wait = driver.wait;
}

/**
* @function waitForPageToLoad
* @param {string} locator CSS Selector of Element.
* @returns {Object} An object representing the page.
* @throws ElementNotFound
*/
async waitForPageToLoad(locator) {
let element = await this.findElement(locator);
await this.driver.wait(this
.until
.elementIsVisable(element), this.timeout);
return this;
}

/**
* @function findElement
* @param {string} locator CSS Selector of Element.
* @returns {Object} A WebElement object for element matching selector.
* @throws ElementNotFound
*/
async findElement(locator) {
this.logger.info(`Looking for element ${locator}`);
this.waitForElement(locator);
this.logger.info(`Found element ${locator}, returning it.`);
if (this.root) {
let root = this.driver.findElement(this.by.css(this.root));
let element = root.findElement(this.by.css(locator));
return element;
} else {
let element = await this.driver.findElement(this.by.css(locator));
return element;
}
}

/**
* @function findElements
* @param {string} locator CSS Selector of Element.
* @returns {Object} A list of WebElements.
* @throws ElementNotFound
*/
async findElements(locator) {
this.logger.info(`Looking for element ${locator}`);
this.waitForElement(locator);
this.logger.info(`Found element ${locator}, returning it.`);
if (this.root) {
let root = this.driver.findElement(this.by.css(this.root));
let elements = root.findElements(this.by.css(locator));
return elements;
}
return await this.driver.findElements(this.by.css(locator));
}

/**
* @function waitForElement
* @param {string} locator CSS Selector of Element.
* @throws ElementNotFound
*/
async waitForElement(locator) {
this.logger.info(`Waiting for locator ${locator} to appear`);
await this.driver.wait(this
.until
.elementLocated(this
.by
.css(locator)), this.timeout
);
this.logger.info(`Locator ${locator} appeared, returning it.`);
}
}
72 changes: 72 additions & 0 deletions test/integration/page_objects/list_page.js
@@ -0,0 +1,72 @@
import BasePage from './base_page';
import NotePage from './note_page';

export default class ListPage extends BasePage {

constructor(driver) {
super(driver);
this.listLocator = '.listView';
this.newNoteBtnLocator = '.newNoteBtn';
}

async waitForPageToLoad() {
this.logger.info('Waiting for list page load.');
let element = await this.findElement(this.listLocator);
await this.driver.wait(this
.until
.elementIsVisible(element), this.timeout);
this.logger.info('List page loaded.');
return this;
}

/**
* @property notesList
* @returns {Object} A list of notes found on the home page
*/
async notesList(){
let elements = [];

await this.findElement(this.listLocator);
await this.findElement('ul');
for(let item in await this.findElements('li')) {
elements.push(new ListNote(this.driver, item));
this.logger.info('Created ListNote class.');
}
return elements;
}

/**
* @function newNoteButton
* @returns {Object} A Note Page Object
* Clicks the 'new note' button
*/
async newNoteButton() {
let button = await this.findElement(this.newNoteBtnLocator);
await button.click();
return await new NotePage(this.driver).waitForPageToLoad();
}
}

class ListNote extends ListPage {

constructor(driver) {
super(driver);
this.titleLocator = 'div > p';
}

/**
* @property clickBackButton
* @returns {Object} The title of an individual note
*/
async getTitle() {
let element = await this.findElement(this.titleLocator);
return element.getText();
}

async click() {
let note = await this.driver.findElement(this.by.css(this.titleLocator));
await note.click();
return await new NotePage(this.driver).waitForPageToLoad()
}

}
66 changes: 66 additions & 0 deletions test/integration/page_objects/note_page.js
@@ -0,0 +1,66 @@
import BasePage from './base_page';
import ListPage from './list_page';

export default class NotePage extends BasePage {

constructor(driver) {
super(driver);
this.backBtnLocator = '.iconBtn';
this.editorLocator = '.ck-editor__editable';
this.noteTitleLocator = 'div > header > p';
}

async waitForPageToLoad() {
this.logger.info('Waiting for note page load.');
let element = await this.findElement(this.backBtnLocator);
await this.driver.wait(this
.until
.elementIsVisible(element), this.timeout);
this.logger.info('Note page loaded.');
return this;
}

/**
* @property noteTitle
* @returns {Object} The title of the note
*/
get noteTitle() { return (async () =>
{
let title = await this.findElement(this.noteTitleLocator);
return title.getText();
})();
}

/**
* @function clickBackButton
* @returns {Object} A List Page object
* Clicks the back button to return to the list page
*/
async clickBackButton() {
let btn = await this.findElement(this.backBtnLocator);
await btn.click();
return await new ListPage(this.driver).waitForPageToLoad();
}

/**
* @function addNote
* @param {string} title Title of the note you want top add
* @param {string} paragraph Paragraph of the note you want to add
* @returns {Object} An object representing the page.
* @throws ElementNotFound
*/
async addNote(title, paragraph) {
this.logger.info('Adding a new note');
let editor = await this.driver.findElement(this.by.css(this.editorLocator));
await editor.sendKeys(title, this.key.ENTER, paragraph);
// Find note title before we wait for it to change
let thisNoteTitle = await this
.driver
.findElement(this.by.css(this.noteTitleLocator));
await this.driver.wait(
this.until.elementTextIs(thisNoteTitle, paragraph),
this.timeout
);
this.logger.info('Note added');
}
}
3 changes: 3 additions & 0 deletions test/integration/setup-webext.sh
@@ -0,0 +1,3 @@
#!/bin/bash
cp ./web-ext-artifacts/*.zip ./firefox_notes.xpi
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be curious. What if there are multiple ZIP files in the web-ext-artifacts directory? Or is it emptied before every build?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it is only done when built, and the build script seems to nuke that web-ext-artifacts dir on call.

echo "Webextension built and moved to root dir."