Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test #3041

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from
Open

test #3041

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/contributing/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,9 @@ project. Please read this section carefully if you are interested in contributin
* Please squash pull requests down to a single commit to simplify review and keep history clean.
* Ensure all new features have unit tests that pass and any bug fixes include tests that
fail without the fix applied
[UI Testing](./ui-testing.md) - How to use UI-Test in sandstone source.
[Screenshot Testing](./screenshot-testing.md) - How to use Screenshot-Test in sandstone source.

* Help keep diffs easy to read by not making unnecessary rearrangements to the source code.
* Make sure not to inadvertently change line ending types from Unix to Windows.
* Ensure inline API documentation exists and is up-to-date (minimum: component summary and descriptions of all
Expand Down
86 changes: 86 additions & 0 deletions docs/contributing/screenshot-testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
title: Screenshot Testing
---

## What is the Screenshot Testing
Screenshot test is visual test that takes a screenshot, then compares it to a reference screenshot file saved. When ui changes, one of the major considerations is whether visual changes exist. Some changes may be easy to check, but some changes are hard to check due to very small change. With Screenshot test, very small changes can easily check due to compare pixel-by-pixel. Additionally, Screenshot testing is the most reliable test for static images.

## Prerequisites
Clone sandstone from GitHub, install dependencies and connect the modules using Lerna:

```shell
# clone the repo!
git clone git@github.com:enactjs/sandstone.git
# move in
cd sandstone
# we're using git flow so develop is our working branch
git checkout develop
# install lerna
npm install
# link dependencies
enact link
```

## Creating tests
Within the UI Library, create an app for testing in `./tests/screenshot/apps` and create a corresponding test in `./tests/screenshot/specs`.

+ src
+ test
+ screenshot
+ apps
+ component
+ testComponent
testComponent.js <-- create app here
+ specs
testComponent-specs.js <-- create spec file here

In screenshot test, create apps to test component. Please refer sample code.
* Button.js

```JS
import Button from '../../../../Button';

const ButtonTests = [
<Button> Hello <Button>
];

export default ButtonTests;
```
And create spec to test additional specification. It is commonly used by all components. Please refer sample code.
* Common-spec.js

```JS
const {Page} = require('@enact/ui-test-utils/utils');
const {runTest} = require('@enact/ui-test-utils/utils');

runTest({
testName: 'Common',
Page: Page,
skin: 'light'
});
```
The generateTestData and Page component from `@enact/ui-test-utils/utils/` generateTestData and Page contain useful methods for screenshot tests.

## Running Tests
For a single-run, execute `npm run test-ss`.

* Filtering Screenshot by Component:

`npm run test-ss -- --component <name-of-component>`

* Re-run tests without building:

`npm run test-ss -- --skip-build`

* Running with filtering by component and without building:

`npm run test-ss -- --component <name-of-component> --skip-build`

## Viewing Test Results
* After a test runs, if new screenshots are generated, a page is created with links to open each of the images. To open this file (on a Mac): `open tests/screenshot/dist/newFiles.html`
* If there are test failures, a failure log is created that contains links to the sets of images. To open this file (on a Mac): `open tests/screenshot/dist/failedTests.html`
Images can be navigated using the keyboard arrow keys. Click on an image to zoom in. Click out of the image to zoom out.
* In the output, the test case button opens the sample app with the parameters that produced the output. This requires that a server be running on port 5000. If you have globally installed the serve command with npm install -g serve you can start the server like this: `serve tests/screenshot/dist`

## Reference and Screen
Screenshot testing is a test that compares and assures two different outputs. For the first test without any criteria, all results are saved in the `tests/screenshot/dist/reference` and they become criteria for later tests. In Second run, all results are saved in the `tests/screenshot/dist/screen` and compares screen with reference.
102 changes: 102 additions & 0 deletions docs/contributing/ui-testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
title: Ui Testing
---
## Prerequisites
Clone sandstone from GitHub, install dependencies and connect the modules using Lerna:

```shell
# clone the repo!
git clone git@github.com:enactjs/sandstone.git
# move in
cd sandstone
# we're using git flow so develop is our working branch
git checkout develop
# install lerna
npm install
# link dependencies
enact link
```

## Creating tests
Within the UI Library, create an app for testing in `./tests/ui/apps` and create a corresponding test in `./tests/ui/specs`.

+ src
+ test
+ ui
+ apps
+testComponent
testComponent-View.js <-- create app here
+ specs
+ testComponent
testComponent-specs.js <-- create spec file here
In UI test, create apps to test view and specs. Please refer sample code.
* Button.js

```JS
import Button from '../../../../Button';
import ThemeDecorator from '../../../../ThemeDecorator';

const app = (props) => <div {...props}>
<div>
<Button
id="button"
>
Default Button
</Button>
</div>
</div>;

export default ThemeDecorator(app);
```

* Button-specs.js

```JS
const Page = require('@enact/ui-test-utils/utils');

describe('Button', function () {

beforeEach(async function () {
await Page.open();
});

describe('5-way', function () {
it('should focus disabled button on 5-way right', async function () {
await Page.spotlightDown();
await Page.spotlightSelect();
});
});
});
```

The Page component from `@enact/ui-test-utils/utils/` Page contains useful methods for loading tests.

## Running Tests
For a single-run, execute `npm run test-ui`.

* Filtering UI by Component:

`npm run test-ui -- --spec <name-of-component>`

* Re-run tests without building:

`npm run test-ui -- --skip-build`

* Running with visible browser:

`npm run test-ui -- --visible`

* Running with filtering by component and without building:

`npm run test-ui -- --spec <name-of-component> --skip-build`

## Viewing Test Results
By default, Test result display on console whether pass or fail.
If fail occurs, fail log display in console and you can see the screenshot about fail in `./test/ui/errorShots/`.
Also, you can see the view you made on chrome. This requires that a server be running on port 5000. If you have globally installed the serve command with npm install -g serve you can start the server like this `serve ./test/ui/dist/`.
To open a specific test app, open the URL path for the test. A specific app display as directory in this page. You can see the app as click the app you wanted.

## Goal of Ui Testing
Manual tests are time consuming and somtimes error occurs. A more efficient approach is to create UI tests so that user tasks are executed in an automated test. With UI test, tests can be executed quickly and reliably in a repeatable way. Simply stated, first priority considering is two things.
* how component handles user actions performed via used input devices.
* whether elements correctly work as intended.