Skip to content

Commit

Permalink
Guidelines for running automation test
Browse files Browse the repository at this point in the history
  • Loading branch information
baotn166 authored and lilyhdo committed Jun 1, 2018
1 parent 4707f5b commit 91fd25f
Show file tree
Hide file tree
Showing 28 changed files with 316 additions and 0 deletions.
Binary file added .DS_Store
Binary file not shown.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.DS_Store
Binary file added content/.DS_Store
Binary file not shown.
114 changes: 114 additions & 0 deletions content/1-triggering-jenkins/guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# 1. Triggering a Jenkins build from a push to GitHub

Guidance on integrating Kobiton service into the build pipeline of mobile app development: Jenkins, HockeyApp, Appium and ReactNative

**Triggering a Jenkins build from a push to GitHub**

Upload the app you want to test to a GitHub repo.

Go to Jenkins and create a new freestyle project.

<!-- Refer to the this tutorial https://medium.com/@marc_best/trigger-a-jenkins-build-from-a-github-push-b922468ef1ae to :
- install the GitHub integration Plugin
- prepare the Github repository
- add Jenkins Github plugin service
- give the Jenkins user access to the Gihub repository
- deploy SSH key
- update Jenkins job with GitHub configuration -->

1. Install the GitHub integration plugin

On Jenkins, go to 'Manage Jenkins' and then click on 'Manage Plugins'.

![manage-jenkins](../../images/2-manage-jenkins.png)

Click on the tab 'Available', search for the GitHub Integration Plugin, and install it.

2. Prepare GitHub repository

Go to your Github, click on Settings, and then go to 'Integration & services'. Add the Jenkins Github plugin.

![jenkins-git](../../images/2-jenkins-git.png)

Once you've added the plugin, click on it. For 'Jenkins hook url', enter in the URL of your Jenkins instance, followed by /github-webhook/

![jenkins-hook](../../images/2-jenkins-hook.png)

3. Add SSH key to GitHub

If you do not already have an SSH key, you can generate one with the following command in your terminal.

```
ssh-keygen
```
Copy the public key. This may vary depending on where the key was generated.
```
cat /var/lib/jenkins/.ssh/id_rsa.pub
```
In GitHub, go to 'Settings' and 'Deploy keys'. Click on 'Add deploy key' and paste the SSH key you just created.

![deploy-key](../../images/2-deploy-key.png)

To check everything is working, enter the following command as the Jenkins user in your console.
```
ssh git@github.com
```
If everything is correct, you should see:
```
PTY allocation request failed on channel 0
Hi lilyhdo! You've successfully authenticated, but GitHub does not provide shell access.
Connection to github.com closed.
```

4. Add GitHub project to Jenkins configuration

In Jenkins, go to the configuration of your project. Under the tab 'General', check the box that says 'GitHub project' and enter in your project url.

![githubproject](../../images/2-githubproject.png)

Under the Source Code Management, add the repository URL with 'git@github.com:' before your repo.

![source-code-management](../../images/2-source-code-management.png)

> Troubleshooting
You may see an error from adding your repository URL if your "Credentials" are set to "None".

![error](../../images/2-error.png)

To fix this, you must set up the credentials under "Source Code Management" under "Repositories" in the configuration.

![credentials](../../images/2-credentials.png)

To add a Credential, click on the 'Add' button next to 'Credentials'.

There are several kinds of credentials you may choose.

- Username with password

Enter in the username that will be used to connect to the git repository. You may leave the ID and Description field empty.

![user-with-pass](../../images/2-user-with-pass.png)

- SSH Username with private key

Enter in the username that will be used to connect to the git repository.

There are three options for the Private Key. If you choose to Enter directly, you may have to retrieve your RSA private key.

Enter in The passphrase for your key.

You may leave the ID and Description empty.

Select the Credential you just created in the Credentials dropdown.

For more information on setting up the credentials, refer to https://www.thegeekstuff.com/2016/10/jenkins-git-setup/

Under the 'Build Triggers' section, check the box with 'GitHub hook trigger for GITScm polling'.

![trigger](../../images/2-trigger.png)

**Jenkins (GitHub plugin)**

Make a push to your GitHub repository and then check Jenkins to see if a build was done successfully.

Binary file not shown.
200 changes: 200 additions & 0 deletions content/2-running-automation-testing/guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
# 2. Running automation test with Kobiton

## 2.1 Prerequisites

* Username
1. Go to portal.kobiton.com
2. In the upper right hand corner, click on your name and in the drop down menu, click on Profile.

![profile](../../images/2-profile.png)

3. You should see the username.

![username](../../images/2-username.png)

* API key
1. Click on your name in the upper righthand corner again and select settings.
2. You should be able to find your API key under 'API Keys'.

![api-key](../../images/2-apikey.png)

* Desired cap
1. In the navigation bar at the top of the Kobiton website, select Devices.
2. Hover over any device and click on the Automation settings button (the gear symbol).

![automation](../../images/2-automation.png)

3. On the right hand side, you can select your preferred language, and in the left side of the window you will see the desiredCaps.

![automation-settings](../../images/2-automation-settings.png)

## 2.2 Setup Jenkins
#### How to setup Jenkins to run automation testing

Jenkins is an open source automation server that provides hundreds of plugins to support building, deploying, and automating any project.

Docker is a desktop app for building, debugging, and testing Dockerized apps on a Mac. We will be running Jenkins through a Docker container.

1. Download Jenkins from https://jenkins.io/
2. Download Docker from https://www.docker.com/

After finishing the installations, run Docker and then go to the terminal and run the following commands.

```
docker run -d -p 8080:8080 jenkins
```

```
docker ps
```
Copy the status and enter it in the next command.

![status](../../images/2-status.png)

```
docker exec -it 0b541bce2145 /bin/bash
```
Copy the admin password that shows up, and go to http://localhost:8080

Enter in your Kobiton username and password. Enter the admin password you got from the terminal when Jenkins prompts you to do so.

### 2.2.1 For Jenkins Freestyle
On the left side on the page, click on New Item to start a new Jenkins project.

![new-item](../../images/2-new-item.png)

Enter in a name for the project and select 'Freestyle project'. Hit 'OK'.

![project](../../images/2-project.png)

### 2.2.2 For Jenkins Pipeline
On the left side on the page, click on New Item to start a new Jenkins project.

Enter in a name for the project and select 'Pipeline'. Hit 'OK'.

## 2.3 Running automation testing

### 2.3.1 Test scripts

* How to write a simple test

For examples of automation tests, go to https://github.com/kobiton/samples . Choose a language for your test script, and decide whether you want to test on Android or iOS, and either do a web test or an app test. Make sure in the code you specify your Kobiton username, API key, and information under desiredCaps.

* How to run that test script

Follow our guide on how to trigger a Jenkins build from a Git push and set up the configurations stated.

Add NodeJS Plugin on Jenkins. To do this, go to 'Manage Jenkins' -> 'Manage Plugins' -> Available. Search for NodeJS and install it.

![manage-plugins](../../images/2-manage-plugins.png)

Go to Configure in your Jenkins project. Add a build step, execute shell.

![build-execute-shell](../../images/2-build-executeshell.png)

Enter this in the command. Enter your own Kobiton username and API key as well as the desired device and platform you wish to test with. The below example will execute the android-web-test script sample from Kobiton.
```
#!/bin/bash
[ ! -d "samples" ] && git clone https://github.com/kobiton/samples.git || echo "Samples already exist"
cd samples/javascript
npm install
npm run android-web-test
```

Under build environment, check the box that says 'Provide Node & npm bin/ folder to PATH'.

![build-environment](../../images/2-build-environment.png)

Try Build now and check Kobiton cloud devices to see if a test session was created.

### 2.3.2 Output

#### Success case

Get all information we can get through public API

**1. Session information**

With this line of code, you can print the session information to the console.
```javascript
console.log(sessionCapabilities)
```
<!-- to access more specific information within the sessionCapabilities -->

How to get this information:

* Session name
- sessionCapabilities.sessionName
* Session description
- sessionCapabilities.sessionDescription
* Device orientation
- sessionCapabilities.deviceOrientation
* Device name
- sessionCapabilities.deviceName
* Platform name
- sessionCapabilities.platformName
* App
- sessionCapabilities.app
* Kobiton session id
- sessionCapabilities.kobitonSessionId

**2. Test logs**

* Prepare basic authentication

One way is to insert the following in your test script.

```javascript
var basicAuth = "Basic " + new Buffer(username + ":" + apiKey).toString("base64");
var response = await fetch(`https://api-test.kobiton.com/v1/sessions/${sessionId}`, {
headers: { 'Authorization': basicAuth }
})
const body = await response.json()
```

How to get the following information:

* App version
```javascript
var appVersionId = body.executionData.desired.appVersionId
var appVersionResponse = await fetch(`https://api-test.kobiton.com/v1/app/versions/${appVersionId}`, {
headers: { 'Authorization': basicAuth }
})
var appVersionBody = await appVersionResponse.json()
console.log(appVersionBody)
```
* Logs url
```javascript
body.log
```
* Video url
```javascript
body.video
```
> Note: Log and video URL may not be available immediately
* Commands
```javascript
var commandsResponse = await fetch(`https://api-test.kobiton.com/v1/sessions/${sessionId}/commands`, {
headers: { 'Authorization': basicAuth }
})
console.log(commandsResponse)
const commandsBody = await commandsResponse.json()
console.log(commandsBody)
```
To get to a certain page in your commands, add the page number to the commands URL. For example:
```javascript
`https://api-test.kobiton.com/v1/sessions/${sessionId}/commands?page=2`
```
> For more details on how to retrieve information about your session, go to https://api.kobiton.com/docs/
**3. Final result**

The test is a either a success or failure.

#### Failure case

* Error: "The environment you requested was unavailable."
- This means that the device you selected is already booked. Either select a different device or wait a few moments until your device becomes available
* Other
- Contact Kobiton for support
1 change: 1 addition & 0 deletions content/3-upload-to-HockeyApp/guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# 3. Using HockeyApp to manage apps
Binary file added images/2-apikey.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-automation-settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-automation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-build-environment.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-build-executeshell.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-credentials.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-deploy-key.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-error.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-githubproject.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-jenkins-git.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-jenkins-hook.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-manage-jenkins.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-manage-plugins.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-new-item.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-profile.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-project.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-source-code-management.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-status.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-trigger.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-user-with-pass.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/2-username.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 91fd25f

Please sign in to comment.