Skip to content

maestro testing doc #995

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 7, 2024
Merged

maestro testing doc #995

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 15 additions & 83 deletions docs/hyperexecute-maestro-testing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ import TabItem from '@theme/TabItem';
This page outlines how to execute your Maestro tests on HyperExecute with [YAML 0.2](/support/docs/hyperexecute-yaml-version0.2/)

## Prerequisites

To run the Tests on HyperExecute from your Local System, you are required:

- Your LambdaTest [Username and Access key](/support/docs/hyperexecute-how-to-get-my-username-and-access-key/)
Expand All @@ -60,23 +59,17 @@ To run the Tests on HyperExecute from your Local System, you are required:
- [HyperExecute YAML](/support/docs/hyperexecute-yaml-version0.2/) file which contains all the necessary instructions.

## Step 1: Setup Your Test Suite

You can use your own project to configure and test it. For demo purposes, we are using the sample repository.

:::tip Sample repo

Download or Clone the code sample for the Maestro framework from the LambdaTest GitHub repository to run the tests on the HyperExecute.

<a href="https://github.com/LambdaTest/maestro-sample-test" className="github__anchor"><img loading="lazy" src={require('../assets/images/icons/github.png').default} alt="Image" className="doc_img"/> View on GitHub</a>

:::

## Step 2: Setup the CLI in your Test Suite

After cloning / downloading the sample repo, you need to setup the CLI and the environment variables.

### Download the HyperExecute CLI

The CLI is used for triggering the tests on HyperExecute. It is recommend to download the CLI binary on the host system and keep it in the root directory of the suite to perform the tests on HyperExecute.

You can download the CLI for your desired platform from the below mentioned links:
Expand All @@ -88,13 +81,11 @@ You can download the CLI for your desired platform from the below mentioned link
| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute |

### Setup Environment Variable

Now, you need to export your environment variables *LT_USERNAME* and *LT_ACCESS_KEY* that are available in the [LambdaTest Profile page](https://accounts.lambdatest.com/detail/profile).

Run the below mentioned commands in your terminal to setup the CLI and the environment variables.

<Tabs className="docs__val">

<TabItem value="bash" label="Linux / MacOS" default>

<div className="lambdatest__codeblock">
Expand All @@ -119,8 +110,7 @@ set LT_ACCESS_KEY="${ YOUR_LAMBDATEST_ACCESS_KEY()}"`}
</Tabs>

## Step 3: Upload your Application

Upload your <b>_android_</b> application (.apk file) to the LambdaTest servers using our <b>REST API</b>. You need to provide your <b>Username</b> and <b>AccessKey</b> in the format `Username:AccessKey` in the <b>cURL</b> command for authentication.
Upload your <b>_android_</b> application (.apk file) or <b>iOS</b> application (.ipa file) to the LambdaTest servers using our <b>REST API</b>. You need to provide your <b>Username</b> and <b>AccessKey</b> in the format `Username:AccessKey` in the <b>cURL</b> command for authentication.

:::info
Enter your local path of the code repository instead of `<YOUR_LOCAL_APP_PATH>` in the below cURL command.
Expand All @@ -133,85 +123,28 @@ Enter your local path of the code repository instead of `<YOUR_LOCAL_APP_PATH>`
</CodeBlock>
</div>


> Response of above cURL will be a **JSON** object containing the `App ID` of the format - `<APP123456789012345678901234567>` and will be used in the next step.

## Step 4: Configure YAML in your Test Suite
Enter your `APP_ID` in the YAML file that you have fetched in the above step.

Enter your `<APP_ID>` in the YAML file (line 52) that you have fetched in the above step.
<Tabs className="docs__val">
<TabItem value="android" label="Android" default>

```yaml
---
# Define the version of the configuration file
version: "0.2"

# Specify the target platform for test execution (Android in this case)
runson: android

# Enable autosplit for test execution
autosplit: true

# Set the concurrency level for test execution (2 devices in parallel)
concurrency: 1

# Only report the status of the test framework
frameworkStatusOnly: true

# Enable dynamic allocation of resources
dynamicAllocation: true

# Pre-install required dependencies using pip
# will need java and maestro inside the container
pre:
- chmod +x ./maestro-test/setup-script.sh

env:
MAESTRO: true

# Test discovery configuration
testDiscovery:
# Command to discover tests from the test.txt file
command: cat ./maestro-test/discover.txt
# Test discovery mode can be static/dynamic
mode: static
# Test type is raw (custom test implementation)
type: raw

# Command to run the tests using the testRunnerCommand
testRunnerCommand: ./maestro-test/runTest.sh $test

# Test framework configuration
framework:
# Name of the test framework (raw in this case)
name: raw
args:
# List of devices to run tests on (two Pixel 5 devices in this case)
devices: ["Galaxy S22 Ultra 5G"]
# Enable or disable video recording support
video: true
# Enable or disable device log support
deviceLog: true
# App ID to be installed (mandatory field, using <app_id>)
# highlight-next-line
appId: lt://<app-id>
# Build name for identification on the automation dashboard
buildName: maestro-t1
# All devices are in a private cloud
privateCloud: true
# Timeout for device queue
queueTimeout: 600
# Configuration fields specific to running raw tests
region: ap
disableReleaseDevice: true
isRealMobile: true
reservation: false
platformName: android

jobLabel: [maestro-testing, android, autosplit]
```yaml reference title="hyperexecute.yaml"
https://github.com/LambdaTest/maestro-sample-test/blob/main/yaml/maestro.yaml
```
</TabItem>

## Step 5: Execute your Test Suite
<TabItem value="ios" label="iOS-Simulator" default>

```yaml reference title="hyperexecute.yaml"
https://github.com/LambdaTest/maestro-sample-test/blob/ios-simulator/maestro.yaml
```
</TabItem>
</Tabs>

## Step 5: Execute your Test Suite
> **NOTE :** In case of MacOS, if you get a permission denied warning while executing CLI, simply run **`chmod u+x ./hyperexecute`** to allow permission. In case you get a security popup, allow it from your **System Preferences** → **Security & Privacy** → **General tab**.

Run the below command in your terminal at the root folder of the project:
Expand All @@ -231,7 +164,6 @@ OR use this command if you have not exported your username and access key in the
<img loading="lazy" src={require('../assets/images/hyperexecute/frameworks/maestro/1.png').default} alt="JUnit HyperExecute Terminal Logs" width="1920" height="868" className="doc_img"/>

## Step 6: Monitor the Test Execution

Visit the [HyperExecute Dashboard](https://hyperexecute.lambdatest.com/hyperexecute) and check your Job status.

<img loading="lazy" src={require('../assets/images/hyperexecute/frameworks/maestro/2.png').default} alt="automation-dashboard" width="1920" height="868" className="doc_img"/>
Loading