diff --git a/assets/images/smart-visual-testing/ci-cd-integration/gitlab/Gitlab_pr_check_dashboard.png b/assets/images/smart-visual-testing/ci-cd-integration/gitlab/Gitlab_pr_check_dashboard.png new file mode 100644 index 000000000..13420fde1 Binary files /dev/null and b/assets/images/smart-visual-testing/ci-cd-integration/gitlab/Gitlab_pr_check_dashboard.png differ diff --git a/assets/images/smart-visual-testing/ci-cd-integration/gitlab/pipeline_gitlabtab.png b/assets/images/smart-visual-testing/ci-cd-integration/gitlab/pipeline_gitlabtab.png new file mode 100644 index 000000000..3471863e0 Binary files /dev/null and b/assets/images/smart-visual-testing/ci-cd-integration/gitlab/pipeline_gitlabtab.png differ diff --git a/at_mention.png b/at_mention.png new file mode 100644 index 000000000..3595d30bd Binary files /dev/null and b/at_mention.png differ diff --git a/autoasveddraft_reject.png b/autoasveddraft_reject.png new file mode 100644 index 000000000..704c2e773 Binary files /dev/null and b/autoasveddraft_reject.png differ diff --git a/autosaveddraft_multiselect.png b/autosaveddraft_multiselect.png new file mode 100644 index 000000000..4a90c192f Binary files /dev/null and b/autosaveddraft_multiselect.png differ diff --git a/autosaveddraft_textbox.png b/autosaveddraft_textbox.png new file mode 100644 index 000000000..e71d8c336 Binary files /dev/null and b/autosaveddraft_textbox.png differ diff --git a/comment_drawer.png b/comment_drawer.png new file mode 100644 index 000000000..ab25c9a19 Binary files /dev/null and b/comment_drawer.png differ diff --git a/comment_icon.png b/comment_icon.png new file mode 100644 index 000000000..b0a60531f Binary files /dev/null and b/comment_icon.png differ diff --git a/commenttag_build.png b/commenttag_build.png new file mode 100644 index 000000000..dc92b0f2d Binary files /dev/null and b/commenttag_build.png differ diff --git a/commenttag_screenshotdrawer.png b/commenttag_screenshotdrawer.png new file mode 100644 index 000000000..98e690d1a Binary files /dev/null and b/commenttag_screenshotdrawer.png differ diff --git a/commenttag_screenshotheader.png b/commenttag_screenshotheader.png new file mode 100644 index 000000000..7226fb888 Binary files /dev/null and b/commenttag_screenshotheader.png differ diff --git a/docs/lambdatest-mcp-server.md b/docs/lambdatest-mcp-server.md index 53c0e5681..fbd594a15 100644 --- a/docs/lambdatest-mcp-server.md +++ b/docs/lambdatest-mcp-server.md @@ -89,7 +89,7 @@ Cline (The Collaborative AI Coder) is an advanced AI-powered coding assistant th automation-dashboard **Step 3:** Add Server Name as **MCP-LambdaTest**.
-**Step 4:** Enter the following server configiration and save it: `http://mcp.lambdatest.com/mcp`
+**Step 4:** Enter the following server configuration and save it: `http://mcp.lambdatest.com/mcp`
**Step 5:** Click on Authenticate and you will be redirected to lambdatest.com for authentication. automation-dashboard **Step 6:** Provide valid credentials and authorise client to use LambdaTest MCP server. diff --git a/docs/smartui-gitlab-pr-checks-exec.md b/docs/smartui-gitlab-pr-checks-exec.md new file mode 100644 index 000000000..6c08b400a --- /dev/null +++ b/docs/smartui-gitlab-pr-checks-exec.md @@ -0,0 +1,687 @@ +--- +id: smartui-gitlab-pr-checks-exec +title: GitLab PR Checks with SmartUI Exec (SDK) +sidebar_label: GitLab PR Checks (Exec) +description: Integrate SmartUI visual regression testing with GitLab merge requests using SmartUI SDK/Exec method for web and mobile testing with Selenium, Playwright, Appium, and more. +slug: smartui-gitlab-pr-checks-exec/ +keywords: + - GitLab PR checks + - SmartUI SDK + - SmartUI Exec + - GitLab merge request integration + - Visual regression testing + - Selenium Playwright Appium + - npx smartui exec +url: https://www.lambdatest.com/support/docs/smartui-gitlab-pr-checks-exec/ +site_name: LambdaTest +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; +import NewTag from '../src/component/newTag'; +import CodeBlock from '@theme/CodeBlock'; +import {YOUR_LAMBDATEST_USERNAME, YOUR_LAMBDATEST_ACCESS_KEY} from "@site/src/component/keys"; + +This guide shows you how to integrate SmartUI visual regression testing with GitLab merge requests using the **SmartUI SDK/Exec method**. This works for both **web testing** (Selenium, Playwright, Cypress, Puppeteer) and **mobile app testing** (Appium, WebdriverIO) across all supported languages. + +:::info SmartUI SDK/Exec vs Hooks + +This guide covers the **SmartUI SDK/Exec approach**, where you use `npx smartui exec -- ` to run your tests. This is different from the Hooks approach: + +**SmartUI SDK/Exec (This Guide):** +- ✅ Uses `npx smartui exec -- ` +- ✅ Works with Java SDK and CLI-based projects +- ✅ Requires SmartUI CLI installation +- ✅ Suitable for Selenium/Playwright/Cypress/Puppeteer/Appium +- ✅ Works with **web testing** (Selenium, Playwright, Cypress, Puppeteer) +- ✅ Works with **mobile app testing** (Appium, iOS/Android) + +**SmartUI Hooks:** +- No `npx smartui exec` command needed +- Tests run normally (e.g., `npm test`, `mvn test`, `pytest`) +- SmartUI integration happens automatically through capabilities +- See [GitLab PR Checks with SmartUI Hooks](/support/docs/smartui-gitlab-pr-checks-hooks) for Hooks approach + +::: + +--- + +## Prerequisites + +Before you begin, ensure you have: + +- LambdaTest account with active subscription +- GitLab repository with CI/CD enabled +- SmartUI project created in [LambdaTest SmartUI Dashboard](https://smartui.lambdatest.com/) +- Test suite configured (Selenium/Playwright/Cypress/Puppeteer/Appium/WebdriverIO) +- Test framework configured in your preferred language (TypeScript/JavaScript/Java/Python/Ruby/C#) +- LambdaTest credentials (`LT_USERNAME` and `LT_ACCESS_KEY`) +- SmartUI CLI installed (or use `npx @lambdatest/smartui-cli`) + +--- + +## Step 1: Integrate GitLab with LambdaTest + +1. Go to [LambdaTest Integrations page](https://integrations.lambdatest.com/) +2. Search for **GitLab** and select the integration +3. Click on **OAuth** as your preferred authentication method +4. Click **Install** and authorize the integration +5. After successful authentication, refresh the Integrations page to verify GitLab is installed + +GitLab integration setup in LambdaTest + +:::tip Integration Status + +You can verify your GitLab integration is active by checking the [Integrations page](https://integrations.lambdatest.com/). The GitLab integration should show as "Installed" or "Active". + +::: + +--- + +## Step 2: Configure SmartUI Project + +Create or update your SmartUI configuration file (`.smartui.json`): + +```json title=".smartui.json" +{ + "projectName": "your-smartui-project-name", + "buildName": "smartui-build-${CI_PIPELINE_ID}", + "baseline": false +} +``` + +:::info Configuration File + +The SmartUI configuration file is used to specify project settings. You can also use environment variables or command-line options to override these settings. + +::: + +--- + +## Step 3: Configure GitLab CI/CD Pipeline + +Create or update your `.gitlab-ci.yml` file. The key difference with the Exec method is that you use `npx smartui exec --gitURL` to run your tests. + +### Complete GitLab CI/CD Configuration + + + + +```yaml title=".gitlab-ci.yml - TypeScript/JavaScript Example" +stages: + - test + +variables: + NODE_VERSION: "18" + LT_USERNAME: $LT_USERNAME + LT_ACCESS_KEY: $LT_ACCESS_KEY + PROJECT_TOKEN: $PROJECT_TOKEN + +visual_regression_tests: + stage: test + image: node:${NODE_VERSION} + + before_script: + - npm ci + - npm install -g @lambdatest/smartui-cli + + script: + # Get GitLab project ID and commit SHA + - | + PROJECT_ID=${CI_PROJECT_ID} + COMMIT_SHA=${CI_COMMIT_SHA} + + # For merge requests, use the merge request commit SHA + if [ -n "$CI_MERGE_REQUEST_IID" ]; then + COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} + fi + + # Construct GitLab API URL for status updates + GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" + + echo "GitLab Project ID: ${PROJECT_ID}" + echo "Commit SHA: ${COMMIT_SHA}" + echo "GitLab Status URL: ${GIT_URL}" + + # Run tests with SmartUI Exec and GitLab integration + npx smartui exec --gitURL "${GIT_URL}" -- npm test + # Or: npx smartui exec --gitURL "${GIT_URL}" -- npx wdio run wdio.conf.ts + # Or: npx smartui exec --gitURL "${GIT_URL}" -- npm run test:mobile + + only: + - merge_requests + - main + - develop + + environment: + name: visual-regression/$CI_COMMIT_REF_NAME +``` + + + + +```yaml title=".gitlab-ci.yml - Java Example" +stages: + - test + +variables: + MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" + LT_USERNAME: $LT_USERNAME + LT_ACCESS_KEY: $LT_ACCESS_KEY + PROJECT_TOKEN: $PROJECT_TOKEN + +visual_regression_tests: + stage: test + image: maven:3.8-openjdk-11 + + cache: + paths: + - .m2/repository/ + + before_script: + - mvn clean install -DskipTests + - npm install -g @lambdatest/smartui-cli + + script: + # Get GitLab project ID and commit SHA + - | + PROJECT_ID=${CI_PROJECT_ID} + COMMIT_SHA=${CI_COMMIT_SHA} + + # For merge requests, use the merge request commit SHA + if [ -n "$CI_MERGE_REQUEST_IID" ]; then + COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} + fi + + # Construct GitLab API URL for status updates + GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" + + echo "GitLab Project ID: ${PROJECT_ID}" + echo "Commit SHA: ${COMMIT_SHA}" + echo "GitLab Status URL: ${GIT_URL}" + + # Run tests with SmartUI Exec and GitLab integration + npx smartui exec --gitURL "${GIT_URL}" -- mvn test + # Or: npx smartui exec --gitURL "${GIT_URL}" -- ./gradlew test (for Gradle) + + only: + - merge_requests + - main + - develop + + environment: + name: visual-regression/$CI_COMMIT_REF_NAME +``` + + + + +```yaml title=".gitlab-ci.yml - Python Example" +stages: + - test + +variables: + PYTHON_VERSION: "3.9" + LT_USERNAME: $LT_USERNAME + LT_ACCESS_KEY: $LT_ACCESS_KEY + PROJECT_TOKEN: $PROJECT_TOKEN + +visual_regression_tests: + stage: test + image: python:${PYTHON_VERSION} + + before_script: + - pip install -r requirements.txt + - npm install -g @lambdatest/smartui-cli + + script: + # Get GitLab project ID and commit SHA + - | + PROJECT_ID=${CI_PROJECT_ID} + COMMIT_SHA=${CI_COMMIT_SHA} + + # For merge requests, use the merge request commit SHA + if [ -n "$CI_MERGE_REQUEST_IID" ]; then + COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} + fi + + # Construct GitLab API URL for status updates + GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" + + echo "GitLab Project ID: ${PROJECT_ID}" + echo "Commit SHA: ${COMMIT_SHA}" + echo "GitLab Status URL: ${GIT_URL}" + + # Run tests with SmartUI Exec and GitLab integration + npx smartui exec --gitURL "${GIT_URL}" -- pytest + # Or: npx smartui exec --gitURL "${GIT_URL}" -- python -m unittest discover + # Or: npx smartui exec --gitURL "${GIT_URL}" -- behave + + only: + - merge_requests + - main + - develop + + environment: + name: visual-regression/$CI_COMMIT_REF_NAME +``` + + + + +```yaml title=".gitlab-ci.yml - Ruby Example" +stages: + - test + +variables: + RUBY_VERSION: "3.1" + LT_USERNAME: $LT_USERNAME + LT_ACCESS_KEY: $LT_ACCESS_KEY + PROJECT_TOKEN: $PROJECT_TOKEN + +visual_regression_tests: + stage: test + image: ruby:${RUBY_VERSION} + + before_script: + - bundle install + - npm install -g @lambdatest/smartui-cli + + script: + # Get GitLab project ID and commit SHA + - | + PROJECT_ID=${CI_PROJECT_ID} + COMMIT_SHA=${CI_COMMIT_SHA} + + # For merge requests, use the merge request commit SHA + if [ -n "$CI_MERGE_REQUEST_IID" ]; then + COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} + fi + + # Construct GitLab API URL for status updates + GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" + + echo "GitLab Project ID: ${PROJECT_ID}" + echo "Commit SHA: ${COMMIT_SHA}" + echo "GitLab Status URL: ${GIT_URL}" + + # Run tests with SmartUI Exec and GitLab integration + npx smartui exec --gitURL "${GIT_URL}" -- bundle exec rspec + # Or: npx smartui exec --gitURL "${GIT_URL}" -- bundle exec cucumber + + only: + - merge_requests + - main + - develop + + environment: + name: visual-regression/$CI_COMMIT_REF_NAME +``` + + + + +### Key Configuration Points + +1. **SmartUI CLI Installation**: Install SmartUI CLI globally or use `npx @lambdatest/smartui-cli` +2. **GitLab Project ID**: Automatically available as `CI_PROJECT_ID` in GitLab CI/CD +3. **Commit SHA**: Use `CI_COMMIT_SHA` for regular commits, or `CI_MERGE_REQUEST_SHA` for merge requests +4. **GitLab API URL**: Construct as `https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId}` +5. **Exec Command**: Use `npx smartui exec --gitURL "${GIT_URL}" -- ` + +:::info Understanding the SmartUI Exec Command + +The `npx smartui exec` command wraps your test execution and provides SmartUI integration: + +```bash +npx smartui exec --gitURL "" -- +``` + +- `--gitURL`: GitLab API URL for status updates (legacy name, works with GitLab) +- `--`: Separator before your test command +- ``: Your normal test command (e.g., `npm test`, `mvn test`, `pytest`) + +::: + +--- + +## Step 4: Set Up GitLab CI/CD Variables + +Configure the following variables in your GitLab project: + +1. Go to your GitLab project → **Settings** → **CI/CD** → **Variables** +2. Add the following variables: + +GitLab CI/CD Variables configuration + +| Variable Name | Description | +|--------------|-------------| +| `LT_USERNAME` | Your LambdaTest username | +| `LT_ACCESS_KEY` | Your LambdaTest access key | +| `PROJECT_TOKEN` | Your SmartUI project token (found in SmartUI project settings) | + +:::info Project Token + +The `PROJECT_TOKEN` is different from `LT_USERNAME` and `LT_ACCESS_KEY`. You can find it in your SmartUI project settings in the [SmartUI Dashboard](https://smartui.lambdatest.com/). + +::: + +--- + +## Step 5: View Pipeline Results in GitLab + +After your pipeline runs, you can view the results in the GitLab Pipelines page: + +GitLab Pipelines page showing SmartUI test results + +The pipeline will show: +- **Pipeline status** (Success/Failed) +- **Job status** for SmartUI tests +- **Screenshot statistics** (Total, Approved, Changes Found) in the job tooltip + +--- + +## Step 6: View PR Check Results in GitLab Merge Request + +After your pipeline runs, you'll see SmartUI status checks in your GitLab merge request: + +GitLab merge request showing SmartUI PR check status + +### Successful Status + +When all visual tests pass: +- ✅ **Status**: Success +- **Details**: Click "Details" to view the SmartUI build in the dashboard +- **Screenshot**: All screenshots match baseline or are approved + +### Failed Status + +When visual differences are detected: +- ❌ **Status**: Failed +- **Details**: Click "Details" to review differences in SmartUI dashboard +- **Action Required**: Review and approve/reject changes in SmartUI dashboard + +--- + +## Complete Working Examples + + + + + + + +```yaml title=".gitlab-ci.yml - Complete Web Testing Example" +stages: + - test + +variables: + NODE_VERSION: "18" + LT_USERNAME: $LT_USERNAME + LT_ACCESS_KEY: $LT_ACCESS_KEY + PROJECT_TOKEN: $PROJECT_TOKEN + +visual_regression_tests: + stage: test + image: node:${NODE_VERSION} + + before_script: + - npm ci + - npm install -g @lambdatest/smartui-cli + + script: + - | + PROJECT_ID=${CI_PROJECT_ID} + COMMIT_SHA=${CI_COMMIT_SHA} + + if [ -n "$CI_MERGE_REQUEST_IID" ]; then + COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} + fi + + GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" + + echo "GitLab Status URL: ${GIT_URL}" + + # Run web tests with SmartUI Exec + npx smartui exec --gitURL "${GIT_URL}" -- npm test + + only: + - merge_requests + - main +``` + + + + +```yaml title=".gitlab-ci.yml - Complete Java Web Testing Example" +stages: + - test + +variables: + MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" + LT_USERNAME: $LT_USERNAME + LT_ACCESS_KEY: $LT_ACCESS_KEY + PROJECT_TOKEN: $PROJECT_TOKEN + +visual_regression_tests: + stage: test + image: maven:3.8-openjdk-11 + + cache: + paths: + - .m2/repository/ + + before_script: + - mvn clean install -DskipTests + - npm install -g @lambdatest/smartui-cli + + script: + - | + PROJECT_ID=${CI_PROJECT_ID} + COMMIT_SHA=${CI_COMMIT_SHA} + + if [ -n "$CI_MERGE_REQUEST_IID" ]; then + COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} + fi + + GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" + + echo "GitLab Status URL: ${GIT_URL}" + + # Run Java tests with SmartUI Exec + npx smartui exec --gitURL "${GIT_URL}" -- mvn test + + only: + - merge_requests + - main +``` + + + + + + + + + + +```yaml title=".gitlab-ci.yml - Complete Mobile Testing Example" +stages: + - test + +variables: + NODE_VERSION: "18" + LT_USERNAME: $LT_USERNAME + LT_ACCESS_KEY: $LT_ACCESS_KEY + PROJECT_TOKEN: $PROJECT_TOKEN + +visual_regression_tests: + stage: test + image: node:${NODE_VERSION} + + before_script: + - npm ci + - npm install -g @lambdatest/smartui-cli + + script: + - | + PROJECT_ID=${CI_PROJECT_ID} + COMMIT_SHA=${CI_COMMIT_SHA} + + if [ -n "$CI_MERGE_REQUEST_IID" ]; then + COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} + fi + + GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" + + echo "GitLab Status URL: ${GIT_URL}" + + # Run mobile tests with SmartUI Exec + npx smartui exec --gitURL "${GIT_URL}" -- npm run test:mobile + # Or: npx smartui exec --gitURL "${GIT_URL}" -- npx wdio run wdio.conf.ts + + only: + - merge_requests + - main +``` + + + + +```yaml title=".gitlab-ci.yml - Complete Java Mobile Testing Example" +stages: + - test + +variables: + MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" + LT_USERNAME: $LT_USERNAME + LT_ACCESS_KEY: $LT_ACCESS_KEY + PROJECT_TOKEN: $PROJECT_TOKEN + +visual_regression_tests: + stage: test + image: maven:3.8-openjdk-11 + + cache: + paths: + - .m2/repository/ + + before_script: + - mvn clean install -DskipTests + - npm install -g @lambdatest/smartui-cli + + script: + - | + PROJECT_ID=${CI_PROJECT_ID} + COMMIT_SHA=${CI_COMMIT_SHA} + + if [ -n "$CI_MERGE_REQUEST_IID" ]; then + COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} + fi + + GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" + + echo "GitLab Status URL: ${GIT_URL}" + + # Run Java mobile tests with SmartUI Exec + npx smartui exec --gitURL "${GIT_URL}" -- mvn test -D suite=mobile-tests.xml + + only: + - merge_requests + - main +``` + + + + + + + +--- + +## Troubleshooting + + + + +**Issue: PR Check Not Appearing in GitLab** + +**Symptoms**: Pipeline runs but no SmartUI status check appears in merge request. + +**Solutions**: +1. Verify GitLab integration is active in [LambdaTest Integrations](https://integrations.lambdatest.com/) +2. Check that `--gitURL` parameter is correctly set in the exec command +3. Verify GitLab API URL format: `https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId}` +4. Ensure `CI_PROJECT_ID` and `CI_COMMIT_SHA` are correctly set +5. For merge requests, use `CI_MERGE_REQUEST_SHA` instead of `CI_COMMIT_SHA` +6. Check pipeline logs to ensure tests completed successfully +7. Verify SmartUI CLI is installed and accessible + + + + +**Issue: Tests Run But No Screenshots in SmartUI** + +**Symptoms**: Pipeline completes successfully but no screenshots appear in SmartUI dashboard. + +**Solutions**: +1. Verify `PROJECT_TOKEN` is correctly set in GitLab CI/CD variables +2. Check that SmartUI configuration file (`.smartui.json`) exists and is valid +3. Ensure `LT_USERNAME` and `LT_ACCESS_KEY` are correctly set +4. Verify SmartUI project exists and is accessible +5. Check test logs for SmartUI execute command errors +6. Ensure screenshot commands are being called in your tests +7. Verify `SMARTUI_SERVER_ADDRESS` is set correctly (if using non-Selenium frameworks) + + + + +**Issue: SmartUI Exec Command Fails** + +**Symptoms**: `npx smartui exec` command fails or doesn't run tests. + +**Solutions**: +1. Verify SmartUI CLI is installed: `npm install -g @lambdatest/smartui-cli` +2. Check that `PROJECT_TOKEN` environment variable is set +3. Verify `.smartui.json` configuration file exists and is valid +4. Ensure test command after `--` is correct +5. Check for port conflicts (default port: 49152) +6. Review pipeline logs for detailed error messages +7. Try running the command locally first to debug + + + + +--- + +## Key Differences: Exec vs Hooks + +| Aspect | SmartUI Exec (This Guide) | SmartUI Hooks | +|--------|---------------------------|---------------| +| **Command** | Use `npx smartui exec --gitURL -- ` | Run tests normally (`npm test`, `mvn test`, `pytest`) | +| **Integration** | Requires CLI wrapper | Automatic via capabilities | +| **Setup** | Install SmartUI CLI, configure `.smartui.json` | Add capabilities to test config | +| **GitLab Integration** | Use `--gitURL` parameter with exec | Add `github.url` capability | +| **Languages** | Java SDK, CLI projects, all frameworks | TypeScript/JS/Java/Python/Ruby/C#/WebdriverIO/Appium | +| **Project Token** | Required (`PROJECT_TOKEN`) | Not required (uses `LT_USERNAME`/`LT_ACCESS_KEY`) | +| **Server Address** | May need `SMARTUI_SERVER_ADDRESS` for non-Selenium | Not required | + +--- + +## Next Steps + +- Learn about [SmartUI CLI Exec Commands](/support/docs/smartui-cli-exec) for detailed exec usage +- Explore [SmartUI Best Practices](/support/docs/smartui-best-practices) for efficient visual testing workflows +- Check the [SmartUI Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/) for common issues +- Review [GitLab CI/CD Documentation](https://docs.gitlab.com/ee/ci/) for advanced pipeline configuration + +--- + +## Related Documentation + +- [GitLab PR Checks with SmartUI Hooks](/support/docs/smartui-gitlab-pr-checks-hooks) - Hooks approach for GitLab integration +- [SmartUI with GitLab](/support/docs/smartui-with-gitlab) - General GitLab integration guide +- [SmartUI CLI Exec Commands](/support/docs/smartui-cli-exec) - Detailed exec command reference +- [SmartUI Project Settings](/support/docs/smartui-project-settings) - Configure SmartUI projects +- [SmartUI Appium Java SDK](/support/docs/smartui-appium-java-sdk) - Java SDK documentation + diff --git a/docs/smartui-gitlab-pr-checks-hooks.md b/docs/smartui-gitlab-pr-checks-hooks.md new file mode 100644 index 000000000..7d567f57b --- /dev/null +++ b/docs/smartui-gitlab-pr-checks-hooks.md @@ -0,0 +1,1232 @@ +--- +id: smartui-gitlab-pr-checks-hooks +title: GitLab PR Checks with SmartUI Hooks +sidebar_label: GitLab PR Checks (Hooks) +description: Integrate SmartUI visual regression testing with GitLab merge requests using SmartUI Hooks for web and mobile testing with Selenium, Playwright, WebdriverIO, Appium, and more. +slug: smartui-gitlab-pr-checks-hooks/ +keywords: + - GitLab PR checks + - SmartUI Hooks + - GitLab merge request integration + - Visual regression testing + - Selenium Playwright WebdriverIO + - Appium mobile testing + - SmartUI webhooks +url: https://www.lambdatest.com/support/docs/smartui-gitlab-pr-checks-hooks/ +site_name: LambdaTest +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; +import NewTag from '../src/component/newTag'; +import CodeBlock from '@theme/CodeBlock'; +import {YOUR_LAMBDATEST_USERNAME, YOUR_LAMBDATEST_ACCESS_KEY} from "@site/src/component/keys"; + +This guide shows you how to integrate SmartUI visual regression testing with GitLab merge requests using the **SmartUI Hooks** approach. This works for both **web testing** (Selenium, Playwright, Cypress, Puppeteer) and **mobile app testing** (Appium, WebdriverIO) across all supported languages. + +:::info SmartUI Hooks vs SDK + +This guide covers the **SmartUI Hooks** approach, where you pass SmartUI capabilities directly in your test configuration. This is different from the SDK approach: + +**SmartUI Hooks (This Guide):** +- ✅ No `npx smartui exec` command needed +- ✅ Tests run normally (e.g., `npm test`, `mvn test`, `pytest`) +- ✅ SmartUI integration happens automatically through capabilities +- ✅ Suitable for TypeScript/JavaScript/Java/Python/Ruby/C#/WebdriverIO/Appium +- ✅ Works with **web testing** (Selenium, Playwright, Cypress, Puppeteer) +- ✅ Works with **mobile app testing** (Appium, iOS/Android) + +**SmartUI SDK:** +- Requires `npx smartui exec -- ` +- Used for Java SDK and CLI-based projects +- See [SmartUI Appium Java SDK](/support/docs/smartui-appium-java-sdk) for SDK approach + +::: + +--- + +## Prerequisites + +Before you begin, ensure you have: + +- LambdaTest account with active subscription +- GitLab repository with CI/CD enabled +- SmartUI project created in [LambdaTest SmartUI Dashboard](https://smartui.lambdatest.com/) +- Test suite configured (Selenium/Playwright/Cypress/Puppeteer/Appium/WebdriverIO) +- Test framework configured in your preferred language (TypeScript/JavaScript/Java/Python/Ruby/C#) +- LambdaTest credentials (`LT_USERNAME` and `LT_ACCESS_KEY`) + +--- + +## Step 1: Integrate GitLab with LambdaTest + +1. Go to [LambdaTest Integrations page](https://integrations.lambdatest.com/) +2. Search for **GitLab** and select the integration +3. Click on **OAuth** as your preferred authentication method +4. Click **Install** and authorize the integration +5. After successful authentication, refresh the Integrations page to verify GitLab is installed + +GitLab integration setup in LambdaTest + +:::tip Integration Status + +You can verify your GitLab integration is active by checking the [Integrations page](https://integrations.lambdatest.com/). The GitLab integration should show as "Installed" or "Active". + +::: + +--- + +## Step 2: Configure SmartUI Capabilities with GitLab Integration + +Configure your test suite with SmartUI capabilities. Since you're using the **Hooks approach**, you'll pass SmartUI capabilities directly in your test configuration, including the GitLab integration capability. + + + + + + + +```typescript title="Example: TypeScript Selenium Configuration with SmartUI Hooks and GitLab" +import { Builder, Capabilities } from 'selenium-webdriver'; + +const capabilities = { + browserName: 'Chrome', + browserVersion: 'latest', + platformName: 'Windows 10', + 'LT:Options': { + username: process.env.LT_USERNAME, + accessKey: process.env.LT_ACCESS_KEY, + project: 'Your Project Name', + w3c: true, + name: 'Web Test Session', + build: process.env.CI + ? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` + : `smartui-local-build-${new Date().toISOString().split('T')[0]}`, + + // SmartUI Hooks Configuration + "smartUI.project": `${process.env.SMARTUI_PROJECT_NAME}-visual`, + "smartUI.build": process.env.CI + ? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` + : `smartui-local-build-${new Date().toISOString().split('T')[0]}`, + "smartUI.baseline": false, + + // GitLab Integration Capability + github: { + url: process.env.GIT_URL // GitLab API URL for status updates + } + } +}; + +const driver = await new Builder() + .usingServer(`https://${process.env.LT_USERNAME}:${process.env.LT_ACCESS_KEY}@hub.lambdatest.com/wd/hub`) + .withCapabilities(capabilities) + .build(); +``` + + + + +```java title="Example: Java Selenium Configuration with SmartUI Hooks and GitLab" +package webhook; + +import org.openqa.selenium.chrome.ChromeOptions; +import org.openqa.selenium.remote.RemoteWebDriver; +import org.testng.annotations.BeforeClass; + +import java.net.MalformedURLException; +import java.net.URL; +import java.util.HashMap; +import java.util.Map; + +public class BaseClassWebhook { + + public RemoteWebDriver driver; + public String githubURL = System.getenv("GITHUB_URL"); // GitLab URL from CI/CD + + @BeforeClass + public void setup() throws MalformedURLException { + String username = System.getenv("LT_USERNAME") == null + ? "Your LT Username" + : System.getenv("LT_USERNAME"); + String authkey = System.getenv("LT_ACCESS_KEY") == null + ? "Your LT AccessKey" + : System.getenv("LT_ACCESS_KEY"); + + ChromeOptions browserOptions = new ChromeOptions(); + HashMap ltOptions = new HashMap(); + + // LambdaTest Options + ltOptions.put("username", username); + ltOptions.put("accessKey", authkey); + ltOptions.put("project", "Your Project Name"); + ltOptions.put("w3c", true); + ltOptions.put("browserName", "Chrome"); + ltOptions.put("browserVersion", "latest"); + ltOptions.put("platformName", "Windows 10"); + + // SmartUI Hooks Configuration + String projectName = System.getenv("SMARTUI_PROJECT_NAME") != null + ? System.getenv("SMARTUI_PROJECT_NAME") + "-visual" + : "Your SmartUI Project Name"; + ltOptions.put("smartUI.project", projectName); + + String buildName = System.getenv("CI") != null + ? System.getenv("CI_PROJECT_NAME") + "-" + System.getenv("CI_PIPELINE_ID") + : "smartui-local-build"; + ltOptions.put("smartUI.build", buildName); + ltOptions.put("smartUI.baseline", false); + + browserOptions.setCapability("LT:Options", ltOptions); + + // GitLab Integration Capability + if (githubURL != null) { + Map github = new HashMap(); + github.put("url", githubURL); + browserOptions.setCapability("github", github); + System.out.println("GitLab URL received successfully: " + githubURL); + } + + String remoteUrl = "https://" + username + ":" + authkey + "@hub.lambdatest.com/wd/hub"; + driver = new RemoteWebDriver(new URL(remoteUrl), browserOptions); + } +} +``` + + + + +```python title="Example: Python Selenium Configuration with SmartUI Hooks and GitLab" +from selenium import webdriver +from selenium.webdriver.common.desired_capabilities import DesiredCapabilities +import os + +# Get GitLab URL from environment +github_url = os.getenv("GITHUB_URL") + +capabilities = { + "browserName": "Chrome", + "browserVersion": "latest", + "platformName": "Windows 10", + "LT:Options": { + "username": os.getenv("LT_USERNAME"), + "accessKey": os.getenv("LT_ACCESS_KEY"), + "project": "Your Project Name", + "w3c": True, + "name": "Web Test Session", + "build": f"{os.getenv('CI_PROJECT_NAME')}-{os.getenv('CI_PIPELINE_ID')}" if os.getenv("CI") else "smartui-local-build", + + # SmartUI Hooks Configuration + "smartUI.project": f"{os.getenv('SMARTUI_PROJECT_NAME')}-visual", + "smartUI.build": f"{os.getenv('CI_PROJECT_NAME')}-{os.getenv('CI_PIPELINE_ID')}" if os.getenv("CI") else "smartui-local-build", + "smartUI.baseline": False, + + # GitLab Integration Capability + "github": { + "url": github_url + } + } +} + +driver = webdriver.Remote( + command_executor=f"https://{os.getenv('LT_USERNAME')}:{os.getenv('LT_ACCESS_KEY')}@hub.lambdatest.com/wd/hub", + desired_capabilities=capabilities +) +``` + + + + + + + + + + +```typescript title="Example: TypeScript/WebdriverIO Mobile Configuration with SmartUI Hooks and GitLab" +import { remote, RemoteOptions } from 'webdriverio'; + +const capabilities: RemoteOptions['capabilities'] = { + deviceName: "iPhone 12", + platformName: "ios", + platformVersion: "14", + isRealMobile: true, + app: "APP_URL", // Your uploaded app URL + visual: true, // Mandatory for SmartUI + name: "Mobile App Test Session", + build: process.env.CI + ? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` + : `smartui-local-build-${new Date().toISOString().split('T')[0]}`, + + // SmartUI Hooks Configuration + "smartUI.project": `${process.env.SMARTUI_PROJECT_NAME}-visual`, + "smartUI.build": process.env.CI + ? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` + : `smartui-local-build-${new Date().toISOString().split('T')[0]}`, + "smartUI.baseline": false, + "smartUI.cropStatusBar": true, + "smartUI.cropFooter": true, + + // GitLab Integration Capability + github: { + url: process.env.GITHUB_URL // GitLab API URL for status updates + // GitLab URL format: https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId} + } +}; + +const driver = await remote({ + hostname: 'mobile-hub.lambdatest.com', + port: 443, + path: '/wd/hub', + protocol: 'https', + user: process.env.LT_USERNAME, + key: process.env.LT_ACCESS_KEY, + capabilities: capabilities as any, +}); +``` + + + + +```java title="Example: Java Appium Configuration with SmartUI Hooks and GitLab" +package webhook; + +import org.openqa.selenium.chrome.ChromeOptions; +import org.openqa.selenium.remote.RemoteWebDriver; +import org.testng.annotations.BeforeClass; + +import java.net.MalformedURLException; +import java.net.URL; +import java.util.HashMap; +import java.util.Map; + +public class BaseClassWebhook { + + public RemoteWebDriver driver; + public String githubURL = System.getenv("GITHUB_URL"); // GitLab URL from CI/CD + + @BeforeClass + public void setup() throws MalformedURLException { + String username = System.getenv("LT_USERNAME") == null + ? "Your LT Username" + : System.getenv("LT_USERNAME"); + String authkey = System.getenv("LT_ACCESS_KEY") == null + ? "Your LT AccessKey" + : System.getenv("LT_ACCESS_KEY"); + + ChromeOptions browserOptions = new ChromeOptions(); + HashMap ltOptions = new HashMap(); + + // LambdaTest Options + ltOptions.put("username", username); + ltOptions.put("accessKey", authkey); + ltOptions.put("project", "Your Project Name"); + ltOptions.put("w3c", true); + ltOptions.put("deviceName", "iPhone 12"); + ltOptions.put("platformName", "ios"); + ltOptions.put("platformVersion", "14"); + ltOptions.put("isRealMobile", true); + ltOptions.put("app", "APP_URL"); + ltOptions.put("visual", true); // Mandatory for SmartUI + + // SmartUI Hooks Configuration + String projectName = System.getenv("SMARTUI_PROJECT_NAME") != null + ? System.getenv("SMARTUI_PROJECT_NAME") + "-visual" + : "Your SmartUI Project Name"; + ltOptions.put("smartUI.project", projectName); + + String buildName = System.getenv("CI") != null + ? System.getenv("CI_PROJECT_NAME") + "-" + System.getenv("CI_PIPELINE_ID") + : "smartui-local-build"; + ltOptions.put("smartUI.build", buildName); + ltOptions.put("smartUI.baseline", false); + ltOptions.put("smartUI.cropStatusBar", true); + + browserOptions.setCapability("LT:Options", ltOptions); + + // GitLab Integration Capability + if (githubURL != null) { + Map github = new HashMap(); + github.put("url", githubURL); + browserOptions.setCapability("github", github); + System.out.println("GitLab URL received successfully: " + githubURL); + } + + String remoteUrl = "https://" + username + ":" + authkey + "@mobile-hub.lambdatest.com/wd/hub"; + driver = new RemoteWebDriver(new URL(remoteUrl), browserOptions); + } +} +``` + + + + +```python title="Example: Python Appium Configuration with SmartUI Hooks and GitLab" +from appium import webdriver +import os + +# Get GitLab URL from environment +github_url = os.getenv("GITHUB_URL") + +capabilities = { + "deviceName": "iPhone 12", + "platformName": "ios", + "platformVersion": "14", + "isRealMobile": True, + "app": "APP_URL", # Your uploaded app URL + "visual": True, # Mandatory for SmartUI + "name": "Mobile App Test Session", + "build": f"{os.getenv('CI_PROJECT_NAME')}-{os.getenv('CI_PIPELINE_ID')}" if os.getenv("CI") else "smartui-local-build", + + # SmartUI Hooks Configuration + "smartUI.project": f"{os.getenv('SMARTUI_PROJECT_NAME')}-visual", + "smartUI.build": f"{os.getenv('CI_PROJECT_NAME')}-{os.getenv('CI_PIPELINE_ID')}" if os.getenv("CI") else "smartui-local-build", + "smartUI.baseline": False, + "smartUI.cropStatusBar": True, + + # GitLab Integration Capability + "github": { + "url": github_url # GitLab API URL for status updates + } +} + +driver = webdriver.Remote( + command_executor=f"https://{os.getenv('LT_USERNAME')}:{os.getenv('LT_ACCESS_KEY')}@mobile-hub.lambdatest.com/wd/hub", + desired_capabilities=capabilities +) +``` + + + + + + + +:::info GitLab Capability Note + +The capability is named `github` (legacy name) but works with GitLab's API endpoint. Use the `GIT_URL` environment variable to pass the GitLab API URL. This is the same capability used for GitHub integration and supports both GitHub and GitLab status APIs. + +::: + +### Taking Screenshots with SmartUI Hooks + +In your test code, use the SmartUI execute command to capture screenshots: + + + + +```typescript title="Taking Screenshots with SmartUI Hooks" +// Viewport screenshot +await driver.execute("smartui.takeScreenshot=Homepage"); + +// Full page screenshot (if supported) +const config = { + screenshotName: 'Homepage', + fullPage: true, + pageCount: 15 // Minimum 1, Maximum 20 +}; +await driver.execute("smartui.takeScreenshot", config); +``` + + + + +```java title="Taking Screenshots with SmartUI Hooks in Java" +import org.openqa.selenium.JavascriptExecutor; +import java.util.HashMap; +import java.util.Map; + +// Viewport screenshot +((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot=Homepage"); + +// Full page screenshot (if supported) +Map config = new HashMap<>(); +config.put("screenshotName", "Homepage"); +config.put("fullPage", true); +config.put("pageCount", 15); // Minimum 1, Maximum 20 +((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot", config); +``` + + + + +```python title="Taking Screenshots with SmartUI Hooks in Python" +# Viewport screenshot +driver.execute_script("smartui.takeScreenshot=Homepage") + +# Full page screenshot (if supported) +config = { + "screenshotName": "Homepage", + "fullPage": True, + "pageCount": 15 # Minimum 1, Maximum 20 +} +driver.execute_script("smartui.takeScreenshot", config) +``` + + + + +```ruby title="Taking Screenshots with SmartUI Hooks in Ruby" +# Viewport screenshot +driver.execute_script("smartui.takeScreenshot=Homepage") + +# Full page screenshot (if supported) +config = { + 'screenshotName' => 'Homepage', + 'fullPage' => true, + 'pageCount' => 15 # Minimum 1, Maximum 20 +} +driver.execute_script("smartui.takeScreenshot", config) +``` + + + + +```csharp title="Taking Screenshots with SmartUI Hooks in C#" +using OpenQA.Selenium; + +// Viewport screenshot +((IJavaScriptExecutor)driver).ExecuteScript("smartui.takeScreenshot=Homepage"); + +// Full page screenshot (if supported) +var config = new Dictionary +{ + { "screenshotName", "Homepage" }, + { "fullPage", true }, + { "pageCount", 15 } // Minimum 1, Maximum 20 +}; +((IJavaScriptExecutor)driver).ExecuteScript("smartui.takeScreenshot", config); +``` + + + + +:::caution Important + +The `visual: true` capability is **mandatory** for SmartUI visual regression testing. Without this capability, screenshots will not be captured and the build status will show as `Error`. + +::: + +--- + +## Step 3: Configure GitLab CI/CD Pipeline + +Create or update your `.gitlab-ci.yml` file. Since you're using **Hooks**, you just need to run your tests normally - no SmartUI CLI exec command required. + +### Complete GitLab CI/CD Configuration + + + + +```yaml title=".gitlab-ci.yml - TypeScript/JavaScript Example" +stages: + - test + +variables: + NODE_VERSION: "18" + LT_USERNAME: $LT_USERNAME + LT_ACCESS_KEY: $LT_ACCESS_KEY + SMARTUI_PROJECT_NAME: $SMARTUI_PROJECT_NAME + +visual_regression_tests: + stage: test + image: node:${NODE_VERSION} + + before_script: + - npm ci + + script: + # Get GitLab project ID and commit SHA + - | + PROJECT_ID=${CI_PROJECT_ID} + COMMIT_SHA=${CI_COMMIT_SHA} + + # For merge requests, use the merge request commit SHA + if [ -n "$CI_MERGE_REQUEST_IID" ]; then + COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} + fi + + # Construct GitLab API URL for status updates + GITHUB_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" + + echo "GitLab Project ID: ${PROJECT_ID}" + echo "Commit SHA: ${COMMIT_SHA}" + echo "GitLab Status URL: ${GITHUB_URL}" + + # Export GITHUB_URL as environment variable for use in test capabilities + export GITHUB_URL="${GITHUB_URL}" + + # Run your tests normally - SmartUI Hooks work automatically through capabilities + npm test + # Or: npx wdio run wdio.conf.ts + # Or: npm run test:mobile + + only: + - merge_requests + - main + - develop + + environment: + name: visual-regression/$CI_COMMIT_REF_NAME +``` + + + + +```yaml title=".gitlab-ci.yml - Java Example" +stages: + - test + +variables: + MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" + LT_USERNAME: $LT_USERNAME + LT_ACCESS_KEY: $LT_ACCESS_KEY + SMARTUI_PROJECT_NAME: $SMARTUI_PROJECT_NAME + +visual_regression_tests: + stage: test + image: maven:3.8-openjdk-11 + + cache: + paths: + - .m2/repository/ + + before_script: + - mvn clean install -DskipTests + + script: + # Get GitLab project ID and commit SHA + - | + PROJECT_ID=${CI_PROJECT_ID} + COMMIT_SHA=${CI_COMMIT_SHA} + + # For merge requests, use the merge request commit SHA + if [ -n "$CI_MERGE_REQUEST_IID" ]; then + COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} + fi + + # Construct GitLab API URL for status updates + GITHUB_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" + + echo "GitLab Project ID: ${PROJECT_ID}" + echo "Commit SHA: ${COMMIT_SHA}" + echo "GitLab Status URL: ${GITHUB_URL}" + + # Export GITHUB_URL as environment variable for use in test capabilities + export GITHUB_URL="${GITHUB_URL}" + + # Run your tests normally - SmartUI Hooks work automatically through capabilities + mvn test + # Or: ./gradlew test (for Gradle) + + only: + - merge_requests + - main + - develop + + environment: + name: visual-regression/$CI_COMMIT_REF_NAME +``` + + + + +```yaml title=".gitlab-ci.yml - Python Example" +stages: + - test + +variables: + PYTHON_VERSION: "3.9" + LT_USERNAME: $LT_USERNAME + LT_ACCESS_KEY: $LT_ACCESS_KEY + SMARTUI_PROJECT_NAME: $SMARTUI_PROJECT_NAME + +visual_regression_tests: + stage: test + image: python:${PYTHON_VERSION} + + before_script: + - pip install -r requirements.txt + + script: + # Get GitLab project ID and commit SHA + - | + PROJECT_ID=${CI_PROJECT_ID} + COMMIT_SHA=${CI_COMMIT_SHA} + + # For merge requests, use the merge request commit SHA + if [ -n "$CI_MERGE_REQUEST_IID" ]; then + COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} + fi + + # Construct GitLab API URL for status updates + GITHUB_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" + + echo "GitLab Project ID: ${PROJECT_ID}" + echo "Commit SHA: ${COMMIT_SHA}" + echo "GitLab Status URL: ${GITHUB_URL}" + + # Export GITHUB_URL as environment variable for use in test capabilities + export GITHUB_URL="${GITHUB_URL}" + + # Run your tests normally - SmartUI Hooks work automatically through capabilities + pytest + # Or: python -m unittest discover + # Or: behave + + only: + - merge_requests + - main + - develop + + environment: + name: visual-regression/$CI_COMMIT_REF_NAME +``` + + + + +```yaml title=".gitlab-ci.yml - Ruby Example" +stages: + - test + +variables: + RUBY_VERSION: "3.1" + LT_USERNAME: $LT_USERNAME + LT_ACCESS_KEY: $LT_ACCESS_KEY + SMARTUI_PROJECT_NAME: $SMARTUI_PROJECT_NAME + +visual_regression_tests: + stage: test + image: ruby:${RUBY_VERSION} + + before_script: + - bundle install + + script: + # Get GitLab project ID and commit SHA + - | + PROJECT_ID=${CI_PROJECT_ID} + COMMIT_SHA=${CI_COMMIT_SHA} + + # For merge requests, use the merge request commit SHA + if [ -n "$CI_MERGE_REQUEST_IID" ]; then + COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}} + fi + + # Construct GitLab API URL for status updates + GITHUB_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}" + + echo "GitLab Project ID: ${PROJECT_ID}" + echo "Commit SHA: ${COMMIT_SHA}" + echo "GitLab Status URL: ${GITHUB_URL}" + + # Export GITHUB_URL as environment variable for use in test capabilities + export GITHUB_URL="${GITHUB_URL}" + + # Run your tests normally - SmartUI Hooks work automatically through capabilities + bundle exec rspec + # Or: bundle exec cucumber + + only: + - merge_requests + - main + - develop + + environment: + name: visual-regression/$CI_COMMIT_REF_NAME +``` + + + + +### Key Configuration Points + +1. **No SmartUI CLI exec needed**: With Hooks, you run your tests normally (e.g., `npm test`, `mvn test`, `pytest`) +2. **GitLab Project ID**: Automatically available as `CI_PROJECT_ID` in GitLab CI/CD +3. **Commit SHA**: Use `CI_COMMIT_SHA` for regular commits, or `CI_MERGE_REQUEST_SHA` for merge requests +4. **GitLab API URL**: Export as `GIT_URL` environment variable, which your test capabilities will use +5. **GitLab API URL Format**: `https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId}` + +:::info How Hooks Work + +With SmartUI Hooks: +- You pass SmartUI capabilities (including `github.url` with `GIT_URL` for GitLab) in your test configuration +- Run your tests normally (no `npx smartui exec` command) +- SmartUI integration happens automatically through the capabilities +- GitLab PR checks are updated automatically when tests complete + +::: + +--- + +## Step 4: Set Up GitLab CI/CD Variables + +Configure the following variables in your GitLab project: + +1. Go to your GitLab project → **Settings** → **CI/CD** → **Variables** +2. Add the following variables: + +GitLab CI/CD Variables configuration + +| Variable Name | Description | +|--------------|-------------| +| `LT_USERNAME` | Your LambdaTest username | +| `LT_ACCESS_KEY` | Your LambdaTest access key | +| `SMARTUI_PROJECT_NAME` | Your SmartUI project name | + +--- + +## Step 5: View Pipeline Results in GitLab + +After your pipeline runs, you can view the results in the GitLab Pipelines page: + +GitLab Pipelines page showing SmartUI test results + +The pipeline will show: +- **Pipeline status** (Success/Failed) +- **Job status** for SmartUI tests +- **Screenshot statistics** (Total, Approved, Changes Found) in the job tooltip + +--- + +## Step 6: View PR Check Results in GitLab Merge Request + +After your pipeline runs, you'll see SmartUI status checks in your GitLab merge request: + +GitLab merge request showing SmartUI PR check status + +### Successful Status + +When all visual tests pass: +- ✅ **Status**: Success +- **Details**: Click "Details" to view the SmartUI build in the dashboard +- **Screenshot**: All screenshots match baseline or are approved + +### Failed Status + +When visual differences are detected: +- ❌ **Status**: Failed +- **Details**: Click "Details" to review differences in SmartUI dashboard +- **Action Required**: Review and approve/reject changes in SmartUI dashboard + +--- + +## Complete Working Examples + + + + + + + +```typescript title="example.spec.ts - Complete Web Test with SmartUI Hooks and GitLab" +import { Builder, Capabilities } from 'selenium-webdriver'; + +describe('Web Visual Regression Tests', () => { + let driver; + + before(async () => { + // Construct GitLab URL (in CI/CD, this would come from environment variable) + const gitUrl = process.env.GIT_URL || + `https://gitlab.com/api/v4/projects/${process.env.CI_PROJECT_ID}/statuses/${process.env.CI_COMMIT_SHA}`; + + const capabilities = { + browserName: 'Chrome', + browserVersion: 'latest', + platformName: 'Windows 10', + 'LT:Options': { + username: process.env.LT_USERNAME, + accessKey: process.env.LT_ACCESS_KEY, + project: 'Your Project Name', + w3c: true, + name: 'Web Visual Tests', + build: process.env.CI + ? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` + : `local-build-${Date.now()}`, + "smartUI.project": `${process.env.SMARTUI_PROJECT_NAME}-visual`, + "smartUI.build": process.env.CI + ? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` + : `local-build-${Date.now()}`, + "smartUI.baseline": false, + // GitLab integration capability + github: { + url: gitUrl + } + } + }; + + driver = await new Builder() + .usingServer(`https://${process.env.LT_USERNAME}:${process.env.LT_ACCESS_KEY}@hub.lambdatest.com/wd/hub`) + .withCapabilities(capabilities) + .build(); + }); + + after(async () => { + if (driver) { + await driver.quit(); + } + }); + + it('should capture homepage screenshot', async () => { + await driver.get('https://example.com'); + await driver.executeScript("smartui.takeScreenshot=Homepage"); + }); + + it('should capture login page screenshot', async () => { + await driver.get('https://example.com/login'); + await driver.executeScript("smartui.takeScreenshot=LoginPage"); + }); +}); +``` + + + + +```java title="BaseClassWebhook.java - Complete Java Web Test with SmartUI Hooks and GitLab" +package webhook; + +import org.openqa.selenium.JavascriptExecutor; +import org.openqa.selenium.chrome.ChromeOptions; +import org.openqa.selenium.remote.RemoteWebDriver; +import org.testng.annotations.AfterClass; +import org.testng.annotations.BeforeClass; +import org.testng.annotations.Test; + +import java.net.MalformedURLException; +import java.net.URL; +import java.util.HashMap; +import java.util.Map; + +public class BaseClassWebhook { + + public RemoteWebDriver driver; + public String githubURL = System.getenv("GITHUB_URL"); // GitLab URL from CI/CD + + @BeforeClass + public void setup() throws MalformedURLException { + String username = System.getenv("LT_USERNAME") == null + ? "Your LT Username" + : System.getenv("LT_USERNAME"); + String authkey = System.getenv("LT_ACCESS_KEY") == null + ? "Your LT AccessKey" + : System.getenv("LT_ACCESS_KEY"); + + ChromeOptions browserOptions = new ChromeOptions(); + HashMap ltOptions = new HashMap(); + + // LambdaTest Options + ltOptions.put("username", username); + ltOptions.put("accessKey", authkey); + ltOptions.put("project", "Your Project Name"); + ltOptions.put("w3c", true); + ltOptions.put("browserName", "Chrome"); + ltOptions.put("browserVersion", "latest"); + ltOptions.put("platformName", "Windows 10"); + + // SmartUI Hooks Configuration + String projectName = System.getenv("SMARTUI_PROJECT_NAME") != null + ? System.getenv("SMARTUI_PROJECT_NAME") + "-visual" + : "Your SmartUI Project Name"; + ltOptions.put("smartUI.project", projectName); + + String buildName = System.getenv("CI") != null + ? System.getenv("CI_PROJECT_NAME") + "-" + System.getenv("CI_PIPELINE_ID") + : "smartui-local-build"; + ltOptions.put("smartUI.build", buildName); + ltOptions.put("smartUI.baseline", false); + + browserOptions.setCapability("LT:Options", ltOptions); + + // GitLab Integration Capability + if (githubURL != null) { + Map github = new HashMap(); + github.put("url", githubURL); + browserOptions.setCapability("github", github); + System.out.println("GitLab URL received successfully: " + githubURL); + } + + String remoteUrl = "https://" + username + ":" + authkey + "@hub.lambdatest.com/wd/hub"; + driver = new RemoteWebDriver(new URL(remoteUrl), browserOptions); + } + + @Test + public void testHomepageScreenshot() { + driver.get("https://example.com"); + ((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot=Homepage"); + } + + @Test + public void testLoginPageScreenshot() { + driver.get("https://example.com/login"); + ((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot=LoginPage"); + } + + @AfterClass + public void tearDown() { + if (driver != null) { + driver.quit(); + } + } +} +``` + + + + + + + + + + +```typescript title="example.spec.ts - Complete Mobile App Test with SmartUI Hooks and GitLab" +import { remote, RemoteOptions } from 'webdriverio'; + +describe('Mobile App Visual Regression Tests', () => { + let driver: WebdriverIO.Browser; + + before(async () => { + // Construct GitLab URL (in CI/CD, this would come from environment variable) + const gitUrl = process.env.GIT_URL || + `https://gitlab.com/api/v4/projects/${process.env.CI_PROJECT_ID}/statuses/${process.env.CI_COMMIT_SHA}`; + + const capabilities: RemoteOptions['capabilities'] = { + deviceName: "iPhone 12", + platformName: "ios", + platformVersion: "14", + isRealMobile: true, + app: process.env.APP_URL || "YOUR_APP_URL", + visual: true, // Mandatory for SmartUI + name: "Mobile App Visual Tests", + build: process.env.CI + ? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` + : `local-build-${Date.now()}`, + "smartUI.project": `${process.env.SMARTUI_PROJECT_NAME}-visual`, + "smartUI.build": process.env.CI + ? `${process.env.CI_PROJECT_NAME}-${process.env.CI_PIPELINE_ID}` + : `local-build-${Date.now()}`, + "smartUI.baseline": false, + "smartUI.cropStatusBar": true, + // GitLab integration capability + github: { + url: gitlabUrl + } + }; + + driver = await remote({ + hostname: 'mobile-hub.lambdatest.com', + port: 443, + path: '/wd/hub', + protocol: 'https', + user: process.env.LT_USERNAME, + key: process.env.LT_ACCESS_KEY, + capabilities: capabilities as any, + }); + }); + + after(async () => { + if (driver) { + await driver.deleteSession(); + } + }); + + it('should capture homepage screenshot', async () => { + // Navigate or perform actions + await driver.execute("smartui.takeScreenshot=Homepage"); + }); + + it('should capture login screen screenshot', async () => { + // Navigate to login screen + await driver.execute("smartui.takeScreenshot=LoginScreen"); + }); +}); +``` + + + + +```java title="BaseClassWebhook.java - Complete Java Mobile Test with SmartUI Hooks and GitLab" +package webhook; + +import org.openqa.selenium.JavascriptExecutor; +import org.openqa.selenium.chrome.ChromeOptions; +import org.openqa.selenium.remote.RemoteWebDriver; +import org.testng.annotations.AfterClass; +import org.testng.annotations.BeforeClass; +import org.testng.annotations.Test; + +import java.net.MalformedURLException; +import java.net.URL; +import java.util.HashMap; +import java.util.Map; + +public class BaseClassWebhook { + + public RemoteWebDriver driver; + public String githubURL = System.getenv("GITHUB_URL"); // GitLab URL from CI/CD + + @BeforeClass + public void setup() throws MalformedURLException { + String username = System.getenv("LT_USERNAME") == null + ? "Your LT Username" + : System.getenv("LT_USERNAME"); + String authkey = System.getenv("LT_ACCESS_KEY") == null + ? "Your LT AccessKey" + : System.getenv("LT_ACCESS_KEY"); + + ChromeOptions browserOptions = new ChromeOptions(); + HashMap ltOptions = new HashMap(); + + // LambdaTest Options + ltOptions.put("username", username); + ltOptions.put("accessKey", authkey); + ltOptions.put("project", "Your Project Name"); + ltOptions.put("w3c", true); + ltOptions.put("deviceName", "iPhone 12"); + ltOptions.put("platformName", "ios"); + ltOptions.put("platformVersion", "14"); + ltOptions.put("isRealMobile", true); + ltOptions.put("app", "APP_URL"); + ltOptions.put("visual", true); // Mandatory for SmartUI + + // SmartUI Hooks Configuration + String projectName = System.getenv("SMARTUI_PROJECT_NAME") != null + ? System.getenv("SMARTUI_PROJECT_NAME") + "-visual" + : "Your SmartUI Project Name"; + ltOptions.put("smartUI.project", projectName); + + String buildName = System.getenv("CI") != null + ? System.getenv("CI_PROJECT_NAME") + "-" + System.getenv("CI_PIPELINE_ID") + : "smartui-local-build"; + ltOptions.put("smartUI.build", buildName); + ltOptions.put("smartUI.baseline", false); + ltOptions.put("smartUI.cropStatusBar", true); + + browserOptions.setCapability("LT:Options", ltOptions); + + // GitLab Integration Capability + if (githubURL != null) { + Map github = new HashMap(); + github.put("url", githubURL); + browserOptions.setCapability("github", github); + System.out.println("GitLab URL received successfully: " + githubURL); + } + + String remoteUrl = "https://" + username + ":" + authkey + "@mobile-hub.lambdatest.com/wd/hub"; + driver = new RemoteWebDriver(new URL(remoteUrl), browserOptions); + } + + @Test + public void testHomepageScreenshot() { + // Navigate or perform actions + ((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot=Homepage"); + } + + @Test + public void testLoginScreenScreenshot() { + // Navigate to login screen + ((JavascriptExecutor) driver).executeScript("smartui.takeScreenshot=LoginScreen"); + } + + @AfterClass + public void tearDown() { + if (driver != null) { + driver.quit(); + } + } +} +``` + + + + + + + +--- + +## Troubleshooting + + + + +**Issue: PR Check Not Appearing in GitLab** + +**Symptoms**: Pipeline runs but no SmartUI status check appears in merge request. + +**Solutions**: +1. Verify GitLab integration is active in [LambdaTest Integrations](https://integrations.lambdatest.com/) +2. Check that `github.url` capability is correctly set in your test configuration +3. Verify `GIT_URL` environment variable is exported in CI/CD pipeline +4. Ensure `CI_PROJECT_ID` and `CI_COMMIT_SHA` are correctly set +5. For merge requests, use `CI_MERGE_REQUEST_SHA` instead of `CI_COMMIT_SHA` +6. Check test logs to ensure tests completed successfully +7. Verify GitLab API URL format: `https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId}` + + + + +**Issue: Tests Run But No Screenshots in SmartUI** + +**Symptoms**: Pipeline completes successfully but no screenshots appear in SmartUI dashboard. + +**Solutions**: +1. Verify `visual: true` is set in capabilities +2. Check `smartUI.project` capability matches your SmartUI project name +3. Ensure `LT_USERNAME` and `LT_ACCESS_KEY` are correctly set +4. Verify you're using the correct LambdaTest grid URL: + - Web testing: `@hub.lambdatest.com/wd/hub` + - Mobile testing: `@mobile-hub.lambdatest.com/wd/hub` +5. Check test logs for SmartUI execute command errors +6. Verify SmartUI project exists and is accessible +7. Ensure screenshot commands are being called in your tests + + + + +**Issue: GitLab URL Not Available in Tests** + +**Symptoms**: Tests run but GitLab PR check doesn't update. + +**Solutions**: +1. Verify `GIT_URL` is exported in CI/CD pipeline before test execution +2. Check that `github.url` capability is reading from `GIT_URL` environment variable +3. Add debug logging to verify URL is set +4. Ensure URL format is correct: `https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId}` +5. Verify the URL is set in the same script block that runs tests + + + + +--- + +## Key Differences: Hooks vs SDK + +| Aspect | SmartUI Hooks (This Guide) | SmartUI SDK | +|--------|---------------------------|-------------| +| **Command** | Run tests normally (`npm test`, `mvn test`, `pytest`) | Use `npx smartui exec -- ` | +| **Integration** | Automatic via capabilities | Requires CLI wrapper | +| **Setup** | Add capabilities to test config | Configure CLI and run with exec | +| **GitLab Integration** | Add `github.url` capability with `GIT_URL` | Use `--gitURL` parameter with exec | +| **Languages** | TypeScript/JS/Java/Python/Ruby/C#/WebdriverIO/Appium | Java SDK, CLI projects | +| **Java Support** | ✅ Yes - Use capabilities with `github` capability | ✅ Yes - Use `npx smartui exec -- mvn test` | + +--- + +## Next Steps + +- Learn about [SmartUI Appium Hooks](/support/docs/smartui-appium-hooks) for detailed mobile testing guide +- Learn about [SmartUI Selenium Hooks](/support/docs/smartui-selenium-hooks) for web testing guide +- Explore [SmartUI Best Practices](/support/docs/smartui-best-practices) for efficient visual testing workflows +- Check the [SmartUI Troubleshooting Guide](/support/docs/smartui-troubleshooting-guide/) for common issues +- Review [GitLab CI/CD Documentation](https://docs.gitlab.com/ee/ci/) for advanced pipeline configuration + +--- + +## Related Documentation + +- [SmartUI with GitLab](/support/docs/smartui-with-gitlab) - General GitLab integration guide +- [SmartUI Appium Hooks](/support/docs/smartui-appium-hooks) - Mobile testing with Appium Hooks +- [SmartUI Mobile Testing Overview](/support/docs/smartui-appium-sdk) - Mobile testing frameworks overview +- [SmartUI Project Settings](/support/docs/smartui-project-settings) - Configure SmartUI projects +- [GitHub App Integration](/support/docs/smartui-github-app-integration) - Similar integration pattern for GitHub diff --git a/docs/smartui-selenium-java-sdk.md b/docs/smartui-selenium-java-sdk.md index 9d4be579a..b50801b70 100644 --- a/docs/smartui-selenium-java-sdk.md +++ b/docs/smartui-selenium-java-sdk.md @@ -227,7 +227,7 @@ Once, the configuration file will be created, you will be seeing the default con Execute `visual regression tests` on SmartUI using the following commands ```bash -npx smartui --config .smartui.json exec -- mvn test -D suite="sdk-cloud.xml +npx smartui --config .smartui.json exec -- mvn test -D suite="sdk-cloud.xml" ``` :::note You may use the `npx smartui --help` command in case you are facing issues during the execution of SmartUI commands in the CLI. diff --git a/sidebars.js b/sidebars.js index 14c19641c..8fdbc1494 100644 --- a/sidebars.js +++ b/sidebars.js @@ -3660,6 +3660,16 @@ module.exports = { label: "GitLab", id: "smartui-with-gitlab", }, + { + type: "doc", + label: "GitLab PR Checks (Hooks)", + id: "smartui-gitlab-pr-checks-hooks", + }, + { + type: "doc", + label: "GitLab PR Checks (Exec)", + id: "smartui-gitlab-pr-checks-exec", + }, { type: "doc", label: "Bitbucket",