diff --git a/assets/images/kane-ai/features/assertion/img1.png b/assets/images/kane-ai/features/assertion/img1.png deleted file mode 100644 index 0ad1b08f2..000000000 Binary files a/assets/images/kane-ai/features/assertion/img1.png and /dev/null differ diff --git a/assets/images/kane-ai/features/assertion/img2.png b/assets/images/kane-ai/features/assertion/img2.png deleted file mode 100644 index 52a9bfe88..000000000 Binary files a/assets/images/kane-ai/features/assertion/img2.png and /dev/null differ diff --git a/assets/images/kane-ai/features/assertion/img3.png b/assets/images/kane-ai/features/assertion/img3.png deleted file mode 100644 index 8a6f9b5b5..000000000 Binary files a/assets/images/kane-ai/features/assertion/img3.png and /dev/null differ diff --git a/assets/images/kane-ai/features/assertion/img4.png b/assets/images/kane-ai/features/assertion/img4.png deleted file mode 100644 index dc6c62b33..000000000 Binary files a/assets/images/kane-ai/features/assertion/img4.png and /dev/null differ diff --git a/assets/images/kane-ai/features/assertion/img5.png b/assets/images/kane-ai/features/assertion/img5.png deleted file mode 100644 index 9fc1a33fa..000000000 Binary files a/assets/images/kane-ai/features/assertion/img5.png and /dev/null differ diff --git a/assets/images/mobile-app-testing/edit-system-fields.png b/assets/images/mobile-app-testing/edit-system-fields.png new file mode 100644 index 000000000..1b4facd57 Binary files /dev/null and b/assets/images/mobile-app-testing/edit-system-fields.png differ diff --git a/assets/images/mobile-app-testing/field-settings.png b/assets/images/mobile-app-testing/field-settings.png new file mode 100644 index 000000000..6e13d8e11 Binary files /dev/null and b/assets/images/mobile-app-testing/field-settings.png differ diff --git a/assets/images/mobile-app-testing/project-dashboard.webp b/assets/images/mobile-app-testing/project-dashboard.webp deleted file mode 100644 index 528a61df8..000000000 Binary files a/assets/images/mobile-app-testing/project-dashboard.webp and /dev/null differ diff --git a/assets/images/mobile-app-testing/system-fields.png b/assets/images/mobile-app-testing/system-fields.png new file mode 100644 index 000000000..abef4248f Binary files /dev/null and b/assets/images/mobile-app-testing/system-fields.png differ diff --git a/assets/images/mobile-app-testing/system-fields.webp b/assets/images/mobile-app-testing/system-fields.webp deleted file mode 100644 index aeb720508..000000000 Binary files a/assets/images/mobile-app-testing/system-fields.webp and /dev/null differ diff --git a/assets/images/mobile-app-testing/test-runs/audit_logs.png b/assets/images/mobile-app-testing/test-runs/audit_logs.png new file mode 100644 index 000000000..e9d2368fe Binary files /dev/null and b/assets/images/mobile-app-testing/test-runs/audit_logs.png differ diff --git a/assets/images/mobile-app-testing/test-runs/audit_logs_step_preview.png b/assets/images/mobile-app-testing/test-runs/audit_logs_step_preview.png new file mode 100644 index 000000000..373a8670b Binary files /dev/null and b/assets/images/mobile-app-testing/test-runs/audit_logs_step_preview.png differ diff --git a/assets/images/mobile-app-testing/test-runs/view_execution_logs.png b/assets/images/mobile-app-testing/test-runs/view_execution_logs.png new file mode 100644 index 000000000..912dfd856 Binary files /dev/null and b/assets/images/mobile-app-testing/test-runs/view_execution_logs.png differ diff --git a/docs/kane-ai-assertions.md b/docs/kane-ai-assertions.md deleted file mode 100644 index 77bcd862d..000000000 --- a/docs/kane-ai-assertions.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -id: kane-ai-assertions -title: KaneAI - Assertions -hide_title: false -sidebar_label: Assertions -description: KaneAI's assertion feature enables hard assertions to stop tests on failure, with detailed logs, while allowing soft assertions for selective test continuation. -keywords: - - lambdatest automation - - lambdatest kaneai - - kaneai assertions - - kaneai hard assertion - - kaneai soft assertion -url: https://www.lambdatest.com/support/docs/kane-ai-assertions/ -site_name: LambdaTest -slug: kane-ai-assertions/ ---- - - -The Assertion Feature in KaneAI enhances test execution control by transitioning from soft assertions (which allow tests to proceed despite failures) to hard assertions, where test execution stops immediately upon failure. This feature ensures that failures are logged with detailed remarks, providing better debugging insights. However, users retain flexibility by marking specific assertions as soft, allowing test execution to continue selectively. - -## Key Features -- **Hard Assertions :** Test execution halts immediately upon assertion failure, marking the test as failed. -- **Soft Assertions :** Specific assertions can be marked as soft, allowing test execution to continue despite failures. -- **Detailed Failure Logging :** Every failed assertion logs comprehensive failure remarks for better debugging. -- **Improved Test Control :** Users can customize assertion behavior based on test requirements. - -## Step-by-Step Guide - -### Step 1: Initiate a Test Session -- Log in to your LambdaTest account. -- Click on Create a Web / App Test to start a new session within Kane AI. - -kaneai - -### Step 2: Perform an Assertion test -- Create an assertion step and KaneAI will perform a visual query to detect if the specified test is valid or not. -- If the assertion passes, it is marked as **Assertion True**. -- If the assertion fails it will be marked as **Assertion Failed**. - -kaneai - -### Step 3: Configure Hard Assertion - ->**NOTE:** From **28 July 2025** onwards we are introducing **Failure Conditions** which will change the test step execution behavior. This change will not affect your existing test cases or the generated code where you’ve used hard/soft assertions. However, when you edit these tests, the new failure conditions will be applied. Refer [Failure Conditions doc](https://www.lambdatest.com/support/docs/kaneai-failure-conditions) - -- Click on the assertion step where the test failed. -- Toggle the button to enable or disable the **Hard assertion**: -- If enabled, test execution stops immediately on failure. -- If disabled, the test continues execution despite the failure. -- Save the configuration and proceed. - -kaneai - -### Step 4: Review and Execute the test -- Navigate to the Test Manager to view the summary of executed tests. -- Review the steps performed, and click on the **save** button. -- Navigate to the **Code** tab, your code is generated to run on HyperExecute. -- Click on the button **Run on HyperExecute**. - -kaneai - -### Step 5: Monitor the Test Execution -- Veriy your test results in the logs as shown below: - -kaneai diff --git a/docs/kane-ai-command-guide.md b/docs/kane-ai-command-guide.md index 26a016041..915e31fc8 100644 --- a/docs/kane-ai-command-guide.md +++ b/docs/kane-ai-command-guide.md @@ -133,14 +133,94 @@ You can add maximum of 300 seconds timeout for an operation. If your element is not visible in the viewport, you can use the `scroll until` command to go to that particular element if that is present currently in the DOM. This can be used in vertical scrollable pages & even on the scrollable sub-sections. -## Assertions and Queries -### Assert Element Text -- `assert if red button text is "subscribe"` +## Assertions +KaneAI supports a range of assertions to make test validations more seamless and effective. Here are the types of assertions currently supported: -### Assert Element Presence -- `assert if KaneAI is present on the "viewport"` +### Driver Assertions +Driver assertions rely on the web driver to validate browser url, page & window properties and dimensions. -### Query Information +**Examples:** +- Assert if the current browser URL is "https://example.com". +- Validate if the client height and width match the expected values. + +### Text Assertions +Text assertions validate the presence or absence of specific text on the screen. + +**Examples:** +- Assert if the text "Welcome Back!" is visible. +- Check if the error message "Invalid password" appears on failed login. + +### Visual Assertions +Visual assertions ensure the visibility of images on the screen. + +**Examples:** +- Assert if the company logo is displayed in the header. +- Verify the visibility of a product image on the product page. + +### Relative Assertions +Relative assertions check the visibility of one element in relation to another. + +**Examples:** +- Assert if the login button is in same column as the username field. +- Check if the submit button and cancel button are in the same row. + +### Mathematical Assertions +Mathematical assertions verify numerical operations or calculations. + +**Examples:** +- Verify if the sum of 3 and 4 equals 7. + +:::note + By default any assertion used has a failure condition set to fail test immediately if assertion fails. This change has been implemented since July, 20, 2025. Any tests which were authored or edited before this date, would have a failure condition set to warn, but continue by default. Existing tests would keep running as it is until you edit in KaneAI agent, post which the new failure conditions will get set. More details about failure conditions are available [here](https://www.lambdatest.com/support/docs/kaneai-failure-conditions) +::: + +## Unsupported Assertions +There are some assertions that KaneAI does not support at this moment. However the support for these kind of assertions is currently under development. +Below are examples of unsupported assertions along with examples. + +### Element State Assertions (To be available soon) +These assertions check for state of elements like being disabled or enabled. + +**Examples:** +- Assert if the submit button is disabled. +- Assert if text input field is enabled. +- Assert if a dropdown is expanded. + +### Element Property Assertions (To be available soon) +Property assertions involve checking styles or attributes of an element. + +**Examples:** +- Assert if the font size of a header is "16px". +- Assert if the padding of a button is "10px". + +### Spatial Assertions (To be available soon) +Spatial assertions validate the position or arrangement of elements. + +**Examples:** +- Assert that the 5th column of a table contains "Jordan.Mathews". + +### Logical Assertions (To be available soon) +Logical assertions are used to combine multiple conditions. + +**Examples:** +- Assert if the user is an admin **and** is logged in. +- Assert if either username or email is filled. + +### Assertions for Actions Being Performed (To be available soon) +These assertions aim to check the state after an action, which is not supported directly. + +**Examples:** +- Assert if the page is scrolled to the bottom. +- Assert if a tooltip appears after hovering over an info icon. + - Currently, you can break this into two steps: Hover on the info icon, then assert if the tooltip is visible. + +### Nested Assertions (To be available soon) +Nested assertions involve multiple layers of validation within a single assertion. + +**Examples:** +- Assert if both the login button is enabled and the welcome message is visible. + +## Query Information - `query the current url` - `query the time mentioned in the poster` diff --git a/docs/kane-ai-supported-assertions.md b/docs/kane-ai-supported-assertions.md deleted file mode 100644 index a955a9582..000000000 --- a/docs/kane-ai-supported-assertions.md +++ /dev/null @@ -1,144 +0,0 @@ ---- -id: kane-ai-supported-assertions -title: KaneAI - Supported Assertions -hide_title: false -sidebar_label: Supported Assertions -description: KaneAI supports a wide range of assertions to validate test cases efficiently. Here's an overview of the supported and unsupported assertions. -keywords: - - lambdatest automation - - lambdatest kaneai - - kaneai assertions -url: https://www.lambdatest.com/support/docs/kane-ai-supported-assertions/ -site_name: LambdaTest -slug: kane-ai-supported-assertions/ ---- - - -KaneAI offers robust assertion support to validate various aspects of your test cases efficiently. Below is an overview of the supported and unsupported assertions. - -## Supported Assertions - -KaneAI supports a range of assertions to make test validations more seamless and effective. Here are the types of assertions currently supported: - -### Driver Assertions -Driver assertions rely on the web driver to validate browser url, page & window properties and dimensions. - -**Examples:** -- Assert if the current browser URL is "https://example.com". -- Check if the page title is "Dashboard". -- Validate if the client height and width match the expected values. - -### Text Assertions -Text assertions validate the presence or absence of specific text on the screen. - -**Examples:** -- Assert if the text "Welcome Back!" is visible. -- Check if the error message "Invalid password" appears on failed login. - -### Selection State Assertions -Selection state assertions verify the selection status of elements like checkboxes and radio buttons. - -**Examples:** -- Assert if the "Remember Me" checkbox is checked. -- Verify if the "Male" radio button is selected. - -### Color Assertions -Color assertions validate the color properties of elements on the screen. - -**Examples:** -- Assert if the submit button has a blue background. -- Check if the error message appears in red. - -### Visual Assertions -Visual assertions ensure the visibility of images on the screen. - -**Examples:** -- Assert if the company logo is displayed in the header. -- Verify the visibility of a product image on the product page. - -### Relative Assertions -Relative assertions check the visibility of one element in relation to another. - -**Examples:** -- Assert if the login button is in same column as the username field. -- Check if the submit button and cancel button are in the same row. - ---- - -## Unsupported Assertions -There are some assertions that KaneAI does not support at this moment. However the support for these kind of assertions is currently under development. -Below are examples of unsupported assertions along with examples. - -### Element State Assertions (To be available soon) -These assertions check for state of elements like being disabled or enabled. - -**Examples:** -- Assert if the submit button is disabled. -- Assert if text input field is enabled. -- Assert if a dropdown is expanded. - -### Element Property Assertions (To be available soon) -Property assertions involve checking styles or attributes of an element. - -**Examples:** -- Assert if the font size of a header is "16px". -- Assert if the padding of a button is "10px". - -### Spatial Assertions (To be available soon) -Spatial assertions validate the position or arrangement of elements. - -**Examples:** -- Assert that the 5th column of a table contains "Jordan.Mathews". - -### Logical Assertions (To be available soon) -Logical assertions are used to combine multiple conditions. - -**Examples:** -- Assert if the user is an admin **and** is logged in. -- Assert if either username or email is filled. - -### Mathematical Assertions (To be available soon) -Mathematical assertions verify numerical operations or calculations. - -**Examples:** -- Verify if the sum of 3 and 4 equals 7. - -### Assertions for Actions Being Performed (To be available soon) -These assertions aim to check the state after an action, which is not supported directly. - -**Examples:** -- Assert if the page is scrolled to the bottom. -- Assert if a tooltip appears after hovering over an info icon. - - Currently, you can break this into two steps: Hover on the info icon, then assert if the tooltip is visible. - -### Nested Assertions (To be available soon) -Nested assertions involve multiple layers of validation within a single assertion. - -**Examples:** -- Assert if both the login button is enabled and the welcome message is visible. - ---- - -For further assistance or questions, feel free to reach out to our support team at kaneaisupport@lambdatest.com. diff --git a/docs/system-and-custom-fields.md b/docs/system-and-custom-fields.md index b4dbd6a3d..9bea7c531 100644 --- a/docs/system-and-custom-fields.md +++ b/docs/system-and-custom-fields.md @@ -43,29 +43,44 @@ slug: system-and-custom-fields/ Explore the structured organization of your test projects through the use of Fields and Values, accessible via the Project's dashboard. Dive into System Fields for essential categorizations or enhance your project's flexibility with Custom Fields to improve your testing workflows. -1. In the Project's dashboard, click on **Settings** in the top right hand side. +To access the System & Custom Fields settings, click on **Settings** in the top right hand side in the Project's dashboard. -Real +Real -2. You will be able to view the **System Fields** and **Custom Fields**. +You can manage your **System** and **Custom Fields** in this Fields page. -- System Fields is a default feature provided by LambdaTest that includes standard fields like **Priority**, **Status**, and **Type**. These fields help organize and track your test cases. They can be seamlessly integrated into your test management process and customized by adding values. To do so, simply click on any field and select **Add Value**. +### System Fields +Are default fields present in the Test Manager. These fields help organize and track your test cases, test runs or instances. They can be seamlessly integrated into your test management process and customized by adding values. Test Cases & Test Runs have their separate System Fields. -Real +- Test Cases have these pre defined fields: **Priority**, **Status**, and **Type**. +- Test Runs can have only one System Field that is the `Status`. -- Custom Fields allow you to store additional information beyond what System Fields offer. To create a new field, enter the required details and choose the appropriate data type from the **Type** function. Available types include String, Textarea, Number, Dropdown (Single Select), Dropdown (Multi Select), Boolean (Checkbox), Date, User, and URL. +To manage these fields, navigate to the System Fields tab, where you'll see dedicated sections for both Test Case and Test Runs fields. + +Real + +These System Fields can have Custom Values which can be added by simply selecting any field and clicking on **Add Value**. + +Real + +:::tip + Every new Status value of Test Runs will have a unique random color defined to it on creation. +::: + +### Custom Fields +Allow you to store additional information beyond what System Fields offer. To create a new field, enter the required details and choose the appropriate data type from the **Type** function. Available types include String, Textarea, Number, Dropdown (Single Select), Dropdown (Multi Select), Boolean (Checkbox), Date, User, and URL. Real -- For Dropdown types (Single Select and Multi Select), you also have the option to add values. +For Dropdown types (Single Select and Multi Select), you also have the option to add values. Real -- Enter the name, placeholder, mark the field, apply it to all future projects if required and click create. +Enter the name, placeholder, mark the field, apply it to all future projects if required and click create. Real -3. You can also link a single or multiple projects of your choice to the custom fields and click on **Save changes**. +You can also link a single or multiple projects of your choice to the custom fields and click on **Save changes**. Real diff --git a/docs/test-instance-audit-logs.md b/docs/test-instance-audit-logs.md new file mode 100644 index 000000000..69eedd7c6 --- /dev/null +++ b/docs/test-instance-audit-logs.md @@ -0,0 +1,65 @@ +--- +id: test-instance-audit-logs +title: Test Instance Audit Logs +hide_title: false +sidebar_label: Test Instance Audit Logs +description: See every thing you update in a test instance through Audit Logs. +keywords: + - audit log + - test instance logs + - execution history +url: https://www.lambdatest.com/support/docs/test-instance-audit-logs/ +site_name: LambdaTest +slug: test-instance-audit-logs/ +--- + + +Test Manager now provides Audit Logs for test instance execution. Audit Logs bring visibility into the **who, what, and when** for every test execution. This is critical for teams working in regulated environments, or those needing high accountability in their QA processes. + +With Audit logs you can store the execution history of your tests and even run parallel execution sessions without losing the context on individual executions as every action will be logged with clear indication of time stamps & executor. + +## Details captured in Audit Logs + +- Test Instance & Steps `Status` changes. +- Test Instance & Steps `Remarks or attachment` changes. +- Test Instance Assignee changes. + +You can view the Audit Logs by clicking on the `View Execution Log`. + +Real + +Real + +For the Steps level execution logs, you can preview the step with respect to which the log was created. + +:::note + As step level Audit logs are with respect to that specific steps in case the step changes the old logs will still show the preview of the older step. +::: + +Real + +:::note + When you delete a configuration or a test case, all associated audit logs are removed. This is because the action deletes the entire instance. If you then add the same test case and configuration again, it creates a brand new instance with no prior audit history. +::: \ No newline at end of file diff --git a/sidebars.js b/sidebars.js index 5375c68fc..42c13b949 100644 --- a/sidebars.js +++ b/sidebars.js @@ -1245,7 +1245,6 @@ module.exports = { "kane-ai-modules", "kaneai-modules-versions-and-enhancement", "kaneai-upload-and-download-files", - "kane-ai-assertions", "kaneai-dynamic-url-replacement", "kaneai-chrome-options", "kaneai-custom-headers", @@ -1261,7 +1260,6 @@ module.exports = { label: "Knowledge Base", items: [ "kane-ai-command-guide", - "kane-ai-supported-assertions", "kaneai-failure-conditions", "kane-ai-web-test-writing-guidelines", "kane-ai-app-test-writing-guidelines", @@ -2007,9 +2005,10 @@ module.exports = { ], }, { - type: "doc", - label: "Test Run", - id: "test-run-creation-and-management", + type: "category", + collapsed: true, + label: "Test Run & Instances", + items: ["test-run-creation-and-management", "test-instance-audit-logs"], }, { type: "doc",