diff --git a/docs/analytics-test-failure-classification.md b/docs/analytics-test-failure-classification.md new file mode 100644 index 000000000..83adf8438 --- /dev/null +++ b/docs/analytics-test-failure-classification.md @@ -0,0 +1,85 @@ +--- +id: analytics-test-failure-classification +title: Failure Categorization AI - Test Intelligence +sidebar_label: Failure Categorization AI +description: LambdaTest Analytics uses Failure Categorization AI to classify test failures into different categories. This helps in identifying the root cause of failures and improving test automation efficiency. +keywords: + - analytics + - test failure classification + - test intelligence + - failure categorization + - AI + - machine learning + - test automation +url: https://www.lambdatest.com/support/docs/analytics-test-failure-classification/ +site_name: LambdaTest +slug: analytics-test-failure-classification/ +--- + + + +# Failure Categorization AI - Test Intelligence + +LambdaTest Analytics uses Failure Categorization AI to classify test failures into different categories. This helps in identifying the root cause of failures and improving test automation efficiency. The AI model analyzes the test execution data and categorizes the failures based on various parameters such as environment, browser, OS, and failure type. This allows you to focus on the most critical issues and prioritize your testing efforts. + +## Pre-requisites For Failure Categorization AI + +1. You should have an active LambdaTest account. +2. This feature is available for users with the HyperExecute or App / Web Automation subscription plan. +3. You should run at least 10 tests on the LambdaTest automation platform. +4. You should have access to the LambdaTest Insights platform. +5. You can see the failure categorization AI on creating a dashboard in the Insights section. +6. Click on the `Create New` button to create a new dashboard. +7. Choose any of the available widgets and add them to the dashboard. +8. Click on the widget drill down to see the failure categorization AI. + +## What is Failure Categorization AI? + +The test failure categorization AI is a machine learning model that classifies test failures into different categories. The AI model uses various parameters such as environment, browser, OS, and failure type to categorize the failures. This helps in identifying the root cause of failures and improving test automation efficiency. + +For the first failure as a user you will need to select the type of failure from the list of categories available. The categories are as follows: +- **Product Bug:** This category is used when the failure is due to a bug in the product being tested. This could be a UI issue, a functional issue, or any other type of bug that affects the product's functionality. +- **Test Automation Bug:** This category is used when the failure is due to a bug in the test automation code. This could be a script issue, a framework issue, or any other type of bug that affects the test automation process. +- **Environment Issue:** This category is used when the failure is due to an issue with the test environment. This could be a configuration issue, a network issue, or any other type of issue that affects the test environment. +- **No Action Required:** This category is used when the failure does not require any action. This could be a known issue, a false positive, or any other type of issue that does not require any action from the user. + +Once the user selects the category for the first failure, the AI model will learn from the user's selection and will automatically categorize the subsequent failures based on the user's selection. This helps in reducing the manual effort required to categorize the failures and improves the overall efficiency of the test automation process. + +## How will the Failure Categorization AI help you? + +The failure categorization AI helps you in the following ways: + +- **Automated Classification:** The AI model automatically classifies the test failures into different categories, reducing the manual effort required to categorize the failures. + +- **Improved Efficiency:** By categorizing the failures, the AI model helps in identifying the root cause of failures and improving test automation efficiency. + +- **Prioritization of Testing Efforts:** The AI model helps in prioritizing the testing efforts by identifying the most critical issues that need to be addressed. + +- **Better Insights:** The AI model provides better insights into the test execution data, helping you to make data-driven decisions. + + +## Support + +For any queries or issues related to the Failure Categorization AI, please reach out to our [24/7 customer support](mailto:support@lambdatest.com). We are here to help you! diff --git a/docs/queuing.md b/docs/queuing.md deleted file mode 100644 index 57aa1af09..000000000 --- a/docs/queuing.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -id: queuing -title: LambdaTest Test Session Queuing -hide_title: true -sidebar_label: Test Session Queuing -description: To prevent over-exploitation of LambdaTest, LamvdaTest has added a capacity constraint on the number of tests that can be queued at our platform. This document will explain how test session queuing works on the LambdaTest platform. -keywords: - - test session queuing - - queuing - - lambdatest queuing -url: https://www.lambdatest.com/support/docs/queuing/ -site_name: LambdaTest -slug: queuing/ ---- - - - -# Test Session Queuing -*** -To prevent over-exploitation of LambdaTest, we have added a capacity constraint on the number of tests that can be queued at our platform. The maximum number of elements that are allowed to be queued for your LambdaTest account will depend upon the number of concurrent sessions you are eligible for. You can figure out the maximum number of test cases you are allowed to queue with the below formula. - -```javascript -Maximum number of test cases that can be queued = n + 150 - -Here, n = number of concurrent sessions. -``` -Here is an example, if your LambdaTest account is eligible for 10 concurrent sessions, then your queue can have a maximum of (10 + 150) queued test cases i.e. 160 queued test cases. The scheduling and execution of test cases in your queue will be taken care of by LambdaTest. - -If more tests are sent for execution than allowed by your concurrency limit then they will be put in a queue until the queue reaches its maximum holding capacity. So, if you have a concurrency of 100 but you send 200 tests, the "extra" tests will be queued and run as the first round of tests finish. In this scenario, when the number of running tests drops to 99, 1 test from the queue will start running, and when the next test finishes, another test from the queue will start running. This continues until all the queued tests are executed. - -Keep in mind the below key points before you start queuing your automated test cases using LambdaTest Selenium grid. - -1. As explained on top, there is a limit to the number of tests that can be queued for a single instance. This limit is dependent on the number of your LambdaTest concurrent sessions and exceeding your maximum queuing capacity will directly reject the exceeding test cases. -2. There is also a maximum limit on the time for which a test is allowed to be in the queue. So, If you have got automated test cases that have been in a queue from more than 15 minutes then they will be timed out and will not be executed. - -## Rate Limiter and Penalty -*** -An account can breach allowed queuing limit 7 times in a month, Example an account has 100 concurrent session plan so that he is allowed to put n+150=250 requests in the queue. If he tries to exceed the limit then the requests will get rejected. Rejection events are counted monthly at the account level. So. user is allowed to request (n+150)*x (default 7 in a month) (where x is a constant per account) before suspending the account. - -1. After (n+150)*2 request rejection due to exceeding allowed queuing threshold in a month, Admin will get an email warning for exploitation of service. -2. After (n+150)*4 request rejection due to exceeding allowed queuing threshold in a month, Admin will get an email warning for exploitation of service. -3. After (n+150)*6 request rejection due to exceeding allowed queuing threshold in a month, Admin will get an email warning for exploitation of service with Last warning content. -4. After (n+150)*7 request rejection due to exceeding allowed queuing threshold in a month, Admin will get an email of account suspension with a support link for re-activation of account. - - \ No newline at end of file diff --git a/sidebars.js b/sidebars.js index 51ea351ec..70fb1ccf5 100644 --- a/sidebars.js +++ b/sidebars.js @@ -2167,7 +2167,6 @@ module.exports = { "Selenium-Grid-Configuration", "supported-browsers-and-operating-systems-for-the-web-interface", "change-individual-test-details", - "queuing", ], }, { @@ -3077,6 +3076,7 @@ module.exports = { }, "analytics-build-comparison", "analytics-smart-tags-test-intelligence", + "analytics-test-failure-classification", "analytics-modules-test-intelligence-flaky-test-analytics", "analytics-dashboard-copilot", "analytics-allure-api-widgets",