From fb4494095f79473ac7bb350c862ada2134489167 Mon Sep 17 00:00:00 2001 From: mseong6251 Date: Thu, 20 Nov 2025 13:58:51 -0500 Subject: [PATCH 1/4] Update and rename adaptive-testing.adoc to smarter-testing.adoc --- ...tive-testing.adoc => smarter-testing.adoc} | 46 +++++++++---------- 1 file changed, 23 insertions(+), 23 deletions(-) rename docs/guides/modules/test/pages/{adaptive-testing.adoc => smarter-testing.adoc} (92%) diff --git a/docs/guides/modules/test/pages/adaptive-testing.adoc b/docs/guides/modules/test/pages/smarter-testing.adoc similarity index 92% rename from docs/guides/modules/test/pages/adaptive-testing.adoc rename to docs/guides/modules/test/pages/smarter-testing.adoc index 3d3732062f..97d94177a9 100644 --- a/docs/guides/modules/test/pages/adaptive-testing.adoc +++ b/docs/guides/modules/test/pages/smarter-testing.adoc @@ -1,36 +1,36 @@ -= Adaptive testing += Smarter Testing :page-badge: Preview :page-platform: Cloud -:page-description: This document describes the adaptive testing feature in CircleCI, which enables only running tests that are impacted by code changes and evenly distributes tests across parallel execution nodes. +:page-description: This document describes the Smarter Testing feature in CircleCI, which enables only running tests that are impacted by code changes and evenly distributes tests across parallel execution nodes. :experimental: :page-noindex: true -CAUTION: *Adaptive testing* is available in closed preview. When the feature is made generally available there will be a cost associated with access and usage. +CAUTION: *Smarter Testing* is available in closed preview. When the feature is made generally available there will be a cost associated with access and usage. NOTE: This page is currently in development and will be updated as the feature is developed. -Use adaptive testing to optimize test runs as follows: +Use Smarter Testing to optimize test runs as follows: * Run only tests that are impacted by code changes. * Evenly distribute tests across parallel execution nodes. -Adaptive testing reduces test execution time while maintaining test confidence. +Smarter Testing reduces test execution time while maintaining test confidence. -== Is my project a good fit for adaptive testing? +== Is my project a good fit for Smarter Testing? -The following list shows some examples of where adaptive testing can be most beneficial: +The following list shows some examples of where Smarter Testing can be most beneficial: * Tests that exercise code within the same repository. -* Projects with comprehensive test coverage. The more thorough your tests, the more precisely adaptive testing can identify which tests are impacted by changes. +* Projects with comprehensive test coverage. The more thorough your tests, the more precisely Smarter Testing can identify which tests are impacted by changes. * Test frameworks with built-in coverage support (Jest, pytest, Go test, Vitest) where generating coverage reports is straightforward. + -TIP: In codebases with sparse test coverage, adaptive testing cannot accurately determine which tests cover changed code. This causes the system to run more tests, reducing the benefits of intelligent test selection. +TIP: In codebases with sparse test coverage, Smarter Testing cannot accurately determine which tests cover changed code. This causes the system to run more tests, reducing the benefits of intelligent test selection. == Limitations -* Generating code coverage data is essential for determining how tests are related to code. If tests are run in a way that makes generating and accessing code coverage data tricky then adaptive testing may not be a good fit. -* Adaptive testing works best when testing a single deployable artifact. For example, a monorepo with integration tests that span multiple packages/services, especially when services run in separate container, makes coverage generation and consolidation difficult. -* Adaptive testing needs to be configured with commands to discover all available tests and run a subset of those tests. If you cannot run commands to discover tests and run a subset of tests on the CLI then adaptive testing may not be a good fit. +* Generating code coverage data is essential for determining how tests are related to code. If tests are run in a way that makes generating and accessing code coverage data tricky then Smarter Testing may not be a good fit. +* Smarter Testing works best when testing a single deployable artifact. For example, a monorepo with integration tests that span multiple packages/services, especially when services run in separate container, makes coverage generation and consolidation difficult. +* Smarter Testing needs to be configured with commands to discover all available tests and run a subset of those tests. If you cannot run commands to discover tests and run a subset of tests on the CLI then Smarter Testing may not be a good fit. == Key benefits @@ -40,7 +40,7 @@ TIP: In codebases with sparse test coverage, adaptive testing cannot accurately * Scale efficiently as test suites grow. == How it works -Adaptive testing operates through two main components that work together to optimize your test execution: +Smarter Testing operates through two main components that work together to optimize your test execution: * Dynamic test splitting * Test impact analysis @@ -50,7 +50,7 @@ Each component is described in more detail below. === Dynamic test splitting Dynamic test splitting distributes your tests across parallel execution nodes. The system maintains a shared queue that each node pulls from to create a balanced workload. -When you configure parallelism in your job, adaptive testing automatically: +When you configure parallelism in your job, Smarter Testing automatically: * Retrieves timing data from previous test runs. * Calculates optimal test distribution across your specified number of parallel nodes. @@ -293,15 +293,15 @@ options: If tests are still slower, share the pipeline link in the closed beta slack channel. -== 2. Enable adaptive testing +== 2. Enable Smarter Testing -We recommend following the steps in <> first before enabling the adaptive testing feature to ensure the `discover` and `run` commands are set up correctly. +We recommend following the steps in <> first before enabling the Smarter Testing feature to ensure the `discover` and `run` commands are set up correctly. -The goal of this section is to enable adaptive testing for your test suite. +The goal of this section is to enable Smarter Testing for your test suite. === 2.1 Update the test suites file -When using adaptive testing for test impact analysis the following commands are used: +When using Smarter Testing for test impact analysis the following commands are used: * The `discover` command discovers all tests in a test suite. * The `run` command runs only impacted tests and a new command. @@ -497,7 +497,7 @@ The following options are available to be defined in the options map in config: | `adaptive-testing` |false -|Enables the adaptive testing features, such as test impact analysis. +|Enables the Smarter Testing features, such as test impact analysis. | `full-test-run-paths` a| @@ -548,7 +548,7 @@ a| * `all` selects and runs all discovered tests, used to run the full test suit |=== -- -== 3. Start using adaptive testing +== 3. Start using Smarter Testing Now the test suite is set up, test selection is working and the test analysis is up to date with the latest changes from the feature branch that ran the first test analysis. @@ -658,7 +658,7 @@ options: == Limitations -The adaptive testing feature has some limitations to consider: +The Smarter Testing feature has some limitations to consider: *Initial setup period*:: Test impact analysis requires an initial analysis run on all tests before intelligent selection can begin. This first analysis run will be slower than normal test execution. @@ -782,7 +782,7 @@ When test selection determines that no existing tests are affected by your chang *Best practice:* Include relevant paths in `full-test-run-paths` to explicitly trigger full test runs for infrastructure changes. -=== How do I know if adaptive testing is working? +=== How do I know if Smarter Testing is working? Look for these indicators in your CircleCI build output: @@ -858,7 +858,7 @@ The `analysis-baseline` command will be run just before running analysis. The co === What test frameworks are supported? -Adaptive testing is runner-agnostic. We provide default configurations for the following test frameworks: +Smarter Testing is runner-agnostic. We provide default configurations for the following test frameworks: * Jest (JavaScript/TypeScript) * gotestsum (Go) From fa1c353bce85dceb94516a304fca2314d3e9c713 Mon Sep 17 00:00:00 2001 From: rosie yohannan Date: Fri, 21 Nov 2025 14:34:52 +0000 Subject: [PATCH 2/4] redirect and style guide update --- docs/guides/modules/test/pages/smarter-testing.adoc | 1 + styles/config/vocabularies/Docs/accept.txt | 1 + 2 files changed, 2 insertions(+) diff --git a/docs/guides/modules/test/pages/smarter-testing.adoc b/docs/guides/modules/test/pages/smarter-testing.adoc index 97d94177a9..6f9ca4b1d3 100644 --- a/docs/guides/modules/test/pages/smarter-testing.adoc +++ b/docs/guides/modules/test/pages/smarter-testing.adoc @@ -4,6 +4,7 @@ :page-description: This document describes the Smarter Testing feature in CircleCI, which enables only running tests that are impacted by code changes and evenly distributes tests across parallel execution nodes. :experimental: :page-noindex: true +:page-aliases: adaptive-testing.adoc CAUTION: *Smarter Testing* is available in closed preview. When the feature is made generally available there will be a cost associated with access and usage. diff --git a/styles/config/vocabularies/Docs/accept.txt b/styles/config/vocabularies/Docs/accept.txt index c7bc4a7efa..8e5a548a07 100644 --- a/styles/config/vocabularies/Docs/accept.txt +++ b/styles/config/vocabularies/Docs/accept.txt @@ -315,6 +315,7 @@ SLAs? Slanger [Ss]ignup SKUs? +Smarter Testing [Ss]nap SSH statsd From f6e929dd996c3618dac891251ee22a5c9a89d42a Mon Sep 17 00:00:00 2001 From: rosie yohannan Date: Fri, 21 Nov 2025 19:18:40 +0000 Subject: [PATCH 3/4] update file name in unlisted pages allow list --- antora-playbook.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/antora-playbook.yml b/antora-playbook.yml index 0b026866fa..fd12063498 100644 --- a/antora-playbook.yml +++ b/antora-playbook.yml @@ -69,7 +69,7 @@ antora: extensions: - require: ./extensions/unlisted-pages-extension.js allowedUnlistedPages: - - 'test:adaptive-testing.adoc' + - 'test:smarter-testing.adoc' - 'test:fix-flaky-tests.adoc' - require: '@sntke/antora-mermaid-extension' mermaid_initialize_options: From 0652aec0483b4bc5133c20d177521958443f3fb6 Mon Sep 17 00:00:00 2001 From: rosie yohannan Date: Fri, 21 Nov 2025 19:34:27 +0000 Subject: [PATCH 4/4] fix lint errors --- .../modules/test/pages/smarter-testing.adoc | 33 +++++++------ styles/AsciiDoc/UnsetAttributes.yml | 49 ------------------- 2 files changed, 18 insertions(+), 64 deletions(-) delete mode 100644 styles/AsciiDoc/UnsetAttributes.yml diff --git a/docs/guides/modules/test/pages/smarter-testing.adoc b/docs/guides/modules/test/pages/smarter-testing.adoc index 6f9ca4b1d3..a6e0acbbb2 100644 --- a/docs/guides/modules/test/pages/smarter-testing.adoc +++ b/docs/guides/modules/test/pages/smarter-testing.adoc @@ -1,7 +1,7 @@ = Smarter Testing :page-badge: Preview :page-platform: Cloud -:page-description: This document describes the Smarter Testing feature in CircleCI, which enables only running tests that are impacted by code changes and evenly distributes tests across parallel execution nodes. +:page-description: This page describes CircleCI's Smarter Testing. Only run tests that are impacted by code changes and evenly distribute tests across parallel execution nodes. :experimental: :page-noindex: true :page-aliases: adaptive-testing.adoc @@ -43,8 +43,8 @@ TIP: In codebases with sparse test coverage, Smarter Testing cannot accurately d == How it works Smarter Testing operates through two main components that work together to optimize your test execution: -* Dynamic test splitting -* Test impact analysis +* Dynamic test splitting. +* Test impact analysis. Each component is described in more detail below. @@ -551,7 +551,10 @@ a| * `all` selects and runs all discovered tests, used to run the full test suit == 3. Start using Smarter Testing -Now the test suite is set up, test selection is working and the test analysis is up to date with the latest changes from the feature branch that ran the first test analysis. +Now the test suite is set up: + +* Test selection is working. +* The test analysis is up to date with the latest changes from the feature branch that ran the first test analysis. *Action Items* @@ -679,9 +682,9 @@ The Smarter Testing feature has some limitations to consider: *Debugging steps:* -. Check that all test files are discovered with the discover command -. Verify parallelism is set correctly in your config.yml -. Look for timing data in previous test runs +. Check that all test files are discovered with the discover command. +. Verify parallelism is set correctly in your config.yml. +. Look for timing data in previous test runs. . Ensure test results are being stored with `store_test_results`. === Test impact analysis not selecting expected tests @@ -692,10 +695,10 @@ The Smarter Testing feature has some limitations to consider: *Debugging steps:* -. Verify analysis has run successfully on your configured branch(es) -. Check that coverage data is being generated correctly -. Review the full-test-run-paths configuration - changes to these paths trigger full test runs -. Confirm the analysis command is producing valid LCOV output +. Verify analysis has run successfully on your configured branch(es). +. Check that coverage data is being generated correctly. +. Review the full-test-run-paths configuration - changes to these paths trigger full test runs. +. Confirm the analysis command is producing valid LCOV output. *When all tests run:* If no impact data exists or all tests are determined to be affected, the system runs all tests as a safety measure. @@ -745,9 +748,9 @@ The frequency depends on your test execution speed and development pace: *Consider re-running analysis:* -* After major refactoring or code restructuring -* When test selection seems inaccurate or outdated -* After adding significant new code or tests +* After major refactoring or code restructuring. +* When test selection seems inaccurate or outdated. +* After adding significant new code or tests. *Remember:* You can customize which branches run analysis through your CircleCI configuration - it does not have to be limited to the main branch. @@ -822,7 +825,7 @@ See the < 0 { - // remove the most recently added match - matches = matches[:len(matches)-1] - } else if text.re_match(unset_attr_suff, line) { - if len(matches) > 0 { - matches = matches[:len(matches)-1] - } - } - } - } - } - } - }