Skip to content

Conversation

NivYarmus
Copy link
Contributor

@NivYarmus NivYarmus commented Jul 24, 2025

Motivation/Description of the PR

Hello there.
Added an option to the run command that shuffles the order in which test suites are run.
Helpful if you want to make sure that your tests are not passing just because of a lucky side effect of a previous test.

Applicable helpers:

  • Playwright
  • Puppeteer
  • WebDriver
  • REST
  • FileHelper
  • Appium
  • TestCafe

Applicable plugins:

  • allure
  • autoDelay
  • autoLogin
  • customLocator
  • pauseOnFail
  • coverage
  • retryFailedStep
  • screenshotOnFail
  • selenoid
  • stepByStepReport
  • stepTimeout
  • wdio
  • subtitles

Type of change

  • 🔥 Breaking changes
  • 🚀 New functionality
  • 🐛 Bug fix
  • 🧹 Chore
  • 📋 Documentation changes/updates
  • ♨️ Hot fix
  • 🔨 Markdown files fix - not related to source code
  • 💅 Polish code

Checklist:

  • Tests have been added
  • Documentation has been added (Run npm run docs)
  • Lint checking (Run npm run lint)
  • Local tests are passed (Run npm test)

@kobenguyent kobenguyent requested a review from Copilot July 25, 2025 07:35
Copilot

This comment was marked as outdated.

@kobenguyent
Copy link
Collaborator

I guess that'll be helpful to update docs as well -> https://github.com/codeceptjs/CodeceptJS/blob/3.x/docs/commands.md

@NivYarmus
Copy link
Contributor Author

Added doc😁

@kobenguyent kobenguyent requested a review from Copilot August 2, 2025 05:19
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a --shuffle option to the run command that randomizes the order of test suite execution, helping to identify tests that depend on side effects from previous tests.

  • Added lodash.shuffle dependency for randomizing test file order
  • Implemented shuffle logic in the Codecept class when the shuffle option is enabled
  • Updated documentation with usage example for the new shuffle option

Reviewed Changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated no comments.

File Description
package.json Added lodash.shuffle dependency for array randomization
lib/codecept.js Implemented test file shuffling logic when shuffle option is enabled
docs/commands.md Added documentation example for the new --shuffle option
Comments suppressed due to low confidence (1)

lib/codecept.js:185

  • The shuffle functionality lacks test coverage. Consider adding tests to verify that when the shuffle option is enabled, the test files are randomized while maintaining the same set of files.
    if (this.opts.shuffle) {

@kobenguyent kobenguyent merged commit 4daabc8 into codeceptjs:3.x Aug 19, 2025
11 of 12 checks passed
@kobenguyent kobenguyent mentioned this pull request Aug 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants