Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Development: Disable redundant cypress tests #8431

Merged
merged 1 commit into from
Apr 27, 2024

Conversation

muradium
Copy link
Contributor

@muradium muradium commented Apr 18, 2024

Checklist

General

Client

Motivation and Context

We migrated our E2E test suite from Cypress to Playwright. As a result, most of the tests in Cypress are now redundant and we want to disable them.

Description

This PR disables majority of the tests in Cypress and only leaves the ones that either don't exist in Playwright yet or are currently faulty in Playwright.

Steps for Testing

  • Code Review: Ensure that disabled/skipped cypress tests have equivalent tests in Playwright.

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked







Review Progress

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1
  • Test 2

Summary by CodeRabbit

  • Tests
    • Temporarily disabled execution of various test suites to prioritize critical testing areas.
    • Enabled exclusive focus on specific test suites for detailed validation in areas like course icon deletion and quiz exercise assessment.

@muradium muradium requested a review from a team as a code owner April 18, 2024 13:38
@github-actions github-actions bot added tests cypress Pull requests that update cypress tests. (Added Automatically!) labels Apr 18, 2024
Copy link

coderabbitai bot commented Apr 18, 2024

Walkthrough

The recent modifications primarily focus on managing the execution of test suites in a Cypress environment. Numerous test suites across different modules like exams, courses, and exercises have been selectively skipped or isolated for execution. This includes significant use of describe.skip to bypass certain tests and describe.only to exclusively run specific tests, streamlining the testing process and likely focusing on critical areas or functionalities.

Changes

File Path Change Summary
.../Login.cy.ts Skipped tests related to the login page.
.../CourseCommunication.cy.ts Isolated a test case focusing on instructor's interactions.
.../course/... Skipped multiple test suites related to courses (CourseExercise, CourseMessages, etc.).
.../course/CourseManagement.cy.ts Isolated the test suite for 'Course icon deletion'.
.../exam/... Skipped numerous exam-related test suites (ExamCreationDeletion, ExamDateVerification, etc.).
.../exam/ExamAssessment.cy.ts Isolated the test suite for 'Quiz exercise assessment'.
.../exercises/... Skipped various exercise-related test suites across different types (FileUpload, Modeling, Programming, etc.).
.../exercises/quiz-exercise/QuizExerciseDropLocation.cy.ts Toggled the execution state and focused on specific test case within the suite.
.../lecture/LectureManagement.cy.ts Skipped the 'Lecture management' test suite.

This table groups files with similar changes, providing a clear and concise summary of the modifications made to the test execution strategy in the Cypress test suites.


Recent Review Details

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between adfbd38 and 2c91f32.
Files selected for processing (35)
  • src/test/cypress/e2e/Login.cy.ts (1 hunks)
  • src/test/cypress/e2e/course/CourseCommunication.cy.ts (1 hunks)
  • src/test/cypress/e2e/course/CourseExercise.cy.ts (1 hunks)
  • src/test/cypress/e2e/course/CourseManagement.cy.ts (1 hunks)
  • src/test/cypress/e2e/course/CourseMessages.cy.ts (1 hunks)
  • src/test/cypress/e2e/exam/ExamAssessment.cy.ts (1 hunks)
  • src/test/cypress/e2e/exam/ExamCreationDeletion.cy.ts (1 hunks)
  • src/test/cypress/e2e/exam/ExamDateVerification.cy.ts (1 hunks)
  • src/test/cypress/e2e/exam/ExamManagement.cy.ts (1 hunks)
  • src/test/cypress/e2e/exam/ExamParticipation.cy.ts (1 hunks)
  • src/test/cypress/e2e/exam/ExamTestRun.cy.ts (1 hunks)
  • src/test/cypress/e2e/exam/test-exam/TestExamCreationDeletion.cy.ts (1 hunks)
  • src/test/cypress/e2e/exam/test-exam/TestExamManagement.cy.ts (1 hunks)
  • src/test/cypress/e2e/exam/test-exam/TestExamParticipation.cy.ts (1 hunks)
  • src/test/cypress/e2e/exam/test-exam/TestExamStudentExams.cy.ts (1 hunks)
  • src/test/cypress/e2e/exam/test-exam/TestExamTestRun.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/ExerciseImport.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/file-upload/FileUploadExerciseAssessment.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/file-upload/FileUploadExerciseManagement.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/file-upload/FileUploadExerciseParticipation.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/modeling/ModelingExerciseAssessment.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/modeling/ModelingExerciseManagement.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/modeling/ModelingExerciseParticipation.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/programming/ProgrammingExerciseAssessment.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/programming/ProgrammingExerciseManagement.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/programming/ProgrammingExerciseParticipation.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/programming/ProgrammingExerciseStaticCodeAnalysis.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/quiz-exercise/QuizExerciseAssessment.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/quiz-exercise/QuizExerciseDropLocation.cy.ts (2 hunks)
  • src/test/cypress/e2e/exercises/quiz-exercise/QuizExerciseManagement.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/quiz-exercise/QuizExerciseParticipation.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/text/TextExerciseAssessment.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/text/TextExerciseManagement.cy.ts (1 hunks)
  • src/test/cypress/e2e/exercises/text/TextExerciseParticipation.cy.ts (1 hunks)
  • src/test/cypress/e2e/lecture/LectureManagement.cy.ts (1 hunks)
Files skipped from review due to trivial changes (27)
  • src/test/cypress/e2e/Login.cy.ts
  • src/test/cypress/e2e/course/CourseExercise.cy.ts
  • src/test/cypress/e2e/course/CourseMessages.cy.ts
  • src/test/cypress/e2e/exam/ExamCreationDeletion.cy.ts
  • src/test/cypress/e2e/exam/ExamManagement.cy.ts
  • src/test/cypress/e2e/exam/ExamParticipation.cy.ts
  • src/test/cypress/e2e/exam/ExamTestRun.cy.ts
  • src/test/cypress/e2e/exam/test-exam/TestExamManagement.cy.ts
  • src/test/cypress/e2e/exam/test-exam/TestExamStudentExams.cy.ts
  • src/test/cypress/e2e/exam/test-exam/TestExamTestRun.cy.ts
  • src/test/cypress/e2e/exercises/ExerciseImport.cy.ts
  • src/test/cypress/e2e/exercises/file-upload/FileUploadExerciseAssessment.cy.ts
  • src/test/cypress/e2e/exercises/file-upload/FileUploadExerciseManagement.cy.ts
  • src/test/cypress/e2e/exercises/file-upload/FileUploadExerciseParticipation.cy.ts
  • src/test/cypress/e2e/exercises/modeling/ModelingExerciseAssessment.cy.ts
  • src/test/cypress/e2e/exercises/modeling/ModelingExerciseManagement.cy.ts
  • src/test/cypress/e2e/exercises/programming/ProgrammingExerciseAssessment.cy.ts
  • src/test/cypress/e2e/exercises/programming/ProgrammingExerciseManagement.cy.ts
  • src/test/cypress/e2e/exercises/programming/ProgrammingExerciseParticipation.cy.ts
  • src/test/cypress/e2e/exercises/programming/ProgrammingExerciseStaticCodeAnalysis.cy.ts
  • src/test/cypress/e2e/exercises/quiz-exercise/QuizExerciseAssessment.cy.ts
  • src/test/cypress/e2e/exercises/quiz-exercise/QuizExerciseManagement.cy.ts
  • src/test/cypress/e2e/exercises/quiz-exercise/QuizExerciseParticipation.cy.ts
  • src/test/cypress/e2e/exercises/text/TextExerciseAssessment.cy.ts
  • src/test/cypress/e2e/exercises/text/TextExerciseManagement.cy.ts
  • src/test/cypress/e2e/exercises/text/TextExerciseParticipation.cy.ts
  • src/test/cypress/e2e/lecture/LectureManagement.cy.ts
Additional comments not posted (8)
src/test/cypress/e2e/exercises/modeling/ModelingExerciseParticipation.cy.ts (1)

8-8: Skipped the entire test suite for "Modeling Exercise Participation".

Ensure that the corresponding Playwright tests cover all functionalities tested here. Consider removing this file if it becomes obsolete.

src/test/cypress/e2e/exercises/quiz-exercise/QuizExerciseDropLocation.cy.ts (2)

9-9: Toggled the test suite "Quiz Exercise Drop Location Spec" from skipped to executed.

Verify that this change is intentional and confirm that the corresponding Playwright tests are not yet ready or need adjustments.


25-25: Enabled the test "Checks drop locations" within the "Quiz Exercise Drop Location Spec".

Ensure this test is critical to run at this stage and check if the Playwright suite has a corresponding test.

src/test/cypress/e2e/exam/test-exam/TestExamCreationDeletion.cy.ts (1)

26-26: Skipped the test suite "Test Exam creation/deletion".

Confirm that all functionalities covered by this suite are adequately tested in the Playwright tests. Consider removing this file if it becomes obsolete.

src/test/cypress/e2e/exam/ExamDateVerification.cy.ts (1)

20-20: Skipped the test suite "Exam date verification".

Ensure that the corresponding Playwright tests cover all scenarios tested here. If this suite is no longer needed, consider its removal.

src/test/cypress/e2e/exam/test-exam/TestExamParticipation.cy.ts (1)

17-17: Skipped the test suite "Test exam participation".

Verify that the Playwright tests are in place to cover the functionalities tested in this suite. Consider removing this file if it becomes obsolete.

src/test/cypress/e2e/exam/ExamAssessment.cy.ts (1)

134-134: Isolated the test suite "Quiz exercise assessment" to run exclusively.

Confirm that this change is intentional and check if it's necessary to focus exclusively on this suite, potentially affecting other tests.

src/test/cypress/e2e/course/CourseManagement.cy.ts (1)

255-255: Isolated the test suite "Course icon deletion" to run exclusively.

Ensure that isolating this suite is necessary and does not unintentionally skip other important tests in this file.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

@@ -47,7 +47,7 @@ courseConfigsToTest.forEach((configToTest) => {
});
});

it('instructor should be able to select answer', () => {
it.only('instructor should be able to select answer', () => {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

The verification process has confirmed that the it.only method is used exclusively in the CourseCommunication.cy.ts file to focus on a specific test case. This usage could unintentionally prevent other tests in the suite from running when the test suite is executed, which might not align with the typical goals of maintaining comprehensive test coverage in a production environment.

Please review and confirm whether the inclusion of it.only is intended for the final merge or if it should be removed to allow all tests in the suite to run.

  • File and Line: src/test/cypress/e2e/course/CourseCommunication.cy.ts line 50
Analysis chain

Focus on a specific test case using it.only.

The use of it.only here isolates this test case, ensuring that only this test is executed when the test suite runs. This is typically used during development for debugging or focusing on a specific issue. If this change is intended for the final merge, it might prevent other tests in the suite from running, which could reduce test coverage unintentionally. Please confirm if this change is meant to be temporary or if it should be included in the final commit.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Search for other occurrences of `it.only` in the Cypress test files to ensure it's not unintentionally left in other tests.
rg --type ts 'it.only' src/test/cypress

Length of output: 167

@Strohgelaender
Copy link
Contributor

Since we want to migrate away from Cypress, would it make sense to remove these tests already instead of skipping them?

Copy link

There hasn't been any activity on this pull request recently. Therefore, this pull request has been automatically marked as stale and will be closed if no further activity occurs within seven days. Thank you for your contributions.

@github-actions github-actions bot added the stale label Apr 26, 2024
@krusche krusche changed the title General: Disable redundant cypress tests Development: Disable redundant cypress tests Apr 27, 2024
@krusche krusche added this to the 7.0.2 milestone Apr 27, 2024
@krusche krusche merged commit c614aee into develop Apr 27, 2024
27 of 31 checks passed
@krusche krusche deleted the test/cypress/disable-redundant-tests branch April 27, 2024 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cypress Pull requests that update cypress tests. (Added Automatically!) stale tests
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

3 participants