Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update cypress guidelines #2563

Merged
merged 1 commit into from
Jun 21, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 32 additions & 26 deletions cypress/CYPRESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Cypress is being used for both end-to-end tests and component tests.

SEE: [README](../README.md) for instructions on setting up and running tests
SEE: [README](../README.md) for instructions on setting up and running tests.

## Coverage

Expand All @@ -27,7 +27,10 @@ End-to-End tests for our project are located in the `cypress/e2e` directory. The
1. **Test Isolation and Independence**:

- Each test should be independent and capable of running on its own without relying on the state created by previous tests.
- If during deployment certain resources were created, those should not be deleted as part of the test runs.
- Ideally, tests should not rely on a previously created resource, and whenever possible, the test should create the resources it needs.
- There are several tests using `Global Organization` and `Global Project` that are created as part of the `login` process. This should be avoided whenever possible, as it was identified as a possible source of flakiness in the tests.
- Create the resources in the test itself or in a `before` block, and clean them up in an `after` block.
- If certain resources were created during deployment, those should not be deleted as part of the test runs.
- Avoid tests that require execution in a specific order.

2. **Environment**:
Expand All @@ -36,6 +39,7 @@ End-to-End tests for our project are located in the `cypress/e2e` directory. The
- Design tests to be environment-agnostic, ensuring they can run in development and production environments.

3. **Resource Management**:

- Each test should create all necessary resources for its execution and clean them up after completion.
- This includes data creation, user logins, and any other setup or teardown tasks.
- Leveraging the `before` and `after` hooks in Cypress can help manage resource creation and cleanup.
Expand All @@ -50,7 +54,7 @@ End-to-End tests for our project are located in the `cypress/e2e` directory. The
2. **Asynchronous Operations**:

- Properly handle the asynchronous nature of web applications, using Cypress's automatic waiting features or custom waits when necessary.
- Avoid to use `cy.wait` as much as possible - mainly with hardcoded wait values, prefer to use `cy.get` with assertions.
- Avoid using `cy.wait` as much as possible, especially with hardcoded wait values. Prefer to use `cy.get` with assertions.

**Example**:

Expand Down Expand Up @@ -82,29 +86,29 @@ End-to-End tests for our project are located in the `cypress/e2e` directory. The

- Utilize dynamic data creation methods to avoid conflicts with existing data.

4. **Resources creation**:
4. **Resource Creation**:

- Avoid creating resources in `beforeEach` or `before` blocks that are not used later in the `it` test block. For cases where the resource is not used during a `it` block then it is preferable to create a new `describe` block.
- Avoid creating resources in `beforeEach` or `before` blocks that are not used later in the `it` test block. For cases where the resource is not used during an `it` block, it is preferable to create a new `describe` block.

**NOT TO DO Example**:

```javascript
describe('Test Click Resource', () => {
let resourceA: Resource;
let resourceA;

beforeEach(() => {
cy.createResourceA().then((r) => {
resourceA = r;
});
});
beforeEach(() => {
cy.createResourceA().then((r) => {
resourceA = r;
});
});

it('click on the resource A button', () => {
//Click on the Resource A button
// Click on the Resource A button
cy.getByDataCy('resource-a').should('contain', resourceA).click();
});

it('click on the resource B button', () => {
//Click on the Resource B button
// Click on the Resource B button
cy.getByDataCy('resource-b').click();
});
});
Expand All @@ -114,23 +118,23 @@ End-to-End tests for our project are located in the `cypress/e2e` directory. The

```javascript
describe('Test Click Resource A', () => {
let resourceA: Resource;
let resourceA;

beforeEach(() => {
cy.createResourceA().then((r) => {
resourceA = r;
});
});
beforeEach(() => {
cy.createResourceA().then((r) => {
resourceA = r;
});
});

it('click on the resource A button', () => {
//Click on the Resource A button
cy.getByDataCy('resource').should('contain', resourceA).click();
// Click on the Resource A button
cy.getByDataCy('resource-a').should('contain', resourceA).click();
});
});

describe('Test Click Resource B', () => {
it('click on the resource B button', () => {
//Click on the Resource B button
// Click on the Resource B button
cy.getByDataCy('resource-b').click();
});
});
Expand All @@ -148,6 +152,7 @@ End-to-End tests for our project are located in the `cypress/e2e` directory. The
- Be cautious with sensitive data in tests, using mock or anonymized data where possible.

3. **Performance Considerations**:

- Keep tests efficient to minimize impact on the development and deployment process.

### Documentation and Collaboration
Expand All @@ -158,15 +163,16 @@ End-to-End tests for our project are located in the `cypress/e2e` directory. The
- Include usage examples and explanations of the purpose of each command.

2. **Team Collaboration**:

- Keep the team informed about testing strategies and updates.
- Encourage team members to contribute to the testing process and documentation.

By following these guidelines, our E2E tests will be more robust, reliable, and aligned with the best practices in automated testing.

## AWX Cleanup during tests
## AWX Cleanup during Tests

Many AWX resources have an organization and if the organization is deleted the resource gets deleted. By creating an organization at the start of the tests and using it with resources and then deleting the organization, most resources clean up.
Many AWX resources have an organization, and if the organization is deleted, the resource gets deleted. By creating an organization at the start of the tests and using it with resources, then deleting the organization, most resources clean up.

There are a few cases where resources do not get cleaned up, such as when a project is syncing, deleting the organization leaves the project around without an associated organization.
There are a few cases where resources do not get cleaned up, such as when a project is syncing. Deleting the organization leaves the project around without an associated organization.

To handle this, the tests delete E2E organizations older than 2 hours old. Then the other tests delete resources that do not have an organization. This makes the system eventually cleanup the E2E resources.
To handle this, the tests delete E2E organizations older than 2 hours. Then the other tests delete resources that do not have an organization. This makes the system eventually clean up the E2E resources.
nixocio marked this conversation as resolved.
Show resolved Hide resolved
Loading