Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce defaultCommandTimeout configuration for E2E tests #8956

Closed
cnotv opened this issue May 23, 2023 · 2 comments · Fixed by #10863
Closed

Reduce defaultCommandTimeout configuration for E2E tests #8956

cnotv opened this issue May 23, 2023 · 2 comments · Fixed by #10863
Assignees
Labels
area/e2e area/test Test (e2e and unit) kind/tech-debt Technical debt QA/dev-automation Issues that engineers have written automation around so QA doesn't have look at this
Milestone

Comments

@cnotv
Copy link
Contributor

cnotv commented May 23, 2023

Description

As temporary solution, last year we increased the defaultCommandTimeout of the waiting response for the E2E tests, giving the chance to solve the cause of the delay.
The vast majority of the errors are although covered in a very shorter time, still keeps the whole process too long.

Reduce timeout to default (2s) or a similar value, instead of current (60s).

To be solved after #9040

@cnotv cnotv added area/test Test (e2e and unit) kind/tech-debt Technical debt labels May 23, 2023
@cnotv cnotv added this to the v2.7.next3 milestone May 23, 2023
@cnotv cnotv added the area/e2e label May 23, 2023
@cnotv cnotv added the QA/dev-automation Issues that engineers have written automation around so QA doesn't have look at this label Jun 22, 2023
@gaktive gaktive modified the milestones: v2.7.next3, v2.8.next4 Jul 19, 2023
@nwmac nwmac modified the milestones: v2.8.0, v2.8.next1 Aug 15, 2023
@cnotv cnotv self-assigned this Oct 10, 2023
@zube zube bot removed the [zube]: Backlog label Oct 10, 2023
@cnotv
Copy link
Contributor Author

cnotv commented Oct 10, 2023

Although in case of errors it avoids to wait between 2 to 3 times the normal time, overall it does not seems to reduce so much on an average of many errors:

  • defaultCommandTimeout 4s, with a lot of errors (CI, dashboard)
    • admin: 59m 18s
    • user: 14m 43s
  • Current defaultCommandTimeout 1min, only 1 flaky case which is kinda best scenario (CI, dashboard)
    • admin 1h 5m 46s
    • user 24m 46s

Branch comparison: master...cnotv:dashboard:feature/8956-reduce-e2e-timeout

PS: For this scenario have been reduced/removed requestTimeout, wait() commands and defaultCommandTimeout.

@cnotv
Copy link
Contributor Author

cnotv commented Oct 11, 2023

In another attempt I tried to remove any fixed waiting time I crossed, although it did not produce very huge improvements as well:

Given these circumstances, we may want to simply tackle down flaky tests issues and handle performances issues (mainly API responses).

We could reduce safely this timeout afterwards just for the flaky/errors sake.
#9233 needs to be completed before start this task then.

@zube zube bot removed the [zube]: Working label Oct 11, 2023
@cnotv cnotv modified the milestones: v2.8.next1, v2.8.x Oct 11, 2023
@cnotv cnotv changed the title Reduce timeout configuration for E2E tests Reduce defaultCommandTimeout configuration for E2E tests Jan 12, 2024
@nwmac nwmac modified the milestones: v2.8.x, v2.9.0 Feb 14, 2024
@nwmac nwmac modified the milestones: v2.9.0, v2.9.x Feb 27, 2024
@richard-cox richard-cox removed this from the v2.9.x milestone Apr 23, 2024
@richard-cox richard-cox added this to the v2.9.0 milestone Apr 23, 2024
@nwmac nwmac assigned nwmac and unassigned cnotv Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/e2e area/test Test (e2e and unit) kind/tech-debt Technical debt QA/dev-automation Issues that engineers have written automation around so QA doesn't have look at this
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants