Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timeout operation priority according to the source #3127

Merged

Conversation

nikocanvacom
Copy link
Contributor

@nikocanvacom nikocanvacom commented Nov 13, 2021

Motivation/Description of the PR

  • Step timeout is set form multiple sources and need to be selected using following logic:
    • between step timeout set on action from code with I.limitTime and timeout set with configuration of stepTimeout plugin one from the code has priority unless stepTimeout.config.force===true
    • test/suite timeout counter update stepTimeout if it is smaller than stepTimeout set from other sources.
  • Resolves #issueId (if applicable).

Applicable helpers:

  • WebDriver
  • Puppeteer
  • Nightmare
  • REST
  • FileHelper
  • Appium
  • Protractor
  • TestCafe
  • Playwright

Applicable plugins:

  • allure
  • autoDelay
  • autoLogin
  • customLocator
  • pauseOnFail
  • coverage
  • retryFailedStep
  • screenshotOnFail
  • selenoid
  • stepByStepReport
  • stepTimeout
  • wdio
  • subtitles

Type of change

  • 🔥 Breaking changes
  • 🚀 New functionality
  • 🐛 Bug fix
  • 📋 Documentation changes/updates
  • ♨️ Hot fix
  • 🔨 Markdown files fix - not related to source code
  • 💅 Polish code

Checklist:

  • Tests have been added
  • Documentation has been added (Run npm run docs)
  • Lint checking (Run npm run lint)
  • Local tests are passed (Run npm test)

@nikocanvacom nikocanvacom force-pushed the niko-set-timeout-priorities-by-sources branch from a2e1ad8 to f879a76 Compare November 13, 2021 14:19
Copy link
Contributor

@DavertMik DavertMik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a question, so the timeout with the higher source is taken, right?
should we call it priority then?

lib/step.js Outdated
* @param {number} timeout - timeout in milliseconds or 0 if no timeout
* @param {number} source - 0 - test/suite, 1 - config force=true, 2 - code, 3 - config force=false
*/
this.setTotalTimeout = function (timeout, source = 3) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

source is not something that is really clear here.
should we call it priority?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, we could call it priority. lower priority value gives higher priority to timeout value. I thought of calling it priority but I ended up calling it source as the logic here has very specific expectations for levels/priorities for timeout values from different sources and I tried to think of scenario where some other source of timeout try to meddle with this and couldn't.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

other plugins could also use this API to add their timeouts... So the priority is cleaner. But if lower value makes things more prioritized then let's call it order. In this case, 0 should be understandable to be the first value to pull out.

@@ -85,6 +85,6 @@ module.exports = (config) => {
}
}
stepTimeout = stepTimeout === undefined ? config.timeout : stepTimeout;
step.totalTimeout = stepTimeout * 1000;
step.setTotalTimeout(stepTimeout * 1000, config.force ? 1 : 3);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does config.force do here?
I thought it will make global step timeouts more prioritized over local step timeouts. Does it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

force makes config values to override values set from code I.limitTime(t)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, got it! But the name is very unclear... How about a more verbose name?

ignoreStepLimits: true

lib/step.js Outdated
new Map([...timeouts.entries()].sort().reverse()).forEach((timeout, order) => {
if (timeout !== undefined && (
// when orders >= 10 - timeout value overrides those set with higher order elements
order >= 10
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this breaks my mind... Magical numbers 🧙 🔮
what the point of 10 number here?

Why not just the lower number is is more priority it gets?

lib/step.js Outdated
* @param {number} timeout - timeout in milliseconds or 0 if no timeout
* @param {number} order - <10 - test/suite, 10-19 - config when set to override timeouts from code, 20-29 - code, 30-39 - config
*/
this.setTimeout = function (timeout, order = 35) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's set 50 at the beginning
so it would be the lowest priority for now
35 is also magical number

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you have a look at the update in the last commit. It resolves the problem of really magical constant 10. Now the timeout override logic changes for orders below zero. Also as discussed introduced constants and jsdoc for them.

Copy link
Contributor

@DavertMik DavertMik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much!

@DavertMik DavertMik merged commit 9f0ca05 into codeceptjs:3.x Nov 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants