Update guides for Cy 13, where assertions are now commands #5081
Conversation
* .readFile() is now a query * Add section on new .readFile() superpowers * Fix page slug for readFile * Prettier run
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
Note: The build here is failing because the updated guides references a new URL, /api/table-of-contents, which is part of #5053. I've opted to leave it failing for now, so that I don't have to keep track of updating URLs as a separate task. Once the other PR is merged in, this build will pass. It is ready for review as-is. |
| to non-deterministic tests - different runs may behave differently, which makes | ||
| them less consistent and useful for verifying your application's correctness. In | ||
| general, there are only a handful of very specific situations where you can | ||
| create control flow using Cypress commands. | ||
|
|
||
| With that said, as long as you are aware of the potential pitfalls with control |
There was a problem hiding this comment.
lol not sure I know what these pitfalls are... i know you didn't write this, but do you know if there is more info on the conditional testing page about these?
There was a problem hiding this comment.
I think they're laid out here - https://docs.cypress.io/guides/core-concepts/conditional-testing#The-problem.
I don't find that page terribly compelling, but not something I want to mess with too much here.
| working properly across the client and server! | ||
| With Cypress, you don't have to write explicit assertions to have a useful test. | ||
| Without a single `expect()` or `.should()`, a few lines of Cypress can ensure | ||
| thousands of lines of code are working properly across the client and server. |
There was a problem hiding this comment.
I actually like the explication points in this guide since it's the intro and trying to get you hyped 🙃
There was a problem hiding this comment.
Hm, I tend to prefer a more steady tone - hype turns me away from docs, and would prefer for features to speak for themselves.
|
|
||
| This is because many commands have a built in | ||
| [Default Assertion](#Default-Assertions) which offer you a high level of | ||
| guarantee. | ||
| [Implicit Assertions](#Implicit-Assertions) which offer you a high level of |
There was a problem hiding this comment.
feels like overkill to link to the next section.
There was a problem hiding this comment.
True. Removing link.
| @@ -1160,7 +1142,7 @@ they can fail, typically by passing a `{force: true}` option. | |||
|
|
|||
| ```js | |||
| cy | |||
There was a problem hiding this comment.
above where it says "all dom commands automatically wait" should that be queries?
There was a problem hiding this comment.
No; there are commands that interact with the DOM (like .click()) that aren't queries which wait, and queries (like .readFile()) which aren't related to the DOM at all.
|
|
||
| ::: | ||
|
|
||
| ### Default Assertions | ||
| ### Implicit Assertions | ||
|
|
||
| Many commands have default, built-in assertions, or rather have requirements | ||
| that may cause it to fail without needing an explicit assertion you've added. |
There was a problem hiding this comment.
should we mention each command will outline these in their docs?
There was a problem hiding this comment.
I'm not sure that needs to be said explicitly - just feels like words for the sake of words.
Would you as a new user not expect each command page's to explain it?
| @@ -1385,10 +1366,10 @@ Remember because assertions are used to describe a condition of the previous | |||
| commands - the `timeout` modification goes on the previous commands _not the | |||
| assertions_. | |||
|
|
|||
| #### Example #1: Default Assertion | |||
| #### Example #1: Implicit Assertion | |||
There was a problem hiding this comment.
Above the section:
### Writing Assertions
There are two ways to write assertions in Cypress:
1. **Implicit Subjects:** Using [`.should()`](/api/commands/should) or
[`.and()`](/api/commands/and).
2. **Explicit Subjects:** Using `expect`.
why is .should()/.and() an implicit subject? it'd just be a queued assertion vs sync assertion right?
There was a problem hiding this comment.
Yeah, I think that section could use a touch-up. Rewriting it.
| While all methods you chain off of `cy` in your Cypress tests are commands, it's | ||
| important to understand the different rules by which they operate. **Queries** | ||
| link up, retrying the entire chain together, while **non-queries** only execute | ||
| once. Assertions are a type of query that's specially displayed in the command |
There was a problem hiding this comment.
Nit: bullet point each type to make this easier to read.
| once. Assertions are a type of query that's specially displayed in the command | |
| once. **Assertions** are a unique query such that it ensures an explicit condition is met to continue executing the Command Queue. |
There was a problem hiding this comment.
Bullet-pointed the types.
| important to understand the different rules by which they operate. **Queries** | ||
| link up, retrying the entire chain together, while **non-queries** only execute | ||
| once. Assertions are a type of query that's specially displayed in the command | ||
| log. For example, there are 7 queries (2 of them assertions) and 2 non-queries |
There was a problem hiding this comment.
IMO the logic of assertions are in parallel with query commands under the hood, but they act differently, so keeping the concepts separate makes it easier to digest their purposes and overall behaviors.
There was a problem hiding this comment.
They don't act differently any more; that's the whole point of this work.
Assertions are 100% queries in all ways - they are implemented as them and they behave as them, because they are them.
| cy.visit('/') | ||
|
|
||
| cy.focused() // query | ||
| .should('have.class', 'new-todo') // assertion | ||
| // The queries .focused() and .should() link together, |
There was a problem hiding this comment.
| // The queries .focused() and .should() link together, | |
| // The .focused() query and .should() assertion link together, |
There was a problem hiding this comment.
.should() is a query, and I'm very intentionally treating it as one in the text. That's why I went through these guides in the first place, to make sure that point is built into our docs.
There was a problem hiding this comment.
taking this suggestion as should() is still an assertion
| - 🚨 If the assertion fails, then Cypress will requery the application's DOM | ||
| again - starting from the top of the chain of linked queries. It will look for | ||
| elements that match `.get().find()` again, and re-run the assertion. If the | ||
| assertion still fails, Cypress continues retrying until the timeout is |
There was a problem hiding this comment.
| assertion still fails, Cypress continues retrying until the timeout is | |
| assertion still fails, Cypress continues retrying the query chain until the timeout is |
There was a problem hiding this comment.
That's already been mentioned twice in the previous two sentences, I'm not sure stating it a third time in a new way is necessary.
| .and(($li) => { | ||
| // 2 more assertions | ||
| // 2 expect() statements inside of the .and() query |
There was a problem hiding this comment.
| // 2 expect() statements inside of the .and() query | |
| // 2 explicit assertions inside of the .and() query |
There was a problem hiding this comment.
Done. Called them 'mocha assertions' rather than 'explicit assertions'.
| Command Log correctly shows that the first encountered assertion | ||
| `should('have.length', 2)` passed, but the second assertion and the command | ||
| itself failed. | ||
| Because the first expect statement |
There was a problem hiding this comment.
| Because the first expect statement | |
| If the first expect statement |
There was a problem hiding this comment.
We're not talking hypothetically ("if") - we have a specific example, and directly below a gif of it executing. "Because" feels appropriate here.
| itself failed. | ||
| Because the first expect statement | ||
| (`expect($li.get(0).textContent, 'first item').to.equal('todo a')`) fails, the | ||
| second statement is never reached. The command fails after timing out, and the |
There was a problem hiding this comment.
| second statement is never reached. The command fails after timing out, and the | |
| second statement is never reached. So the `.and()` command fails after timing out, and the |
There was a problem hiding this comment.
Tweaked the wording a bit.
| Command Log correctly shows that `.get()` and `.should()` passed, but the "first | ||
| item" assertion failed. |
There was a problem hiding this comment.
I expected this to say something like the explicit assertion will not cause the .and command to retry"...going into the explanation of the command seems like expected behavior (even though I think we just fixed this).
There was a problem hiding this comment.
I'm not sure I'm following you here. The expect() statement is inside the callback function for .and() - it 100% does cause it to retry, and keep retrying until it times out.
| queries to be retried. For example, the [`.eq()`](/api/commands/eq) query will | ||
| be retried even without any attached assertions until it finds an element with | ||
| the given index. | ||
| Often a Cypress command has built-in assertions that will cause the command to |
There was a problem hiding this comment.
| Often a Cypress command has built-in assertions that will cause the command to | |
| Often a Cypress command has [Implicit Assertions](link) that will cause the command to |
There was a problem hiding this comment.
I'm not sure it makes sense to link back to the "Getting Started" guide here - this is almost the same content, but written this time with a more granular focus on how commands and queries retry. This is already more detailed than the place it would link.
I have renamed the section "Implicit Assertions" rather than "Built-in Assertions" to better match the terminology in the other guide.
| @@ -187,7 +184,7 @@ cy.get('.todo-list li') // query | |||
| /> | |||
|
|
|||
| Only queries can be retried, but most other commands still have built-in | |||
| _waiting_. For example, as described in the "Assertions" section of | |||
| assertions. For example, as described in the "Assertions" section of | |||
There was a problem hiding this comment.
From a marketing perspective, I do think keeping references to auto-awaiting is good buzz words to keep in here even if implicit assertions are how we accomplish this
| @@ -201,8 +198,9 @@ Cypress tries to act like a human user would using the browser. | |||
| - Is the element behind another element? | |||
| - Does the element have the `disabled` attribute? | |||
|
|
|||
| Actions - such as `.click()` - automatically wait until multiple built-in | |||
| assertions like these pass, and then it will attempt to click just once. | |||
| Action commands - such as `.click()` - automatically wait until multiple | |||
There was a problem hiding this comment.
This is the first mention of action commands, also missing link. Seems like it should be bolded
There was a problem hiding this comment.
Link added. I haven't bolded it though, randomly bolded words are a pet peeve of mine.
| Actions - such as `.click()` - automatically wait until multiple built-in | ||
| assertions like these pass, and then it will attempt to click just once. | ||
| Action commands - such as `.click()` - automatically wait until multiple | ||
| built-in assertions like these pass, and then it will attempt to click just |
There was a problem hiding this comment.
| built-in assertions like these pass, and then it will attempt to click just | |
| built-in assertions like these pass, and then it will only ever be attempted once. |
mixing example and def of action command
There was a problem hiding this comment.
Good catch. Applied as
and then it will attempt the action once.
to keep active voice.
| form the subject for later commands the way queries do. Cypress will retry any | ||
| queries _leading up to_ a command, and retry any assertions _after_ a command, | ||
| but commands themselves only execute once. After they've executed, nothing |
There was a problem hiding this comment.
| form the subject for later commands the way queries do. Cypress will retry any | |
| queries _leading up to_ a command, and retry any assertions _after_ a command, | |
| but commands themselves only execute once. After they've executed, nothing | |
| form the subject for later commands the way queries do. Cypress will retry any | |
| queries _leading up to_ a failed query, | |
| but commands themselves only execute once. After they've executed, nothing |
There was a problem hiding this comment.
This is an incorrect edit. I'm talking here about non-query commands, and we do in fact retry queries leading up to the command.
cy.get().children().next().click().parent().should() - we retry the queries leading up to the command (.get().children().next()) while waiting for the implicit assertions to pass. Then we retry the queries after it (.parent().should()).
That's what this paragraph is trying to explain.
|
|
||
| #### <Icon name="check-circle" color="green" /> Correctly waiting for the stub to be called | ||
|
|
||
| We recommend aliasing the stub using the [`.as`](/api/commands/as) command and | ||
| using `cy.get('@alias').should(...)` assertions. | ||
| using `cy.get('@alias')` to run assertions. |
There was a problem hiding this comment.
| using `cy.get('@alias')` to run assertions. | |
| using `cy.get('@alias')` to re-run a set of queries and assertions. |
There was a problem hiding this comment.
This change doesn't make much sense to me - we're giving advice directly on how to run assertions about stubs.
|
|
||
| See the [Retry-Ability Guide](/guides/core-concepts/retry-ability) for more | ||
| details on how queries chain together and retry. | ||
|
|
There was a problem hiding this comment.
Should there also be a section about not being able to overwrite .should() and .and() with Cypress.Commands.overwrite() anymore, similar to the one for .readFile()?
There was a problem hiding this comment.
changes should be removed due to cypress-io/cypress#25738
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
… into issue-25134-should-as-guide-updates
… and and() will not be queries in v13. see cypress-io/cypress#25738
… is no longer applicable due to cypress-io/cypress#25738
* BREAKING: update documentation by setting default video: false for v13.0.0 (#5300) * BREAKING: update documentation by setting default video: false * Update docs/guides/overview/why-cypress.mdx Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> * Update docs/guides/end-to-end-testing/migration/protractor-to-cypress.mdx Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> * Update docs/guides/continuous-integration/introduction.mdx Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> * Update docs/guides/guides/screenshots-and-videos.mdx Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> * chore: re-order video bits * Update docs/guides/end-to-end-testing/migration/protractor-to-cypress.mdx * chore: linting --------- Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> * breaking: update documentation by setting videoCompression: false (#5301) * BREAKING: update documentation by setting default video: false * BREAKING: update documentation by setting videoCompression: false * Update docs/guides/guides/screenshots-and-videos.mdx Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com> * apply suggestion from code review * chore: attempt to rework the video encoding section * Update docs/guides/guides/screenshots-and-videos.mdx Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> * chore: address comments in code review * chore: update video encoding guide to flow a bit more evenly --------- Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com> Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> * BREAKING: remove the videoUploadOnPasses configuration option (#5306) * BREAKING: remove the videoUploadOnPasses configuration option * chore: add history entry of removed videoUploadOnPasses config option * .readFile() is now a query (#5017) (#5316) * .readFile() is now a query * Add section on new .readFile() superpowers * Fix page slug for readFile * Prettier run Co-authored-by: Blue F <blue@everblue.info> * Update guides for Cy 13, where assertions are now commands (#5081) * .readFile() is now a query (#5017) * .readFile() is now a query * Add section on new .readFile() superpowers * Fix page slug for readFile * Prettier run * Update guides to better support users in Cy 13, where assertions are now commands * Review updates * Apply suggestions from code review Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> * Partial review feedback (more to come) * Review updates * Prettier run * fix: update retry ability guide and migration guide now that should() and and() will not be queries in v13. see cypress-io/cypress#25738 * chore: correct number of queries in example * revert: images changes due to should() and and() being queries, which is no longer applicable due to cypress-io/cypress#25738 * chore: lint guides --------- Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> Co-authored-by: Bill Glesias <bglesias@gmail.com> * Apply suggestions from code review Co-authored-by: Mike McCready <66998419+MikeMcC399@users.noreply.github.com> * Remove nodeVersion documentation (#5318) * Remove reference to 'nodeVersion' * added history, removed changes to legacy * docs: v13 video and TR migration info (#5383) * added video config migration info * added tr migration info * PR feedback * PR feedback * PR feedback * Apply suggestions from code review Co-authored-by: Bill Glesias <bglesias@gmail.com> * Update docs/guides/references/migration-guide.mdx Co-authored-by: Jennifer Shehane <jennifer@cypress.io> --------- Co-authored-by: Bill Glesias <bglesias@gmail.com> Co-authored-by: Jennifer Shehane <jennifer@cypress.io> * Update results for module API and after:spec handler (#5379) * docs: adding documentation for runner-ui cli flags (#5424) * docs: test replay docs (#5349) * sidebar content shuffle * main feature content & link updates * add screenshots * add url to vpn whitelist * add TR banner * add & reorder faq * build test replay partial * add TR partial to related content * add TR link to related "see more" sections * content tweaks * include TR content in body copy * add troubleshooting * remove old images * add PR feedback * add command log notes * remove dead link * update images * replace runs page screenshots * Apply suggestions from code review Co-authored-by: Jennifer Shehane <jennifer@cypress.io> Co-authored-by: Bill Glesias <bglesias@gmail.com> Co-authored-by: Jennifer Shehane <jennifer@cypress.io> * add more faqs * restore missing partial export * fix links * swap in screenshot of new network panel * Apply suggestions from code review Co-authored-by: Bill Glesias <bglesias@gmail.com> * fix command line merge issue * lint * Update docs/guides/cloud/debugging/recorded-runs.mdx Co-authored-by: Bill Glesias <bglesias@gmail.com> * Apply suggestions from code review Co-authored-by: Bill Glesias <bglesias@gmail.com> --------- Co-authored-by: Jennifer Shehane <jennifer@cypress.io> Co-authored-by: Bill Glesias <bglesias@gmail.com> * Fix JavaScript caps * Fix caps of JavaScript * Fix typo * Write a document around Cloud data storage and controls (#5425) Co-authored-by: Paul Jaffre <jaffrepaul@gmail.com> Co-authored-by: Mike McCready <66998419+MikeMcC399@users.noreply.github.com> * add changelog, remove node 16 support, and update docker image refere… (#5434) * add changelog, remove node 16 support, and update docker image reference to be on at least node 18 * Update docs/guides/references/changelog.mdx * docs: minor v13 updates (#5438) * match data callout with data storage page * remove mention from artifacts panel * remove submitting feature requests callout * build & insert cloud free plan partial * docs: update v13 content urls (#5428) * update content urls * add todo * replace v13 changelog urls * docs: v13 changelog summary (#5441) * changelog summary * Update docs/guides/references/changelog.mdx Co-authored-by: Jennifer Shehane <jennifer@cypress.io> * tweak verbaige & lint --------- Co-authored-by: Jennifer Shehane <jennifer@cypress.io> * Apply suggestions from code review Co-authored-by: Bill Glesias <bglesias@gmail.com> * lint * Update docs/faq/questions/cloud-faq.mdx Co-authored-by: Mike McCready <66998419+MikeMcC399@users.noreply.github.com> * include TR note in cypress screenshot api options (#5445) * docs: additional test replay content (#5447) * add TR cost faq * add centos troubleshooting * sync up changelog * Update _cloud_free_plan.mdx minor text update --------- Co-authored-by: Bill Glesias <bglesias@gmail.com> Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com> Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com> Co-authored-by: Blue F <blue@everblue.info> Co-authored-by: Mike McCready <66998419+MikeMcC399@users.noreply.github.com> Co-authored-by: Paul Jaffre <jaffrepaul@gmail.com> Co-authored-by: Jennifer Shehane <jennifer@cypress.io> Co-authored-by: Matt Schile <mschile@cypress.io> Co-authored-by: Chris Breiding <chrisbreiding@gmail.com> Co-authored-by: Paul Jaffre <paul@cypress.io>
This PR has been pulled out of #5053, in order to ease review of different sections.
In this PR, I've updated the our various
/guides/to help users understand how assertions and retryability work in Cypress. #5053 contains reorganization and updates of the/api/docs.