Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: add snapshot testing (this is not screenshot diffing) #772

Open
bahmutov opened this issue Oct 19, 2017 · 9 comments

Comments

@bahmutov
Copy link
Collaborator

commented Oct 19, 2017

Similar to Jest / Ava snapshot testing (see https://glebbahmutov.com/blog/snapshot-testing/ and https://glebbahmutov.com/blog/picking-snapshot-library/) it would be nice to save / compare values to snapshots.

For example this test can save the wrapped objects on the first run, then we would check in the generated snapshots.js file. Next time Cypress runs it would compare new values with saved snapshots and throw an error if they are different.

Here is a proof of concept

describe('my tests', () => {
  it('works', () => {
    cy.log('first snapshot')
    cy.wrap({foo: 42}).snapshot()

    cy.log('second snapshot')
    cy.wrap({bar: 101}).snapshot()
  })
})

The test runner

screen shot 2017-10-19 at 1 52 55 am

When clicking on "snapshot" in the reporter, it shows the object in the dev tools

Similarly, if previous command yields a jQuery element, then it is converted into JSON and saved. For example, we could snapshot focused element before and after typing (TodoMVC example at todomvc.com/examples/react/)

describe('focused input field', () => {
  it('is empty and then typed into', () => {
    cy.visit('http://todomvc.com/examples/react/')
    cy.focused()
      .snapshot('initial') // given human names as well
      .type('eat healthy breakfast')
      .snapshot('after typing')
  })
})

When clicking on the snapshot in this case, prints original element and JSON object

screen shot 2017-10-19 at 1 53 07 am

The snapshots are all loaded at the start and saved together at the end. Here is snapshot file

module.exports = {
  "my tests": {
    "works": {
      "1": {
        "foo": 42
      },
      "2": {
        "bar": 101
      }
    }
  },
  "focused input field": {
    "is empty and then typed into": {
      "initial": {
        "tagName": "input",
        "attributes": {
          "class": "new-todo",
          "placeholder": "What needs to be done?",
          "value": ""
        }
      },
      "after typing": {
        "tagName": "input",
        "attributes": {
          "class": "new-todo",
          "placeholder": "What needs to be done?",
          "value": "eat healthy breakfast"
        }
      }
    }
  },
  "__version": "1.0.2"
}

If a run time value during test does not match value loaded from snapshot, it throws an exception.
For example, changed code to cy.wrap({foo: 45}).snapshot() leads to

screen shot 2017-10-19 at 2 01 27 am

@brian-mann

This comment has been minimized.

Copy link
Member

commented Oct 19, 2017

Loving this. Can we also add it for elements?

// should serialize the HTML as a string and inline 
// certain things like value, checked, selected, etc
cy.get('form').snapshot() 

Bonus points if you format the HTML nicely 👍

@bahmutov

This comment has been minimized.

Copy link
Collaborator Author

commented Oct 19, 2017

@brian-mann

This comment has been minimized.

Copy link
Member

commented Oct 19, 2017

I see... is this what jest does as opposed to string HTML?

@bahmutov

This comment has been minimized.

Copy link
Collaborator Author

commented Oct 19, 2017

I think Jest converts to HTML, but I am not sure. But I think this is great user space feature, so I can make this public under @bahmutov/cypress-snapshot, see how it goes and then we can move into @cypress/<..> or build into Cypress itself.
Plus we can have options how to stringify values before comparing (already snapshot supports it under the hood), which is something Jest does too.

@brian-mann

This comment has been minimized.

Copy link
Member

commented Oct 19, 2017

I want to see this in cypress core... it's incredibly useful and powerful for a myriad of things.

Using the console as a comparison is super nice to explore the object structures.

We could even potentially compare the HTML using the actual application under test by inlining all of the styles or capturing them as well separately and applying the styles to the elements.

@bahmutov

This comment has been minimized.

Copy link
Collaborator Author

commented Oct 19, 2017

@jennifer-shehane

This comment has been minimized.

Copy link
Member

commented Oct 19, 2017

Just wanted to bring up that we use the terminology 'snapshot' for explaining the 'time-travel' process of restoring the DOM when hovering over commands and 'pinning' - we will have to rename this to something else, since I think most people define 'snapshots' as the feature explained in this proposal.

@bondz

This comment has been minimized.

Copy link

commented Nov 18, 2017

Yes @jennifer-shehane. New to Cypress, and every time I read snapshot in the docs, I thought it had support for jest snapshots, which I thought was awesome. Seeing that is uses chai is what led me here.

@newtack

This comment has been minimized.

Copy link

commented Aug 21, 2018

We use Jest for the rest of our testing, so if Cypress could integrate with Jest that would be awesome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
5 participants
You can’t perform that action at this time.