New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for "first class" Unit Tests #318

Open
brian-mann opened this Issue Dec 1, 2016 · 22 comments

Comments

@brian-mann
Member

brian-mann commented Dec 1, 2016

To date Cypress has always represented itself as an integration / e2e testing tool.

Nearly all of the cy commands drive the browser in this manner.

Almost There
However with automatic ES2015 support we are also tantalizingly close to being able to add first class unit testing support. This would enable users to import their application specific libs and test them in Cypress.

Thinking Ahead
And in fact the entire reason we created a cypress/integration folder is so that we could add unit testing support alongside this as cypress/unit. By distinguishing these two types of tests means that Cypress could handle them both differently and optimize against each.

Today's workaround
As it stands right now, and as per the cypress-example-recipes you currently just write "unit tests" alongside your integration tests.

But this is not ideal for several reasons:

The problems

  • Unit tests are just different than integration/e2e tests. Currently we enforce a certain lifecycle to integration tests - meaning between the tests we do things like clear cookies, local storage, and tear stuff down. This has no place in a unit test and is a massive waste of CPU clock cycles and network traffic. In other words, unit tests won't run nearly as fast in this manner.
  • There is no concept of visiting the application in a unit test. For instance the default view before you cy.visit makes no sense. Perhaps we could repurpose this area as like a "scratch-pad" of sorts for tossing HTML into that does still get torn down between tests.
  • Most of the cy commands are useless, but others are still really helpful. cy.spy, cy.stub, cy.fixture could all still be used.
  • Handling things like coverage reports would be possible to automatically since a unit test inlines all of the JS files, whereas an integration/e2e test does not.
  • Likely sourcemaps could be "on" by default in a unit test.
  • We could possibly add more commands like cy.render which takes an HTML string and "renders" it to the scratchpad area.

@brian-mann brian-mann changed the title from Support first class Unit Tests to Support for "first class" Unit Tests Dec 1, 2016

@brian-mann brian-mann added the roadmap label Dec 22, 2016

@jhanink

This comment has been minimized.

jhanink commented Jun 1, 2017

is there demand for this? I would love to see "first class" unit test support.

@lukemadera

This comment has been minimized.

lukemadera commented Jun 6, 2017

Seconded! @brian-mann is unit testing coming to Cypress and if so, any ETA? Or should we switch to other tools for now?

@brian-mann

This comment has been minimized.

Member

brian-mann commented Jun 6, 2017

It is coming, there is no ETA, but you can do this right now. Just import your modules in the spec files and it'll run. It'll just run slower / not be as ideal as other node based tools. But it will work.

We have examples of this (even using Enzyme here).

https://github.com/cypress-io/cypress-example-recipes#contents

@lukemadera

This comment has been minimized.

lukemadera commented Jun 6, 2017

Thanks for the prompt reply! I'll take a look!

@lgandecki

This comment has been minimized.

lgandecki commented Oct 30, 2017

Hey guys, could someone elaborate a bit about what this would be useful for? Why not use cypress for ui testing, and tools like mocha/jest/whatever else for unit/integration?
Having one test results would be nice (and easier to consume by CIs), but I'm not sure if advantage of that is bigger comparing to using the battle-tested unit testing frameworks. (watch mode of jest is fantastic, for example, and ability to run it with wallabyjs makes it even better)

I hope I'm missing something :)

@paulfalgout

This comment has been minimized.

Contributor

paulfalgout commented Oct 30, 2017

I personally hope to move my current mocha/chai unit tests to cypress and share the resources more. I am doing little bits of the same work when I would not need to.

@lgandecki

This comment has been minimized.

lgandecki commented Oct 30, 2017

interesting. I can't think of things that I redoing between e2e tests and unit/integration (I use cucumber with webdriverio/chimp and jest in majority of my projects).
By sharing the resources you mean speeding up the test run?

@paulfalgout

This comment has been minimized.

Contributor

paulfalgout commented Oct 30, 2017

I don't yet run Cypress for e2e tests. I find it most useful for stubbing the server endpoints. I restub a few of those endpoints for some my mocha tests.

@lgandecki

This comment has been minimized.

lgandecki commented Oct 30, 2017

oh, I see. Very cool. Thanks for sharing!

@daKmoR

This comment has been minimized.

daKmoR commented Nov 15, 2017

As it's almost a year old now - I'm just wondering has there been any process on unit tests becoming a "first class" citizen?

We may try e2e test with cypress and while we are add it - it would be nice to have also the unit test in one package :p
if not it's fine too - just asking if it would make sense to also convert unit test to cypress or keep unit test in a separate system for now?

@bahmutov

This comment has been minimized.

Collaborator

bahmutov commented Nov 15, 2017

@daKmoR can I ask you what you are trying to unit test that you cannot do already? For example we have unit tests directly in Cypress in the examples https://github.com/cypress-io/cypress-example-recipes/blob/master/cypress/integration/unit_test_application_code_spec.js

It is just a little bare, because the iframe that normally shows your web UI stays empty

@daKmoR

This comment has been minimized.

daKmoR commented Nov 15, 2017

sure unit test work fine - that is not the problem.
However the problems/inconvenience in the original message still remain right?
I would be especially interested in

  • "remove" enforce a certain lifecycle to integration tests
  • test coverage
  • actually be able to put them in a separate folder then the integration tests
@bahmutov

This comment has been minimized.

Collaborator

bahmutov commented Nov 15, 2017

No, sorry, nothing was done in this regard, although with release of preprocessor plugins, you could add code coverage. For separate folder you could put them into a different folder and run with integration folder CLI option https://docs.cypress.io/guides/references/configuration.html#Folders

@junyper

This comment has been minimized.

junyper commented Dec 7, 2017

I would really love to write React/Enzyme tests that render to the DOM and use some of the Cypress commands (especially screenshot). The missing piece for me is the default view/"scratch-pad" area so that I can see what my test is rendering.

@cwohlman

This comment has been minimized.

cwohlman commented Mar 13, 2018

After playing around with cypress-react-unit-test, I'd really like to see the ability to render my ui tests in the same iframe they're running in & show that iframe when unit tests are running (vs. integration tests). This clears up several side issues I ran into (e.g. react components interacting with the wrong document).

@cwohlman

This comment has been minimized.

cwohlman commented Mar 13, 2018

Also, many of the points originally raised are still relevant, especially when running in non-dev mode.

@bahmutov

This comment has been minimized.

Collaborator

bahmutov commented Mar 14, 2018

I don't like the idea of shared environment between the tests and the code / element @cwohlman. Yes, there is danger of doing the wrong thing to the wrong document, but at least there is no easy way to "dirty the water". We probably need a better way to isolate code bundled for mounting from the test code!

@egucciar

This comment has been minimized.

egucciar commented Jul 24, 2018

The scratch pad is a really cool idea, @bahmutov , but is something like this readily/easily possible today?

@jeffscottward

This comment has been minimized.

jeffscottward commented Aug 1, 2018

@brian-mann issue was created quite some time ago. whats the status on this?
Would love to run our App's redux tests (with latest babel v7 features)

Maybe there is a way to isolate mocha testing with a flag on the CLI?
--unit perhaps

@brian-mann

This comment has been minimized.

Member

brian-mann commented Aug 1, 2018

@jeffscottward people are using Cypress to write unit tests today, and it works just fine. You just configure use the webpack preprocessor or typescript or whatever it is that you do for your own app files and then you can import your components directly in your spec files.

We have example recipes of this (i think) or at the very least @bahmutov has a ton of repos that show examples of this for react, angular, vue, and a bunch of others.

@amirrustam

This comment has been minimized.

@silbinarywolf

This comment has been minimized.

silbinarywolf commented Dec 12, 2018

I'd like to just point out an issue I hit while playing with importing libraries directly into the browser.

Apologies if this is wrong, but I'm going off my vague memory from a few weeks ago. The "test running" code is running in a <iframe> seperate to where the element is rendered, any non-test code that uses document.querySelector will not work as expected. To work around this, I simply changed my non-test code... but ideally I shouldn't have to! I want to be able to provide an API similar to https://flatpickr.js.org/, so being able to use document.querySelector would be good.

I can imagine this would cause issues for people who are trying to install Cypress on a legacy project that utilizes document.querySelector for whatever reason. (Mix of jQuery / VDOM-framework)

Code:

if (!buttonEl) {
	throw new Error('Collapo: buttonEl cannot be null');
}
const ariaControls = buttonEl.getAttribute('aria-controls')
if (!ariaControls) {
	throw new Error('Collapo: aria-controls missing on button element given.');
}
// NOTE: Jake: 2018-11-11
// I'm doing this so that I'm using the correct document context when running
// cypress... this is another compelling reason to stop doing everything in Cypress as
// they suggest!
const document = buttonEl.ownerDocument;
if (!document) {
	throw new Error('Collapo: ownerDocument missing on button.');
}
const containerEl = document.getElementById(ariaControls)
if (!containerEl) {
	throw new Error('Collapo: Cannot get element by id: '+'#'+ariaControls+'. Are you missing aria-controls on the button element?');
}
if (!styles) {
	throw new Error('Missing styles param. TODO: Make this optional')
}
buttonEl.innerHTML = "~~~~~~~~~~~~~collapo initializing~~~~~~~~~~~~~~~"
console.warn(styles);
buttonEl.addEventListener('click', function(e) {
	containerEl.classList.add(styles['container--is-active'])
})

Repo:
https://github.com/silbinarywolf/collapo/blob/a6de78b1bbf336fa683514e44b5582579ddc272e/src/Collapo.ts#L26

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment