Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running tests in the browser #139

Closed
pspeter3 opened this issue Sep 30, 2014 · 55 comments
Closed

Running tests in the browser #139

pspeter3 opened this issue Sep 30, 2014 · 55 comments

Comments

@pspeter3
Copy link

@pspeter3 pspeter3 commented Sep 30, 2014

It would be great for a TDD setup to be able to run the tests in the browser and see the component you are trying to test. Is it possible to run jest in the browser?

@pspeter3
Copy link
Author

@pspeter3 pspeter3 commented Sep 30, 2014

Specifically I just want the ability to run a single jest test file

@danvk
Copy link

@danvk danvk commented Oct 16, 2014

There was some discussion about this on the mailing list.

@lostrouter
Copy link

@lostrouter lostrouter commented Jan 3, 2015

is there a way that jest could make use of the jasmine spec runner?

@facundocabrera
Copy link

@facundocabrera facundocabrera commented Jan 21, 2015

Anyone has any update on this? Is there any work in progress?

@danvk
Copy link

@danvk danvk commented Jan 21, 2015

We migrated our tests to Mocha for this reason, among others. I'd highly recommend doing the same (or to Jasmine). While we still can't run our tests in-browser (they depend on fs and other node modules that don't make sense there), we can at least use node-inspector, which is almost as good for debugging.

@pspeter3
Copy link
Author

@pspeter3 pspeter3 commented Jan 21, 2015

We migrated our tests to mocha as well. We use browserify to run them in the browser and jsdom to run them on the command line.

@facundocabrera
Copy link

@facundocabrera facundocabrera commented Jan 21, 2015

Thanks a lot for the feedback guys 😄

@leebyron
Copy link
Contributor

@leebyron leebyron commented Jan 21, 2015

Nothing done here yet, as the auto-mocking requires the node runtime environment.

@facundocabrera
Copy link

@facundocabrera facundocabrera commented Jan 22, 2015

Why do not provide a working version without auto-mocking? I remember https://remysharp.com/2014/05/30/commonjs-with-devtools-live-edit so it should not be impossible to do this limited version.

I know PR are welcome!

@malliapi
Copy link

@malliapi malliapi commented Jun 3, 2015

i would also really appreciate using the specrunner!

@mikepc
Copy link

@mikepc mikepc commented Jul 24, 2015

The important issue here is the ability to use the browser to debug the unit test in the browser. It's very useful to debug the code you're testing, rather than a run-and-see-if-it-works setup. Really helps in a TDD environment

@kiki-le-singe
Copy link

@kiki-le-singe kiki-le-singe commented Aug 4, 2015

if there is no possibility to test in browsers, how can we debug?

@ghost
Copy link

@ghost ghost commented Aug 5, 2015

Thank you for reporting this issue and appreciate your patience. We've notified the core team for an update on this issue. We're looking for a response within the next 30 days or the issue may be closed.

@buzinas
Copy link

@buzinas buzinas commented Oct 30, 2015

Any news?

@danvk
Copy link

@danvk danvk commented Oct 30, 2015

It's a truism: if you're going to run your code in the browser then you should test it in the browser, too. Running your tests under node with jsdom isn't the same thing. It's just going to give you headaches.

My preferred approach for testing client-side JS is currently Mocha + mocha-phantomjs for running tests on the command line, but there are many ways to do it.

@quantuminformation
Copy link

@quantuminformation quantuminformation commented Jan 25, 2016

+1

@quantuminformation
Copy link

@quantuminformation quantuminformation commented Jan 26, 2016

So is there no simple way to run jest (using jasmine 2) through karma? What is so difficult about it if Jasmine already can? Seems like a MVP requirement for any test runner for unit testing enterprise apps.

@cpojer
Copy link
Collaborator

@cpojer cpojer commented Jan 27, 2016

Oh I haven't commented here. The new architecture of jest proposed in #599 will make it easier to build a server environment around Jest. This is unfortunately not a priority for Facebook however, so I'm not going to actively push for this. If anyone wants to work on it, I'd be happy to help.

@quantuminformation
Copy link

@quantuminformation quantuminformation commented Jan 27, 2016

thanks Chris.

What about running Jest on node webkit and chromimum web driver?

@cpojer
Copy link
Collaborator

@cpojer cpojer commented Jan 27, 2016

Seems like that is the same concept?

@quantuminformation
Copy link

@quantuminformation quantuminformation commented Jan 27, 2016

Yeah I guess so. I'll see what I can dig up on it.

@quantuminformation
Copy link

@quantuminformation quantuminformation commented Jan 27, 2016

I tried just running in the browser with jest like so:

https://github.com/QuantumInformation/JestBrowserTest

but I get this error in the console:

Uncaught (in promise) SyntaxError: Unexpected token H

@cpojer
Copy link
Collaborator

@cpojer cpojer commented Mar 30, 2016

Merging this task into #848.

@cpojer cpojer closed this Mar 30, 2016
@vvo
Copy link
Contributor

@vvo vvo commented Jun 27, 2016

One very interesting information from this thread is that it seems running unit tests in browsers not to be a priority for Facebook. Which means: Facebook, one of the biggest website in the world does not need to run unit testing in browsers, only in nodejs, with jsdom and auto mocks.

Could maybe one Facebook engineer explain how they came with this reasoning. Is facebook using solely jest for unit testing of their components base?

As for me I believe most important bugs nowadays will come from your own coding, not because of a bug in a browser.

Unit testing in node with mocks should be sufficient to test the flow of your program. Once you are able to mock any browser quirk then you can mock it in nodejs. If you need to test that your login form is working in latest chrome then it's a job for selenium and a whole different subject.

@cpojer
Copy link
Collaborator

@cpojer cpojer commented Jun 29, 2016

@vvo Hey! I'm happy to share some thoughts. Five years ago we actually used to have a browser unit test suite (that I used to work on, hah!) but it wasn't good, slow to use and not how people built code at FB.

We then built a command line testing utility with a mock-dom and people loved the iteration speed and how well it worked. The browser test suite was killed and we never looked back. To cover browser testing we do a lot of things ranging from web driver tests, manual QA and especially employee dogfooding. These things have in general provided more value than a flaky browser test engine that is hard to scale for thousands of tests.

However, I do recognize that people enjoy writing tests using a browser environment. Node's debugging story is getting a lot better and JS frameworks provide the necessary abstractions to take care of this. There is an ongoing conversation about the potential of Jest running in the browser some day: #848.

@vvo
Copy link
Contributor

@vvo vvo commented Jun 29, 2016

Awesome feedback, thanks a lot :)

@FredyC
Copy link

@FredyC FredyC commented Jun 29, 2016

I'll just add my two cents here. I've started working on a new project with React. I've decided to write unit tests for my Redux code only. Instead of browser tests I am giving a go to shiny new toy called React Storybook which essentially allows me to create show case (story) of my components and how would they behave/render based on various inputs. What I like most about it is the way how it forces you to create decoupled small components. You have to think about that single component only when designing it.

Of course it cannot be compared to webdriver tests as it's not automated in any way. Also it's harder to imagine using it for a huge project like Facebook with hundreds of components. On other hand you have a nice "catalog" of available components and you can immediately see its use cases. It is probably gonna be even bigger beast once https://storybooks.io/ is ready. And who knows, perhaps one it will be automated some way too, eg. comparing changes and validating it hasn't changed from last time.

@mhuggins
Copy link

@mhuggins mhuggins commented Sep 2, 2018

I was just bitten by this. I have a test passing in Jest, but the behavior is different in the browser. This is specifically because the URL object does not match between these two environments.

https://stackoverflow.com/questions/52141486/testing-url-usage-for-browser-in-node-js

@stockenja
Copy link

@stockenja stockenja commented Nov 27, 2018

It's a truism: if you're going to run your code in the browser then you should test it in the browser, too. Running your tests under node with jsdom isn't the same thing. It's just going to give you headaches.

My preferred approach for testing client-side JS is currently Mocha + mocha-phantomjs for running tests on the command line, but there are many ways to do it.

So, true! So much pain to do something simple like testing ES6 code in a browser environment.

FACEBOOK, please just make a Facebook browser. A browser that would run ES6 code instead of the whole transpile BS. It is time for Javascript to mature and have a real solid standard.

The browser will be good for your stock price too. Trust me! If you make a Facebook browser, I will go buy your stocks immediately.

Yao from Stockenja

@brucou
Copy link

@brucou brucou commented Nov 27, 2018

Can't agree more with running tests in a real browser. I use good old Qunit and react-testing-library for my react code. Does not work too badly, but more importantly I avoid the quirks related of browser reimplementations. I don't avoid the quirks of React DOM though... And I loose snapshots. Oh well...
My main reason, apart from I already mentioned, was that it is much more convenient to debug in the browser, and then because I actually play a sequence of inputs in the browser, I can see what is being displayed at what time on a real screen, that is invaluable on some bugs. I don't how much time is lost running the test in the browser but I do believe a lot of time is also saved by finding bugs quicker instead of playing detective around browser abstractions. In any case, for the sanity of my mind, that is a much better solution

@jasonworden
Copy link

@jasonworden jasonworden commented Feb 25, 2019

Noting this as a blocker for testing Ember with Jest

#2714 (comment)

@i5ar
Copy link

@i5ar i5ar commented Apr 7, 2019

For the people interested in testing React in the browser:

Those are the best solutions I had experience with when I couldn't use Node.js and therefore Jest.

@kvendrik
Copy link

@kvendrik kvendrik commented Jul 22, 2019

For anyone still interested in running Jest in the browser, check out jest-lite which lets you do just that. 🙂

@taktakpeops
Copy link

@taktakpeops taktakpeops commented Sep 27, 2019

Hello, for anyone interested in running his tests with Jest in Browserstack, I released this environment: https://www.npmjs.com/package/jest-environment-browserstack

@iliakan
Copy link

@iliakan iliakan commented Jan 7, 2020

Dear Sirs,

Is anyone working on that? Are there any estimations when jest will be runnable in-browser? Are there any crucial obstacles to that?

Thanks!

@trusktr
Copy link

@trusktr trusktr commented Feb 28, 2020

If anyone needs to run tests in a browser, just use Karma! Jest is great if you are testing only Node.js APIs.

You can pair Karma with the karma-electron plugin so that your tests will run in a headless Electron window so you can run any tests in an environment with actual Node.js and Browser APIs, AND use import/require to import modules or files.

Take a look at karma-webpack for bundling each test file (or compiling other language features), as the current version of Electron (and hence karma-electron) doesn't (yet) support ES Module syntax.

Note that very soon you won't need karma-webpack once Electron upgrades to a recent version of Node v12 with native ES Modules. Electron skips non-LTS versions of Node (odd numbered versions), but in April Node 14 LTS is coming out, and Electron will soon after move to Node 14 with all modern glory. At that point the version of Electron used by karma-electron just needs to be updated.

Tip: use the xvfb-maybe package to run karma-electron tests in CI where a windowing system is not available.

TLDR: If you need real APIs from real environments, instead of mocks, use Karma. It has a higher learning curve than Jest, but in my opinion it is worth it.

@FezVrasta
Copy link

@FezVrasta FezVrasta commented Feb 28, 2020

Nowadays there are better solutions such as jest-puppeteer or jest-playwright

@trusktr
Copy link

@trusktr trusktr commented Feb 28, 2020

@FezVrasta jest-puppeteer and jest-playwright are for a different purpose: they are for controlling a browser from outside, using code inside your Jest tests which are running in Node (not in a Browser).

Those tools do not run your Jest code inside a browser. Instead they allow you to script a browser from the outside. That serves the use cases of end-to-end or integration testing, and I would totally recommend Jest for that purpose. 👍

What I mentioned above about using Karma is good for unit tests whose code needs to run inside a browser environment (as opposed to Node with fake browser APIs provided by JSDom or Undom).

As an example, if you have functions that directly access WebGL APIs, or certain CSSOM APIs, or other browser APIs that otherwise Jest (via JSDom, Undom) does not currently provide or only partially implements, and your unit tests will trigger those functions (which would otherwise crash Jest) then you'll want to run your unit tests inside an actual browser that has those APIs.

Some people spent valuable time making fake canvas and webgl mocks. They could have instead simply ran tests in a browser.

You may just want to use a real browser environment for your unit tests so that you have higher confidence that your stuff works properly in browsers.

The jest-puppeteer and jest-playwright tools won't serve that purpose.

Do you see what I mean? I hope I can convince you to change your thumbs down to a thumbs up. :)

@iliakan
Copy link

@iliakan iliakan commented Feb 28, 2020

I don't see a reason why jest tests may not run in browser environment, while they do run in jsdom.

A lot of node.js modules run in browser quite well.

For really node.js-specific stuff, there may be indeed compat issues, but most jest tests should just work.

@trusktr
Copy link

@trusktr trusktr commented Feb 28, 2020

For really node.js-specific stuff, Jest is great. It's already running your code in Node!

A lot of node.js modules run in browser quite well.

True, but Jest is a test runner that gives you a CLI, to run tests with, and it runs those tests in parallel using Node.js machinery, not browser machinery.

How would you tell Jest CLI to run your tests in a browser?

In the meantime, we can do it simply with Karma, with code executed in Electron, Chrome, and other browsers.

I definitely would like to see where #848 goes, but that's been open for 4 years, and I don't want to try any of the working hacks there, when Karma just works.

In case it helps, here's my karma.config.js

Notes

  • It uses karma-jasmine for describe/it grouping APIs, along with familiar expect and assert assertion APIs.
  • It currently uses Webpack (karma-webpack) for bundling each entry point without any source code transformation, and it is relatively fast. I need this because my source code and test code use ES Module syntax, which Electron does not yet understand.
  • Once native ES Modules are released in Electron (later this year hopefully! Possibly beginning of 2021), then karma-webpack will no longer be a requirement and test code will run as-is without any bundling needed.
  • The code that runs in Karma with this configuration is plain JS, with no transpilation (other than handling the import/export syntax, but otherwise Webpack is only for bundling). If you need to transpile code, you'd need to add Babel (or similar) to the Webpack config, but that could increase the time it takes to run tests.
  • If your test code and source code are written with CommonJS require(), then you don't need karma-webpack. I only need it because my code uses ES Module syntax. But again, soon this karma-webpack won't be needed!

karma.config.js

// @ts-check
const CWD = process.cwd()

console.log(`
##################################################################
NOTE: You may see Electron security warnings for local files. These are
harmless, as they are not from the web.
##################################################################
`)
console.log('')

const debugMode = !!process.env.KARMA_DEBUG

module.exports = function(config) {
	config.set({
		frameworks: ['jasmine', 'stacktrace'],
		reporters: ['spec'],
		port: 9876, // karma web server port
		colors: true,
		logLevel: config.LOG_INFO,
		autoWatch: false,
		singleRun: debugMode ? false : true,
		concurrency: Infinity,

		basePath: CWD,

		// Set up a "CustomElectron" launcher that extends karma-electron's
		// default "Electron" launcher, so that we can control options
		// (docs: // https://github.com/twolfson/karma-electron/tree/5.1.1#launcher-configuration)
		browsers: ['CustomElectron'],
		customLaunchers: {
			CustomElectron: {
				base: 'Electron',

				flags: debugMode
					? [
							// If in debug mode, this makes the electron window visible
							// so that we can step through tests using Chromium devtools
							'--show',

							// Alternatively to the --show option, we can use this
							// option, then open chrome://inspect in Google Chrome and
							// inspect from there.
							// '--remote-debugging-port=9222',
					  ]
					: [],
			},
		},

		files: [{pattern: 'dist/**/*.test.js', watched: false}],
		preprocessors: {
			'dist/**/*.test.js': ['electron', 'webpack'],
		},

		// We need Webpack support because the code running through Karma uses
		// ES Module syntax. Once native ES Modules are released in Electron,
		// then karma-webpack will no longer be a requirement and test code will
		// run as-is without any bundling needed.
		webpack: {
			// Make it FAST with development mode, for testing purposes. We
			// don't need to compile test code in production mode with
			// minification or other features that will slow tests down. All we
			// want is to bundle each test's dependencies (unit tests should
			// have minimal dependencies, and not import the entire library
			// being tested, only import specific parts being tested) and send
			// it to the browser.
			mode: 'development',
		},
		client: debugMode
			? {}
			: {
					// Prevent a "require is not defined" error in
					// karma-electron
					useIframe: false,
					loadScriptsViaRequire: true,
			  },
	})
}

Then I run karma with

xvfb-maybe karma start ./path/to/karma.config.js

xvfb-maybe allows Electron to run headlessly in a unix environment without a windowing system (Xorg/X11), if needed (for example on a continuous integration server).

That's it. Simple!

@trusktr
Copy link

@trusktr trusktr commented Feb 28, 2020

why jest tests may not run in browser environment, while they do run in jsdom.

Let me clarify this: JSDom is not a browser environment. JSDom is simply a set of fake APIs that the jsdom package simply adds to your Node.js environment as globals; it adds a fake global window variable along with fake APIs on that window variable. And similar with undom

Jest is not running inside JSDom. Jest is running inside Node.js after the fake globals have already been created by jsdom or undom.

This by no means you can simply run Jest inside a browser, as Jest is designed to run inside Node, with fake global variables created before your tests run.

@trusktr
Copy link

@trusktr trusktr commented Feb 28, 2020

For example of APIs that are missing in JSDom (and why you'd want to run in an actual browser environment), see these bugs (or missing features):

If you run in a browser, and not in Node.js+JSDom or Node.js+Undom, then you will avoid all the problems of fake API with missing features and therefore causes your code not to work (which defeats the purpose of testing that code).

JSDom and Undom are trying to... implement a browser... without implementing a browser. Eventually you hit edge cases that make that whole idea not worth it (in my opinion).

If you're okay with all of the above issues, then use Jest. If you're not, I recommend Karma.

@tom-sherman
Copy link

@tom-sherman tom-sherman commented Jul 28, 2020

I have published a post that worked for us to take our Jest tests and run them in a browser with minimal effort: https://github.com/tom-sherman/blog/blob/master/posts/02-running-jest-tests-in-a-browser.md#the-good-stuff

Hopefully this will be helpful for some!

@AtofStryker AtofStryker mentioned this issue Aug 3, 2020
6 of 13 tasks complete
@naruaway
Copy link

@naruaway naruaway commented Aug 11, 2020

I think making existing test suits using, for example, react-testing-library + jest runnable in real web browsers is super useful.
We can "reuse" existing jest tests for different purposes like: visually following test sequences in browser or take screenshots on every test completion automatically.

@tom-sherman 's above approach or Jest electron runner looks promising for this purpose though.

@Shingaz
Copy link

@Shingaz Shingaz commented Nov 18, 2020

I have published a post that worked for us to take our Jest tests and run them in a browser with minimal effort: https://github.com/tom-sherman/blog/blob/master/posts/02-running-jest-tests-in-a-browser.md#the-good-stuff

Hopefully this will be helpful for some!

@tom-sherman Interesting post. How did tou manage to use the expect as it is not bundled for the browser anymore?

@tom-sherman
Copy link

@tom-sherman tom-sherman commented Nov 18, 2020

@Shingaz If it didn't work for you please leave a comment here and we'll see if we can figure it out.

I seem to remember mocking fs for one version, and another version I didn't have to but I could be misremembering.

@bard
Copy link

@bard bard commented Mar 7, 2021

@tom-sherman @Shingaz and anyone who may end up here:

I got expect to work in the browser by disabling the modules that otherwise prevent expect from loading in a non-node environment. karma.conf excerpt:

module.exports = function (config) {
  config.set({

    frameworks: ['webpack', 'mocha'],

    plugins: [
      'karma-webpack',
      'karma-mocha',
      // ...
    ],
    
    // ...
    
    webpack: {
      node: {
        // handles stack-utils looking for 'module'
        module: 'empty',
      },
      module: {
        rules: [
          // handles jest-message-utils importing 'graceful-fs'
          { test: /graceful-fs/, use: 'null-loader' },

          // ...

Note also that jest's expect signals failures by throwing, hence the use of Mocha above instead of Jasmine.

  • expect@26.6.2
  • karma@6.1.1
  • webpack@4.44.2
  • karma-webpack@5.0.0
@github-actions
Copy link

@github-actions github-actions bot commented May 10, 2021

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 10, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet