Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What is the official way to run jest programmatically? #5048

Open
IgorAufricht opened this issue Dec 11, 2017 · 59 comments · May be fixed by #14062
Open

What is the official way to run jest programmatically? #5048

IgorAufricht opened this issue Dec 11, 2017 · 59 comments · May be fixed by #14062

Comments

@IgorAufricht
Copy link

My question is how to run jest programmatically. There are some examples on the internet:
https://stackoverflow.com/a/30562836/1652273
https://stackoverflow.com/a/33669530/1652273
#3848

All of them reference requiring jest or jest-cli and calling runCLI(), but it does not seem to be documented anywhere (I haven't found anything in the jest docs nor is there any documentation for the jest-cli package).

Is this the official supported way? Thanks.

@IgorAufricht
Copy link
Author

IgorAufricht commented Dec 11, 2017

I guess my main question is: is the runCli() a part of public API?

For example this commit 200b032 changed the runCli() - it removed the onComplete callback and replaced it with a Promise, which is not reflected in the commit message and/or the changelog.

@thymikee
Copy link
Collaborator

I think there's no official way, that's why it's not referenced in the docs and changelog.
Other way to do it would be jest-editor-support package by @orta

@cpojer what do you think about an "officially supported" way of invoking Jest as a node process?
@IgorAufricht would you be interesting in writing about this?

@IgorAufricht
Copy link
Author

To add a bit of context - I'm adding jest into our grunt-based build process. I basically have these options:

  1. Run jest outside grunt (e.g. the build script could be something like npm run jest && grunt build)
  2. Use the deprecated grunt-jest
  3. Run jest from grunt as a process (e.g. using grunt-run)
  4. Run jest programmatically from a grunt task
    From these options the nicest would be the last one, hence my question.

If there is currently not an official way to run jest from node, then adding it would involve more than just a documentation change (probably some more thinking about the public API, adding tests and documentation, etc.). While I personally think it might be useful to add an official way to run jest programmatically, it's up to the jest maintainers to decide whether to add support for this or not.

@thymikee I'm not entirely sure what you're asking (writing about what?). But yes, I'm happy to help.

@cpojer
Copy link
Member

cpojer commented Dec 11, 2017

@thymikee yeah, I'm not opposed, Jest already has an API that can be used but it requires people to read the source to understand how it works, so it's not optimal. The other thing is that we may choose to break the API from major to major.

@orta
Copy link
Member

orta commented Dec 11, 2017

FWIW, I'd love this - I think exposing a consistent JS API to run tests in jest-editor-support would mean that jest-x deps APIs can break between majors, but editor-support could provide a facade for whatever has to actually happen under the hood.

@cpojer
Copy link
Member

cpojer commented Dec 15, 2017

I'm gonna close this because the issue is a question. If you'd like to use Jest's programmatic API, check out the main module of jest-cli. If you'd like it to do things that it doesn't currently do, please send as a well-tested PR that adds or changes behavior.

@tquinlan1992
Copy link

const jest = require('jest');

jest.run(argv);

@slavafomin
Copy link

I'm also interested in running jest programmatically from node.js, sadly it's not documented anywhere and the API doesn't look very thoughtful for this use case.

@SimenB
Copy link
Member

SimenB commented Jan 17, 2019

It's not. That's something we want to improve though (although a bit down the priority list at the moment)

@slavafomin
Copy link

@SimenB I've just found out that Jest is working pretty fine in this scenario actually, the only problem is the documentation and typings support.

I've posted a small example on how to use Jest in this scenario:
— How to run Jest programmatically in node.js (Jest JavaScript API)

@cpojer Is there an issue where the progress on this feature could be tracked?

@SimenB
Copy link
Member

SimenB commented Jan 17, 2019

No issue currently. We'll try to split up jest-cli for the next major and clean up the programmatic API at least a little bit (so that yargs and argv stuff is removed). After that, we'll probably create an issue trying to gather feedback for what use cases people have for using Jest programmatically in order to decide how to best move forward

@slavafomin
Copy link

@SimenB that sounds great. Thank you for your hard work!

We are calling Jest as part of our build framework inside of the existing node.js process. The current implementation works great for us, we get the console output as well as results object with more information regarding tests outcome.

My only suggestion would be to have a better naming for the API and a typings support, so the module could be used in the TypeScript environment directly.

@SimenB
Copy link
Member

SimenB commented Jan 17, 2019

We're going to be migrating to TS (#7554), so typings will definitely be there 🙂

@brendon-codes
Copy link

brendon-codes commented Jan 18, 2019

Just in case anybody is wondering, I was able to successfully do this:

npm install --save-dev jest jest-cli
const jest = require("jest-cli/build/cli");
jest
  .runCLI(
    {json: false},
    [process.cwd()]
  )
  .catch((error) => {
    console.log("Error:");
    console.log(error);
  })
  .then(() => {
    console.log("Done");
  });

EDIT: Fixed typo.

@SimenB
Copy link
Member

SimenB commented Jan 25, 2019

Teeny, tiny start here: #7696. Not ready for consumption (don't rely on us not breaking its API without bumping major), but it will happen at some point 🙂

@JSONRice
Copy link

Just in case anybody is wondering, I was able to successfully do this:

npm install --save-dev jest jest-cli
const jest = require("jest-cli/build/cli");
Jest
  .runCLI(
    {json: false},
    [process.cwd()]
  )
  .catch((error) => {
    console.log("Error:");
    console.log(error);
  })
  .then(() => {
    console.log("Done");
  });

Small typo Jest should be jest

@SimenB
Copy link
Member

SimenB commented Feb 19, 2019

Please don't reach into the build directory - that's bound to break.

We've created a @jest/core package in master (the PR linked above - will be part of the next release, whenever that happens) that will evolve into the programmatic API. It's API should be considered experimental (aka, don't come complaining if it breaks outside of a major), but it's moving forward at least 🙂

@JSONRice
Copy link

Please don't reach into the build directory - that's bound to break.

We've created a @jest/core package in master (the PR linked above - will be part of the next release, whenever that happens) that will evolve into the programmatic API. It's API should be considered experimental (aka, don't come complaining if it breaks outside of a major), but it's moving forward at least 🙂

@SimenB hey that sounds really good. Along with the new code a tutorial would be very nice. Thank you.

@SimenB
Copy link
Member

SimenB commented Feb 19, 2019

Whenever we've landed on an API we're happy with, we'll document it properly as well 🙂

@JSONRice
Copy link

Whenever we've landed on an API we're happy with, we'll document it properly as well 🙂

@SimenB also I'd be happy to be your use case tester as I'm sure others here would be. 💯

@ghost
Copy link

ghost commented Mar 1, 2019

@SimenB I would also love some documentation on this area. Looking forward to it! 👍

@SimenB
Copy link
Member

SimenB commented Mar 1, 2019

The core team will be meeting up in London later this month, and I've written this issue down as one of the things I'd like to discuss at that time. So hopefully we'll have a plan for programmatic API after that 🙂

@another-guy
Copy link

@SimenB ❤️ it's very nice to hear. Is there any way to affect the priority on this issue?

@SimenB
Copy link
Member

SimenB commented Mar 4, 2019

Until we gave a plan (which we'll hopefully have in 3 weeks or so), no.

In the meantime, maybe you can provide som API suggestions? What do you want from a programmatic API?

The first thing that comes to mind as difficult (the way things currently are) is to trigger events in watch mode - the only interface is internal FS watchers and watch plugins, nothing that can be interacted with from the outside. But is that something you need, or is just spinning up Jest and awaiting the promise it returns enough?

So if we can focus on "run jest" and "run tests multiple times (and interact with a running version of Jest)" as separate things, that'd probably be good.

For one-time runs, do you need an argv parser? Should you be able to provide partial config, or mix in defaults yourself?

@SimenB SimenB added this to the Jest 25 milestone Mar 7, 2019
@pauldraper
Copy link

pauldraper commented Nov 18, 2020

instantiating jest then calling run is an idea I can get behind!

For comparison, see Jasmine

import * as Jasmine from 'jasmine';

const jasmine = new Jasmine();
jasmine.loadConfigFile('spec/support/jasmine.json'); // or jasmine.loadConfig
jasmine.configureDefaultReporter(...); // by default, console
jasmine.execute();

https://jasmine.github.io/setup/nodejs.html

@nicojs
Copy link
Contributor

nicojs commented Dec 1, 2020

Whenever this gets implemented it would be awesome if we could leverage --bail without the process exiting on us. Right now this is not possible without monkey patching process.exit. It would be awesome if we could leverage --bail in Stryker.

@OR13
Copy link

OR13 commented Dec 3, 2020

I'm not sure if this is needed to get what I want, but I landed here searching for it.

I love jest, and I want to write a web server which will run jest tests against user supplied input, instead of fixtures checked into version control.

consider a function magicOperation which takes an input and produces an output.

everyone implements magicOperation differently, and the input and out are complicated, so I want to describe magicOperation in terms of its input and output and call some helper functions in it and cover positive and negative tests, then I want to produce JSON associated with the test run and return it in the response to the web request.

now the web server will run the magicOperation test suite on any input and output supplied by a client and report failures over http.

I was not able to figure out how to do this with Jest, so I ended up implementing it with vanilla JS functions... and its feeling a bit awkward...

Here I apply the approach to demonstrate normative statements of a specification are testable with positive and negative tests... https://w3c.github.io/did-test-suite/#did-parameters

If anyone has a better solution to this problem that does not involve running jest programmatically, I would be very grateful... here is the repo with my current solution: https://github.com/w3c/did-test-suite

EDIT: I ended up pooling some of the code from the comments here, and building this: https://github.com/transmute-industries/vc.transmute.world/tree/master/packages/jest-test-server

@evenstensberg
Copy link

Any plans to make a programatic api of some sort?

@JustinGrote
Copy link

Came here wanted to test my vscode extension E2E UI with Jest, but finding the runCLI method rather limiting

@socketopp
Copy link

Any progress here? I want to run njsTrace. However, it requires Jest to be run similar to Jasmine

var Jasmine = require('jasmine');
var jasmine = new Jasmine();
jasmine.loadConfigFile(config);
jasmine.execute([], filters);

@pauldraper
Copy link

See #9233 for a request for custom instrumentations.

@socketopp
Copy link

See #9233 for a request for custom instrumentations.

I'm not sure that's 100% related. So when using a test suite, like jest, you will have to run your own "test runner", say create a "test-runner.js" file, which will include njsTrace and call inject, then you will have to somehow "require" jest and run your tests.
Unfortunately, there is no way of require jest, is there?

@JustinGrote
Copy link

@socketopp not really, you can use the RunCLI method to invoke tests, but it's not supported. Why on earth they haven't invested in a simple programmatic interface is beyond me.

@manuman94
Copy link

Hi people!
We are planning to migrate our Jasmine projects to Jest. @JustinGrote what is the problem of using runCli()?

I have done some tests and it seems to work without problems. I can build the configuration and pass it to the runCli function, useful for us.

We want to know if we are making a bad approach. Thanks!

@JustinGrote
Copy link

@manuman94 it's not supported and could break at any time, that said it's the "best" approach if you need to run tests using jest.

@manuman94
Copy link

manuman94 commented Sep 16, 2021

@JustinGrote oh! What a pity, it would be awesome to count on a reliable API to do this. Thanks for your response.

We will be using this for the moment, but we are waiting for the final solution as many of you.

Greetings

@pauldraper
Copy link

pauldraper commented Sep 19, 2021

I'm not sure that's 100% related.

@socketopp for your particular use case of using njsTrace, you can plug into jest's instrumentation framework. jest will have main(), so to speak, but njsTrace or instanbul or whatever else can be called.


It is a pity that Jest didn't take the same approach as React, but is instead is a tangled framework that does everything from compilation to module resolution to test helpers.

@rodrigogs
Copy link

So apparently it is still a no-no, am I wrong?

@mark-wiemer
Copy link

mark-wiemer commented Jan 29, 2022 via email

@samuelkitazume
Copy link

Just to support this feature 👍

@max-b
Copy link

max-b commented Aug 17, 2022

FWIW Create React App is depending on this implicit API: https://github.com/facebook/create-react-app/blob/3880ba6cfd98d9f2843217fd9061e385274b452f/packages/react-scripts/scripts/test.js#L129

@nicojs nicojs linked a pull request Apr 9, 2023 that will close this issue
3 tasks
@tachyon83
Copy link

tachyon83 commented Sep 3, 2023

Is jest team(?) still working on this?
It seems to me that the official way to run jest programmtically still has not come out yet.
Here I am just adding use case to help anyone who is working on this.

my use case:
I need to run jest programmatically inside a Serverless AWS lambda function.
Some APIs with separate DB are deployed to the Serverless lambda functions.
My separate jest test files will be called by one of the lambda function which is not open to public, is created just for this test purpose, and will be invoked only one time in CD(continuous deployment) process.

I am just calling the jest runCLI function to do that for now.
If any updates come out, I will follow.

Thank you

@mark-wiemer
Copy link

mark-wiemer commented Sep 3, 2023

@tachyon83 PR #14062 for this was opened in April 2023 and conversation slowed until halting in late June. It's worth following that PR as well for updates, but I'm sure this issue will be updated as well. Otherwise I'm not optimistic for any fix within 6 months--someone would have to take initiative and that's only happened a couple times in the 5+ years this issue has been open.

@everett1992
Copy link
Contributor

I'm particularly interested in an interface that would let met configure a jest instance and run tests multiple times, and cancel on an abort signal. Something like

const jest = new Jest(options);
const signal = AbortSignal.timeout(60_000);

for (await (const _ of setInterval(1_000, { signal })) {
  jest.run({ signal })
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.