Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Global state is accessible when running with runInBand #5731

Open
ovidiu-lapadus opened this Issue Mar 6, 2018 · 16 comments

Comments

Projects
None yet
10 participants
@ovidiu-lapadus
Copy link

commented Mar 6, 2018

How can I get access to the variables I set in globalSetup if tests run in parallel?

I have an issue where the global object I set in globalSetup is not available if there are multiple test suites ran in parallel (this is by default). Running a single suite test or if I set --runInBand to run tests serially, the object is available. I tried to store the global object on node 'process' object and I also tried the version where I am using a custom TestEnvironment but I had no luck:

Custom test environment to pass global variable to test suites, is getting a global variable set on globalSetup.

const PuppeteerJsdomEnvironment = require('jest-puppe-shots/lib/jsdom-environment');

class JestPuppeShotsEnv extends PuppeteerJsdomEnvironment {

  async setup(config) {
    await super.setup(config);
    const { allThemesCss } = global;

    // make the allThemesCss object available in test suites
    Object.assign(this.global, {
      allThemesCss
    });
  }
}
module.exports = JestPuppeShotsEnv;

Allows the use of a custom global setup module which exports an async function that is triggered once before all test suites.

const jestPuppeEnvGlobalSetup = require('jest-puppe-shots/lib/global-setup');
const ScreenShotTestUtils = require('./screenShotUtils');

module.exports = async function globalSetup() {
  await jestPuppeEnvGlobalSetup();
  const allThemesCss = ScreenShotTestUtils.getAllThemesCss();

  global.allThemesCss = allThemesCss;
  // process.testSetup = { allThemesCss };
};

There is no proper documentation about this new feature.

jest: v22.4.2
yarn: 1.3.2
OS: Ubuntu 14
node: v8.9.4

@stephenash

This comment has been minimized.

Copy link

commented Apr 5, 2018

Ran into the same issue. Looking at https://github.com/vladgolubev/jest-mongodb (which is linked to as the working example on https://facebook.github.io/jest/docs/en/mongodb.html), also uses --runInBand in order to execute tests.

Without that, errors like the following show.

 FAIL  ./mongo-aggregate.test.js
  ● Test suite failed to run

    TypeError: Cannot read property 'getConnectionString' of undefined

       9 |     console.log('Setup MongoDB Test Environment');
      10 |
    > 11 |     this.global.__MONGO_URI__ = await global.__MONGOD__.getConnectionString();
      12 |     this.global.__MONGO_DB_NAME__ = global.__MONGO_DB_NAME__;
      13 |
      14 |     await super.setup();

      at MongoEnvironment.setup (mongo-environment.js:11:57)
@cyberwombat

This comment has been minimized.

Copy link

commented Apr 7, 2018

Same issue with me as well - SO post here

With some loose testing I can see that if I use the jsdon environment the setup runs when doing parallel tests but the node environment upon which the mongo one is build does not.

@rickhanlonii

This comment has been minimized.

Copy link
Member

commented Apr 12, 2018

@ovidiu-lapadus if you believe this is a bug, can you update the OP to follow the issue template?

@ojongerius

This comment has been minimized.

Copy link

commented Apr 25, 2018

Do you want to request a feature or report a bug?
A bug.

What is the current behavior?
If there are more than 1 tests, running Jest with--runInBand will succeed, but without will fail.

▶ yarn jest
yarn run v1.5.1
$ /Users/ojongerius/repos/jest-bug/node_modules/.bin/jest

 RUNS  ./mongo-insert.test.js
 FAIL  ./mongo-aggregate.test.js
  ● Test suite failed to run

    TypeError: Cannot read property 'getConnectionString' of undefined

       9 |     console.log('Setup MongoDB Test Environment');
      10 |
    > 11 |     this.global.__MONGO_URI__ = await global.__MONGOD__.getConnectionString();
      12 |     this.global.__MONGO_DB_NAME__ = global.__MONGO_DB_NAME__;
      13 |
      14 |     await super.setup();

      at MongoEnvironment.setup (mongo-environment.js:11:57)

 FAIL  ./mongo-insert.test.js
  ● Test suite failed to run

    TypeError: Cannot read property 'getConnectionString' of undefined

       9 |     console.log('Setup MongoDB Test Environment');
      10 |
    > 11 |     this.global.__MONGO_URI__ = await global.__MONGOD__.getConnectionString();
      12 |     this.global.__MONGO_DB_NAME__ = global.__MONGO_DB_NAME__;
      13 |
      14 |     await super.setup();

      at MongoEnvironment.setup (mongo-environment.js:11:57)

Test Suites: 2 failed, 2 total
Tests:       0 total
Snapshots:   0 total
Time:        0.454s
Ran all test suites.
Teardown mongod
error An unexpected error occurred: "Command failed.
Exit code: 1
Command: sh
Arguments: -c /Users/ojongerius/repos/jest-bug/node_modules/.bin/jest
Directory: /Users/ojongerius/repos/jest-bug
Output:
".
info If you think this is a bug, please open a bug report with the information provided in "/Users/ojongerius/repos/jest-bug/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

If the current behavior is a bug, please provide the steps to reproduce and
either a repl.it demo through https://repl.it/languages/jest or a minimal
repository on GitHub that we can yarn install and yarn test.

Pull down https://github.com/ojongerius/jest-bug, running yarn test will fail and yarn test-inband will succeed.

What is the expected behavior?
Running more n+1 tests will succeed

Please provide your exact Jest configuration

module.exports = {
  globalSetup: './setup.js',
  globalTeardown: './teardown.js',
  testEnvironment: './mongo-environment.js'
};

Run npx envinfo --preset jest in your project directory and paste the
results here

▶ npx envinfo --preset jest
npx: installed 1 in 3.967s

  System:
    OS: macOS High Sierra 10.13.4
    CPU: x64 Intel(R) Core(TM) i7-3615QM CPU @ 2.30GHz
  Binaries:
    Node: 8.10.0 - ~/.nvm/versions/node/v8.10.0/bin/node
    Yarn: 1.5.1 - /usr/local/bin/yarn
    npm: 5.8.0 - ~/.nvm/versions/node/v8.10.0/bin/npm
  npmPackages:
    jest: ^22.4.3 => 22.4.3
@rickhanlonii

This comment has been minimized.

Copy link
Member

commented Apr 26, 2018

@ojongerius thanks a ton

@SimenB

This comment has been minimized.

Copy link
Collaborator

commented Apr 27, 2018

FWIW, I think it's a bug that it does work when running in band, not the other way around.

Tests are supposed to run in a sandbox, if they can all access the same global variable, it breaks the contract.

I wonder if we should explicitly add a field which we pass into the environments so it's accessible? That way it's an explicit opt-in to a given object. E.g.

// my-setup.js

module.exports = context => {
  context.something = 'yay';
}

or

// my-setup.js

module.exports = () => {
  return { something: 'yay' };
}
// test.js
console.log(jest.context); // -> { something: 'yay' }

Maybe try to enforce some immutability up until globalTeardown, but that might be overkill.

nikpundik added a commit to nikpundik/jest that referenced this issue Jun 15, 2018

Using Jest with MongoDB example - wrong variables
In the example file setup.js the variables `mongod` and `mongoFileConfigPath` are undefined. I think they are `mongoserver` and `globalConfigPath`.
And the linked full working example repo comes with a different implementation failing with multiple tests (facebook#5731).
@jjjj222

This comment has been minimized.

Copy link

commented Jul 23, 2018

I have the same issue here.
I tried the v23.4.1 and still can't access the data that I stored to "global" in globalSetup.
How can I pass a variable from globalSetup to TestEnvironment without using a file?

@SimenB

This comment has been minimized.

Copy link
Collaborator

commented Jul 25, 2018

You can't

@jjjj222

This comment has been minimized.

Copy link

commented Aug 1, 2018

Here is the issue.

My tests require to access a shared resource (let's say it's a file). Right now I use a fixed file name but it will cause some issues if I launch 2 jest simultaneously. So I want to decide the file name in globalSetup, in case that the file name has been used (which mean there has been a jest running). So what can I do to let all the tests know the file name I chose in the globalSetup?

@dbartholomae

This comment has been minimized.

Copy link

commented Aug 23, 2018

I've got a similar issue: We are using jest across our whole stack including backend. For backend e2e tests it would be very expensive to start up the app for each test individually, as this would require to connect to the database for each test separately which isn't feasible. Therefore we start the app in globalSetup. Unfortunately this makes it impossible to efficiently mock external APIs with modules like nock because these modules just patch http.request, but since globalSetup is run in a different process, mocking this inside the tests doesn't help. If I could e. g. set up nock in globalSetup and handover a reference to the tests, this would solve the problem.

@FanAs

This comment has been minimized.

Copy link

commented Aug 27, 2018

I have the same issue here

@davidgilbertson

This comment has been minimized.

Copy link

commented Oct 13, 2018

I have a similar issue; I want a variable available to all tests, but only want to write to it.

I want to be able to report the names of each test to an external system. Ideally I could hook into it() and just push the names to a globally-available array, then do something with that when the tests are finished.

Any other ideas would be most welcome.

@rickhanlonii

This comment has been minimized.

Copy link
Member

commented Oct 13, 2018

Hey all - the answer to sharing globals in parallel is that you cannot do it right now

I understand the use cases (and have run into them myself), and I think we should consider some proposals to support them. The key difficulty is how do we allow you to share state between parallel tests without giving you a footgun that makes your tests interdependent and flaky

If anyone wants to think of a solution and submit a proposal we're happy to consider

Let's keep this issue open only for the purposes of inconsistency between runInBand and in parallel 👌

@rickhanlonii rickhanlonii changed the title Parallel testing using the new globalSetup feature - issues. Global state is accessible when running with runInBand Oct 13, 2018

@dbartholomae

This comment has been minimized.

Copy link

commented Oct 16, 2018

@rickhanlonii Where's the best spot to discuss such proposals? Didn't find any info in CONTRIBUTING.md how to handle feature requests.

@SimenB

This comment has been minimized.

Copy link
Collaborator

commented Oct 16, 2018

We have a template for it 🙂 See https://github.com/facebook/jest/issues/new?template=feature.md

@dbartholomae

This comment has been minimized.

Copy link

commented Oct 16, 2018

Thanks! For those looking, I set up a feature request at #7184.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.