Workers #781

Closed
tomitrescak opened this Issue Sep 9, 2016 · 7 comments

Projects

None yet

2 participants

@tomitrescak
tomitrescak commented Sep 9, 2016 edited

Hello. I'm trying to wrap my head around workers setup.
I am runnign tests that execute MongoDB calls.
I have a bit issues with parallelisation so I tried disabling parallel workers.
Yet, with every test run, wallaby executes three parallel processes.
Is this normal? Tests seem to work ok, but sometimes wallaby does not do cleanup and leaves temporary databases hanging. This I have limited by throttling, but still happens time to time.

Here is the repo:

https://github.com/tomitrescak/apollo-connector-mongodb

Here is my wallaby config:

module.exports = function (w) {
  return {
    files: [
      'src/*.ts'
    ],
    tests: [
      'src/tests/*.ts'
    ],
    compilers: {
      "**/*.ts*": w.compilers.typeScript({module: "es6", target: "es6"})
    },
    env: {
      type: "node"
    },
    testFramework: "mocha",
    preprocessors: {
      "**/*.js*": file => { 
        return transform(file.content); 
      }
    },
    workers: {
       initial: 1,
       regular: 1
     },
    delays: {
      run: 500
    }
}

[Edit] A bit more info.
The Mongo log in each pass is this

2016-09-10T00:15:00.025+1000 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:60213 #88 (2 connections now open)
2016-09-10T00:15:00.035+1000 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:60214 #89 (3 connections now open)
2016-09-10T00:15:00.065+1000 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:60215 #90 (4 connections now open)
2016-09-10T00:15:00.074+1000 I COMMAND  [conn88] dropDatabase tmp972 starting
2016-09-10T00:15:00.075+1000 I COMMAND  [conn88] dropDatabase tmp972 finished
2016-09-10T00:15:00.076+1000 I NETWORK  [conn90] end connection 127.0.0.1:60215 (3 connections now open)
2016-09-10T00:15:00.076+1000 I NETWORK  [conn89] end connection 127.0.0.1:60214 (3 connections now open)
2016-09-10T00:15:00.076+1000 I NETWORK  [conn88] end connection 127.0.0.1:60213 (2 connections now open)
@ArtemGovorov
Member

Yet, with every test run, wallaby executes three parallel processes.

This shouldn't happen with the

workers: {
       initial: 1,
       regular: 1
     },

setting. If you add a setup function like:

    setup: (w) => {
      console.log(w.workerId);
    }

you'll only see one test runner process.

Tests seem to work ok, but sometimes wallaby does not do cleanup and leaves temporary databases hanging.

With the way you create/cleanup the database (using mocha before and after hooks), it is expected.

Imagine you launch plain mocha from a command line to run your tests, and then in a middle of the run you stop it by terminating the run. Now imagine you are doing it frequently. Sometimes (depending on the moment you stopped it), it will not do proper cleanup and will leave temporary databases hanging.

That's what wallaby does - it runs your mocha tests with mocha runner, but if you change your code while your tests are still running, it'll stop the previous run and do the next one.

Wallaby has the teardown function, that unlike mocha hooks will run in most of the cases. But there're still valid cases when wallaby can't guarantee that it's going to be triggered. For example when you introduce an infinite loop while changing your code. You may read the post for more details.

One thing that is absolutely guaranteed is that your setup function will be invoked before each test run. If you're after 100% consistency, then it is the best place to check if a database exists, clean/set it up if required, etc. Wallaby also passes the workerId to the function, and you may use it to enable parallelisation by creating a unique database per workerId.

You may read more about this strategy in #774 and #256.

If you need to make the strategy work with plain mocha as well, you move the code that handles your db setup code into a separate file (like setupDb.js), and --require it in your mocha command. In wallaby setup function you'll have:

    setup: w => {
      global.workerId = w.workerId;
      require('./setupDb');
    }

and in the setupDb.js:

const workerId = global.workerId || 0; // real worker ID or 0 if plain mocha
// your DB setup code
// where the DB/tables/other entities name can be parametrized with workerId
@tomitrescak

@ArtemGovorov I can never thanks you for the amazing work that you do and the support you provide. My little thank you is by spreading the word about your software which hopefully resulted into purchased licenses. I got quite a few people at Meteor world hooked, and one "funny" story ..

Two nights ago I went out for a beer in Darling Harbour with a friend that just came from Austria where he works for a large IT corporation. He was telling me about the amazing testing solution that they are developing in-house. I mentioned wallaby.js and that he should check it out. Next morning his only message was ... "fu.k, we just lost 7 months and god knows how many man hours trying to develop something not even marginally that good as Wallaby". ;)

I'll go and check out the tear down now, sound like something I exactly need.

@ArtemGovorov
Member

@tomitrescak Thanks a lot Tomas, much appreciated! Spreading the word is a really great way to help us with the tool.

@tomitrescak

Artem, I still can't make it work ;( Do you have any example where Mongo is initlaised in setup and torn down as well? I am getting all kinds of weird errors such as "topology was destroyed" ;(

@ArtemGovorov
Member

My experience with Mongo is very limited, but if you can create a sample repo with some instructions how/what to run and reproducible errors, I'm happy to have a look.

@tomitrescak

Artem, for some reason the terdwon function is not executed .. or at least does not print anything into console:

module.exports = function (w) {
  return {
    files: [
      'src/*.ts',
      './setup.js'
    ],
    tests: [
      'src/tests/**/*.ts'
    ],
    compilers: {
      "**/*.ts*": w.compilers.typeScript({module: "commonjs", target: "es6"})
    },
    env: {
      type: "node"
    },
    testFramework: "mocha",
    workers: {
       initial: 1,
       regular: 1,
       recycle: true
     },
    delays: {
      run: 500
    },
    setup: function(w) {
      // handle db
      global.workerId = w.workerId;
      console.log('setting up ...');
    },
    teardown: function (wallaby) {
      console.log('tearing down');
      if (db) {
        console.log('tearing down db');
        db.dropDatabase();
        db.close();
      }
    }
  };
};
@ArtemGovorov
Member

Yes, with the workers.recycle: true setting the teardown function doesn't get called. You'll need to set workers.recycle: false.

However, a better solution like I have been suggesting, is to use the setup function to check connect and drop any DBs if required. As I have mentioned, there're valid cases when wallaby can't guarantee that the teardown is going to be triggered. For example when you introduce an infinite loop while changing your code. You may read the post for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment