Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mocha 4 doesn't exit unlike mocha 3 #3044

Closed
sagiegurari opened this Issue Oct 3, 2017 · 49 comments

Comments

Projects
None yet
@sagiegurari
Copy link

sagiegurari commented Oct 3, 2017

Prerequisites

  • [x ] Checked that your issue isn't already filed by cross referencing issues with the common mistake label
  • Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn't just a feature that actually isn't supported in the environment in question or a bug in your code.
  • [ x] 'Smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
  • [ x] Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with:
    node node_modules/.bin/mocha --version(Local) and mocha --version(Global). We recommend avoiding the use of globally installed Mocha.

Description

I have been running specific set of tests for few years now and have been always upgrading to the latest mocha and everything was ok.
With mocha 4 suddenly all tests are passing but it doesn't end as if the --no-exit is automatically added although I never added it.

Steps to Reproduce

Expected behavior:
After all tests end, the process should stop even if there are timeoutes or sockets that prevent the process from existing.

Actual behavior:
Mocha 4 process waits forever like mocha 3 with --no-exit flag

Reproduces how often:
With our tests always. I have 700 tests so it is hard to pinpoint which one casuses the issue or if maybe it is in our codebase.

Versions

mocha 4.0.0 fails. before that everything works good.

@mkrufky

This comment has been minimized.

Copy link

mkrufky commented Oct 3, 2017

I am seeing the exact same problem. Tests will pass and then mocha just hangs. See my Travis CI:
https://travis-ci.org/mkrufky/node-dvbtee/builds/282593109

I also noticed this same problem on video-dev/hls.js - mocha just hangs after passing tests:
https://travis-ci.org/video-dev/hls.js/builds/282590422

@pgilad

This comment has been minimized.

Copy link

pgilad commented Oct 3, 2017

Did you guys read the release notes? https://boneskull.com/mocha-v4-nears-release/#mochawontforceexit

@sagiegurari

This comment has been minimized.

Copy link
Author

sagiegurari commented Oct 3, 2017

Thanks. tok bad the mocha website is not updated with this breaking change. The cli args there do not mention it.

@pgilad

This comment has been minimized.

Copy link

pgilad commented Oct 3, 2017

Other than providing a quick fix (use --exit), I do agree they left the core issue of finding the faulty tests to the user. I'm struggling with that myself now, but when upgrading major versions, I make sure to read the release notes and not blindly upgrade

@borisovg

This comment has been minimized.

Copy link

borisovg commented Oct 3, 2017

The release notes suggest using why-is-node-running except it doesn't work due to mafintosh/why-is-node-running#7

@mceachen

This comment has been minimized.

Copy link

mceachen commented Oct 3, 2017

Hit by this too. I understand the reasoning behind the change to the default, and thank you for incrementing the major version, but given that why-is-node-running is abandoned and broken, this may not be the most user-friendly change.

@ScottFreeCode ScottFreeCode changed the title mocha 4 doesn't exist unlike mocha 3 mocha 4 doesn't exit unlike mocha 3 Oct 3, 2017

@ScottFreeCode

This comment has been minimized.

Copy link
Contributor

ScottFreeCode commented Oct 3, 2017

Hi all,

First off, I want to apologize for the rough upgrade path on this point -- we definitely agree that Mocha ought to tell the user what's keeping the tests running (and then it wouldn't even need to keep the process open; it could just be a reason to return failure after they're done), but haven't found an entirely satisfactory way to do so yet. Version 4 didn't get the amount of time we wanted to put into it, as it was prompted by our CI failing due to changes in PhantomJS 1.x's installer (package-lock.json probably would have prevented this if we'd had it set up beforehand, but we still can't get it working). why-is-node-running was the one tool we found to help, but we don't think we can integrate it (between the requirement of --expose-internals and lack of a good way to programmatically get its output); I have found that it does work if you follow a couple steps:

  • run Mocha with --expose-internals (node --expose-internals node_modules/mocha/bin/_mocha)
  • make your first test file (e.g. listed in mocha.opts) contain (or at least begin with) after(require('why-is-node-running'))

...so it's better than nothing (although I'll see if I can update the release notes to describe this in more detail), but if anyone knows a better option please let us know!

Also sorry about missing the site documentation -- we'll get it updated as soon as possible. (Once #2987 is done we can even make it an automatic part of our version/publish scripting!)

@borisovg

This comment has been minimized.

Copy link

borisovg commented Oct 3, 2017

@ScottFreeCode

borisov@glossy:~/test/mocha $ node --version
v6.11.3

borisov@glossy:~/test/mocha $ ./node_modules/.bin/mocha --version
4.0.0

borisov@glossy:~/test/mocha $ ./node_modules/.bin/mocha --expose-internals test.spec.js 
  error: unknown option `--expose-internals'

Edit:

This works:

node --expose-internals ./node_modules/.bin/mocha test.spec.js
@ScottFreeCode

This comment has been minimized.

Copy link
Contributor

ScottFreeCode commented Oct 3, 2017

Right, sorry I wasn't clear about that -- --expose-internals is a Node option that can be used by running node --expose-internals ./node_modules/mocha/bin/_mocha instead of ./node_modules/.bin/mocha. That's also something we can fix, since Mocha passes certain flags on to Node and we can add --expose-internals to those flags.

@ScottFreeCode

This comment has been minimized.

Copy link
Contributor

ScottFreeCode commented Oct 3, 2017

Created mochajs/old-mochajs-site#81 and #3045 to track updating the documentation and Mocha's Node flags, respectively. Will keep this issue open at least till the documentation is updated.

@mceachen

This comment has been minimized.

Copy link

mceachen commented Oct 3, 2017

@ScottFreeCode you may want to mention that --expose-internals only works for node <= 6. Hopefully people can downgrade to node 6 temporarily so they can find what timer needs to be cancelled or unref'ed, or sockets needing to be unref'ed. You may also want to point people to after and afterEach hooks to do cleanup.

(is there a global "after" hook that mocha calls when all tests are complete?)

In any event, I appreciate the help, and thanks for Mocha!

@ScottFreeCode

This comment has been minimized.

Copy link
Contributor

ScottFreeCode commented Oct 3, 2017

you may want to mention that --expose-internals only works for node <= 6.

Are you sure? I tested with Node 8.6.0. Albeit not on all OSes, so I guess I should fire up every box I have and triple-check...

@boneskull

This comment has been minimized.

Copy link
Member

boneskull commented Oct 4, 2017

@boneskull boneskull added question and removed documentation labels Oct 4, 2017

@boneskull

This comment has been minimized.

Copy link
Member

boneskull commented Oct 4, 2017

Are you sure? I tested with Node 8.6.0. Albeit not on all OSes, so I guess I should fire up every box I have and triple-check...

This "works" in that it triggers the output from the module, but it doesn't actually give me much information, regardless of the version of Node.js.

I'm going to add some updates to the site, but please see my forthcoming comments on #3045.

@ScottFreeCode

This comment has been minimized.

Copy link
Contributor

ScottFreeCode commented Oct 4, 2017

This "works" in that it triggers the output from the module, but it doesn't actually give me much information, regardless of the version of Node.js.

It gives a useful stacktrace for setTimeout and setImmediate, but no real info at all for other things such as a listening server (as I just recently found out while trying to understand how it's doing what it's doing). The "async-dump" example in the gist has a real advantage in terms of working for everything (and not requiring --expose-internals), even though it's only useable on newer versions of Node.

@boneskull

This comment has been minimized.

Copy link
Member

boneskull commented Oct 4, 2017

I suppose my general advice would be "if you're pulling your hair out, just add --exit and relax". Then don't use it for your next project 😉

@boneskull

This comment has been minimized.

Copy link
Member

boneskull commented Oct 4, 2017

...and here are the docs

@boneskull boneskull closed this Oct 4, 2017

@boneskull boneskull added the faq label Oct 4, 2017

mkozjak added a commit to elpheria/rpc-websockets that referenced this issue Oct 10, 2017

package.json: exit mocha after tests are done; mochajs/mocha#3044
Signed-off-by: Mario Kozjak <kozjakm1@gmail.com>
@andywer

This comment has been minimized.

Copy link

andywer commented Oct 10, 2017

Sorry to comment on the closed PR again, but there might a user-friendly solution here:

One could log a warning about the changed behavior after the runner has finished for 3s or so, but the process didn't exit.
Would just need to add a setTimeout() after the runner finished and call timeout.unref() to make sure this timeout doesn't prevent the process from exiting. If the timeout gets executed it's time for a warning 😉

@boneskull

This comment has been minimized.

Copy link
Member

boneskull commented Oct 10, 2017

I’ve considered that but I’m wary that it will cause other problems with reporters or other integrations... I can’t prove it won’t.

@ekulabuhov

This comment has been minimized.

Copy link

ekulabuhov commented Apr 20, 2018

wtfnode didn't really work for me. All I could see was that some PID started by mocha was hanging.

On the other hand, @boneskull gist worked: #3044 (comment). Thank you!

@boneskull

This comment has been minimized.

Copy link
Member

boneskull commented Apr 20, 2018

wtfnode needs to be run against _mocha, not mocha.

@A-lee-201

This comment has been minimized.

Copy link

A-lee-201 commented Jun 1, 2018

Thanks for the --exit. It fixed it for me.

mtth added a commit to mtth/avsc that referenced this issue Jun 9, 2018

Update dependencies
Also tweak test to terminate even with mocha 4+ [1].

[1] mochajs/mocha#3044

@y-luis y-luis referenced this issue Jun 11, 2018

Open

Mocha tests do not exit #108

1 of 3 tasks complete

mbana added a commit to OpenBankingUK/tpp-reference-server that referenced this issue Jun 20, 2018

@kibertoad kibertoad referenced this issue Jul 9, 2018

Merged

Update mocha #2701

@ProfJigsaw

This comment has been minimized.

Copy link

ProfJigsaw commented Aug 13, 2018

enter this script in package.json:
"scripts": {
"test": "mocha --exit"
}

@mjgs

This comment has been minimized.

Copy link

mjgs commented Sep 19, 2018

I tried various remedies mentioned in this thread:

  • why-is-node-running produces no output
  • wtfnode produces output but not enough to determine where the File descriptors/Sockets/Servers/Timers are created in the code
  • async_hooks debug method errors
  • adding –exit to mocha cmd line flag results in exit

So follow @ProfJigsaw and use –exit, that’s what I am doing. Would be great if there was an improvement by mocha though, at least a way to figure out where issues are and how to resolve them.

@boneskull

This comment has been minimized.

Copy link
Member

boneskull commented Sep 19, 2018

That's what a debugger's for, IMO. How would you debug your own scripts if they never exited?

https://github.com/GoogleChromeLabs/ndb is a cool project.

@borisovg

This comment has been minimized.

Copy link

borisovg commented Sep 19, 2018

Off-topic: I love this ticket, just for the information on troubleshooting tools it keeps on giving! :)

@mjgs

This comment has been minimized.

Copy link

mjgs commented Sep 19, 2018

@borisovg Imagine how great this ticket would be if it actually provided a solution! :)
@boneskull Is there some "list File descriptors/Sockets/Servers/Timer" debugger feature that I am not aware of?

@borisovg

This comment has been minimized.

Copy link

borisovg commented Sep 19, 2018

@mjgs it did for me - wtfnode worked but only when I used it with the _mocha binary, not the mocha one:

$ wtfnode ./node_modules/.bin/_mocha my-shitty-code.spec.js 


  tests
    ✓ some test


  1 passing (5ms)

^C[WTF Node?] open handles:
- File descriptors: (note: stdio always exists)
  - fd 1 (tty) (stdio)
  - fd 2 (tty) (stdio)
- Servers:
  - :::8080 (HTTP)
    - Listeners:
      - request: (anonymous) @ /home/borisov/test/my-shitty-code.js:4
- Intervals:
  - (5000 ~ 5 s) (anonymous) @ /home/borisov/test/my-shitty-code.js:8
@mjgs

This comment has been minimized.

Copy link

mjgs commented Sep 20, 2018

@borisovg Thanks for your really super example. In my case wtfnode shows there is a mongo connection open, but all mongo connections in my my-shitty-code.js have been closed, so the problem is from somewhere else, and wtfnode doesn't give enough info as to where it is from.

@evert

This comment has been minimized.

Copy link

evert commented Sep 20, 2018

If wtfnode indicates that a mongo db connection is open, then it's open. Why would you assume it's wrong?

@mjgs

This comment has been minimized.

Copy link

mjgs commented Sep 20, 2018

@evert I think you might have misinterpreted what I wrote

Minimal examples are great for illustration, and it's actually useful to see similar output to what you are seeing, but in real code that has been written over many years these connections aren't so easy to find. It's good to know that there are mongo connections still open, but to actually find them is non-trivial. I eventually found some open connections, but they were much much deeper in the code than the surface level where all connections had been verifiably closed, wtfnode didn't help in identifying where they were only that they existed by listing the mongoose module path. I still have some to track down, based only on port number, but I am hopeful.

Igor-Lopes added a commit to Nucleus-Inc/MeanStarter that referenced this issue Oct 15, 2018

@randonia

This comment has been minimized.

Copy link

randonia commented Oct 19, 2018

I ran into this very issue writing a Serverless function that communicated with Firebase. Turns out their admin SDK maintains an open handle indefinitely. Because of this change (and the subsequent suggestion to use wtfnode, I was able to identify that fact and save myself a ton of headache (and costs) down the road.

In my opinion, it would be very helpful to include some sort of "if it hangs for X long and doesn't have any output throw some text to stdout" logic. I'm unsure how feasible that is or how much bandwidth is available to produce such an improvement, but I think that might help alleviate some initial frustration of "wtf is going on with my mocha!"

imlucas added a commit to mongodb-js/data-service that referenced this issue Jan 16, 2019

chore: We can now upgrade to mocha@5 🌮
## Symptoms

Upgrading to `mocha@5` now causes my tests to hang without any context. [See this Travis build as an example](https://travis-ci.org/mongodb-js/index-model/builds/451375910)

## Resolution

We were *not* explicitly closing driver connections after tests that needed it finished which kept the driver's underlying sockets to remain open. These `after(client.close)` blocks have been added in this PR.

## Root Cause

`mocha@4` introduced a reenforcement for better testing and quality with the removal of mocha killing its process when it *thinks* the tests complete. [See release notes](https://boneskull.com/mocha-v4-nears-release/#mochawontforceexit)

For us, this is a huge win. `mocha` has always included its own process management which allowed us to play fast and loose releasing fds/sockets/timeouts/etc properly.

## Diagnosis

Googling lead me to [this mocha issue](mochajs/mocha#3044) where the maintainer provides several solutions to debug this and make your tests even *more* correct. Here's what worked for me:

```bash
npm i -g wtfnode;
wtfnode ./node_modules/.bin/_mocha test/data-service.test.js;
# Wait for hang to appear
# Hit ctrl+c which produces the output below
```

```
  34 passing (1s)

^C[WTF Node?] open handles:
- File descriptors: (note: stdio always exists)
  - fd 1 (tty) (stdio)
  - fd 2 (tty) (stdio)
- Sockets:
  - 127.0.0.1:57887 -> 127.0.0.1:27018
  - 127.0.0.1:57888 -> 127.0.0.1:27018
  - 127.0.0.1:57890 -> 127.0.0.1:27018
  - 127.0.0.1:57891 -> 127.0.0.1:27018
- Timers:
  - (5000 ~ 5 s) (anonymous) @ /Users/lucas/data-service/node_modules/mongodb-core/lib/topologies/server.js:373
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.