"Timed out while running tests" #3036
-
|
I've just started receiving more and more timed out. TBH It is rare to run proper tests... My config: process.env.TS_NODE_PROJECT = './tsconfig.test.json';
module.exports = {
failWithoutAssertions: false,
extensions: [ 'js', 'ts' ],
files: [ './src/**/tests/**/*.ts', '!./src/**/tests/_utils/**/*.ts' ],
require: [ 'ts-node/register', 'source-map-support/register' ]
};I'm not sure what else I could add here - because there is no errors in the console, only Increasing the timeout in config or in the single test does not help at all... It looks like the async tests cause this trouble: AVA: Originally posted by @sculpt0r in #2494 (comment) |
Beta Was this translation helpful? Give feedback.
Replies: 14 comments 31 replies
-
|
I've managed to stabilize tests by adding: workerThreads: false,
timeout: '80s'But I would rather say that it looks like a workaround rather than a solution. Does the test cases total amount, test files amount or test cases per file somehow impact the performance? And that might influence the amount of the timeout? One additional thing: I'm managing the nodeJS versions via |
Beta Was this translation helpful? Give feedback.
-
|
Timeouts occur when there is no single test passing for the timeout duration (defaulting to 10 seconds if I'm not mistaken). Essentially if your test suite isn't making progress, then AVA gives up.
I wonder if this is related to Have you tried with precompiled files using |
Beta Was this translation helpful? Give feedback.
-
|
Not yet, but I will give it a try - thanks. I will provide te results here or any other observations - thank you for the response :) |
Beta Was this translation helpful? Give feedback.
-
|
Added, the @ava/typescript, and in the config I put an extra: Still need to have Also, I'm pasting the text for sake of copy/paste Funny thing, the CI seems to be more stable (or there is no issue at all) 🤔 I didn't mentioned, but I have mac with M1, however the problem also appears on linux 🤔 |
Beta Was this translation helpful? Give feedback.
-
|
Would be interesting to try with Your crash is something in Node.js itself unfortunately. |
Beta Was this translation helpful? Give feedback.
-
|
Could you guide me a little bit on how to properly set up the |
Beta Was this translation helpful? Give feedback.
-
|
@sculpt0r this may be worth a separate discussion. I'd be eager to see where you're getting stuck so we can improve the documentation. |
Beta Was this translation helpful? Give feedback.
-
|
I'm seeing this issue too: sindresorhus/is#167
|
Beta Was this translation helpful? Give feedback.
-
|
@sindresorhus the issue @sculpt0r is describing below could be related to globbing timing out. I've tried with your PR and that's not the case. I'm inclined to blame |
Beta Was this translation helpful? Give feedback.
-
|
Another fact: and the project root directory is: Via VSCode plugin the command is run in the terminal: cd /Users/XXX/XXXzzzzz/yyy-qqqqqqqqq && npm test /XXX/XXXzzzzz/yyy-qqqqqqqqq/src/ccc-bbbbbb/tests/index.tswhich leads to run: ava --verbose "/Users/XXX/XXXzzzzz/yyy-qqqqqqqqq/src/ccc-bbbbbb/tests/index.ts"The file contains: import test from 'ava';
test( 'pkg exports public api', async t => {
t.pass();
} );Result: However, if I run the same high-level command, but without cd /Users/XXX/XXXzzzzz/yyy-qqqqqqqqq && npm test /XXX/XXXzzzzz/yyy-qqqqqqqqq/src/ccc-bbbbbb/tests/which leads to run: ava --verbose "/Users/XXX/XXXzzzzz/yyy-qqqqqqqqq/src/ccc-bbbbbb/tests/"I received the time out: Maybe sth in selecting files to tests leads to timeout? Sorry for the weird path obfuscating - but the project is private and I'm not allowed to publish any information about it. |
Beta Was this translation helpful? Give feedback.
-
|
☝️ The same happens whether I set the config files: [ './src/yyy-qqqqqqqqq/tests/**/*.ts' ],but this one is working: files: [ './src/yyy-qqqqqqqqq/tests/**/index.ts' ], |
Beta Was this translation helpful? Give feedback.
-
|
For my project - it started when we upgrade to version: |
Beta Was this translation helpful? Give feedback.
-
|
Interesting! The timeout is started before we search for the test files. I think the globbing could do with its own timeout and error message. @sculpt0r could you install Let's see if the timeout occurs here. |
Beta Was this translation helpful? Give feedback.
-
|
Just run the "full tests run" with the flag |
Beta Was this translation helpful? Give feedback.
-
|
The same for the run with the 'directory' filter: tests passed & no timed outs. |
Beta Was this translation helpful? Give feedback.
-
Looks like it does - all files are selected 👍 |
Beta Was this translation helpful? Give feedback.
-
|
As for the test run with directory filter: the timed out starts to appear when there are 5 test files. If I decrease the amount to 4 files - the tests run properly 🤔 <- With my standard config - so without |
Beta Was this translation helpful? Give feedback.
-
|
And today it hits also the CI 🤔 This time |
Beta Was this translation helpful? Give feedback.
-
|
@novemberborn - I was able to receive the results you were asking for before. With just Please don't be mad that I'm posting so many times in this discussion - but the project is important and I want to provide as many details as I'm able to. Hopefully, they can help to find a bug. Is there a way to tell AVA to output much more details? I mean besides |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the explanation - I will try again :) |
Beta Was this translation helpful? Give feedback.
-
|
One thing I just noticed with |
Beta Was this translation helpful? Give feedback.
-
|
Well 😅 I just look into the activity monitor while I run a full test run and... EDIT: |
Beta Was this translation helpful? Give feedback.
-
|
Yes, I'm seeing the same with |
Beta Was this translation helpful? Give feedback.
-
|
And one more observation - the time out also appears whether some test files have errors - like importing a module that is not exported from another file 🤔 |
Beta Was this translation helpful? Give feedback.
-
|
👋 I have finally managed to precompile the test - thank you for the support in avajs/typescript#43. This required a lot of 'fun' and I need to change the way how some tests are written, but... The prebuilded tests are many times faster than before. Also - there are no timeouts! So I adjust the compile with But if you are still interested in some debugging why the timeouts come out - I'll be happy to help with my 'problematic' project ;p For the next week I won't respond anything - but after that, I'll be back :) Thank you for your efforts with this issue. |
Beta Was this translation helpful? Give feedback.
-
|
I managed to solve the problem, at least for me. The solution was annoyingly simple. Just add |
Beta Was this translation helpful? Give feedback.
-
Yes
The old behavior. It times out. And if I increase the timeout to 5 minutes, it eventually crashes for overuse of memory. |
Beta Was this translation helpful? Give feedback.
-
I mean, what is it trying to do? You'd think transpiling is the default behavior! |
Beta Was this translation helpful? Give feedback.
-
|
@novemberborn - Actually I marked as a valid answer your original first answer some time ago. Your answer seems to be more valid to me. Tests run fine whether they are precompiled and it is not depend on what stuff was used by te user ( |
Beta Was this translation helpful? Give feedback.
-
Looks like it skips type checking? |
Beta Was this translation helpful? Give feedback.
-
FWIW, it's already there, but more explanation is probably needed, and it should probably be recommended as the default option
|
Beta Was this translation helpful? Give feedback.
-
|
I've been battling this for a while but finally figured it out (in my instance). Ava intercepts timers in some way and doesn't allow tests to finish until timers (i.e. I understand why this is something you'd want, particularly for async tests, but I'm not sure the way Ava does it right now is the best. Here are a few suggestions... Error messagesThe tests timing out is a very generic error, as this thread illustrates. I've seen other test frameworks (not in JS/TS) that either raise exceptions by default at the end of a test if there's still ongoing async work like timers, or they log unowned computation in some way. Saying at the end of a test "Warning: there's still a timer running" would have pointed to the solution immediately. DocumentationIt was unclear from documentation that this happens. A mention or callout of some kind on the troubleshooting docs or the timeouts docs would be great in helping to track this down. I lost a few hours to this bug, and probably checked the documentation only a few minutes in. If this exists and I've just not searched for the right thing, apologies, although maybe the text can get a few more searchable keywords in it. Assertion behaviourThe real benefit from waiting on timers would be for tests to be able to assert. My problem was that I ran a bunch of tests that set timers, and the tests all passed, but then the process timed out. What should have happened was that the individual tests that set timers should have timed out and failed. If this had happened I'd have spotted the issue much earlier, in the end I only figured out the issue after commenting out tests progressively until the tests no longer timed out. Control over timersOnce I figured out the issue I tried to find a good way to solve it. In an environment that I have more familiarity with (Python, Java, etc), I'd use a fake timer of some kind or a system framework hook to control the time and fast-forward time in the tests so that I could test the timer behaviour. That way the actual time or the time that tests take to run has no impact on the reliability of the test or its ability to assert about the system behaviour. Unfortunately for now I've had to just set the timer to 1s in tests, knowing it's unlikely for any test to take longer than that, and I've had to skip testing the case where the timer fires for now (or would need to write a test that would take >1s to run). Some support from Ava in this would be great. |
Beta Was this translation helpful? Give feedback.
-
This isn't what's happening. The long-running timer is preventing the test worker from exiting on its own. That's why the error message says "Failed to exit…". Happy for a PR that changes that message to include some suggestions. I don't know how feasible it is to describe the specific handles that prevent Node.js from exiting. Even "there's a long-running timer" may not be helpful since it could come from a dependency you've loaded. There are fake timer implementations available, Also, you can use |
Beta Was this translation helpful? Give feedback.
-
|
I had some inconsistent hang up on tests while using Typescript with AVA. On a clean repo, tests would take forever or simply hang indefinitely. But subsequent tests run would resolve in ms. I personally soved it by starting Might be related to the fact "that import declarations happen in parallel before the code is executed" as stated in the |
Beta Was this translation helpful? Give feedback.
-
|
Hey guys, for those running out of patience, here is a nasty workaround that worked for me: import test from 'ava';
const originalSetTimeout = global.setTimeout;
const originalClearTimeout = global.clearTimeout;
const activeTimeouts = new Set();
global.setTimeout = (callback, delay, ...args) => {
const timeoutId = originalSetTimeout(() => {
activeTimeouts.delete(timeoutId);
callback(...args);
}, delay);
activeTimeouts.add(timeoutId);
return timeoutId;
};
global.clearTimeout = (timeoutId) => {
if (activeTimeouts.has(timeoutId)) {
activeTimeouts.delete(timeoutId);
originalClearTimeout(timeoutId);
}
};
function clearAllTimeouts() {
for (const timeoutId of activeTimeouts) {
originalClearTimeout(timeoutId);
activeTimeouts.delete(timeoutId);
}
}
global.clearAllTimeouts = clearAllTimeouts;
test.afterEach.always('cleanup', (t) => {
clearAllTimeouts();
});Just load this file before running your tests. |
Beta Was this translation helpful? Give feedback.
Timeouts occur when there is no single test passing for the timeout duration (defaulting to 10 seconds if I'm not mistaken). Essentially if your test suite isn't making progress, then AVA gives up.
asyncor not depends on what your test is actually doing. It sounds like it's synchronous (otherwise why would it pass withoutasync?). Technically then the tests still run synchronously, it's just the outcome that is observed asynchronously.I wonder if this is related to
ts-node, #3031 suggests a timeout problem withts-node. That's a little outside of AVA though.Have you tried with precompiled files using
@ava/typescript?