Output logs for tests that remain pending when AVA exits #3125

kevo1ution · 2022-10-25T01:28:49Z

What
This PR plumbs test logs to be printed when tests timeout due to the global timeout set by ava --timeout <timeout> command.

Why
t.log is a very useful utility for debugging what happens when a test fails. When a test times out, these logs are useful for figuring out why the tests timed out.

unecessary dif temp add logs for pending tests fix linting issues undo unecessary change

kevo1ution · 2022-10-25T04:06:17Z

@novemberborn

This is the output with the same code snippet on #3122 . Before I attempt writing test cases for this, do you think this looks good implementation-wise and output-wise?

 % npm run test

> ava-logging-timeout-bug@1.0.0 test
> npx ava --timeout=1s --verbose


start promise that never resolves...
  
  ✘ Timed out while running tests

  1 tests were pending in test/example.js

  ◌ example where logs are not provided by ava when test time out
    ℹ {
        message: 'Very important debug statement, that developer would love to see in output after test run',
      }

  ─

  1 test remained pending after a timeout

kevo1ution · 2022-10-25T04:07:31Z

lib/run-status.js

@@ -9,6 +9,7 @@ export default class RunStatus extends Emittery {
 		super();

 		this.pendingTests = new Map();
+		this.pendingTestsLogReference = new Map();


separate map from this.pendingTests because they are modified and initialized in different events with different objects.

Why Reference?

oops, I think pendingTestsLogs sounds better

novemberborn · 2022-11-01T09:29:30Z

@kevo1ution this is on my list to look at more closely, but at first glance it seems reasonable.

kevo1ution · 2022-11-01T16:46:40Z

@kevo1ution this is on my list to look at more closely, but at first glance it seems reasonable.

thanks I'll start adding several tests to confirm that the output works, and mark this ready for review when it's ready to be looked at

kevo1ution · 2022-11-01T18:52:40Z

@novemberborn This is ready for review. It took some time to figure out how testing the testing framework works (lol), so hopefully, I did it up to your standards. Let me know if there is anything I missed

kevo1ution · 2022-11-04T21:01:29Z

@novemberborn bumping on this, let me know if there's anything else I need to do, thanks!

novemberborn

Nice! Left some comments and suggestions around optional chaining.

novemberborn · 2022-11-07T16:13:16Z

lib/reporters/default.js

@@ -370,8 +370,11 @@ export default class Reporter {
 			}

 			this.lineWriter.writeLine(`${testsInFile.size} tests were pending in ${this.relativeFile(file)}\n`);
+			const testTitleToLogs = evt.pendingTestsLogReference.get(file) ?? new Map();


Suggested change

const testTitleToLogs = evt.pendingTestsLogReference.get(file) ?? new Map();

const testTitleToLogs = evt.pendingTestsLogReference.get(file);

novemberborn · 2022-11-07T16:13:16Z

lib/reporters/default.js

 			for (const title of testsInFile) {
+				const logs = testTitleToLogs.get(title) ?? [];


writeLogs() can handle falsey logs values. So with optional chaining and the above we can do:

Suggested change

const logs = testTitleToLogs.get(title) ?? [];

const logs = testTitleToLogs?.get(title);

novemberborn · 2022-11-07T16:13:16Z

lib/run-status.js

@@ -9,6 +9,7 @@ export default class RunStatus extends Emittery {
 		super();

 		this.pendingTests = new Map();
+		this.pendingTestsLogReference = new Map();


Why Reference?

novemberborn · 2022-11-07T16:13:16Z

lib/run-status.js

+		if (this.pendingTestsLogReference.has(event.testFile)) {
+			this.pendingTestsLogReference.get(event.testFile).set(event.title, event.logs);
+		}


Suggested change

if (this.pendingTestsLogReference.has(event.testFile)) {

this.pendingTestsLogReference.get(event.testFile).set(event.title, event.logs);

}

this.pendingTestsLogReference.get(event.testFile)?.set(event.title, event.logs);

novemberborn

Lovely!

kevo1ution · 2022-11-13T22:02:44Z

Lovely!

Thank you for the helpful reviews and insight on fixing the issue! Looking forward to using this fix in my repos

adding notes before making change

ce2b12e

unecessary dif temp add logs for pending tests fix linting issues undo unecessary change

kevo1ution force-pushed the fix-issue-3222-log-after-timeout branch from 0ff9efc to ce2b12e Compare October 25, 2022 04:03

kevo1ution commented Oct 25, 2022

View changes

novemberborn marked this pull request as ready for review November 1, 2022 09:28

novemberborn marked this pull request as draft November 1, 2022 09:28

kevo1ution added 2 commits November 1, 2022 11:34

add tests

1937048

add tests for sigint printing pending tests

98bbc25

kevo1ution marked this pull request as ready for review November 1, 2022 18:50

kevo1ution changed the title ~~Output logs for each test after global timeout is triggered~~ Output logs for pending tests after global timeout or SIGINT is triggered Nov 3, 2022

chore: address feedback

29ed720

novemberborn reviewed Nov 7, 2022

View changes

kevo1ution requested a review from novemberborn November 7, 2022 22:26

novemberborn changed the title ~~Output logs for pending tests after global timeout or SIGINT is triggered~~ Output logs for tests that remain pending when AVA exits Nov 13, 2022

Merge branch 'main' into fix-issue-3222-log-after-timeout

82231fc

novemberborn approved these changes Nov 13, 2022

View changes

novemberborn merged commit 9206928 into avajs:main Nov 13, 2022
13 checks passed

kevo1ution deleted the fix-issue-3222-log-after-timeout branch November 13, 2022 22:02

Output logs for tests that remain pending when AVA exits #3125

Output logs for tests that remain pending when AVA exits #3125

kevo1ution commented Oct 25, 2022 •

edited

kevo1ution commented Oct 25, 2022 •

edited

kevo1ution Oct 25, 2022

novemberborn Nov 7, 2022

kevo1ution Nov 7, 2022

novemberborn commented Nov 1, 2022

kevo1ution commented Nov 1, 2022

kevo1ution commented Nov 1, 2022 •

edited

kevo1ution commented Nov 4, 2022

novemberborn left a comment

novemberborn Nov 7, 2022

novemberborn Nov 7, 2022

novemberborn Nov 7, 2022

novemberborn Nov 7, 2022

novemberborn left a comment

kevo1ution commented Nov 13, 2022 •

edited

	const testTitleToLogs = evt.pendingTestsLogReference.get(file) ?? new Map();
	const testTitleToLogs = evt.pendingTestsLogReference.get(file);

		for (const title of testsInFile) {
		const logs = testTitleToLogs.get(title) ?? [];

	const logs = testTitleToLogs.get(title) ?? [];
	const logs = testTitleToLogs?.get(title);

Output logs for tests that remain pending when AVA exits #3125

Output logs for tests that remain pending when AVA exits #3125

Conversation

kevo1ution commented Oct 25, 2022 • edited

kevo1ution commented Oct 25, 2022 • edited

kevo1ution Oct 25, 2022

Choose a reason for hiding this comment

novemberborn Nov 7, 2022

Choose a reason for hiding this comment

kevo1ution Nov 7, 2022

Choose a reason for hiding this comment

novemberborn commented Nov 1, 2022

kevo1ution commented Nov 1, 2022

kevo1ution commented Nov 1, 2022 • edited

kevo1ution commented Nov 4, 2022

novemberborn left a comment

Choose a reason for hiding this comment

novemberborn Nov 7, 2022

Choose a reason for hiding this comment

novemberborn Nov 7, 2022

Choose a reason for hiding this comment

novemberborn Nov 7, 2022

Choose a reason for hiding this comment

novemberborn Nov 7, 2022

Choose a reason for hiding this comment

novemberborn left a comment

Choose a reason for hiding this comment

kevo1ution commented Nov 13, 2022 • edited

kevo1ution commented Oct 25, 2022 •

edited

kevo1ution commented Oct 25, 2022 •

edited

kevo1ution commented Nov 1, 2022 •

edited

kevo1ution commented Nov 13, 2022 •

edited