Repo: Our EXPERIMENTAL_useProjectService unit tests are much slower in CI

[JoshuaKGoldberg]

Suggestion

Taking 18ea3b1's https://github.com/typescript-eslint/typescript-eslint/actions/runs/5588135268/ as an example:

• eslint-plugin run without the flag: 8m 6s
• eslint-plugin run with the flag: 34m 9s

The Node version is 18 with the flag and 16 or 20 without, but that's just a quirk of the CI file. I've found it to be slower locally as well. This should be investigated.

@bradzacher and I noticed this when we first merged. cc @jakebailey - if you have the time, we'd appreciate any tips you have on it. We haven't investigated deeply yet.

[jakebailey]

Does each test fully spin up a new ProjectService? It's not a cheap operation; starting tsserver takes a bit to get going so if it's fresh each time I suspect that it won't be cheap.

But, if it's all in one process, I should be able to profile it pretty easily too.

[JoshuaKGoldberg]

each test fully spin up a new ProjectService

Yeah, I believe so. We've previously tried to keep tests pretty isolated from each other. Would it be reasonable to look into speeding up tsserver creation?

[rubiesonthesky]

Jest is not using isolated modules nor reset modules, so I think it's reusing a lot things. But because we implemented the worker idle memory limit, jest is creating new worker when going over the memory limit. This means that they probably are creating new ProjectService too. We could pump the memory limit to 500 MB instead of 300 MB and check does that help. I'm not sure what the max memory in that CI runner is. Like, you could also try 700 MB but then it may get to point, where the whole test setup crashes.

[bradzacher]

typescript-eslint/packages/rule-tester/src/RuleTester.ts

Lines 100 to 109 in 67f93b1

	constructor.afterAll(() => {
	try {
	// instead of creating a hard dependency, just use a soft require
	// a bit weird, but if they're using this tooling, it'll be installed
	const parser = require(TYPESCRIPT_ESLINT_PARSER) as typeof ParserType;
	parser.clearCaches();
	} catch {
	// ignored on purpose
	}
	});

We are clearing the project service after each test block finishes.