-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Vitest run times very slow on a AWS T3.xlarge instance vs Jest with same configuration #5668
Comments
The latest version of Vitest 1.6.0 made the performance on AWS slower. Jest takes only 500 seconds. Below the best time is [2024-06-11T23:31:20.136Z] Benchmark 1: CI=true node --no-compilation-cache --max-old-space-size=1024 node_modules/vitest/dist/cli.js --coverage.enabled=true --minWorkers=1 --maxWorkers=1 [2024-06-12T01:42:57.414Z] Time (mean ± σ): 1944.480 s ± 63.104 s [User: 1802.588 s, System: 232.582 s] [2024-06-12T01:42:57.414Z] Range (min … max): 1901.200 s … 2016.887 s 3 runs [2024-06-12T01:42:57.414Z] Benchmark 2: CI=true node --no-compilation-cache --max-old-space-size=1024 node_modules/vitest/dist/cli.js --coverage.enabled=true --minWorkers=2 --maxWorkers=2 [2024-06-12T02:58:59.407Z] Time (mean ± σ): 1130.736 s ± 33.154 s [User: 1986.188 s, System: 270.670 s] [2024-06-12T02:58:59.407Z] Range (min … max): 1094.696 s … 1159.938 s 3 runs [2024-06-12T02:58:59.407Z] Benchmark 3: CI=true node --no-compilation-cache --max-old-space-size=1024 node_modules/vitest/dist/cli.js --coverage.enabled=true --minWorkers=3 --maxWorkers=3 [2024-06-12T04:01:51.877Z] Time (mean ± σ): 944.267 s ± 14.383 s [User: 2240.765 s, System: 318.537 s] [2024-06-12T04:01:51.877Z] Range (min … max): 928.660 s … 956.989 s 3 runs [2024-06-12T04:01:51.877Z] Benchmark 4: CI=true node --no-compilation-cache --max-old-space-size=1024 node_modules/vitest/dist/cli.js --coverage.enabled=true --minWorkers=4 --maxWorkers=4 [2024-06-12T04:56:39.297Z] Time (mean ± σ): 822.763 s ± 6.932 s [User: 2346.223 s, System: 347.783 s] [2024-06-12T04:56:39.297Z] Range (min … max): 818.183 s … 830.738 s 3 runs [2024-06-12T04:56:39.297Z] Benchmark 5: CI=true node --no-compilation-cache --max-old-space-size=1024 node_modules/vitest/dist/cli.js --coverage.enabled=true --minWorkers=5 --maxWorkers=5 [2024-06-12T05:58:32.352Z] Time (mean ± σ): 924.836 s ± 21.396 s [User: 2408.339 s, System: 384.649 s] [2024-06-12T05:58:32.352Z] Range (min … max): 904.333 s … 947.025 s 3 runs |
Try using |
[sheremet-va] I tried vmThreads and I can't anywhere close to 500 seconds that jest offers. Jest is set to use only 25% of CPUs 2024-06-12T21:29:17.058Z] > insights@0.0.0-build.2215 test:CI [2024-06-12T21:29:17.058Z] > hyperfine --parameter-scan num_threads 1 15 -m 3 -w 1 'CI=true node --no-compilation-cache --max-old-space-size=1024 node_modules/vitest/dist/cli.js --coverage.enabled=true --minWorkers={num_threads} --maxWorkers={num_threads}' [2024-06-12T21:29:17.058Z] [2024-06-12T21:29:17.479Z] Benchmark 1: CI=true node --no-compilation-cache --max-old-space-size=1024 node_modules/vitest/dist/cli.js --coverage.enabled=true --minWorkers=1 --maxWorkers=1 [2024-06-12T23:20:46.224Z] Time (mean ± σ): 1670.071 s ± 31.991 s [User: 1676.073 s, System: 211.337 s] [2024-06-12T23:20:46.224Z] Range (min … max): 1635.569 s … 1698.753 s 3 runs [2024-06-12T23:20:46.224Z] [2024-06-12T23:20:46.224Z] Benchmark 2: CI=true node --no-compilation-cache --max-old-space-size=1024 node_modules/vitest/dist/cli.js --coverage.enabled=true --minWorkers=2 --maxWorkers=2 [2024-06-13T00:26:24.496Z] Time (mean ± σ): 983.366 s ± 1.193 s [User: 1886.827 s, System: 246.599 s] [2024-06-13T00:26:24.496Z] Range (min … max): 981.991 s … 984.117 s 3 runs [2024-06-13T00:26:24.496Z] [2024-06-13T00:26:24.496Z] [2024-06-13T00:26:24.496Z] Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. [2024-06-13T00:26:24.496Z] Benchmark 3: CI=true node --no-compilation-cache --max-old-space-size=1024 node_modules/vitest/dist/cli.js --coverage.enabled=true --minWorkers=3 --maxWorkers=3 [2024-06-13T01:22:56.433Z] [2024-06-13T01:22:56.434Z] Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. [2024-06-13T01:22:56.434Z] Time (mean ± σ): 816.370 s ± 7.728 s [User: 2123.947 s, System: 300.907 s] [2024-06-13T01:22:56.434Z] Range (min … max): 807.455 s … 821.154 s 3 runs [2024-06-13T01:22:56.434Z] [2024-06-13T01:22:56.434Z] Benchmark 4: CI=true node --no-compilation-cache --max-old-space-size=1024 node_modules/vitest/dist/cli.js --coverage.enabled=true --minWorkers=4 --maxWorkers=4 [2024-06-13T02:13:26.833Z] Time (mean ± σ): 755.576 s ± 3.423 s [User: 2278.970 s, System: 334.686 s] [2024-06-13T02:13:26.833Z] Range (min … max): 751.833 s … 758.549 s 3 runs [2024-06-13T02:13:26.833Z] [2024-06-13T02:13:26.833Z] Benchmark 5: CI=true node --no-compilation-cache --max-old-space-size=1024 node_modules/vitest/dist/cli.js --coverage.enabled=true --minWorkers=5 --maxWorkers=5 [2024-06-13T03:08:30.456Z] Time (mean ± σ): 826.621 s ± 2.273 s [User: 2315.224 s, System: 369.573 s] [2024-06-13T03:08:30.456Z] Range (min … max): 824.226 s … 828.748 s 3 runs |
Describe the bug
Hello,
I am writing because I see high collection times when I run our 1100 React tests with Vitest in docker on AWS T3.xlarge instance.
On my local environment I get the following metrics on Macbook Pro M1 (8 CPUs). The test run takes here only 130 seconds.
In Docker on AWS T3.xlarge instance (4CPU) I see the following. The test run takes 680 seconds. Why is the collect number os high?
In Docker on AWS T3.xlarge instance (4CPU) using Jest. The test run takes 495 seconds.
Any Idea why this would happen?
PS:
I am using the following versions:
I use the following setting in package.json to run the test on the AWS machine and I have tried
pool=theads
but it gets slower!CI=true vitest run --coverage.enabled=true --pool=forks --minWorkers=8 --maxWorkers=8
Below is my vitets.config.ts file
setup.js
looks like the following:
Reproduction
I have 1111 test and I am not able to reproduce this in smaller scale
System Info
Used Package Manager
npm
Validations
The text was updated successfully, but these errors were encountered: