Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: fix jest memory flakiness #18859

Merged
merged 5 commits into from
Apr 27, 2023
Merged

ci: fix jest memory flakiness #18859

merged 5 commits into from
Apr 27, 2023

Conversation

millsp
Copy link
Member

@millsp millsp commented Apr 21, 2023 •
edited

My goal is to be able to transpile esm and cjs modules via jest, while also keeping the performance good. As it turned out, I tried to flip this switch by transpiling node_modules recently and that made our CI unstable because transpiling is expensive on memory. With @swc/jest it's kinda all or nothing unless we can dynamically craft a regex that includes esm modules to be transpiled - that's not ideal.

https://www.notion.so/prismaio/Findings-on-flaky-tests-de5b979a8441402784ae6eedd270f9dd?pvs=4#4218c00bcb304675a1e9671372de3d6c

Goals

So 3 goals for me: reasonable heap growth, good performance, and esm transpilation. Here's some rough benchmarks on my machine via pnpm jest --clearCache && pnpm run test:functional --provider postgresql (+ --logHeapUsage --runInBand).

Benchmarks

  1. As things were before recent changes on main (jest.config.js)
Details 
    transformIgnorePatterns: [
      '[\\/]node_modules[\\/]',
      escapeRegex(runtimeDir),
      `${escapeRegex(packagesDir)}[\\/][^\\/]+[\\/]dist[\\/]`,
    ],
 PASS  tests/functional/extensions/enabled/query.ts (163 MB heap size)
 PASS  tests/functional/tracing/tests.ts (10.115 s, 273 MB heap size)
 PASS  tests/functional/interactive-transactions/tests.ts (11.424 s, 273 MB heap size)
 PASS  tests/functional/extensions/enabled/model.ts (289 MB heap size)
 PASS  tests/functional/extensions/enabled/defineExtension.ts (455 MB heap size)
 PASS  tests/functional/methods/upsert/native-atomic/tests.ts (569 MB heap size)
 PASS  tests/functional/relationMode-17255-same-actions/tests.ts (9.069 s, 589 MB heap size)
 PASS  tests/functional/extensions/enabled/result.ts (687 MB heap size)
 PASS  tests/functional/referentialActions-setDefault/tests_1-to-1.ts (891 MB heap size)
 PASS  tests/functional/referentialActions-setDefault/tests_1-to-n.ts (1001 MB heap size)
 PASS  tests/functional/issues/16390-relation-mode-m-n-dangling-pivot/tests.ts (777 MB heap size)
 PASS  tests/functional/extensions/enabled/client.ts (974 MB heap size)
 PASS  tests/functional/fluent-api-null/tests.ts (1090 MB heap size)
 PASS  tests/functional/prisma-promise/tests.ts (1212 MB heap size)
 PASS  tests/functional/blog-update/tests.ts (1313 MB heap size)
 PASS  tests/functional/issues/8832/tests.ts (13.627 s, 1321 MB heap size)
 PASS  tests/functional/logging/tests.ts (1513 MB heap size)
 PASS  tests/functional/extensions/enabled/itx.ts (1185 MB heap size)
 PASS  tests/functional/field-reference/string/tests.ts (997 MB heap size)
 PASS  tests/functional/json-null-types/tests.ts (1111 MB heap size)
 PASS  tests/functional/fluent-api/tests.ts (1214 MB heap size)
 PASS  tests/functional/fulltext-search/tests.ts (1026 MB heap size)
 PASS  tests/functional/extensions/enabled/tx.ts (1140 MB heap size)
 PASS  tests/functional/filter-count-relations/tests.ts (995 MB heap size)
 PASS  tests/functional/relationMode-17255-mixed-actions/tests.ts (1094 MB heap size)
 PASS  tests/functional/field-reference/json/tests.ts (1050 MB heap size)
 PASS  tests/functional/extended-where/validation.ts (1159 MB heap size)
 PASS  tests/functional/metrics/enabled/tests.ts (947 MB heap size)
 PASS  tests/functional/optimistic-concurrency-control/tests.ts (1060 MB heap size)
 PASS  tests/functional/batch-transaction-isolation-level/tests.ts (1169 MB heap size)
 PASS  tests/functional/methods/findUniqueOrThrow/tests.ts (979 MB heap size)
 PASS  tests/functional/methods/findFirstOrThrow/tests.ts (1089 MB heap size)
 PASS  tests/functional/methods/count/tests.ts (1206 MB heap size)
 PASS  tests/functional/issues/14271/tests.ts (1092 MB heap size)
 PASS  tests/functional/query-error-logging/tests.ts (1126 MB heap size)
 PASS  tests/functional/raw-queries/send-type-hints/tests.ts (1228 MB heap size)
 PASS  tests/functional/order-by-null/tests.ts (1329 MB heap size)
 PASS  tests/functional/multi-schema/tests.ts (5.913 s, 1200 MB heap size)
 PASS  tests/functional/methods/create-many/tests.ts (1302 MB heap size)
 PASS  tests/functional/views/tests.ts (1400 MB heap size)
 PASS  tests/functional/mixed-string-uuid-datetime-list-inputs/tests.ts (1505 MB heap size)
 PASS  tests/functional/issues/5952-decimal-batch/tests.ts (1251 MB heap size)
 PASS  tests/functional/tracing-no-sampling/tests.ts (1358 MB heap size)
 PASS  tests/functional/field-reference/numeric/tests.ts (5.631 s, 1465 MB heap size)
 PASS  tests/functional/issues/13766/primary-key/tests.ts (1410 MB heap size)
 PASS  tests/functional/issues/12862-errors-are-obfuscated-by-interactive-transactions/tests.ts (1518 MB heap size)
 PASS  tests/functional/extended-where/create.ts (1395 MB heap size)
 PASS  tests/functional/issues/9326/tests.ts (1564 MB heap size)
 PASS  tests/functional/_example/tests.ts (1765 MB heap size)
 PASS  tests/functional/issues/14954-date-batch/tests.ts (1858 MB heap size)
 PASS  tests/functional/issues/13766/at-unique/tests.ts (1563 MB heap size)
 PASS  tests/functional/batching/tests.ts (1662 MB heap size)
 PASS  tests/functional/raw-queries/typed-results/tests.ts (1764 MB heap size)
 PASS  tests/functional/issues/12557/tests.ts (1723 MB heap size)
 PASS  tests/functional/issues/11233/tests.ts (1825 MB heap size)
 PASS  tests/functional/issues/6578/tests.ts (1920 MB heap size)
 PASS  tests/functional/runtime-import/tests.ts (1730 MB heap size)
 PASS  tests/functional/issues/4004/tests.ts (1927 MB heap size)
 PASS  tests/functional/issues/12378/tests.ts (1697 MB heap size)
 PASS  tests/functional/issues/11740-transaction-stored-query/tests.ts (1903 MB heap size)
 PASS  tests/functional/decimal/precision/tests.ts (10.03 s, 1472 MB heap size)
 PASS  tests/functional/raw-queries/typed-results-advanced-and-native-types/tests.ts (1572 MB heap size)
 PASS  tests/functional/decimal/scalar/tests.ts (1764 MB heap size)
 PASS  tests/functional/binary-engine/restart/tests.ts (1534 MB heap size)
 PASS  tests/functional/extended-where/findUnique.ts (1642 MB heap size)
 PASS  tests/functional/issues/9678/tests.ts (1841 MB heap size)
 PASS  tests/functional/tracing-disabled/tests.ts (1812 MB heap size)
 PASS  tests/functional/binary-engine/reuse-binary-engine/tests.ts (1638 MB heap size)
 PASS  tests/functional/field-reference/list/tests.ts (1741 MB heap size)
 PASS  tests/functional/extended-where/upsert.ts (1939 MB heap size)
 PASS  tests/functional/issues/11974/tests.ts (1642 MB heap size)
 PASS  tests/functional/find-unique-or-throw-batching/tests.ts (1751 MB heap size)
 PASS  tests/functional/issues/13097-group-by-enum/tests.ts (1856 MB heap size)
 PASS  tests/functional/issues/15044/tests.ts (1688 MB heap size)
 PASS  tests/functional/invalid-env/tests.ts (1883 MB heap size)
 PASS  tests/functional/issues/15176/tests.ts (1986 MB heap size)
 PASS  tests/functional/issues/10229/tests.ts (6.573 s, 1815 MB heap size)
 PASS  tests/functional/json-fields/tests.ts (1918 MB heap size)
 PASS  tests/functional/methods/upsert/simple/tests.ts (1736 MB heap size)
 PASS  tests/functional/extended-where/update.ts (1844 MB heap size)
 PASS  tests/functional/too-many-instances-of-prisma-client/tests.ts (1937 MB heap size)
 PASS  tests/functional/issues/15177/tests.ts (1724 MB heap size)
 PASS  tests/functional/json-list-push/tests.ts (1824 MB heap size)
 PASS  tests/functional/decimal/list/tests.ts (1925 MB heap size)
 PASS  tests/functional/extended-where/findFirstOrThrow.ts (2023 MB heap size)
 PASS  tests/functional/extended-where/findUniqueOrThrow.ts (1744 MB heap size)
 PASS  tests/functional/extended-where/aggregate.ts (1843 MB heap size)
 PASS  tests/functional/extended-where/findFirst.ts (1949 MB heap size)
 PASS  tests/functional/extended-where/findMany.ts (1875 MB heap size)
 PASS  tests/functional/large-floats/tests.ts (1976 MB heap size)
 PASS  tests/functional/issues/12003-order-by-self/tests.ts (1920 MB heap size)
 PASS  tests/functional/issues/9007/tests.ts (1825 MB heap size)
 PASS  tests/functional/missing-env/tests.ts (1924 MB heap size)
 PASS  tests/functional/handle-int-overflow/tests.ts (2033 MB heap size)
 PASS  tests/functional/extended-where/delete.ts (2095 MB heap size)
 PASS  tests/functional/metrics/disabled/tests.ts (1989 MB heap size)
 PASS  tests/functional/17797-no-env-error-init/tests.ts (2084 MB heap size)
 PASS  tests/functional/extensions/enabled/extends.ts (2182 MB heap size)
 PASS  tests/functional/17810-direct-url-missing/tests.ts (1987 MB heap size)
 PASS  tests/functional/issues/18598-select-count-true/tests.ts (2091 MB heap size)
 PASS  tests/functional/issues/17030-args-type-conflict/tests.ts (2185 MB heap size)
 PASS  tests/functional/issues/16535-select-enum/tests.ts (2285 MB heap size)
 PASS  tests/functional/issues/14373-batch-tx-error/tests.ts (2087 MB heap size)
 PASS  tests/functional/extensions/disabled/tests.ts (2182 MB heap size)
 PASS  tests/functional/issues/12572/tests.ts (2283 MB heap size)
 PASS  tests/functional/issues/16195-index-out-of-bounds/tests.ts (2034 MB heap size)
 PASS  tests/functional/issues/17005-args-type-conflict/tests.ts (2132 MB heap size)

Test Suites: 44 skipped, 107 passed, 107 of 151 total
Tests:       3197 skipped, 581 passed, 3778 total
Snapshots:   84 passed, 84 total
Time:        457.923 s
  1. As things currently are on main (jest.config.js)
Details 
    transformIgnorePatterns: [
      '[\\.]generated[\\/]@prisma[\\/]client[\\/]',
      escapeRegex(runtimeDir),
      `${escapeRegex(packagesDir)}[\\/][^\\/]+[\\/]dist[\\/]`,
    ],
 PASS  tests/functional/extensions/enabled/query.ts (10.701 s, 298 MB heap size)
 PASS  tests/functional/tracing/tests.ts (10.916 s, 351 MB heap size)
 PASS  tests/functional/interactive-transactions/tests.ts (11.913 s, 441 MB heap size)
 PASS  tests/functional/extensions/enabled/model.ts (525 MB heap size)
 PASS  tests/functional/extensions/enabled/defineExtension.ts (754 MB heap size)
 PASS  tests/functional/methods/upsert/native-atomic/tests.ts (896 MB heap size)
 PASS  tests/functional/relationMode-17255-same-actions/tests.ts (10.117 s, 946 MB heap size)
 PASS  tests/functional/extensions/enabled/result.ts (1074 MB heap size)
 PASS  tests/functional/referentialActions-setDefault/tests_1-to-1.ts (1322 MB heap size)
 PASS  tests/functional/referentialActions-setDefault/tests_1-to-n.ts (1464 MB heap size)
 PASS  tests/functional/issues/16390-relation-mode-m-n-dangling-pivot/tests.ts (1599 MB heap size)
 PASS  tests/functional/extensions/enabled/client.ts (1364 MB heap size)
 PASS  tests/functional/fluent-api-null/tests.ts (1501 MB heap size)
 PASS  tests/functional/prisma-promise/tests.ts (1652 MB heap size)
 PASS  tests/functional/blog-update/tests.ts (1460 MB heap size)
 PASS  tests/functional/issues/8832/tests.ts (14.474 s, 1974 MB heap size)
 PASS  tests/functional/logging/tests.ts (2228 MB heap size)
 PASS  tests/functional/extensions/enabled/itx.ts (1799 MB heap size)
 PASS  tests/functional/field-reference/string/tests.ts (1943 MB heap size)
 PASS  tests/functional/json-null-types/tests.ts (1683 MB heap size)
 PASS  tests/functional/fluent-api/tests.ts (1815 MB heap size)
 PASS  tests/functional/fulltext-search/tests.ts (1953 MB heap size)
 PASS  tests/functional/extensions/enabled/tx.ts (1821 MB heap size)
 PASS  tests/functional/filter-count-relations/tests.ts (2211 MB heap size)
 PASS  tests/functional/relationMode-17255-mixed-actions/tests.ts (5.064 s, 1410 MB heap size)
 PASS  tests/functional/field-reference/json/tests.ts (1672 MB heap size)
 PASS  tests/functional/extended-where/validation.ts (1476 MB heap size)
 PASS  tests/functional/metrics/enabled/tests.ts (1738 MB heap size)
 PASS  tests/functional/optimistic-concurrency-control/tests.ts (1557 MB heap size)
 PASS  tests/functional/batch-transaction-isolation-level/tests.ts (1695 MB heap size)
 PASS  tests/functional/methods/findUniqueOrThrow/tests.ts (1836 MB heap size)
 PASS  tests/functional/methods/findFirstOrThrow/tests.ts (1677 MB heap size)
 PASS  tests/functional/methods/count/tests.ts (1823 MB heap size)
 PASS  tests/functional/issues/14271/tests.ts (2082 MB heap size)
 PASS  tests/functional/query-error-logging/tests.ts (2162 MB heap size)
 PASS  tests/functional/raw-queries/send-type-hints/tests.ts (1667 MB heap size)
 PASS  tests/functional/order-by-null/tests.ts (1795 MB heap size)
 PASS  tests/functional/multi-schema/tests.ts (6.495 s, 1932 MB heap size)
 PASS  tests/functional/methods/create-many/tests.ts (1643 MB heap size)
 PASS  tests/functional/views/tests.ts (1776 MB heap size)
 PASS  tests/functional/mixed-string-uuid-datetime-list-inputs/tests.ts (1906 MB heap size)
 PASS  tests/functional/issues/5952-decimal-batch/tests.ts (2033 MB heap size)
 PASS  tests/functional/tracing-no-sampling/tests.ts (5.442 s, 1736 MB heap size)
 PASS  tests/functional/field-reference/numeric/tests.ts (6.71 s, 1882 MB heap size)
 PASS  tests/functional/issues/13766/primary-key/tests.ts (1802 MB heap size)
 PASS  tests/functional/issues/12862-errors-are-obfuscated-by-interactive-transactions/tests.ts (1948 MB heap size)
 PASS  tests/functional/extended-where/create.ts (2074 MB heap size)
 PASS  tests/functional/issues/9326/tests.ts (5.186 s, 1934 MB heap size)
 PASS  tests/functional/_example/tests.ts (5.031 s, 2187 MB heap size)
 PASS  tests/functional/issues/14954-date-batch/tests.ts (2315 MB heap size)
 PASS  tests/functional/issues/13766/at-unique/tests.ts (1905 MB heap size)
 PASS  tests/functional/batching/tests.ts (2038 MB heap size)
 PASS  tests/functional/raw-queries/typed-results/tests.ts (2169 MB heap size)
 PASS  tests/functional/issues/12557/tests.ts (2043 MB heap size)
 PASS  tests/functional/issues/11233/tests.ts (2169 MB heap size)
 PASS  tests/functional/issues/6578/tests.ts (2300 MB heap size)
 PASS  tests/functional/runtime-import/tests.ts (2037 MB heap size)
 PASS  tests/functional/issues/4004/tests.ts (2294 MB heap size)
 PASS  tests/functional/issues/12378/tests.ts (1982 MB heap size)
 PASS  tests/functional/issues/11740-transaction-stored-query/tests.ts (2245 MB heap size)
 PASS  tests/functional/decimal/precision/tests.ts (10.842 s, 2056 MB heap size)
 PASS  tests/functional/raw-queries/typed-results-advanced-and-native-types/tests.ts (2181 MB heap size)
 PASS  tests/functional/decimal/scalar/tests.ts (2435 MB heap size)
 PASS  tests/functional/binary-engine/restart/tests.ts (2176 MB heap size)
 PASS  tests/functional/extended-where/findUnique.ts (2312 MB heap size)
 PASS  tests/functional/issues/9678/tests.ts (2212 MB heap size)
 PASS  tests/functional/tracing-disabled/tests.ts (2272 MB heap size)
 PASS  tests/functional/binary-engine/reuse-binary-engine/tests.ts (2403 MB heap size)
 PASS  tests/functional/field-reference/list/tests.ts (2532 MB heap size)
 PASS  tests/functional/extended-where/upsert.ts (2384 MB heap size)
 PASS  tests/functional/issues/11974/tests.ts (2512 MB heap size)
 PASS  tests/functional/find-unique-or-throw-batching/tests.ts (2647 MB heap size)
 PASS  tests/functional/issues/13097-group-by-enum/tests.ts (2431 MB heap size)
 PASS  tests/functional/issues/15044/tests.ts (2563 MB heap size)
 PASS  tests/functional/invalid-env/tests.ts (2488 MB heap size)
 PASS  tests/functional/issues/15176/tests.ts (2618 MB heap size)
 PASS  tests/functional/issues/10229/tests.ts (6.84 s, 2874 MB heap size)
 PASS  tests/functional/json-fields/tests.ts (2575 MB heap size)
 PASS  tests/functional/methods/upsert/simple/tests.ts (2705 MB heap size)
 PASS  tests/functional/extended-where/update.ts (2830 MB heap size)
 PASS  tests/functional/too-many-instances-of-prisma-client/tests.ts (5.43 s, 2578 MB heap size)
 PASS  tests/functional/issues/15177/tests.ts (2707 MB heap size)
 PASS  tests/functional/json-list-push/tests.ts (2837 MB heap size)
 PASS  tests/functional/decimal/list/tests.ts (2638 MB heap size)
 PASS  tests/functional/extended-where/findFirstOrThrow.ts (2772 MB heap size)
 PASS  tests/functional/extended-where/findUniqueOrThrow.ts (2896 MB heap size)
 PASS  tests/functional/extended-where/aggregate.ts (2708 MB heap size)
 PASS  tests/functional/extended-where/findFirst.ts (2845 MB heap size)
 PASS  tests/functional/extended-where/findMany.ts (2725 MB heap size)
 PASS  tests/functional/large-floats/tests.ts (2858 MB heap size)
 PASS  tests/functional/issues/12003-order-by-self/tests.ts (2679 MB heap size)
 PASS  tests/functional/issues/9007/tests.ts (2722 MB heap size)
 PASS  tests/functional/missing-env/tests.ts (2850 MB heap size)
 PASS  tests/functional/handle-int-overflow/tests.ts (2684 MB heap size)
 PASS  tests/functional/extended-where/delete.ts (2868 MB heap size)
 PASS  tests/functional/metrics/disabled/tests.ts (2936 MB heap size)
 PASS  tests/functional/17797-no-env-error-init/tests.ts (2800 MB heap size)
 PASS  tests/functional/extensions/enabled/extends.ts (2933 MB heap size)
 PASS  tests/functional/17810-direct-url-missing/tests.ts (3060 MB heap size)
 PASS  tests/functional/issues/18598-select-count-true/tests.ts (2774 MB heap size)
 PASS  tests/functional/issues/17030-args-type-conflict/tests.ts (2908 MB heap size)
 PASS  tests/functional/issues/16535-select-enum/tests.ts (2865 MB heap size)
 PASS  tests/functional/issues/14373-batch-tx-error/tests.ts (3005 MB heap size)
 PASS  tests/functional/extensions/disabled/tests.ts (2918 MB heap size)
 PASS  tests/functional/issues/12572/tests.ts (3048 MB heap size)
 PASS  tests/functional/issues/16195-index-out-of-bounds/tests.ts (2997 MB heap size)
 PASS  tests/functional/issues/17005-args-type-conflict/tests.ts (3129 MB heap size)

Test Suites: 44 skipped, 107 passed, 107 of 151 total
Tests:       3197 skipped, 581 passed, 3778 total
Snapshots:   84 passed, 84 total
Time:        533.491 s
  1. With the changes from this PR (jest.config.js)

Notice how we don't manage transformIgnorePatterns anymore.

Details 
    transformIgnorePatterns: [],
 PASS  tests/functional/extensions/enabled/query.ts (6.65 s, 174 MB heap size)
 PASS  tests/functional/tracing/tests.ts (10.379 s, 293 MB heap size)
 PASS  tests/functional/interactive-transactions/tests.ts (11.458 s, 279 MB heap size)
 PASS  tests/functional/extensions/enabled/model.ts (282 MB heap size)
 PASS  tests/functional/extensions/enabled/defineExtension.ts (453 MB heap size)
 PASS  tests/functional/methods/upsert/native-atomic/tests.ts (565 MB heap size)
 PASS  tests/functional/relationMode-17255-same-actions/tests.ts (8.958 s, 580 MB heap size)
 PASS  tests/functional/extensions/enabled/result.ts (678 MB heap size)
 PASS  tests/functional/referentialActions-setDefault/tests_1-to-1.ts (873 MB heap size)
 PASS  tests/functional/referentialActions-setDefault/tests_1-to-n.ts (725 MB heap size)
 PASS  tests/functional/issues/16390-relation-mode-m-n-dangling-pivot/tests.ts (830 MB heap size)
 PASS  tests/functional/extensions/enabled/client.ts (1025 MB heap size)
 PASS  tests/functional/fluent-api-null/tests.ts (810 MB heap size)
 PASS  tests/functional/prisma-promise/tests.ts (931 MB heap size)
 PASS  tests/functional/blog-update/tests.ts (1030 MB heap size)
 PASS  tests/functional/issues/8832/tests.ts (13.842 s, 1217 MB heap size)
 PASS  tests/functional/logging/tests.ts (1409 MB heap size)
 PASS  tests/functional/extensions/enabled/itx.ts (1020 MB heap size)
 PASS  tests/functional/field-reference/string/tests.ts (1126 MB heap size)
 PASS  tests/functional/json-null-types/tests.ts (868 MB heap size)
 PASS  tests/functional/fluent-api/tests.ts (972 MB heap size)
 PASS  tests/functional/fulltext-search/tests.ts (1082 MB heap size)
 PASS  tests/functional/extensions/enabled/tx.ts (895 MB heap size)
 PASS  tests/functional/filter-count-relations/tests.ts (904 MB heap size)
 PASS  tests/functional/relationMode-17255-mixed-actions/tests.ts (1011 MB heap size)
 PASS  tests/functional/field-reference/json/tests.ts (1209 MB heap size)
 PASS  tests/functional/extended-where/validation.ts (1044 MB heap size)
 PASS  tests/functional/metrics/enabled/tests.ts (1241 MB heap size)
 PASS  tests/functional/optimistic-concurrency-control/tests.ts (1350 MB heap size)
 PASS  tests/functional/batch-transaction-isolation-level/tests.ts (1462 MB heap size)
 PASS  tests/functional/methods/findUniqueOrThrow/tests.ts (1198 MB heap size)
 PASS  tests/functional/methods/findFirstOrThrow/tests.ts (1304 MB heap size)
 PASS  tests/functional/methods/count/tests.ts (1418 MB heap size)
 PASS  tests/functional/issues/14271/tests.ts (1376 MB heap size)
 PASS  tests/functional/query-error-logging/tests.ts (1438 MB heap size)
 PASS  tests/functional/raw-queries/send-type-hints/tests.ts (1543 MB heap size)
 PASS  tests/functional/order-by-null/tests.ts (1646 MB heap size)
 PASS  tests/functional/multi-schema/tests.ts (6.027 s, 1056 MB heap size)
 PASS  tests/functional/methods/create-many/tests.ts (1149 MB heap size)
 PASS  tests/functional/views/tests.ts (1253 MB heap size)
 PASS  tests/functional/mixed-string-uuid-datetime-list-inputs/tests.ts (1056 MB heap size)
 PASS  tests/functional/issues/5952-decimal-batch/tests.ts (1157 MB heap size)
 PASS  tests/functional/tracing-no-sampling/tests.ts (1263 MB heap size)
 PASS  tests/functional/field-reference/numeric/tests.ts (6.048 s, 1102 MB heap size)
 PASS  tests/functional/issues/13766/primary-key/tests.ts (1299 MB heap size)
 PASS  tests/functional/issues/12862-errors-are-obfuscated-by-interactive-transactions/tests.ts (1155 MB heap size)
 PASS  tests/functional/extended-where/create.ts (1264 MB heap size)
 PASS  tests/functional/issues/9326/tests.ts (1428 MB heap size)
 PASS  tests/functional/_example/tests.ts (1628 MB heap size)
 PASS  tests/functional/issues/14954-date-batch/tests.ts (1729 MB heap size)
 PASS  tests/functional/issues/13766/at-unique/tests.ts (1401 MB heap size)
 PASS  tests/functional/batching/tests.ts (1495 MB heap size)
 PASS  tests/functional/raw-queries/typed-results/tests.ts (1600 MB heap size)
 PASS  tests/functional/issues/12557/tests.ts (1421 MB heap size)
 PASS  tests/functional/issues/11233/tests.ts (1522 MB heap size)
 PASS  tests/functional/issues/6578/tests.ts (1624 MB heap size)
 PASS  tests/functional/runtime-import/tests.ts (1383 MB heap size)
 PASS  tests/functional/issues/4004/tests.ts (1579 MB heap size)
 PASS  tests/functional/issues/12378/tests.ts (1251 MB heap size)
 PASS  tests/functional/issues/11740-transaction-stored-query/tests.ts (1449 MB heap size)
 PASS  tests/functional/decimal/precision/tests.ts (9.823 s, 1302 MB heap size)
 PASS  tests/functional/raw-queries/typed-results-advanced-and-native-types/tests.ts (1391 MB heap size)
 PASS  tests/functional/decimal/scalar/tests.ts (1591 MB heap size)
 PASS  tests/functional/binary-engine/restart/tests.ts (1379 MB heap size)
 PASS  tests/functional/extended-where/findUnique.ts (1478 MB heap size)
 PASS  tests/functional/issues/9678/tests.ts (1387 MB heap size)
 PASS  tests/functional/tracing-disabled/tests.ts (1406 MB heap size)
 PASS  tests/functional/binary-engine/reuse-binary-engine/tests.ts (1512 MB heap size)
 PASS  tests/functional/field-reference/list/tests.ts (1602 MB heap size)
 PASS  tests/functional/extended-where/upsert.ts (1506 MB heap size)
 PASS  tests/functional/issues/11974/tests.ts (1600 MB heap size)
 PASS  tests/functional/find-unique-or-throw-batching/tests.ts (1707 MB heap size)
 PASS  tests/functional/issues/13097-group-by-enum/tests.ts (1512 MB heap size)
 PASS  tests/functional/issues/15044/tests.ts (1608 MB heap size)
 PASS  tests/functional/invalid-env/tests.ts (1520 MB heap size)
 PASS  tests/functional/issues/15176/tests.ts (1616 MB heap size)
 PASS  tests/functional/issues/10229/tests.ts (6.628 s, 1810 MB heap size)
 PASS  tests/functional/json-fields/tests.ts (1583 MB heap size)
 PASS  tests/functional/methods/upsert/simple/tests.ts (1685 MB heap size)
 PASS  tests/functional/extended-where/update.ts (1791 MB heap size)
 PASS  tests/functional/too-many-instances-of-prisma-client/tests.ts (1610 MB heap size)
 PASS  tests/functional/issues/15177/tests.ts (1712 MB heap size)
 PASS  tests/functional/json-list-push/tests.ts (1811 MB heap size)
 PASS  tests/functional/decimal/list/tests.ts (1653 MB heap size)
 PASS  tests/functional/extended-where/findFirstOrThrow.ts (1751 MB heap size)
 PASS  tests/functional/extended-where/findUniqueOrThrow.ts (1849 MB heap size)
 PASS  tests/functional/extended-where/aggregate.ts (1686 MB heap size)
 PASS  tests/functional/extended-where/findFirst.ts (1782 MB heap size)
 PASS  tests/functional/extended-where/findMany.ts (1699 MB heap size)
 PASS  tests/functional/large-floats/tests.ts (1801 MB heap size)
 PASS  tests/functional/issues/12003-order-by-self/tests.ts (1994 MB heap size)
 PASS  tests/functional/issues/9007/tests.ts (2001 MB heap size)
 PASS  tests/functional/missing-env/tests.ts (1849 MB heap size)
 PASS  tests/functional/handle-int-overflow/tests.ts (1957 MB heap size)
 PASS  tests/functional/extended-where/delete.ts (1810 MB heap size)
 PASS  tests/functional/metrics/disabled/tests.ts (2003 MB heap size)
 PASS  tests/functional/17797-no-env-error-init/tests.ts (1798 MB heap size)
 PASS  tests/functional/extensions/enabled/extends.ts (1896 MB heap size)
 PASS  tests/functional/17810-direct-url-missing/tests.ts (1995 MB heap size)
 PASS  tests/functional/issues/18598-select-count-true/tests.ts (1827 MB heap size)
 PASS  tests/functional/issues/17030-args-type-conflict/tests.ts (1931 MB heap size)
 PASS  tests/functional/issues/16535-select-enum/tests.ts (2030 MB heap size)
 PASS  tests/functional/issues/14373-batch-tx-error/tests.ts (2130 MB heap size)
 PASS  tests/functional/extensions/disabled/tests.ts (1916 MB heap size)
 PASS  tests/functional/issues/12572/tests.ts (2017 MB heap size)
 PASS  tests/functional/issues/16195-index-out-of-bounds/tests.ts (2118 MB heap size)
 PASS  tests/functional/issues/17005-args-type-conflict/tests.ts (1941 MB heap size)

Test Suites: 44 skipped, 107 passed, 107 of 151 total
Tests:       3197 skipped, 581 passed, 3778 total
Snapshots:   84 passed, 84 total
Time:        461.38 s

Result summary

As you can see

  1. Before: 458s and a heap of ~2100MB
  2. Current: 533s and a heap of ~3100MB (things got much worse)
  3. With PR: 461s and a heap of ~1900MB

How the solution works

I couldn't meet all my requirements with current tooling.

reasonable heap growth, good performance, and esm transpilation

So I implemented a jest transformer for esbuild that actually works and does what I needed.

  • esbuild is equivalent @swc/jest in terms of performance/cpu usage.
  • Heap growth is similar to the one of @swc/jest, but also has smaller footprint.
  • ES modules are transpiled on demand and only if they need to be transpiled.
Some more runs but on multiple cores (8) 
# swc Run without cache
Test Suites: 44 skipped, 107 passed, 107 of 151 total
Tests:       3197 skipped, 581 passed, 3778 total
Snapshots:   84 passed, 84 total
Time:        64.085 s

# swc Run with cache
Test Suites: 44 skipped, 107 passed, 107 of 151 total
Tests:       3197 skipped, 581 passed, 3778 total
Snapshots:   84 passed, 84 total
Time:        64.132 s


# esbuild Run without cache
Test Suites: 44 skipped, 107 passed, 107 of 151 total
Tests:       3197 skipped, 581 passed, 3778 total
Snapshots:   84 passed, 84 total
Time:        66.122 s

# esbuild Run with cache
Test Suites: 44 skipped, 107 passed, 107 of 151 total
Tests:       3197 skipped, 581 passed, 3778 total
Snapshots:   84 passed, 84 total
Time:        65.561 s

@socket-security
Copy link

socket-security bot commented Apr 21, 2023 •
edited

New dependency changes detected. Learn more about Socket for GitHub ↗︎

👍 No new dependency issues detected in pull request

Bot Commands

To ignore an alert, reply with a comment starting with @SocketSecurity ignore followed by a space separated list of package-name@version specifiers. e.g. @SocketSecurity ignore foo@1.0.0 bar@* or ignore all packages with @SocketSecurity ignore-all

Pull request alert summary
Issue Status
Install scripts ✅ 0 issues
Native code ✅ 0 issues
Bin script shell injection ✅ 0 issues
Unresolved require ✅ 0 issues
Invalid package.json ✅ 0 issues
HTTP dependency ✅ 0 issues
Git dependency ✅ 0 issues
Potential typo squat ✅ 0 issues
Known Malware ✅ 0 issues
Telemetry ✅ 0 issues
Protestware/Troll package ✅ 0 issues

📊 Modified Dependency Overview:

➕ Added Package Capability Access +/- Transitive Count Publisher
@jest/create-cache-key-function@29.5.0 filesystem, environment +0 simenb

@codspeed-hq
Copy link

codspeed-hq bot commented Apr 21, 2023 •
edited

CodSpeed Performance Report

Merging #18859 ci/fix-jest-memory-flakiness (b39bf4f) will not alter performances.

Summary

🔥 0 improvements
❌ 0 regressions
✅ 3 untouched benchmarks

🆕 0 new benchmarks
⁉️ 0 dropped benchmarks

@millsp millsp marked this pull request as ready for review April 21, 2023 03:12
@millsp millsp requested a review from a team April 21, 2023 03:12
@millsp millsp requested review from aqrln and removed request for a team April 21, 2023 03:12
@millsp millsp added this to the 4.14.0 milestone Apr 21, 2023
@millsp millsp requested review from SevInf and a team April 21, 2023 03:12
Jolg42
Jolg42 approved these changes Apr 25, 2023
View reviewed changes
Copy link
Member

@Jolg42 Jolg42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting PR 👀
I didn't try it running it, though it looks good to me

@millsp millsp merged commit 8690c5b into main Apr 27, 2023
56 checks passed
@millsp millsp deleted the ci/fix-jest-memory-flakiness branch April 27, 2023 00:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Jolg42 Jolg42 Jolg42 approved these changes

@jkomyno jkomyno Awaiting requested review from jkomyno

@aqrln aqrln Awaiting requested review from aqrln

@SevInf SevInf Awaiting requested review from SevInf

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
4.14.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@millsp @Jolg42