test:lib task-caching w/ Nx

[ZackDeRose]

What This Does

• Implements local task-caching [currently] only for the test:lib task (with Nx)
• I also update the root test-lib script to run with nx's run-many command, as well as the Test workflow in .github/workflows/pr.yml
  • While not directly affected, the Run Test step of the ci pipeline is adjusted as well as test:ci will call pnpm run test:lib
  • My testing shows a 20s time boost to the Test job of the pr pipeline (see Task caching ZackDeRose/query#1 (comment) for my findings)
• Note that any test:lib:dev commands are not affected and work exactly as they did before

What this PR doesn't do, but could!

I wanted to make this PR bite-sized (as opposed to my last one!) but would be happy to expand this with more task-caching features:

• I believe most of the "testing" tasks could be cached in a similar way (this would mainly take a few lines to the nx.json file, and some coordination to make sure I correctly understand what files are created by those tasks, and which files should/shouldn't invalidate the cache)
• builds similarly should be cacheable without too much more work
• We could introduce distributed caching (w/ Nx Cloud) - this would mainly be there to enable caching across github action runs [open source projects get a free unlimited license to Nx Cloud]
  • I'd recommend a public read-only key along with a private write key to be stored in github secrets (happy to help set that up)
  • alternatively, we can persist the node_modules/.cache between ci runs via the workflow yml files, but that's much more manual process (and we'd have to eventually introduce pruning before this cache just gets too large)
• We can introduce distributed task execution to further parallelize the ci workflows

@TkDodo (or others!) let me know if any of the above would be appealing! I'd be happy to work on introducing any of these (either in this PR or in future PRs)

[Also note that I work full-time for Nx!]

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated
query	⬜️ Ignored (Inspect)			Mar 11, 2023 at 11:52AM (UTC)

[codesandbox-ci]

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit 7995733:

Sandbox	Source
@tanstack/query-example-react-basic-typescript	Configuration
@tanstack/query-example-solid-basic-typescript	Configuration
@tanstack/query-example-svelte-basic	Configuration
@tanstack/query-example-vue-basic	Configuration

[codecov-commenter]

Codecov Report

Patch and project coverage have no change.

Comparison is base (b8b0562) 91.90% compared to head (7995733) 91.90%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5116   +/-   ##
=======================================
  Coverage   91.90%   91.90%           
=======================================
  Files         111      111           
  Lines        4188     4188           
  Branches     1083     1083           
=======================================
  Hits         3849     3849           
  Misses        318      318           
  Partials       21       21           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[TkDodo]

love the idea of nx-cloud ❤️

[ZackDeRose]

Hey @TkDodo! Looks like this failed in CI due to some flakiness (the error comes right after a await sleep(20) - and I confirmed after pulling down the new changes that this passes locally.)

Can we give this another try? Also let me know if anything else you want to see adjusted in this branch!

[TkDodo]

Yeah I'm trying to fix flakiness as I see it. Some tests are not written in a robust way 😅

[ZackDeRose]

Yeah I'm trying to fix flakiness as I see it

That's great. If there's an effort to go after these please let me know! I think this one for example, we should use waitFor instead!

[TkDodo]

yeah all the newer tests are written with waitFor instead of sleeps:

query/packages/react-query/src/__tests__/useQuery.test.tsx

Lines 6057 to 6063 in 4aaaa51

	const rendered = renderWithClient(queryClient, <Page />)
	const fetchBtn = rendered.getByRole('button', { name: 'refetch' })
	await waitFor(() => rendered.getByText('data: 1'))
	fireEvent.click(fetchBtn)
	await waitFor(() => rendered.getByText('data: 2'))
	fireEvent.click(fetchBtn)
	await waitFor(() => rendered.getByText('data: 3'))

[ZackDeRose]

Yeah I noticed that! I like that alot. Seems like there might some potential for flakiness based on this timing out, but I've actually never had a test fail on me because of that in any of the testing I've looked at for this project so far!

[TkDodo]

Just watched your video - amazing, thank you so much for this ❤️ . I'll merge the PR and then merge it to alpha. Whatever you wanna do next, I'm game. adding caching for the lint task, or going nx cloud for what we have already 🎉

[TkDodo]

please let me know if I fixed that correctly on alpha 😅

ac31fc0

[TkDodo]

@ZackDeRose I can now see some post-install task failing in our pipelines:

it just logs this error and it still succeeds, but the install times seem higher now (27s vs 11s before)

[TkDodo]

okay never mind the longer times, this was just a cache miss on pnpms side. It's back to 11s with a warm cache. But do you know what the error is?

[ZackDeRose]

okay never mind the longer times, this was just a cache miss on pnpms side. It's back to 11s with a warm cache. But do you know what the error is?

Ah yes - the Nx Daemon is a daemon process that runs in the background to eagerly determine Nx project configuration whenever files change. Looks like the daemon stopped because of an issue parsing pnpm's lockfile.

I'll bring this to our core engineers, as we should be handling this better - but the daemon failing shouldn't be considered a failing event. If it ever does fail, it should just restart itself the next time Nx would need it!

[ZackDeRose]

@TkDodo - I'll be tracking this here: nrwl/nx#15656

[JamesHenry]

Just FYI NX_DAEMON=false environment variable could be used in the immediate term if the issues is daemon specific

[ZackDeRose]

Thanks @JamesHenry! I'll get something up with this added now :)

[ZackDeRose]

@TkDodo some quick fixes:

#5128 to fix the error messages re: the nx daemon
#5129 quick adjustment to alpha branch

Whatever you wanna do next, I'm game. adding caching for the lint task, or going nx cloud for what we have already 🎉

Let's go for adding the rest of our tasks first! I think that should actually be pretty easy! I'll send you a PR soon.

Let's plan for Nx cloud following that because we may have to coordinate that a bit!