add clearCache cli command #4430

tabrindle · 2017-09-05T18:37:45Z

Summary

Occasionally Jest's cache needs to be cleared for debugging purposes.

This seems to be a fairly common question, to say nothing about caching being a difficult thing:

https://stackoverflow.com/questions/44866994/how-to-clear-jest-cache

It has also been mentioned in issues here: #3705

Even with a PR to explain how to do so. #4232

Seems silly to have a multi step operation to clear a cache - so I wrote a bash script to do it
jest --showConfig | grep cacheDirectory | awk -F\" '{ print $4 }' | xargs rm -rf which in the end is dumb because jest has a cli already, and parsing JSON in bash like this is a bad idea.

Test plan

jest --clearCache

This should do nothing more than delete the cacheDirectory and exit 0.

thymikee · 2017-09-05T21:05:01Z

Thanks! Can you make sure yarn test passes? There's lint issue reported by CI

thymikee

We need an integration test for this.

thymikee · 2017-09-05T21:06:53Z

packages/jest-cli/src/cli/args.js

+  clearCache: {
+    default: undefined,
+    description:
+      'Clear default directory Jest uses for cache and exits.' +


Nit: Clears or exit, whatever the convention is right now (hard to access on a phone)

thymikee · 2017-09-05T21:09:12Z

packages/jest-cli/src/cli/index.js

+      clearDirectory(configs[0].cacheDirectory);
+      process.stdout.write(`Cleared ${configs[0].cacheDirectory}\n`);
+      process.exit(0);
+    }


We should be able to clear the cache for certain project or all projects, right now it's only for the first one.

I wasn't sure how this worked: should I just map over the configs array?

yup, should be fine

thymikee · 2017-09-05T21:12:38Z

packages/jest-util/src/index.js

+const clearDirectory = (path: string) => {
+  if (fs.existsSync(path)) {
+    fs.readdirSync(path).map(file => {
+      const filePath = `${path}/${file}`;


This won't work on windows, use path.join or path.resolve

Didnt even think of that. Will research.

tabrindle · 2017-09-06T15:52:56Z

@thymikee I'm really not sure where to start on the integration test- Im assuming we just need to mock jest for the config object it generates, run the clearDirectory function then verify the directory is empty. Open to suggestions.

thymikee · 2017-09-06T15:56:59Z

You can check integration_tests for in this repo to see how to write one. You don't mock anything, just spawn a real jest instance with desired config and test the output

SimenB · 2017-09-07T07:00:41Z

packages/jest-util/src/index.js

@@ -31,11 +33,26 @@ const createDirectory = (path: string) => {
  }
 };

+const clearDirectory = (directoryPath: string) => {


Why not just rimraf? (or del or one other of the many other modules that does this)

Only because I don't generally add dependencies to someone else's project unless they mention it. rimraf is fine.

Migrated to rimraf.

tabrindle · 2017-09-08T17:34:14Z

@thymikee Added an integration test, and it's passing. Refactored to use rimraf. Should be read for re-review.

Test Suites: 166 passed, 166 total
Tests:       1574 passed, 1574 total
Snapshots:   691 passed, 691 total
Time:        48.293s, estimated 53s
Ran all test suites.

=============================== Coverage summary ===============================
Statements   : 53.72% ( 3766/7010 )
Branches     : 55.61% ( 2131/3832 )
Functions    : 51.1% ( 746/1460 )
Lines        : 57.18% ( 3689/6451 )
================================================================================

Looks like appveyor is failing, but it doesn't appear to have anything to do with me. Everything else is passing.

thymikee · 2017-09-08T17:45:56Z

Looks like you didn't push the changes?

tabrindle · 2017-09-08T17:50:26Z

Derp. Sorry.

codecov-io · 2017-09-08T18:26:36Z

Codecov Report

Merging #4430 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #4430   +/-   ##
=======================================
  Coverage   57.13%   57.13%           
=======================================
  Files         194      194           
  Lines        6565     6565           
  Branches        3        3           
=======================================
  Hits         3751     3751           
  Misses       2814     2814

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1ece140...4680586. Read the comment docs.

thymikee

Sorry to hold this, but it just needs a bit more work. We're almost there 🙂

thymikee · 2017-09-08T21:22:18Z

packages/jest-cli/src/cli/index.js

+      let clearCacheError;
+
+      configs.map(config => {
+        rimraf(config.cacheDirectory, [], () => {


I'm afraid this is completely wrong and doesn't work at all.
Please refer to rimraf tests on how to use it:
https://github.com/isaacs/rimraf/blob/d84fe2cc6646d30a401baadcee22ae105a2d4909/test/basic.js#L48-L55.

Also I see no point in using async API, since we're not doing anything else in this time. Please use rimraf.sync, this way you'll avoid temporary clearCacheError variable and perf overhead of async call.

Not sure what I was thinking - process exited before async call finished. Will switch to sync. Will be easier than using a promise all similar.

thymikee · 2017-09-08T21:25:50Z

integration_tests/__tests__/clear_cache.test.js

+      `--cacheDirectory=${CACHE}`,
+    ]);
+
+    expect(fs.existsSync(CACHE)).toBe(false);


It's a false positive. This folder doesn't exist, because Jest didn't run any tests. We need to populate this cache earlier with a sample run, then run it once more. Or just create the dir and make sure it is present prior to running test.

Sorry I thought the folder was created when calling runJest regardless of whether there are tests or not. Will add a stub test, verify the cache dir has contents, then runJest again with clearCache

can you add

expect(fs.existsSync(CACHE)).toBe(true);

above the runJest call?

In the second test correct? Makes sense.

tabrindle · 2017-09-08T22:13:16Z

No problem, if it's not right you shouldn't pass it :)

SimenB · 2017-09-09T00:45:57Z

packages/jest-cli/src/cli/index.js

@@ -67,6 +68,15 @@ const runCLI = async (
      outputStream,
    );

+    if (argv.clearCache) {
+      configs.map(config => {


forEach over map. (You ignore the return value anyways)

I usually prefer map to prevent accidental mutation. I suppose here it doesn't matter as the process is about to end. I will defer to you on this.

SimenB · 2017-09-09T00:47:01Z

packages/jest-cli/src/cli/args.js

+    default: undefined,
+    description:
+      'Clears the configured Jest cache directory and then exits.' +
+      'Default directory can be found by calling jest --showConfig',


Missing space before "Default"?

Good catch.

SimenB · 2017-09-09T00:47:54Z

integration_tests/__tests__/clear_cache.test.js

+
+const CACHE = path.resolve(os.tmpdir(), 'clear_cache_directory');
+
+skipOnWindows.suite();


Necessary, or copypasta? (Just asking, fine if needed 😄)

skipOnWindows? I assumed it was part of the test convention as I saw it everywhere. I can remove it if it isn't necessary.

tabrindle · 2017-10-02T20:10:31Z

There is currently an issue on master where we get the following stack trace messing up quite a few tests --including mine-- where tests are looking for a specific stderr output.

Looks related to yargs, but the last breaking upgrade to yargs was before I started on this PR.

YError: Invalid first argument. Expected string or function but received boolean.
    at argumentTypeError (/Users/tabrindle/Developer/jest/packages/jest-cli/node_modules/yargs/lib/argsert.js:70:9)
    at /Users/tabrindle/Developer/jest/packages/jest-cli/node_modules/yargs/lib/argsert.js:52:39
    at Array.forEach (<anonymous>)
    at module.exports (/Users/tabrindle/Developer/jest/packages/jest-cli/node_modules/yargs/lib/argsert.js:45:21)
    at Object.Yargs.self.version (/Users/tabrindle/Developer/jest/packages/jest-cli/node_modules/yargs/yargs.js:718:5)
    at _buildArgv (/Users/tabrindle/Developer/jest/packages/jest-cli/build/cli/index.js:118:3)
    at Object.<anonymous> (/Users/tabrindle/Developer/jest/packages/jest-cli/build/cli/index.js:37:18)
    at Generator.next (<anonymous>)
    at step (/Users/tabrindle/Developer/jest/packages/jest-cli/build/cli/index.js:45:2466)
    at /Users/tabrindle/Developer/jest/packages/jest-cli/build/cli/index.js:45:2696

SimenB · 2017-10-08T08:52:52Z

@tabrindle this PR seems like it's gotten lots of commits it shouldn't have. Care to rebase?

Regarding you issue, does it help to do yarn clean-all && yarn (make sure you're on yarn@1.1.0 as well)? Seems like something is weird with your dependencies locally, so maybe doing a clean install will help 🙂

- map over all configs

- rename path arg to directoryPath for less confusion

- yarn run jest -- -t 'jest --clearCache' $ node ./packages/jest-cli/bin/jest.js "-t" "jest --clearCache" PASS integration_tests/__tests__/clear_cache.test.js Test Suites: 165 skipped, 1 passed, 1 of 166 total Tests: 1573 skipped, 1 passed, 1574 total Snapshots: 0 total

- actually create directory with real test - check for directory, then delete - use rimraf.sync - remove temp var

tabrindle · 2017-10-11T13:55:53Z

@SimenB Thanks for the tip - Rebased & looks like my new tests are passing, and the only test failing is the same one failing on master.

 FAIL  packages/jest-worker/src/__tests__/child.test.js
  ● lazily requires the file

    TypeError: Cannot read property 'concat' of undefined
      
      at handle (node_modules/worker-farm/lib/child/index.js:44:24)
      at emitOne (events.js:120:20)
      at process.emit (events.js:210:7)
          at Promise (<anonymous>)

SimenB · 2017-10-11T14:14:46Z

CI is happy, that's great. The failing test you see is tracked in #4630

tabrindle · 2017-10-11T14:20:52Z

In an effort to double check, I was able to confirm the new cache directory creation, and it's deletion via jest -t "jest --clearCache"

On my machine: /var/folders/9h/tl11z92d59sd2zw5c3zcxn740000gn/T/clear_cache_directory is created then immediately destroyed via integration test.

SimenB · 2017-10-11T14:24:41Z

integration_tests/__tests__/clear_cache.test.js

+/**
+ * Copyright (c) 2014-present, Facebook, Inc. All rights reserved.
+ *
+ * This source code is licensed under the BSD-style license found in the


change the license header to be MIT, please

Also found jest/integration_tests/__tests__/pass_with_no_tests.test.js with BSD license. Should I take care of that here?

SimenB · 2017-10-11T15:06:34Z

CI is red. Circle is not friendly to mobile, but I'm guessing a lint error?

tabrindle · 2017-10-11T15:10:53Z

Indeed - appears .vscode/settings.json has prettier configured "editor.formatOnSave": true, which spaced out the brackets. I can look into this later.

edit: didnt have prettier-vscode installed before, and somehow it was still formatting - once installed the config seems to work correctly.

tabrindle · 2017-10-11T16:32:26Z

@thymikee @SimenB I believe this is ready to go - sorry for the long process!

points adressed

SimenB

LGTM! Thanks for sticking with us

cpojer · 2017-10-11T17:31:13Z

Awesome! Can you also send a PR to update the documentation with this arg?

SimenB · 2017-10-11T17:31:52Z

Haha.

tabrindle · 2017-10-11T18:08:21Z

Will do!

* add to changelog for PR #4430 * add —clearCache to docs

github-actions · 2021-05-13T05:10:59Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

facebook-github-bot added the CLA Signed ✔️ label Sep 5, 2017

thymikee requested changes Sep 5, 2017

View reviewed changes

SimenB reviewed Sep 7, 2017

View reviewed changes

thymikee previously requested changes Sep 8, 2017

View reviewed changes

SimenB reviewed Sep 9, 2017

View reviewed changes

felipeochoa mentioned this pull request Oct 4, 2017

Sourcemaps #3458

Merged

3 tasks

tabrindle added 9 commits October 11, 2017 09:32

add clearCache cli command

3036f7e

fix lint issue - sort-keys

7615c79

clear all projects cacheDirectory

04b495d

- map over all configs

improve wording in arg description

d63c815

use path.join instead of string concat

0a7a2c2

- rename path arg to directoryPath for less confusion

remove clearDirectory function in favor of rimraf

4595517

allow clearCache loop to run before failing with exit 1

40efb06

fix integration test

613feca

- actually create directory with real test - check for directory, then delete - use rimraf.sync - remove temp var

tabrindle force-pushed the clear-cache-command branch from f6a577a to eeaa476 Compare October 11, 2017 13:34

changes per PR comments

82d01b3

tabrindle force-pushed the clear-cache-command branch from 6f51489 to 82d01b3 Compare October 11, 2017 14:02

SimenB reviewed Oct 11, 2017

View reviewed changes

tabrindle added 3 commits October 11, 2017 10:36

update license headers to MIT

2cc04eb

add check for cache_dir before deletion

cf503fb

update license to MIT for pass_with_no_tests.test.js

072428c

fix lint error

4680586

SimenB approved these changes Oct 11, 2017

View reviewed changes

cpojer merged commit d82925e into jestjs:master Oct 11, 2017

tabrindle mentioned this pull request Oct 11, 2017

Moving src files doesn't bust the Jest cache #3705

Closed

tabrindle added a commit to tabrindle/jest that referenced this pull request Oct 11, 2017

add to changelog for PR jestjs#4430

06df8fe

tabrindle mentioned this pull request Oct 11, 2017

Clear cache docs #4667

Merged

cpojer pushed a commit that referenced this pull request Oct 11, 2017

Clear cache docs (#4667)

1c2dba6

* add to changelog for PR #4430 * add —clearCache to docs

renovate-tests mentioned this pull request Feb 6, 2018

Update dependency jest to v22 - autoclosed renovate-tests/changelog-md#3

Closed

renovate bot mentioned this pull request Feb 14, 2018

fix(deps): update dependency jest to ^22.0.0 manekinekko/devoxx-assistant#3

Open

github-actions bot locked as resolved and limited conversation to collaborators May 13, 2021


		const CACHE = path.resolve(os.tmpdir(), 'clear_cache_directory');

		skipOnWindows.suite();

add clearCache cli command #4430

add clearCache cli command #4430

Conversation

tabrindle commented Sep 5, 2017

thymikee commented Sep 5, 2017

thymikee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tabrindle commented Sep 6, 2017

thymikee commented Sep 6, 2017

SimenB Sep 7, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tabrindle commented Sep 8, 2017

thymikee commented Sep 8, 2017

tabrindle commented Sep 8, 2017

codecov-io commented Sep 8, 2017 • edited

Codecov Report

thymikee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tabrindle Sep 8, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tabrindle commented Sep 8, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tabrindle commented Oct 2, 2017

SimenB commented Oct 8, 2017 • edited

tabrindle commented Oct 11, 2017 • edited

SimenB commented Oct 11, 2017

tabrindle commented Oct 11, 2017

Choose a reason for hiding this comment

tabrindle Oct 11, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SimenB commented Oct 11, 2017

tabrindle commented Oct 11, 2017 • edited

tabrindle commented Oct 11, 2017

SimenB left a comment

Choose a reason for hiding this comment

cpojer commented Oct 11, 2017

SimenB commented Oct 11, 2017

tabrindle commented Oct 11, 2017

github-actions bot commented May 13, 2021

SimenB Sep 7, 2017 •

edited

codecov-io commented Sep 8, 2017 •

edited

tabrindle Sep 8, 2017 •

edited

SimenB commented Oct 8, 2017 •

edited

tabrindle commented Oct 11, 2017 •

edited

tabrindle Oct 11, 2017 •

edited

tabrindle commented Oct 11, 2017 •

edited