cache.save omitting empty directories breaks pre-commit on restore #5030

danielfdickinson · 2023-05-28T22:30:08Z

Describe the bug

cache.save omits directories with no 'leaf' files (that is any directory or multiple levels of directory which have not files at the bottom (empty dirs and dir trees)). This results in pre-commit failing for some pre-commit hooks (like markdownlint-cli, stylelint, eslint, cspell, and more).

The omitting of empty directories should be an optional feature.

Steps to reproduce

Create a .pre-commit-config.yaml file in the root of a Netlify site's Git repository which uses one or more of the mentioned pre-commit hooks.
Verify your pre-commit configuration works
Use a build plugin with uses cache.save(\<$XDG_CACHE_HOME>/pre-commit)
Remove \<$XDG_CACHE_HOME>/pre-commit
Use a build plugin which uses cache.restore(\<$XDG_CACHE_HOME>/pre-commit)
Attempt to use your previously working pre-commit configuration
It will fail with errors.

It only affects Node.js pre-commit hooks that I have found, to date.

CLI command and flags

N/A

Configuration

##############################

Netlify configuration file

##############################

# List of Build plugins
[[plugins]]
# Use the current plugin in builds for testing and debugging.
# Local path to the plugin.
package = "."
	# Plugin configuration
	#  [plugins.inputs]
	#  foo = "bar"

[build]
command = "bash ./scripts/netlify_build.sh"
publish = "public"

[build. Environment]
GO_VERSION = "1.20.4"
HUGO_ENABLEGITINFO = "true"
HUGO_ENV = "production"
HUGO_VERSION = "0.111.3"
NODE_VERSION = "v16.20.0"
TZ="America/Toronto"

Example pre-commit hook: https://github.com/danielfdickinson/netlify-plugin-pre-commit/tree/e9f92c60dc02f1f38e3a8cfbd897d02b9ba56326

CLI output

ubuntu@hugo-dev:~/dfd-zen-mp-workspace/netlify-plugin-pre-commit$ npm run test

> netlify-plugin-pre-commit@0.2.0-alpha.1 test
> npm run lint && npm run ava


> netlify-plugin-pre-commit@0.2.0-alpha.1 lint
> npm run eslint && npm run prettier


> netlify-plugin-pre-commit@0.2.0-alpha.1 eslint
> eslint --ignore-path .gitignore --fix --cache --format=codeframe --max-warnings=0 "{src,init}/**/*.{cjs,mjs,js}"


> netlify-plugin-pre-commit@0.2.0-alpha.1 prettier
> prettier --ignore-path .gitignore --write --loglevel warn "{.github,src,init}/**/*.{cjs,mjs,js}" "*.{cjs,mjs,js}"


> netlify-plugin-pre-commit@0.2.0-alpha.1 ava
> cross-env FORCE_COLOR=1 NETLIFY=true ava --verbose --timeout 5m



Netlify Build                                                 
────────────────────────────────────────────────────────────────

❯ Version
  @netlify/build 29.11.5

❯ Flags
  config: /home/ubuntu/dfd-zen-mp-workspace/netlify-plugin-pre-commit/netlify.toml

❯ Current directory
  /home/ubuntu/dfd-zen-mp-workspace/netlify-plugin-pre-commit

❯ Config file
  /home/ubuntu/dfd-zen-mp-workspace/netlify-plugin-pre-commit/netlify.toml

❯ Context
  production

❯ Loading plugins
   - .@0.2.0-alpha.1 from netlify.toml

. (onPreBuild event)                                          
────────────────────────────────────────────────────────────────

Removing existing local user cache in "/home/ubuntu/.cache/pre-commit"
Restoring user cache at "/home/ubuntu/.cache/pre-commit"
Restored user cache directory.
pre-commit installed at .git/hooks/pre-commit
check for added large files..............................................
Passed
check for case conflicts.................................................
Passed
check for merge conflicts................................................
Passed
check for broken symlinks............................(no files to check)Skipped
check toml...............................................................
Passed
check vcs permalinks.....................................................
Passed
detect private key.......................................................
Passed
file contents sorter.....................................................
Passed
fix utf-8 byte order marker..............................................
Passed
forbid new submodules................................(no files to check)Skipped
mixed line ending........................................................
Passed
don't commit to branch...................................................
Passed
pretty format json.......................................................
Passed
trim trailing whitespace.................................................
Passed
eslint...................................................................
Passed
markdownlint.............................................................
Failed
- hook id: markdownlint
- exit code: 1

node:internal/modules/cjs/loader:1073
  throw err;
  ^

Error: Cannot find module './package.json'
Require stack:
- /home/ubuntu/.cache/pre-commit/repo600nkozx/node_env-default/bin/markdownlint
    at Module._resolveFilename (node:internal/modules/cjs/loader:1070:15)
    at Module._load (node:internal/modules/cjs/loader:923:27)
    at Module.require (node:internal/modules/cjs/loader:1137:19)
    at require (node:internal/modules/helpers:121:18)
    at Object.<anonymous> (/home/ubuntu/.cache/pre-commit/repo600nkozx/node_env-default/bin/markdownlint:15:13)
    at Module._compile (node:internal/modules/cjs/loader:1255:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1309:10)
    at Module.load (node:internal/modules/cjs/loader:1113:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/home/ubuntu/.cache/pre-commit/repo600nkozx/node_env-default/bin/markdownlint'
  ]
}

Node.js v20.2.0
node:internal/modules/cjs/loader:1073
  throw err;
  ^

Error: Cannot find module './package.json'
Require stack:
- /home/ubuntu/.cache/pre-commit/repo600nkozx/node_env-default/bin/markdownlint
    at Module._resolveFilename (node:internal/modules/cjs/loader:1070:15)
    at Module._load (node:internal/modules/cjs/loader:923:27)
    at Module.require (node:internal/modules/cjs/loader:1137:19)
    at require (node:internal/modules/helpers:121:18)
    at Object.<anonymous> (/home/ubuntu/.cache/pre-commit/repo600nkozx/node_env-default/bin/markdownlint:15:13)
    at Module._compile (node:internal/modules/cjs/loader:1255:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1309:10)
    at Module.load (node:internal/modules/cjs/loader:1113:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:83:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/home/ubuntu/.cache/pre-commit/repo600nkozx/node_env-default/bin/markdownlint'
  ]
}

Node.js v20.2.0


Plugin "." failed                                             
────────────────────────────────────────────────────────────────
  ✘ [fail]: Netlify Build should succeed repeatedly

  Error message
  Error: Error message
  Hooks installation failed

  Plugin details
  Package:        .
  Version:        0.2.0-alpha.1
  Repository:     git+https://github.com/danielfdickinson/netlify-plugin-pre-commit.git
  Report issues:  https://github.com/danielfdickinson/netlify-plugin-pre-commit/issues

  Error location
  In "onPreBuild" event in "." from netlify.toml
      at applyHooks (file:///src/index.js:58:16)
      at async onPreBuild (file:///src/index.js:163:13)

  Resolved config
  build:
    command: bash ./scripts/netlify_build.sh
    commandOrigin: config
    environment:
      - HUGO_ENABLEGITINFO
      - HUGO_ENV
      - HUGO_VERSION
      - NODE_VERSION
      - TZ
    publish: /home/ubuntu/dfd-zen-mp-workspace/netlify-plugin-pre-commit/public
    publishOrigin: config
  plugins:
    - inputs: {}
      origin: config
      package: .
  ─

  Netlify Build should succeed repeatedly

  test/index.js:43

   42:   // }           
   43:   t.true(success)
   44: });              

  Value is not `true`:

  false

  › file://test/index.js:43:8

Environment

ubuntu@hugo-dev:~/dfd-zen-mp-workspace/netlify-plugin-pre-commit$ envinfo --system --binaries --npmPackages @netlify/build,@netlify/config,@netlify/git-utils,@netlify/cache-utils,@netlify/functions-utils,@netlify/run-utils,netlify-cli --npmGlobalPackages @netlify/build,netlify-cli
envinfo: command not found
ubuntu@hugo-dev:~/dfd-zen-mp-workspace/netlify-plugin-pre-commit$ npx envinfo --system --binaries --npmPackages @netlify/build,@netlify/config,@netlify/git-utils,@netlify/cache-utils,@netlify/functions-utils,@netlify/run-utils,netlify-cli --npmGlobalPackages @netlify/build,netlify-cli

  System:
    OS: Linux 5.15 Ubuntu 22.04.2 LTS 22.04.2 LTS (Jammy Jellyfish)
    CPU: (8) x64 Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
    Memory: 1.75 GB / 3.83 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 16.20.0 - ~/.nvm/versions/node/v16.20.0/bin/node
    npm: 8.19.4 - ~/.nvm/versions/node/v16.20.0/bin/npm
  npmPackages:
    @netlify/build: ^29.11.5 => 29.11.5 
    netlify-cli: ^15.2.0 => 15.2.0 
  npmGlobalPackages:
    netlify-cli: 15.2.0

Note that this also occurs on the Netlify web app (i.e. on Focal Fossa instance hosted by Netlify for doing deploys).

The text was updated successfully, but these errors were encountered:

Includes a workaround for netlify/build#5030 which prevented caching from working Signed-off-by: Daniel F. Dickinson <dfdpublic@wildtechgarden.ca>

danielfdickinson added the type: bug code to address defects in shipped code label May 28, 2023

lukasholzer added the Ecosystem: Dev Foundations label May 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cache.save omitting empty directories breaks pre-commit on restore #5030

cache.save omitting empty directories breaks pre-commit on restore #5030

danielfdickinson commented May 28, 2023

cache.save omitting empty directories breaks pre-commit on restore #5030

cache.save omitting empty directories breaks pre-commit on restore #5030

Comments

danielfdickinson commented May 28, 2023

Describe the bug

Steps to reproduce

CLI command and flags

Configuration

Netlify configuration file

CLI output

Environment