Heroku custom cacheing is not working

[croraf]

I have toplevel package.json where I set cacheDirectories as per https://devcenter.heroku.com/articles/nodejs-support#custom-caching

  "scripts": {
    "start": "npm --prefix frontend start && npm --prefix mockserver start",
  },
  "dependencies": {
  },
  "devDependencies": {
  },
  "cacheDirectories": ["frontend/node_modules", "mockserver/node_modules"],

So no dependencies in toplevel, but I have frontend/package.json and mockserver/package.json with a lot of dependencies that get installed in their respective node_modules. This is how the respective package.jsons look like (for example mockserver):

  "scripts": {
    "start": "npm install && npm run build && node dist/server.js",
  },
 dependencies...
 devdependencies...

This works fine and deploys, but it does not cache the the two node_modules defined to be cached.

Here is the output from heroku build:

-----> Creating runtime environment
       
       NPM_CONFIG_LOGLEVEL=error
       NPM_CONFIG_PRODUCTION=true
       NODE_VERBOSE=false
       NODE_ENV=demo
       NODE_MODULES_CACHE=true
       
       npm scripts will see NODE_ENV=production (not 'demo')
       https://docs.npmjs.com/misc/config#production
-----> Installing binaries
       engines.node (package.json):  7.10.0
       engines.npm (package.json):   unspecified (use default)
       
       Downloading and installing node 7.10.0...
       Using default npm version: 4.2.0
-----> Restoring cache
       Loading 2 from cacheDirectories (package.json):
       - frontend/node_modules (not cached - skipping)
       - mockserver/node_modules (not cached - skipping)
-----> Building dependencies
       Installing node modules (package.json)
-----> Caching build
       Clearing previous node cache
       Saving 2 cacheDirectories (package.json):
       - frontend/node_modules (nothing to cache)
       - mockserver/node_modules (nothing to cache)
-----> Build succeeded!

Just to note that the two node_modules were installed in the previous build but it says (not cached - skipping).
Thanks in advance!

[jmorrell]

How are you invoking npm install in the subdirectories? If you do it as part of the heroku-postbuild hook then they should get cached:

"scripts": {
  ...
  "heroku-postbuild": "cd frontend && npm install && cd ../mockserver && npm install"
}

If you aren't invoking npm install (or yarn install) in the subdirectories then their dependencies will never be downloaded, and so node_modules in those directories won't exist.

[croraf]

Top level package.json runs:
npm --prefix mockserver start
meaning it runs the "start" script from mockserver/package.json.
That script is:
"npm install && npm run build && node dist/server.js"
So node_modules gets installed. Indeed my application works fine.
Hopefully it gets installed at mockserver/node_modules location.
But it is not cached for the next build.

[jmorrell]

Top level package.json runs: npm --prefix mockserver start

Yes, but if it does that using the start script instead of the heroku-postbuild script, then it will be after the cache is calculated. By the time the server is started the cache is already set.

[croraf]

Thank you very much! I now used heroku-postbuild script in top level package.json to install dependencies from subprojects and they get cached as expected.

In fact, I think I could use any of heroku-prebuild preinstall postinstall heroku-postbuild, as they are all run in the "building dependencies" stage, after "restoring cache" stage and before "caching" stage.