Parcel 2 doesn't detect changes in linked dependencies.

[samuelgozi]

🐛 bug report

When working on a dependency locally by linking it, Parcel 2 doesn't detect changes and therefore doesn't rebuild. You also need to remove the cache manually in order for it to work on re-runs.

🎛 Configuration (.babelrc, package.json, cli command)

Package.json

{
	"name": "parcel-tamplate",
	"version": "1.1.0",
	"description": "My personal Parcel Template",
	"main": "public/index.html",
	"source": "src/index.html",
	"author": "Samuel Elgozi <samu.elgozi@gmail.com>",
	"license": "none",
	"private": true,
	"scripts": {
		"dev": "parcel src/index.html",
		"build": "parcel build src/index.html"
	},
	"targets": {
		"main": {
			"includeNodeModules": true
		}
	},
	"devDependencies": {
		"@babel/plugin-proposal-class-properties": "^7.8.3",
		"babel-eslint": "^10.0.2",
		"eslint": "^6.0.1",
		"less": "^3.9.0",
		"parcel": "^2.0.0-alpha.3.2"
	},
	"dependencies": {
		"firebase-auth-lite": "^0.4.1"
	}
}

The commands I use:

parcel src/index.html

🤔 Expected Behavior

Like in version 1, Parcel should detect changes in linked dependencies, invalidate the cache and rebuild.

😯 Current Behavior

No changes are made, which require manual removal of .parcel-cache and then building again.

🔦 Context

I use Parcel to work on npm modules often. Without detecting the changes in linked packages there is no reason for me to use parcel. In fact, it becomes painful to do so...

🌍 Your Environment

Software	Version(s)
Parcel	2.0.0-alpha.3.2
Node	v13.10.1
Yarn	1.22.1
Operating System	MacOs Catalina 10.15.3

[DeMoorJasper]

Could you create a repo with some instructions on how I'd reproduce this? Would help a lot in fixing this

[samuelgozi]

Ok, here you go: https://github.com/samuelgozi/Parcel2-issue-4332
Further instructions can be found in the repo's README file.

[DeMoorJasper]

@samuelgozi thanks for creating the repo, I can reproduce it pretty consistently, will try and figure out why it happens and hopefully fix it

[samuelgozi]

@DeMoorJasper Thank you!

[kalaschnik]

I develop a local npm package and experience the same issue using the npm link functionality. Is there any news on this?

[devongovett]

This is because the file watcher only watches for changes in the project root. It would be somewhat hard to implement support for symlinks outside the project I think. If we implemented this in the watcher we'd need to somehow crawl the whole file system to detect symlinks in the first place which would be very slow. We could potentially do it by detecting files that resolve to symlinks as we build the asset graph and keep nodes in the request graph to track watch roots. Then we'd have to re-watch all of those on startup and invalidate each of them for changes as opposed to just the project root.

(parcel-bundler/watcher#32)

[ramirofages]

This is because the file watcher only watches for changes in the project root. It would be somewhat hard to implement support for symlinks outside the project I think. If we implemented this in the watcher we'd need to somehow crawl the whole file system to detect symlinks in the first place which would be very slow. We could potentially do it by detecting files that resolve to symlinks as we build the asset graph and keep nodes in the request graph to track watch roots. Then we'd have to re-watch all of those on startup and invalidate each of them for changes as opposed to just the project root.

(parcel-bundler/watcher#32)

This was actually working in parcel 1. Currently I had to revert back to parcel 1 while keeping its dependencies in parcel 2 to correctly build as module.

[krnlde]

This issue keeps me from migrating to v2, too. I wonder how other developers handle this workspaces issue - also considering monorepos are now en vogue. We can't possibly be alone here @ramirofages

[krnlde]

You guys should have a look at #6039 regarding the yarn.lock / package-lock.json files. This probably solves the issue for you too.

[kristojorg]

That linked issue doesn't resolve this for me. I don't have a dangling yarn.lock file, just the one in my primary repository I'm working on. Linked dependency changes don't trigger a recompile, and it seems even restarting parcel and running it with --no-cache don't trigger a full rebuild. I have the delete .parcel-cache in order to have the changes picked up : /

[krnlde]

Afaik parcel can't look outside its start context so if you want to watch all workspaces in a workspace root you'll need to start parcel in that root. Did you do that? Be extra sure to not have any lock files inside any of the workspaces. Parcel unfortunately has no warning for that and will just silently fall back to the workspace scope instead of root scope, which took a lot of lifetime from me in the past the point where I wanted to quit my job and start a lumber business :D

[kristojorg]

I am trying to link a dependency that is located in a separate folder and repository on my machine, but installed via npm. The node_modules/my-dep folder does change when I update the linked dependency, but parcel doesn't seem to pick up the changes there. Do you know if there is a way to have it watch and rebuild node_modules as well?

[krnlde]

As long as the node_modules is inside the parcel start context it works. Again parcel picks its start context by probing all folders for .lock files starting with the most-inner one from your start path: parcel serve workspaces/workspace-a/index.html probes:

1. workspaces/workspace-a/*.lock
2. workspaces/*.lock
3. *.lock

wherever a lock file is found it will not recognize changes beyond that folder. Ever. And they are not planning to change that.

[kristojorg]

Yep I hear you, but let me be more explicit about my setup. I have:

project-a/
    src/                      # code lives here
    example/
        index.html       # no lockfile here, no package.json, no node_modules
    package.json      # here is my script "npm run example": "parcel example/index.html"
    package-lock.json
    node_modules/
        project-b/
project-b/                # linked to ^
    src/
    dist/

It is the linked project-b dependency which is changing and not triggering a rebuild inside project-a when I run npm run example. So what I'm trying to say is that this isn't an issue of a misplaced or extra *.lock file it seems.

[krnlde]

So its linked by npm link and not by workspace logic? I couldn't repeat myself more but:

wherever a lock file is found it will not recognize changes beyond that folder

Edit: also to clarify, parcel doesn't watch the node_modules (since its essentially a black hole), it watches source files and triggers a reload when they change.

[mischnic]

If project-a contains a lockfile or a .git directory, it will be the project root and only files inside of that are watched.

Edit: also to clarify, parcel doesn't watch the node_modules (since its essentially a black hole), it watches source files and triggers a reload when they change.

No, it watches everything except for VCS directories and .parcel-cache:

parcel/packages/core/core/src/RequestTracker.js

Lines 918 to 920 in ff6b7b4

	let vcsDirs = ['.git', '.hg'].map(dir => path.join(options.projectRoot, dir));
	let ignore = [options.cacheDir, ...vcsDirs];
	return {ignore};

[kristojorg]

Okay yes it is the "Edit" there that is my problem then. The confusing thing is that this did work with parcel v1. When I would npm link package-b, parcel would rebuild package-a whenever it changed.

As mentioned, project-a does contain a lockfile and a .git directory, but node_modules are within that folder so I expected them to be watched, or at least not permanently cached when I restart with npm run example. I currently have to delete .parcel-cache in order for it to include the updated node_modules.

[krnlde]

Ah cool! Didn't know that. How about symlinks then?

[kristojorg]

@mischnic so it does appear that this is a legitimate bug and not intended behavior, then. Should node_modules possibly be excluded from being written to the .parcel-cache?

[mischnic]

How about symlinks then?

"only files inside of that are watched" essentially ignores symlinks. This is mostly because the file watching APIs from the various OSs only fire change events for the actual parent directories and don't traverse through every symlink

So node_modules is watched, but node_modules/symlinked-package/* isn't

So this is more of a technical limitation at the moment, as explained in #4332 (comment) and not a design decision

a legitimate bug and not intended behavior, then. Should node_modules possibly be excluded

What do you mean? It intentionally watches node_modules so that updating dependencies and rerunning Yarn triggers a Parcel rebuild for changed files.

[kristojorg]

I see, thanks for the explanation. I wonder if it would be possible to make an option to explicitly tell parcel about a symlinked dependency you would like to be watched, as this does seem like it would be a common use case.

What do you mean? It intentionally watches node_modules so that updating dependencies and rerunning Yarn triggers a Parcel rebuild for changed files.

I meant should node_modules be watched but not cached, such that when you restart parcel it is reading node_modules fresh and not using a cached version. I don't think I totally understand the value of including node_modules files in the cache if they are not bundled. Seems like I'm missing something tho : )

[devongovett]

That's not the issue. Symlinks can still resolve to locations outside Parcel's project root, which is defined to be the nearest directory with a yarn.lock or package-lock.json file. So if those are not at your repo root, but you reference a file outside of this directory, it won't be watched.

Is there a reason not to put your lock file at the root of your repo?

[bminer]

@Systemcluster - As discussed above, allowing Parcel to watch symlinks outside the project root is trickier to implement. Allowing users to specify additional paths to watch outside of Parcel's project root seems like a fair compromise - and it sounds rather easy to do. As many have mentioned, the "project root" means the folder containing the package lock file.

[rodrigo-nexudus]

This used to work without issue in V1 (parcel-bundler) just by adding a link in package.json of the main project. Any changes on the linked package caused a refresh. This has now been open for 3 years and it was something that worked flawlessly on the first version of Parcel. It's an incredibly useful feature when you're working on a library.

[bminer]

Wow it really has been 3 years! Like I've said, I'd be happy to open a PR to resolve this one if I can get some feedback from the devs that it's likely to be merged. If the devs explicitly don't want this feature, I don't want to waste my time.

Thoughts? @devongovett

[maxx0r]

I for one would be really happy with a fix. I've built a script in my webapp that copies on change, not really the solutions you want

[rodrigo-nexudus]

@bminer I don't see why at least giving developers an option through a flag as you suggested would be an issue, if respecting symlinks outside the project root as before is out of the question. In our case, the package I'm working on is a dependency on more than one in-house project, and is not an integral part of any of the parent projects. It doesn't make sense to build all as "workspaces" as the parent projects have no relation between them.

[tpompon]

I'm getting the same issue. Building a package and I have a demo app using Parcel 2 in an folder example, I don't want to mix my .lock files and I would like to be able to get the app refreshing when I apply changes on the package components.

[AmrAhmedA]

I run into this issue recently, the fact that it is standing for 3 years is amazing. unfortuntly, such limitition in parcel 2 makes it less productive when working with linked dependencies.
Possible workaround I found is to refresh package.json for triggering new build.
any convenient solution proposed so far?

[ThomasZeman]

I agree it would be great to have this fixed. My scenario seems even simpler:

some_root_dir/
│
├─ client/
│ ├─ index.html
│ ├─ package.json
│ ├─ package-lock.json
│ └─ client.js
│
└─ shared/shared.js

where client.js imports shared.js -- When I change client.js Parcel does its thing, updates the browser and life is good. When I updated shared.js nothing happens. So to me, the file-watching issue comes down to:
Parcel should watch all files referenced starting from index.html as entry point no matter which directories these are in.

[rowthan]

I agree it would be great to have this fixed. My scenario seems even simpler:

some_root_dir/ │ ├─ client/ │ ├─ index.html │ ├─ package.json │ ├─ package-lock.json │ └─ client.js │ └─ shared/shared.js

where client.js imports shared.js -- When I change client.js Parcel does its thing, updates the browser and life is good. When I updated shared.js nothing happens. So to me, the file-watching issue comes down to: Parcel should watch all files referenced starting from index.html as entry point no matter which directories these are in.

some issue, any way to solve this problem?

[rowthan]

I agree it would be great to have this fixed. My scenario seems even simpler:

some_root_dir/ │ ├─ client/ │ ├─ index.html │ ├─ package.json │ ├─ package-lock.json │ └─ client.js │ └─ shared/shared.js

where client.js imports shared.js -- When I change client.js Parcel does its thing, updates the browser and life is good. When I updated shared.js nothing happens. So to me, the file-watching issue comes down to: Parcel should watch all files referenced starting from index.html as entry point no matter which directories these are in.

@rodrigo-nexudus there is the case what we want. example serve by parcel is just a side part of parent lib dir. we just want limit all test or dev code in example dir. But it should be rebuild after file changed in src.
So it's a very important for us to listen the change from other dir.

[nullablemind]

@jteppinette Thanks for your message. #4332 (comment)
Thanks to you, at least I was able to fasten the crutch so that it all works fine.
I have added a script to watch files using chokidar and simply call a touch package.json when have a change.

The best I could do at the moment. Since inside the folder, there are still their dependencies with package-lock.json.

[jfcere]

I personally worked around this using nodemon and concurrently with npm scripts, this solution is not perfect but works for me so I thought it worth sharing.

npm i concurrently nodemon --save-dev

See start script below which will concurrently...

• watch my lib src folder and rebuild it when it changes (I use rollup to bundle my lib)
• link my local library to node_module (I use the old linklocal package but you can use npm-link for that step)
• and finally start the web page using parcel

  "scripts": {
    "prestart": "rimraf .parcel-cache",
    "start": "concurrently \"npm run watch:lib\" \"npm run link:lib && npm run start:demo\"",
    "start:demo": "parcel demo/index.html --open",
    "build:lib": "rollup -c rollup.config.ts --configPlugin @rollup/plugin-typescript",
    "link:lib": "rimraf node_modules/my-lib && linklocal",
    "watch:lib": "nodemon -e ts --watch src --exec npm run build:lib",
  },

Hope that helps anyone!

[devongovett]

FYI, thanks for @jondlm, there is a new --watch-root CLI option to configure the root directory where the watcher listens which will be available in the next release. See #9424 for details.

[remifroger]

Thanks, it works for me with --watch-dir

I've the app and, outside, I've shared js files (assets dir).

app/
├─ src/client/js/index.js
├─ package.json
├─ package-lock.json
assets/js

The dir assets is installed with npm i ../assets in the app
Then I import my modules in the app/src/client/js/index.js, for example : import { myFunc1, myFunc2 } from 'assets/js/map.js'

And now with this config in the app/package.json and the new --watch-dir :

"scripts": {
  "build-watch": "parcel watch ./src/client/js/index.js --watch-dir ../assets --host localhost --no-cache --public-url ./ --dist-dir ./dist && npm cache clean --force",
  "start-watch": "nodemon ./src/server/app.js",
   "dev": "concurrently --kill-others \"npm run start-watch\" \"npm run build-watch\""
}

The HMR is working again, like in parcel1. I mean, if I make a change in my assets js files, the app rebuild automatically.

[everdimension]

For those having issues, I have created a CLI-tool that can help deal with local npm packages: physical-link

It's not parcel-specific, but it solves the problem of aliased packages not being watched. What it does is basically watch your package files and copy changes to the node_modules directory of the project where you're using that package.

It's a handy tool when you can't have a monorepo and develop your packages as separate projects, and use them in your other projects.

[zeel01]

I really hoped that when I Googled this issue I would find a solution. I went off on a tangent one night writing a bunch of utilities to simplify a project, then realized it made no sense for them to belong to that project instead of being somewhere I could share with other projects. So I pulled it out into its own package... and now Parcel ignores any changes I make to it because the linked package doesn't trigger the watcher.

[bminer]

Since --watch-dir is a good workaround, is it safe to close this issue?

[bradleat]

This still does not work for me when I do a parcel build in a monorepo using the latest version (2.12.0)

I am using a lerna monorepo and this is my full parcel command:

NODE_ENV=development npx parcel build --no-autoinstall --no-optimize --target frontends --cache-dir ./.parcel-cache --watch-dir ../../,

I didn't really want to setup a dev / watch command although I might now...

[bminer]

Interesting. I am doing something quite similar, and it seems to work fine: parcel --watch-dir ../ index.html

[zeel01]

I don't think --watch-dir even comes close to solving the original issue. If you use npm link to symlink a package into your project, Parcel doesn't follow the link. Adding --watch-dir to watch the linked package that's somewhere else on your machine (possibly not in the same parent directory as your project even) is at best a hacky work-around since you might need to resort to an absolute file path. But it also doesn't seem to work reliably anyway.

The entire purpose if npm link is so you can include a local version of a dependency that might be unrelated to the current project without replicating a specific directory structure. As such, the bundler should support the feature as it is intended to be used without need for a work-around. Even if --watch-dir is a workable work-around for some users, the core issue remains.

Personally, I think the original labeling of this as a bug was correct since it's a regression from Parcel 1. But even as a feature request, it's still more than valid and is not adequately satisfied by --watch-dir.

[bradleat]

I agree. However, this especially noticeable when using parcel build over parcel watch.

[bminer]

I agree. However, this especially noticeable when using parcel build over parcel watch.

@bradleat - Are you saying that parcel build does not detect changes to paths outside of the Parcel "project root?" Are you absolutely sure about this? I was under the impression that this issue is specific to parcel watch only.

[bradleat]

@bminer Yes. When I parcel build with the command above and have updated linked build files, they are sometimes not included in the build. I really can't figure out why it is sometimes.

When watching, it will work.

[zeel01]

It seems like it just uses it's cached version since it doesn't detect that anything is changed. I'm constantly needing to use rimraf to clear the cache out and get it to rebuild.

[brianhelba]

@bminer In my experience with this bug, the problem was that Parcel watching didn't follow symlinks. If watching .. coincidentally also included the real destination of the symlink, then everything might work, but other project layouts could still cause problems.

In my case, I wasn't using npm link, but I had some application code which depended on an associated library via a local path. On Linux, npm or Yarn Classic (I can't remember which 😕) would create a symlink within node_modules, which Parcel wouldn't follow.

I recall that I was able to bisect the origin of this bug to some refactor of how symlinks were resolved when traversing node_modules. At that point, I gave up and just used --no-cache. Sorry that I can't be more precise or certain, since this occurred for me several years ago.