[Feature] Automatically detect safe-write #221

edo-codes · 2017-12-11T20:40:41Z

🐛 Bug report

While the parcel dev server or the watcher is running, changes in a file are sent to the browser correctly, but only the first time I save changes in the file. The second time, parcel doesn't rebuild (there is no console output) and the module doesn't reload in the browser. Other files will still reload, but also only once.

Even when refreshing in the browser, the changes aren't shown. I have to restart parcel for it to show the latest changes again.

Reproduction

Here's a minimal sample that shows the problem for me: https://github.com/Edootjuh/parcelrepro

To reproduce:

Start parcel with parcel index.html
Load http://localhost:1234 in browser. The page shows a b
Edit module-a and save. The page changes. Edit it again. The page doesn't change.
Edit module-b and save. The page changes. etc.

The same happens with parcel watch index.html

🌍 Your Environment

Software	Version(s)
Parcel	Latest on master (commit `2d680c0`)
Node	node v8.9.1
npm/Yarn	yarn 1.2.1
Operating System	Ubuntu 17.10

The text was updated successfully, but these errors were encountered:

davidnagli · 2017-12-11T20:55:06Z

I was able to reproduce this on Mac OS as well. However, the problem goes away when I remove module.hot.accept

edo-codes · 2017-12-11T20:56:52Z

That's weird. I just tried without module.hot.accept, and the problem is still there for me. (I've cleared the cache)

davidnagli · 2017-12-11T20:58:24Z

Can you commit the new code (the code without module.hot.accept) to a new branch in your example repo?

edo-codes · 2017-12-11T21:02:03Z

I'd just changed the code in the example repo to remove module.hot.accept in the master branch before your first comment, so if you pull/reset again it should update

DeMoorJasper · 2017-12-11T21:27:16Z

I can't seem to reproduce this in any way

edo-codes · 2017-12-11T22:06:11Z

I've done some poking around, and it seems like chokidar isn't firing change events for the subsequent edits in Bundler.js, but it does when I set usePolling to true. Obviously that isn't a fix, since that option results in poorer performance, but maybe that helps?

diff --git a/src/Bundler.js b/src/Bundler.js
index bb21004..cd0cf1f 100644
--- a/src/Bundler.js
+++ b/src/Bundler.js
@@ -176,7 +176,8 @@ class Bundler extends EventEmitter {
       // FS events on macOS are flakey in the tests, which write lots of files very quickly
       // See https://github.com/paulmillr/chokidar/issues/612
       this.watcher = new FSWatcher({
-        useFsEvents: process.env.NODE_ENV !== 'test'
+        useFsEvents: process.env.NODE_ENV !== 'test',
+        usePolling: true
       });
 
       this.watcher.on('change', this.onChange.bind(this));

edo-codes · 2017-12-11T22:46:08Z

I've found the problem. It has to do with the way many text editors write files, and is actually addressed by the webpack documentation: https://webpack.github.io/docs/webpack-dev-server.html#working-with-editors-ides-supporting-safe-write. To quote:

Working with editors/IDEs supporting “safe write”

Note that many editors support “safe write” feature and have it enabled by default, which makes dev server unable to watch files correctly. “Safe write” means changes are not written directly to original file but to temporary one instead, which is renamed and replaces original file when save operation is completed successfully. This behaviour causes file watcher to lose the track because the original file is removed. In order to prevent this issue, you have to disable “safe write” feature in your editor.

VIM - set :set backupcopy=yes (see documentation)

IntelliJ - Settings ▶︎ System Settings ▶︎ Synchronization ▶︎ disable safe write (may differ in various IntelliJ IDEs, but you can still use the search feature)

I suppose the reason I'd never had this problem before is because I'd always configured webpack to watch the directory instead of individual files.

This means this isn't really a parcel bug, but it is likely an issue that other people will face as well (I've tried editing the file with IntelliJ IDEA, vim and gedit, and they all behaved the same), so it might be worth adding an extra recursive watcher for e.g. the root directory so you can receive events for these 'new' files and rebuild.

davidnagli · 2017-12-11T22:49:33Z

Wow thanks for hunting this down! I would have never figured that out.

Do you guys think we should add a warning about this in the documentation?

DeMoorJasper · 2017-12-11T22:50:45Z

We should maybe add an option to the cli to enable polling for the people who have this activated in their IDE? (because this isn't really an option as standard for performance reasons)
And document it

edo-codes · 2017-12-11T22:57:13Z

Personally, I think watching the directory would solve it completely (though I could be wrong), but in the meantime adding a warning and a cli flag for usePolling seem like good options to me.

DeMoorJasper · 2017-12-11T22:59:12Z

isn't use polling poking the files every now and then, you might be right though?

davidnagli · 2017-12-11T22:59:55Z

Is there a way to detect “safe write”? Maybe we can dynamically fallback to polling when we detect it.

edo-codes · 2017-12-13T11:43:20Z

When you look at the raw fs events from chokidar, at least on Linux, on a safe-written file change you get a change event, and then two rename events (as opposed to a single change event).

DeMoorJasper · 2017-12-13T11:52:02Z

~~@Edootjuh Have you tried listening for the 'add' event, this would trigger if a new file gets written what happens with safe write i guess?~~
~~I can't unfortunately test any of this myself due to using mac OS, would be cool to have a fix in place that doesn't require any additional configuration or performance issues~~
Edit: this didn't work

juniovitorino · 2017-12-18T19:08:19Z

Here using VSCode the same behavior happen, VSCode + OS X High Sierra, HRM only works once.

DeMoorJasper · 2017-12-18T19:50:53Z

This is an IDE related issue @juniovitorino turn off safe-write and it should work

frhagn · 2018-10-23T18:36:25Z

I'm also having problems with safe-write when using Visual Studio 2017 combined with parcel watch.
The problem with Visual Studio is that safe-write can't be turned of..

From what I can tell the safe-write process in Visual Studio looks like this when saving a file (file.js in this example):

Create an empty temp file. 5jrvappx.wht~
Write the content to the temp file. 5jrvappx.wht~
Create a second empty temp file. file.js~RF46beda4b.TMP
Delete the second temp file. file.js~RF46beda4b.TMP
Rename the original file to the same filename as in step 3. file.js > file.js~RF46beda4b.TMP
Rename the first temp file from to the original filename. 5jrvappx.wht~ > file.js
Delete the temp file from step 5. file.js~RF46beda4b.TMP

@DeMoorJasper @kasbah maybe this will break the test:

   const temp = __dirname + '/input/5jrvappx.wht~';
   const file = __dirname + '/input/local.js';
   const backup = __dirname + '/input/local.js~RF46beda4b.TMP';
   function simulateSafewrite(content) {
      fs.writeFileSync(temp, content);
      fs.renameSync(file, backup);
      fs.renameSync(temp, file);
      fs.unlinkSync(backup);
    }

mitar · 2019-02-08T06:24:45Z

I would also add that coming from Meteor which works well with editors which use safe write, this seems like something which should be fixable and it should not require disabling safe write, which is at most a workaround and cannot be seen as simply a solution.

mitar · 2019-02-08T06:30:01Z

See here relevant code in Meteor.

mitar · 2019-02-08T07:01:21Z

Some related issues:

It seems chokidar has option atomic which we could try to make it work correctly when it is set to true: https://github.com/paulmillr/chokidar#errors I have tested it and it seems it does not work.

So it seems we should try to fix chokidar here, not parcel.

mitar · 2019-02-08T08:42:46Z

I think I fixed this. If you want to test it, just install git://github.com/mitar/chokidar.git in your app instead of the original chokidar.

See paulmillr/chokidar#791 for more details.

DeMoorJasper · 2019-02-08T16:47:39Z

@mitar awesome work, hope it'll get merged into chokidar 🎉

mikolajpp · 2019-03-04T10:28:19Z

@mitar I have tried to install chokidar from your repo, as a fix for #2661, but it unfortunately does not work, at least in neovim. Only the first write is detected, afterwards reloading does not work.

mitar · 2019-03-04T16:47:36Z

Strange. It works great for me for PyCharm. Make sure you really use that forked version and that it npm does not restore to previous version.

So I added to my package.json:

  "devDependencies": {
    "chokidar": "git://github.com/mitar/chokidar.git"
  }

And it works then.

mikolajpp · 2019-03-05T02:27:24Z

I have copied your entry verbatim.
Now my package.json looks like this:

{
  "dependencies": {
    "parcel-bundler": "^1.11.0"
  },
  "devDependencies": {
    "[chokidar](https://npmjs.com/package/chokidar)": "git://github.com/mitar/c$
    "elm-hot": "^1.0.1",
    "node-elm-compiler": "^5.0.3"
  }
}

Still, hot reload works only on the first edit in neovim. Afterwards it is broken.

mitar · 2019-03-05T02:31:26Z

Sorry. Not sure why there was that Markdown in JSON. I edited my comment before accordingly.

mikolajpp · 2019-03-05T02:34:47Z

Ah right. Still, I have fixed this, removed node_modules, did a reinstall and it is still broken.
Is there any way I can verify that the modified chokidar is indeed used/run by parcel?

mikolajpp · 2019-03-05T02:37:55Z

I have installed chokidar-cli, which I presume uses the version from your repo, and running
it from the command line reproduces the same behaviour in neovim. First write is detected, subsequent writes are not.

mitar · 2019-03-07T21:20:11Z

My PR has been merged into upstream chokidar.

DeMoorJasper · 2019-03-08T16:45:10Z

Should be fixed in Chokidar, closing this issue.

Feel free to open a new issue if this persists. (Do note that this fix has not been released yet and might take some time before your node_modules are up to date with the chokidar fix.)

mitar · 2019-03-08T17:07:18Z

Shouldn't then this issue be closed once chokidar is released with that fix and parcel updates its package.json to point to it?

Also, before closing this issue, documentation for parcel should be updated to not advise anymore to disable safe write.

DeMoorJasper · 2019-03-08T17:27:18Z

@mitar if it’s a minor release the package.json shouldn’t be updated although we probably will. But docs should probably wait untill chokidar had the update.

_{Sent with GitHawk}

cmcaine · 2019-05-28T11:54:35Z

chokidar reverted @mitar's workaround because it broke another tool and this is broken again.

paulmillr/chokidar@b79120b

My setup:

$ yarn list parcel chokidar
yarn list v1.16.0
warning Filtering by arguments is deprecated. Please use the pattern option instead.
├─ chokidar@2.1.6
└─ parcel@1.12.3
Done in 0.74s.

Writing a tiny chokidar script shows that chokidar isn't seeing the changes when it's watching particular files:

const chokidar = require('chokidar')

chokidar.watch('src/content.ts').on('change', console.log)

Chokidar sees the first change to the file, but not the second and subsequent. Watching the directory instead fixes this. I think parcel should watch the whole directory and filter names on linux. This gives a quick fix that won't break any other tools.

cmcaine · 2019-05-28T12:03:38Z

Obviously another fix would be to just pin chokidar at 2.1.3 or 2.1.4 for now.

I'm doing that locally with:

// package.json
  "resolutions": {
    "parcel/**/chokidar": "2.1.3"
  }

Which is a yarn feature: https://yarnpkg.com/en/docs/selective-version-resolutions

Augustin82 · 2020-12-24T13:15:47Z

Obviously another fix would be to just pin chokidar at 2.1.3 or 2.1.4 for now.

I'm doing that locally with:
// package.json
  "resolutions": {
    "parcel/**/chokidar": "2.1.3"
  }
Which is a yarn feature: https://yarnpkg.com/en/docs/selective-version-resolutions

This fixed it for me, many thanks.

gk9848970 · 2021-03-08T16:54:54Z

I was able to reproduce this on Mac OS as well. However, the problem goes away when I remove module.hot.accept

How and from where to remove it?

gk9848970 · 2021-03-08T16:57:20Z

I've found the problem. It has to do with the way many text editors write files, and is actually addressed by the webpack documentation: https://webpack.github.io/docs/webpack-dev-server.html#working-with-editors-ides-supporting-safe-write. To quote:

Working with editors/IDEs supporting “safe write”

Note that many editors support “safe write” feature and have it enabled by default, which makes dev server unable to watch files correctly. “Safe write” means changes are not written directly to original file but to temporary one instead, which is renamed and replaces original file when save operation is completed successfully. This behaviour causes file watcher to lose the track because the original file is removed. In order to prevent this issue, you have to disable “safe write” feature in your editor.

VIM - set :set backupcopy=yes (see documentation)

IntelliJ - Settings ▶︎ System Settings ▶︎ Synchronization ▶︎ disable safe write (may differ in various IntelliJ IDEs, but you can still use the search feature)

I suppose the reason I'd never had this problem before is because I'd always configured webpack to watch the directory instead of individual files.

This means this isn't really a parcel bug, but it is likely an issue that other people will face as well (I've tried editing the file with IntelliJ IDEA, vim and gedit, and they all behaved the same), so it might be worth adding an extra recursive watcher for e.g. the root directory so you can receive events for these 'new' files and rebuild.

How to do this in VS Code?

piyushk · 2021-06-29T20:39:41Z

chokidar reverted @mitar's workaround because it broke another tool and this is broken again.

paulmillr/chokidar@b79120b

Should this issue be reopened since the fix in chokidar was reverted?

DeMoorJasper added the 🐛 Bug label Dec 11, 2017

davidnagli added the HMR Hot Module Reloading label Dec 11, 2017

davidnagli added the ✔️ Confirmed Bug label Dec 11, 2017

davidnagli added this to To Do in Bugs via automation Dec 11, 2017

DeMoorJasper moved this from To Do to PR in Bugs Dec 12, 2017

DeMoorJasper mentioned this issue Dec 12, 2017

Add CLI option to enable FSWatcher polling #238

Closed

DeMoorJasper self-assigned this Dec 12, 2017

DeMoorJasper changed the title ~~HMR only reloads once for each file~~ 🐛 HMR only reloads once for each file Dec 13, 2017

DeMoorJasper mentioned this issue Dec 17, 2017

Add documentation safe-write parcel-bundler/website#53

Closed

DeMoorJasper closed this as completed Dec 18, 2017

Bugs automation moved this from PR to Done Dec 18, 2017

DeMoorJasper mentioned this issue Dec 22, 2017

Reloading is not working for changes inside folder #382

Closed

brandon93s mentioned this issue Dec 25, 2017

[bug🐛] live-reload is not working sometime #397

Closed

DeMoorJasper mentioned this issue Dec 28, 2017

Nothing happens after source code change. #424

Closed

DeMoorJasper closed this as completed Mar 8, 2019

Bugs automation moved this from To Do to Done Mar 8, 2019

cmcaine mentioned this issue May 28, 2019

A better workaround for atomic writes paulmillr/chokidar#791

Merged

jehkinen mentioned this issue Jul 27, 2019

Watch hangs / doesn't watch anymore in parcel-bundler@1.11.0 #2727

Closed

paco3346 mentioned this issue Sep 28, 2019

Rollup watch not reloading on change sveltejs/sapper#617

Closed

milesfrain mentioned this issue May 5, 2020

Atomic copyFile to foreign.js breaks filewatchers purescript/purescript#3862

Closed

vkottler mentioned this issue Nov 20, 2022

Set up an initial project with parcel and typescript vkottler/console#1

Closed

largis21 mentioned this issue Oct 14, 2023

feat: Chokidar use polling on rollup watch sanity-io/pkg-utils#268

Merged

[Feature] Automatically detect safe-write #221

[Feature] Automatically detect safe-write #221

Comments

edo-codes commented Dec 11, 2017 • edited Loading

🐛 Bug report

Reproduction

🌍 Your Environment

davidnagli commented Dec 11, 2017

edo-codes commented Dec 11, 2017

davidnagli commented Dec 11, 2017

edo-codes commented Dec 11, 2017

DeMoorJasper commented Dec 11, 2017 • edited Loading

edo-codes commented Dec 11, 2017 • edited Loading

edo-codes commented Dec 11, 2017 • edited Loading

Working with editors/IDEs supporting “safe write”

davidnagli commented Dec 11, 2017

DeMoorJasper commented Dec 11, 2017 • edited Loading

edo-codes commented Dec 11, 2017

DeMoorJasper commented Dec 11, 2017 • edited Loading

davidnagli commented Dec 11, 2017

edo-codes commented Dec 13, 2017

DeMoorJasper commented Dec 13, 2017 • edited Loading

juniovitorino commented Dec 18, 2017 • edited Loading

DeMoorJasper commented Dec 18, 2017

frhagn commented Oct 23, 2018

mitar commented Feb 8, 2019

mitar commented Feb 8, 2019

mitar commented Feb 8, 2019 • edited Loading

mitar commented Feb 8, 2019

DeMoorJasper commented Feb 8, 2019

mikolajpp commented Mar 4, 2019

mitar commented Mar 4, 2019 • edited Loading

mikolajpp commented Mar 5, 2019 • edited Loading

mitar commented Mar 5, 2019

mikolajpp commented Mar 5, 2019

mikolajpp commented Mar 5, 2019

mitar commented Mar 7, 2019

DeMoorJasper commented Mar 8, 2019 • edited Loading

mitar commented Mar 8, 2019

DeMoorJasper commented Mar 8, 2019 • edited Loading

cmcaine commented May 28, 2019

cmcaine commented May 28, 2019 • edited Loading

Augustin82 commented Dec 24, 2020

gk9848970 commented Mar 8, 2021

gk9848970 commented Mar 8, 2021

Working with editors/IDEs supporting “safe write”

piyushk commented Jun 29, 2021 • edited Loading

edo-codes commented Dec 11, 2017 •

edited

Loading

DeMoorJasper commented Dec 11, 2017 •

edited

Loading

edo-codes commented Dec 11, 2017 •

edited

Loading

edo-codes commented Dec 11, 2017 •

edited

Loading

DeMoorJasper commented Dec 11, 2017 •

edited

Loading

DeMoorJasper commented Dec 11, 2017 •

edited

Loading

DeMoorJasper commented Dec 13, 2017 •

edited

Loading

juniovitorino commented Dec 18, 2017 •

edited

Loading

mitar commented Feb 8, 2019 •

edited

Loading

mitar commented Mar 4, 2019 •

edited

Loading

mikolajpp commented Mar 5, 2019 •

edited

Loading

DeMoorJasper commented Mar 8, 2019 •

edited

Loading

DeMoorJasper commented Mar 8, 2019 •

edited

Loading

cmcaine commented May 28, 2019 •

edited

Loading

piyushk commented Jun 29, 2021 •

edited

Loading