Transient read failure may cause permanent failure to watch file

[willdean]

V3.1.18
Tailwind CLI, Visual Studio 2022 as editor.
Node v14.18.1
Chrome
Window 10 (No Docker or WSL)

This issue is a very close relative of #7759, but I think it might be quite specialised so I don't want to pollute that one.

I have a rather odd web project, which has a single HTML file. So the content section of tailwind.config.js looks like this:

  content: [
      "./wwwroot/index.html"
  ],

I run a tailwind CLI watcher with a command like this:

npx tailwindcss --output wwwroot/css/tailwind.css --watch -i css/twbase.css

I am suffering from the problem described in #7759 where the watcher stops watching after a while - anywhere between almost immediately and after lots and lots of successful rebuilds.

By using the "Process Monitor" utility to observe the filesystem activity on the index.html file, I have observed that when Visual Studio (devenv.exe) is saving index.html, it goes through a rename/replace process which means there is a window of time in which the index.html file does not actually exist. It appears that node.exe (i.e. Tailwind) sometimes tries to read the file during this window and finds it to be missing.

This apparently causes Tailwind to stop watching that file.

The blue selected line in the image is Node getting a file-not-found failure doing a CreateFile (it's actually an open-to-read) on index.html in the middle of devenv doing a rename dance. I think that's a smoking gun.

If I change the content file specification to be a wildcard, then my first impression is that things are much more robust:

  content: [
      "./wwwroot/*.html"
  ],

So:

1. Is it possible/likely that TW is converting a transient read-failure on a watched file into a permanent failure to watch that file?
2. If so, could this be fixed?

Even without the specific Visual Studio rename thing, transient read failures are probably an inescapable feature of Windows, because it's so common that people have tools like anti-virus and cloud-backup and indexers which do lots of automatic file opening.

[thecrypticace]

@willdean Could you provide some details on the specs of your system? Like are you running off of a hard drive instead of an SSD? I'm trying but I can't get this to fail and I'm betting it's a super weird timing thing.

[thecrypticace]

@willdean Okay I did a bit more testing. I can reproduce something similar-ish to this when using Visual Studio. But other processes writing files in a similar manner do not cause issues. I'm not sure why this is but it seems specific to Visual Studio and how it's writing files. Doing a write + atomic rename does not appear to cause problems generally.

[willdean]

Thanks for looking at this - I'm on a high-end SSD dev machine, but that doesn't mean that there aren't weird timing problems.

My own naïve approach to this would be to massively boost-up the logging in Tailwind, so that we could actually see A. what change events it's getting (that might mess-up the timing) and B. see when it suffers failures to open files, because it seems possible to me that one of those fs.readFileSync calls in, say, getChangedContent is failing in this situation, but I'm unclear where the exception from that ends-up - I'm not really clear what the call stack into those watcher event handlers looks like.

Like I said elsewhere, I know zip about Node, and I'm not even clear how I would modify the source (which I have) and run a modified version of the cli, but I'll have a play.

Is there some simple node ./src/cli.js sort of thing I could be running, or is there a build process I'd need to go through?

[thecrypticace]

yeah you have to run a build process generally but, if you want, you can directly edit node_modules/tailwindcss/lib/cli.js — though it may be a bit weird to do so but yeah that's exactly what I'm gonna do. I suspect that we're probably not getting notified at all after a point but I'm not certain yet. And thanks for the details.

[willdean]

I've just had a bit of a play with Chokidar by itself, to see what events it's seeing when VS edits that same index.html file.

I ran a minimal example of this code:

const chokidar = require('chokidar');
chokidar.watch('n:/xxx/wwwroot/index.html').on('all', (event, path) => {
  console.log(event, path);
});

This reports a quite interesting sequence of events:
Upon starting the Chokidar test app:

add n:\xxx\wwwroot\index.html

The first time I save the file in VS:

unlink n:\xxx\wwwroot\index.html

The second time I save the file in VS:

change n:\xxx\wwwroot\index.html
add n:\xxx\wwwroot\index.html

All subsequence saves in VS:

change n:\xxx\wwwroot\index.html

Next I will try reading the file in the event handler, to see what happens.

[thecrypticace]

Yeah I just poked around and chokidar is not firing change events when saved by Visual Studio. It is however firing a "raw" event with details which I can see:

It should be firing a change event but I'm not sure why it's not. I'd imagine this is a chokidar bug but maybe we can do something to work around it. I'll keep poking around.

[willdean]

Looking through the issues on Chokidar's GH, it may just not work very well on Windows - apparently VS Code tried to use it and gave up for Windows and did their own thing.

[thecrypticace]

So it appears that this fixes it for the CLI:

watcher.on("unlink", (file) => watcher.add(file))

The rename operation firing an unlink causes it to be removed from the list of watched files.

[thecrypticace]

the problem is that, for renames, it's supposed to fire an "add" event internally which it is not doing for some reason. Chokidar uses this unlink / add dance with a timer to detect renames and it doesn't appear to work here for some reason.

[willdean]

Unfortunately your fix doesn't work for me - I added it (with some logging), and I get an unlink event for every save, but nothing else.

I have some other lines of enquiry, because my little tester is showing a different series of events to those that I get in TW when I remove all of TW's event handlers and just log Chokidar events. This may be something about Chokidar configuration.

[thecrypticace]

What version of chokidar are you testing?

[willdean]

3.5.3. I think that's the same as TW's using.
I think the differences might be to do with the write-event-coalescence stuff, but I'm still seeing my simple three line test keep working after unlink events without needing to re-add the file.
The last difference is the exact list of files passed in, so I'll look at that.

[thecrypticace]

@willdean Can you clone this repo: https://github.com/thecrypticace/tw-cli-read-failure-repo

1. Run node .\watch.mjs
2. Open the folder and repeatedly save src/index.html in Visual Studio

And paste the output here? I see a few changes and unlinks for the first couple of saves (depends on how much and how quickly you save) and then they stop being delivered. I also left the "fix" that I mentioned commented. Could you run it with it not commented too and post those as well? Really appreciate your time here — I know this is a lot :)

This is what I see for both of these cases:

Windows.PowerShell.2022-08-11.18-12-30.mp4

[willdean]

@thecrypticace

The blank lines represent each separate ctrl-s in VS.
This is pretty much what I see running a similar trial with logging in TW: the raw change events are sometimes translated into actual change events, but frequently not.

I spent a lot of time in the debugger last night trying to work out where and why the conversion of raw into "cooked" events is supressed, but it got very late and I gave up. I might have another go at this.

Event [raw] { event: 'rename' }
Event [raw] { event: 'rename' }
Event [raw] { event: 'change' }
Event [change] { file: 'src\\index.html' }

Event [raw] { event: 'rename' }
Event [raw] { event: 'rename' }
Event [raw] { event: 'change' }
Event [unlink] { file: 'src\\index.html' }

Event [raw] { event: 'rename' }
Event [raw] { event: 'rename' }
Event [raw] { event: 'change' }
Event [change] { file: 'src\\index.html' }

Event [raw] { event: 'rename' }
Event [raw] { event: 'rename' }
Event [raw] { event: 'change' }

Event [raw] { event: 'rename' }
Event [raw] { event: 'rename' }
Event [raw] { event: 'change' }
Event [change] { file: 'src\\index.html' }

Event [raw] { event: 'rename' }
Event [raw] { event: 'rename' }
Event [raw] { event: 'change' }

Event [raw] { event: 'rename' }
Event [raw] { event: 'rename' }
Event [raw] { event: 'change' }

Event [raw] { event: 'rename' }
Event [raw] { event: 'rename' }
Event [raw] { event: 'change' }
Event [change] { file: 'src\\index.html' }

Really appreciate your time here — I know this is a lot :)

Not at all - I've maintained OSS too, so I know how much one needs the people with the unusual repro scenarios to stick around for the debugging!

I don't see this problem nearly so much on a much larger project I also use TW on, but after a while you get all the selectors you're likely to use already compiled-in, so flaky rebuilds are less noticeable - my guess would be that this issue might be quite prevalent, at least on Windows.