Skip to content

add a option --watch-buffer-size to dynamic change fs event buffer size #58506

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
coderall2 opened this issue May 30, 2025 · 3 comments
Closed
Labels
cli Issues and PRs related to the Node.js command line interface. feature request Issues that request new features to be added to Node.js. watch-mode Issues and PRs related to watch mode

Comments

@coderall2
Copy link

What is the problem this feature will solve?

we did a test using fs.watch with a backgroud thread which write or modify files in watched-dir. The write or modify QPS is closed to 10K/s. We observed some fs event loss.We refered to fsnotify with Go, modify the source code of node to adjust the buffer size to 65536, the event loss is gone.

What is the feature you are proposing to solve the problem?

Windows ReadDirectoryChangeW and Linux inotify both need a buffer to read fs event. Now, the buffer size is const (4096 bytes) and can't be larger to process fs burst request well. So, could we add an option (the name may be --watch-buffer-size ) to tell node the suitable buffer size?

Image Image

What alternatives have you considered?

Get the node's source code, modify the buffer size manually and rebuild a new binary, but it's better to add a command-line option to solve this problem.

@coderall2 coderall2 added the feature request Issues that request new features to be added to Node.js. label May 30, 2025
@github-project-automation github-project-automation bot moved this to Awaiting Triage in Node.js feature requests May 30, 2025
@juanarbol
Copy link
Member

Hey, I loved how detailed the reason behind the feature was. Let me try implement this one.

@juanarbol juanarbol added cli Issues and PRs related to the Node.js command line interface. watch-mode Issues and PRs related to watch mode labels May 30, 2025
@bnoordhuis
Copy link
Member

That's implemented in libuv - separate project. The buffer can maybe be made bigger (but not too big lest it overflow the stack) but I don't plan on adding a tunable, whether that's a flag, config option, or something else.

Also, growing the buffer only helps so much. If enough file events are generated in a short enough time, it's inevitable some get lost; that's just intrinsic to asynchronous events without unbounded queues.

@juanarbol
Copy link
Member

Ok, then I'll close this as not planned.

@juanarbol juanarbol closed this as not planned Won't fix, can't repro, duplicate, stale Jun 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cli Issues and PRs related to the Node.js command line interface. feature request Issues that request new features to be added to Node.js. watch-mode Issues and PRs related to watch mode
Projects
Archived in project
Development

No branches or pull requests

3 participants