-
Notifications
You must be signed in to change notification settings - Fork 5.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deno.copyFile triggers the file watcher infinitely #19425
Comments
I can't reproduce, please provide a minimal example with reproduction steps tested with
const watcher = Deno.watchFs(".");
for await (const event of watcher) {
console.log(event);
event.paths.forEach((path) => {
Deno.copyFileSync(path, "/dev/null"); // without this line I get the same events
});
}
result
|
I have reproduced it in this zip file:
|
My guess is this is due to the large file size, and maybe the os copying it in chunks and each chunk triggers a modify event (its not infinite loop, its just a lot of events) I imagine you get the same things without deno, by using inotify directly in linux |
I tried inotify directly (with C) and its the same number of events, so it seems this is just how it works |
Just clarify my OS is no Linux but macOS, and in my case it never stops (or at least it takes many seconds). |
you can try with this file (courtesy of chatgpt)
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/inotify.h>
#define EVENT_SIZE (sizeof(struct inotify_event))
#define BUF_LEN (1024 * (EVENT_SIZE + 16))
int main() {
int fd, wd, length;
char buffer[BUF_LEN];
fd = inotify_init();
if (fd < 0) {
perror("inotify_init");
exit(EXIT_FAILURE);
}
wd = inotify_add_watch(fd, "./src", IN_ALL_EVENTS);
if (wd < 0) {
perror("inotify_add_watch");
exit(EXIT_FAILURE);
}
printf("Watching current directory for events...\n");
while (1) {
length = read(fd, buffer, BUF_LEN);
if (length < 0) {
perror("read");
exit(EXIT_FAILURE);
}
int i = 0;
while (i < length) {
struct inotify_event *event = (struct inotify_event *) &buffer[i];
if (event->len) {
if (event->mask & IN_CREATE)
printf("File/directory created: %s\n", event->name);
if (event->mask & IN_DELETE)
printf("File/directory deleted: %s\n", event->name);
if (event->mask & IN_MODIFY)
printf("File modified: %s\n", event->name);
if (event->mask & IN_ATTRIB)
printf("File attributes modified: %s\n", event->name);
if (event->mask & IN_MOVE)
printf("File/directory moved: %s\n", event->name);
}
i += EVENT_SIZE + event->len;
}
}
inotify_rm_watch(fd, wd);
close(fd);
return 0;
} |
oh actually mac doesn't use inotify probably |
I can't test this but here is what chatgpt says for macos (you can alwyas fix the code if needed) #include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/event.h>
int main() {
int kq, dirfd, num_events, i;
struct kevent event;
struct timespec timeout = {0, 0};
// Open the current directory
dirfd = open("./src", O_RDONLY);
if (dirfd == -1) {
perror("open");
exit(EXIT_FAILURE);
}
// Create a new kernel event queue
kq = kqueue();
if (kq == -1) {
perror("kqueue");
exit(EXIT_FAILURE);
}
// Register a new event in the event queue
EV_SET(&event, dirfd, EVFILT_VNODE, EV_ADD | EV_ENABLE | EV_CLEAR, NOTE_WRITE | NOTE_DELETE | NOTE_RENAME, 0, NULL);
// Attach the event to the event queue
if (kevent(kq, &event, 1, NULL, 0, &timeout) == -1) {
perror("kevent");
exit(EXIT_FAILURE);
}
printf("Watching current directory for events...\n");
while (1) {
// Wait for events to occur
num_events = kevent(kq, NULL, 0, &event, 1, &timeout);
if (num_events == -1) {
perror("kevent");
exit(EXIT_FAILURE);
} else if (num_events > 0) {
// Process the event
if (event.fflags & NOTE_WRITE)
printf("File modified: %s\n", event.udata);
if (event.fflags & NOTE_DELETE)
printf("File/directory deleted: %s\n", event.udata);
if (event.fflags & NOTE_RENAME)
printf("File/directory renamed: %s\n", event.udata);
}
}
close(kq);
close(dirfd);
return 0;
} |
Thanks, but I'm not familiarized with C code. I don't know if the bug is in the OS or Deno, but I think Deno should fix these issues and try to work the same in all operating systems. The way it works now is: on modify a file, copy it to other folder, which modify the file again, so copy the file again... According to the documentation, the available event types are "any", "access", "create", "modify", "remove" and "other". Copying a file shouldn't modify the original file, but only access to it. This would allow to filter out the "access" events in the watcher and break the infinite loop. |
But it not modifying the original file, it's modifying the file being created (by writing to it in chunks) Just to make sure , is what you get an infinite loop or a lot of events (as in does the logging stops after a while or doesn't?) |
In this video, you can see what happens in my computer.
screen.recording.mov |
Seems to be a bug in the mac implementation @dsherret maybe you can give it a look (or at least I think issue should be labeled) to reproduce
In another terminal
Now the terminal show an infinite loop with modify events, doesn't happen on linux, but happens on macos |
seems like Deno.watch is just a simple wrapper over notify crate Line 96 in e1be2bb
which had deadlocks with macos before notify-rs/notify#118 @oscarotero maybe you can give a shot to upgrading to the latest stable version https://github.com/denoland/deno/blob/main/Cargo.toml#L109 changing it to 6 |
@sigmaSd Thanks for your time researching on this issue. |
I have the following code:
Every time a file is modified, the watcher detects the change, so the file is copied to the destination folder. But seems like
Deno.copyFileSync
somehow triggers a new "modify" event to the original file in the watcher, so it's an infinite loop.As a workaround, I can read and write the file content
I think this is a bug, because
copyFile
shouldn't modify the original file.My Deno version:
The text was updated successfully, but these errors were encountered: