Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File system change events (inotify) support for mounted directories #615

Closed
charleskorn opened this issue Jan 30, 2022 · 19 comments · Fixed by #1913
Closed

File system change events (inotify) support for mounted directories #615

charleskorn opened this issue Jan 30, 2022 · 19 comments · Fixed by #1913
Labels
enhancement New feature or request expert

Comments

@charleskorn
Copy link

Description

From abiosoft/colima#151:

We're using Colima to run Docker in a VM on our macOS laptops. Colima uses Lima under the hood to run this VM.

Are there any plans to support file system change events for mounted directories through inotify?

This is a key feature for us as we use containers in scenarios like auto-reloading dev servers and re-running unit tests on changes.

If there are no plans to support it at the moment, what kind of effort would be involved in adding this? Depending on the effort required, we might be open to putting together a PR to add support for this.

@jandubois
Copy link
Member

what kind of effort would be involved in adding this

As I already commented in the colima issue, I'm not sure this is a possibility at all (generating inotify events). Even if you capture them on the host (fsevents), I think you would need to implement a custom notification mechanism in the guest and can't expect to use inotify, which is a kernel mechanism.

@AkihiroSuda
Copy link
Member

In far future we may support inotify with virtiofs (not virtio-9p-pci).
https://lwn.net/Articles/874000/

@afbjorklund
Copy link
Contributor

afbjorklund commented Jan 31, 2022

There were some earlier series, about the various limits of inotify.
https://lwn.net/Articles/605128/

Inotify reports only events that a user-space program triggers through the filesystem API. This constitutes a fairly serious limitation of the API. For example, it means that inotify does not inform us of events on monitored objects via a remote filesystem (e.g. NFS) operation.

Previously we "solved" it by keeping the files locally on the VM.

@atipugin
Copy link

atipugin commented Jan 11, 2023

Looks like virtiofs is already supported (although marked as experimental), but I still can't get FS events working. Tools like entr (which relies on inotify and kqueue) and webpack (in watch mode) don't see file changes. Interesting that FS notifications work fine with Docker Desktop (with same settings - VZ + virtiofs).

@lobshunter
Copy link
Contributor

Looks like Apple's virtio-fs document doesn't mention anything about inotify. Inotify in docker desktop container does work(although it would drop DELETE event, and inject additional WRITE event). docker.log shows messages like level=info msg="injected MODIFY event (ts: 1677314215061760000) on /host_mnt/private/xxx, seems like it's doing some kind of inotify event injection 🤔.

@abiosoft
Copy link

abiosoft commented Feb 25, 2023

docker.log shows messages like level=info msg="injected MODIFY event (ts: 1677314215061760000) on /host_mnt/private/xxx

That shows that Docker is manually handling the notify events i.e. the feature is missing in virtiofs.

@lobshunter
Copy link
Contributor

i.e. the feature is missing in virtiofs.

Looks like so. I just tried qemu-virtiofsd on a Linux host, host to guest inotify doesn't work either(so it's not Apple's issue). There was some work trying to add inotify support for FUSE, but seems like it wasn't merged in mainline.

@lobshunter
Copy link
Contributor

I bet docker desktop watches files in host, and forwards events into guest(perhaps by invoking something like chmod/write).
Start a docker destop container with a volume, then execute lskq(1) in host, it would show docker is watching files.

It might not be a great solution to do the same in lima.

@henrik242
Copy link

I'm trying to get node/next.js development working with colima, and next.js doesn't pick up changes I do on the host. However, if I touch the changed file from within the container, next.js picks up the change and my web page refreshes.

So, as a (silly) workaround, maybe one can use a script or daemon in the container to continuously scan the files and touch any changed files?

@lobshunter
Copy link
Contributor

However, if I touch the changed file from within the container, next.js picks up the change and my web page refreshes.

Yes, guest to guest inotify works as expected, but host to guest inotify doesn't(most likely because FUSE protocol doesn't support it).

maybe one can use a script or daemon in the container to continuously scan the files and touch any changed files?

As a personal workaround, that would work(just less performant, performance degrades linearly with the number of files being added).

@henrik242
Copy link

henrik242 commented Feb 28, 2023

As a personal workaround, that would work(just less performant, performance degrades linearly with the number of files being added).

Yeah, it's not ideal at all. This is the easiest I could come up with, in case others need it:

FOLDER=src
CHKPT=/tmp/checkpoint
touch $CHKPT
while true; do
  find "$FOLDER" -newer $CHKPT -type f -print -a -exec touch {} \; | grep -q . && touch $CHKPT
  sleep 1
done

The file system seems to only update about every 10 seconds, so changes aren't immediate. I'm using grep to give find a non-zero exit code, only touching /tmp/checkpoint after a change has been seen.

@henrik242
Copy link

henrik242 commented Feb 28, 2023

Some more testing reveals that running the script on the host is a lot more responsive:

FOLDER=$1
TOUCH="docker exec -ti my-docker-guest touch"
CHKPT=/tmp/checkpoint
touch $CHKPT
while true; do
  find "$FOLDER" -newer $CHKPT -type f -print -a -exec $TOUCH {} \; | grep . && touch $CHKPT
  sleep 1
done

hgwood added a commit to Zenika/sensei that referenced this issue May 22, 2023
This can help get hot reloading work when running sensei with a VM-backed Docker from a host file system folder. For example, when running sensei with Docker for Windows with the WSL2 back-end from a Windows file system folder (see #117) or when running sensei with Colima from a macOS file system folder (see lima-vm/lima#615). The recommendation still is to work off the VM file system to get native file watching.
@johan-seesaw
Copy link

I figure'd I'd reference another issue from the colima repo for all those who are using lima via colima. The new options in colima are working for my usecase.
abiosoft/colima#261

colima stop
brew unlink colima
brew install --HEAD colima
colima start --mount-inotify

@henrik242
Copy link

You don't need to use the HEAD version. The latest regular version already supports --mount-inotify

@linnik
Copy link

linnik commented Aug 12, 2023

You don't need to use the HEAD version. The latest regular version already supports --mount-inotify

option is here, but it does not work

@AkihiroSuda
Copy link
Member

@aeifn
Copy link

aeifn commented Oct 22, 2023

the latest regular version already supports --mount-inotify

Is it possible to use something like that with lima only?

@balajiv113
Copy link
Member

@aeifn There is a draft PR #1913

@aeifn
Copy link

aeifn commented Oct 22, 2023

@aeifn There is a draft PR #1913

Thank you! It works for me (vz, virtiofs).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request expert
Projects
None yet
Development

Successfully merging a pull request may close this issue.