Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NonrecursiveTree does not watch recreated folders #200

Closed
FabianKramm opened this issue Jan 27, 2021 · 0 comments · Fixed by syncthing/notify#3 or syncthing/syncthing#7444
Closed

Comments

@FabianKramm
Copy link
Contributor

FabianKramm commented Jan 27, 2021

Hey there! I think I found a rather severe issue on operating systems that use the NonrecurseTree watcher like linux for recursive watches. The problem is that after you create and delete a folder several times, no subsequent events for that folder will be logged anymore.

I think that the underlying problem is that the non-recursive tree implementation only adds directories to watch and never deletes them from the internal tree structure (except if you call Stop() manually from the outside). This can be seen in the func (t *nonrecursiveTree) dispatch(c <-chan EventInfo) function:

// If the event describes newly leaf directory created within
if !isrec || ei.Event() != Create {
return
}
if ok, err := ei.(isDirer).isDir(); !ok || err != nil {
return
}
t.rec <- ei
}(ei)

Here only 'Create' events are forwarded to the internal handler that adds new watchpoints. However, when a directory is actually removed and recreated, the underlying inotify watch does not work anymore and the tree implementation does not re-add it because it thinks the folder is still being watched, because there is a node for it already in the internal in-memory tree. This means that all subsequent events are lost for this folder and it is essentially unwatched until you restart the complete watcher.

I guess this could be also the reason for some other issues like #190 or syncthing/syncthing#7198

The problem can be reproduced on linux with this snippet:

package main

import (
	"context"
	"fmt"
	"github.com/rjeczalik/notify"
	"io/ioutil"
	"log"
	"os"
	"path/filepath"
	"time"
)

func main() {
        // Cleanup test folder if it is still there
	_ = os.RemoveAll("test")
	err := os.Mkdir("test", 0777)
	if err != nil {
		log.Fatal(err)
	}
	time.Sleep(time.Second)

        // Start watching
	ctx, _ := context.WithCancel(context.Background())
	eventChan := make(chan notify.EventInfo, 1000)
	err = notify.Watch("./test/...", eventChan, notify.All)
	if err != nil {
		log.Fatal(err)
	}
	
	// Print events
	go func(){
		for {
			select{
			case e := <-eventChan:
				fmt.Println(e.Path() + " - " + e.Event().String())
			}
		}
	}()
	
	// 1. Create a folder and a file within it
        // This will create a node in the internal tree for the subfolder test/folder
        // Will create a new inotify watch for the folder
	fmt.Println("######## First ########")
	err = recreateFolder("test/folder")
	if err != nil {
		log.Fatal(err)
	}
	
	// 2. Create a folder and a file within it again 
        // This will set the events for the subfolder test/folder in the internal tree
        // Will create a new inotify watch for the folder because events differ
	time.Sleep(time.Second)
	fmt.Println("######## Second ########")
	err = recreateFolder("test/folder")
	if err != nil {
		log.Fatal(err)
	}
	
	// 3. Create a folder and a file within it yet again
        // This time no new inotify watch will be created, because the events
        // and node already exist in the internal tree and all subsequent events 
        // are lost, hence there is no event for the created file here anymore
	time.Sleep(time.Second)
	fmt.Println("######## Third ########")
	err = recreateFolder("test/folder")
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println("Done")
	<-ctx.Done()
}

func recreateFolder(dir string) error {
	// Give the sync some time to process events
	_ = os.RemoveAll(dir)
	err := os.Mkdir(dir, 0777)
	if err != nil {
		return err
	}

	time.Sleep(time.Second)

	// Create a file
	fmt.Println("######## Create File ########")
	err = createFile(filepath.Join(dir, "file"))
	if err != nil {
		log.Fatal(err)
	}
	return nil
}

func createFile(file string) error {
	// Now create a file to check if we see that
	err := ioutil.WriteFile(file, []byte("abc"), 0666)
	if err != nil {
		return err
	}

	time.Sleep(time.Second)
	return nil
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant