Not looking for files in subdirectories dynamically

[BullHacks3]

Issue

Traefik file handlers not looking for the files in the subdirectories
https://docs.traefik.io/configuration/backends/file/#multiple-separated-files
In documentation ,it is clearly mentioned ,that file handlers have capability to look into the subdirectories for the files dynamically

How to reproduce the issue

Step 1:- Contents of the main.toml file

defaultEntryPoints = ["http"]
[entryPoints]
  [entryPoints.http]
  address = ":80"
  [api]
 entrypoint="dashboard"
  dashboard= true
  debug=true

[file]
  directory= "/home/traefik/hostRules/"
  watch=true

Step 2:- Now I am dynamically creating the subdirectories inside the mounted directory

For example the dummy configuration of one of the traefik file is:-

[file]
[frontends]
    [frontends.front_end_abc]
         backend="backend3"
        passHostHeader = true
    [frontends.front_end_abc.routes.first]
         rule="Host: testingpurpose.com"
   [backends]
    [backends.backend3]
         [backends.backend3.servers.first]
           url = "http://192.168.1.2:39832"

But the configurations are not loaded dynamically until ,I restart the docker image
Command used to up the docker container

docker run -d -v /home/traefik:/home/traefik -p 80:80  -p 443:443 -p 172.31.30.68:8080:8080 testing-traefik /usr/bin/traefik -c /home/traefik/main.toml

What did you expect to see?

I expected that the configurations inside the subdirectories should be loaded dynamically without restarting the docker image

What did you see instead?

I have to restart the docker image ,in order to load the configurations inside the subdirectories that are creating dynamically.

Output of traefik version: (What version of Traefik are you using?)

Used the latest traefik image ,for building own docker image on the top of centos

[nmengin]

Hello @BullHacks3 ,

Many thanks for interest in the project.

I guess the problem comes from the watcher we use which is not recursive.
Indeed, Træfik can load file from subbdirectories but can not listen to events from these subdirectories.

As a workaround, by waiting for the fix, you can touch a file in the root directory when a modification is done in a subdirectory to load all the configuration (or make a modification in an existing file).
It is not perfect bu more dynamic than restarting Træfik.

[GuyFromMars365]

I'm guessing this isn't going to get fixed, since last update was 1.5 years ago? :(

[jspdown]

The issue is due to fsnotify/fsnotify not capable of watching file recursively. To provide a cross platform solution, fsnotify relies on adapters (inotify, ReadDirectoryChangesW, kqueue) but recursive file watching is not supported everywhere. There is some work in progress (fsnotify/fsnotify#18, fsnotify/fsnotify#114, fsnotify/fsnotify#12) to bring support but we are not yet there.

An alternative would be to implement a user-space solution based on filepath.Walk. It wouldn't be great in term of performance an addressing edge cases (as there are several) might lead to a complex implementation.

[rizlas]

Same in Traefik v2.5

[elgarfo]

same for traefik v2.9.1

based on @nmengins suggestion we're using an alpine container with inotify-tools to watch for file changes recursively and touch an empty file in traefiks dynamic conf root:

docker run  --rm -it \
  -v $PWD/traefik.d/:/watchme:ro \
  -v $PWD/traefik.d/.please_reread_files_traefik:/touchme \
  alpine \
  sh -c 'apk update && apk add inotify-tools && while true; do inotifywait -e modify,create,delete,attrib -r /watchme && touch /touchme; done'

[nmengin]

Hello,

I link this issue to the PR #9993.
Even if sending a signal doesn't solve the issue, IMO it's a good enough workaround.

If any contributor finds a better solution, we would love community support to address it.
Let us know, and we will work with you to make sure you have all the information needed so that it can be merged.

[centja1]

Not sure if responding on a 6 month old thread is the right place, but is there a reason why there is code in the File provider to specifically exclude adding a watcher for sub directories?

I understand that you cannot recursively monitor a sub-directory that is created after traefik has started, but adding a file to a sub-directory that existed at startup also does not work, and it would appear that this is intentional.

I forked the repo, commented out those lines and built an image and can confirm that creating a new file within a sub-directory that existed at start does automatically load the config from that file.

Curious as to why lines 69-72 exist