[BUG] x-develop watch not working

[MauriceArikoglu]

Description

The x-develop / watch configuration in my compose file does not work at all when run with docker compose up --wait -d && docker compose alpha watch.

I get this output, after that - infinite silence

watch command is EXPERIMENTAL
watching .
watching .
watching ./infra/nginx/

Compose.yml

services:
  proxy:
    container_name: 'nginx-reverseproxy'
    hostname: proxy
    build:
      context: './infra/nginx/'
      dockerfile: 'Dockerfile.dev'
    x-develop:
      watch:
        - action: rebuild
          path: './infra/nginx/'
    restart: unless-stopped
    ports:
      - '80:80'
      - '443:443'
    volumes:
      - './infra/nginx/data:/data'
    healthcheck:
      test: "service nginx status || exit 0"
      interval: 1s
      timeout: 5s
      retries: 10

Inside my infra/nginx directory I have my nginx.conf and the following Dockerfile (Dockerfile.dev)

FROM nginx:1.25.0-alpine

COPY nginx.conf /etc/nginx/nginx.conf

RUN mkdir -p /var/lib/nginx/cache && \
    chown nginx /var/lib/nginx/cache && \
    chmod 700 /var/lib/nginx/cache

Steps To Reproduce

Copy compose.yml from above
setup ./infra/nginx with any nginx.conf and the Dockerfile
Run docker compose up --wait -d && docker compose alpha watch

Compose Version

Docker Compose version v2.18.1

Docker Environment

Client:
 Version:    24.0.2
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.5
    Path:     /Users/me/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.18.1
    Path:     /Users/me/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/me/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.19
    Path:     /Users/me/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.4
    Path:     /Users/me/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/me/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/me/.docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  v0.12.0
    Path:     /Users/me/.docker/cli-plugins/docker-scout

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 7
 Server Version: 24.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.49-linuxkit-pr
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 7.667GiB
 Name: docker-desktop
 ID: 69622abd-668e-42e2-acfc-33a649da49d9
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

[MauriceArikoglu]

If I change the path to '.' or './' or '/' from './infra/nginx/' it still fails.

x-develop:
    watch:
      - action: rebuild
        path: '.'

[ndeloof]

path is indeed relative to build context, so need to be set to .

Which file do you edit, and don't see watch trigger a rebuild ?

I wonder there could be some side effect as your volume source ./infra/nginx/data is included in the watch path, so that any file modified there would trigger a filesystem event on client and trigger a rebuild.

[MauriceArikoglu]

@ndeloof in this case I could edit my nginx.conf or dockerfile which should both trigger a rebuild but don't.

The complete compose file has serviced for backend and frontend as well. In this case the services have build context . with the compose and dockerfile in the root. The respective code is in subdirectories. Whatever watchpath I put for these services, they also neither get rebuild nor synced.

[ndeloof]

The weird part here is you get watching . in your logs. This should show the absolute path, which demonstrate something goes wrong setting up the filesystem events listener.

Could you confirm this by (temporary) setting path to an absolute file path?

[MauriceArikoglu]

The weird part here is you get watching . in your logs. This should show the absolute path, which demonstrate something goes wrong setting up the filesystem events listener.

Could you confirm this by (temporary) setting path to an absolute file path?

@ndeloof
I did that also previously. On Docker Compose 2.17.3 with the same configuration I got the output watching absolutePath - now on 2.18.1 I get watching . no matter the path I put there (absolute, relative, dir, file ....)

[ndeloof]

Can you please attach output for docker compose config command? Seems build context is not converted into an absolute path for some reasons

[MauriceArikoglu]

@ndeloof - content redacted for privacy reasons. Paths kept intact. As far as I can assess it looks correct. The paths all lead to the correct directories intended for watching. Sadly doesnt work

name: x
services:
  backend:
    build:
      context: /Users/me/code/org/x
      dockerfile: ./infra/backend/Dockerfile.backend.dev
    command:
    - lerna
    - run
    - --scope
    - '@x/backend'
    - --stream
    - dev
    container_name: x-backend
    depends_on:
      postgres:
        condition: service_healthy
    environment:
      APP_ENV: development
    expose:
    - "3333"
    hostname: backend
    healthcheck:
      test:
      - CMD
      - curl
      - -f
      - healthcheck
      timeout: 10s
      interval: 30s
      retries: 3
      start_period: 30s
    networks:
      nbackend: null
      npostgres: null
    ports:
    - mode: ingress
      target: 3333
      published: "9999"
      protocol: tcp
    x-develop:
      watch:
      - action: rebuild
        path: ./packages/x-backend/package.json
      - action: rebuild
        path: ./infra/backend/Dockerfile.backend.dev
      - action: rebuild
        path: ./packages/x-backend/prisma/
      - action: sync
        path: ./packages/x-backend/src/
        target: ./app/packages/x-backend/src/
  frontend:
    build:
      context: /Users/me/code/org/x
      dockerfile: ./infra/frontend/Dockerfile.frontend.dev
    command:
    - lerna
    - run
    - --scope
    - '@x/frontend'
    - --stream
    - dev
    container_name: x-frontend
    depends_on:
      backend:
        condition: service_started
    expose:
    - "3000"
    hostname: frontend
    healthcheck:
      test:
      - CMD
      - curl
      - -f
      - healthcheck
      timeout: 10s
      interval: 30s
      retries: 3
      start_period: 30s
    networks:
      nfrontend: null
    ports:
    - mode: ingress
      target: 3000
      published: "9998"
      protocol: tcp
    x-develop:
      watch:
      - action: rebuild
        path: ./packages/x-frontend/package.json
      - action: rebuild
        path: ./infra/frontend/Dockerfile.frontend.dev
      - action: sync
        path: ./packages/x-frontend/src/
        target: ./app/packages/x-frontend/src/
  postgres:
    container_name: postgres-db
    environment:
      x
    expose:
    - "5432"
    hostname: postgres
    healthcheck:
      test:
      - CMD
      - pg_isready
      - -U
      - x
      - -d
      - x
      timeout: 5s
      interval: 1s
      retries: 10
    image: postgres:15.3-alpine3.18
    networks:
      npostgres: null
    restart: unless-stopped
    user: postgres
    volumes:
    - type: bind
      source: /Users/me/code/org/x/infra/postgresql/data
      target: /var/lib/postgresql/data
      bind:
        create_host_path: true
  proxy:
    build:
      context: /Users/me/code/org/x/infra/nginx
      dockerfile: Dockerfile.dev
    container_name: nginx-reverseproxy
    depends_on:
      backend:
        condition: service_healthy
      frontend:
        condition: service_healthy
    hostname: proxy
    healthcheck:
      test:
      - CMD-SHELL
      - service nginx status || exit 0
      timeout: 5s
      interval: 1s
      retries: 10
    networks:
      nbackend: null
      nfrontend: null
    ports:
    - mode: ingress
      target: 80
      published: "80"
      protocol: tcp
    - mode: ingress
      target: 443
      published: "443"
      protocol: tcp
    restart: unless-stopped
    volumes:
    - type: bind
      source: /Users/me/code/org/x/infra/nginx/data
      target: /data
      bind:
        create_host_path: true
    x-develop:
      watch:
      - action: rebuild
        path: ./infra/nginx/nginx.conf
      - action: rebuild
        path: ./infra/nginx/Dockerfile.dev
networks:
  nbackend:
    name: x_nbackend
  nfrontend:
    name: x_nfrontend
  npostgres:
    name: x_npostgres

[MauriceArikoglu]

@ndeloof any news? I couldn't get it working still.

[ndeloof]

I have no idea what's going on here, I've asked teammates to have a look

[simonmallet]

I have the same problem.
Not long ago everything was working just fine.
I upgraded Docker desktop (and probably docker compose at the same time)
and now whatever I use for watch sync
it writes:

watch command is EXPERIMENTAL
watching ../typescript/app-ui-v4
watching ../typescript/current
watching ../typescript/current
watching 
watching ../typescript/current
watching ../
watching ../dotnet
watching ../typescript/app-ui-v3
watching .
watching ../typescript/current
watching ../dotnet

I tried setting some x-develop: (so it no longer watches anything), and the output is always the same!
I tried to change the directories to full-paths like suggested in comments above, and in the end it always shows the same watching output...

It looks like something is stuck in the background

[glours]

Hey there 👋
Did you try with the latest v2.19.1 ? We introduced several fixes such as #10599 and #10710 which could fix your issue.
I tried on my side and wasn't able to reproduce it

[MauriceArikoglu]

@glours @ndeloof it seems to work with 2.19.1 - thanks for caring :)

[glours]

Great news 👍 , I close the issue

[MauriceArikoglu]

@glours @ndeloof this seems to have regressed in v2.20.2-desktop.1 - I again have the same issues

FYI Apparently switching to a different git branch fixed the issue o_O ?! It seems there is some caching going on that can get "stuck" and persists even across restarts of docker engine & the host machine & subsequent rebuilds.