[BUG] Failed to evaluate path "C:\\<Mounted Drive>\\<Directory on Drive>"

[valaphee]

Description

When trying to run docker compose up --build it fails with Failed to evaluate path. It might have to do with that "Projects" is a NTFS mount point where my drive is mounted, and doesn't has a drive letter assigned.

docker build works just fine, docker compose build not.

Console output:

PS C:\Projects\CadaShop> docker compose up --build     
[+] Building 0.0s (1/1) FINISHED
 => [internal] load local bake definitions                                                                                                                                                          0.0s
 => => reading from stdin 340B                                                                                                                                                                      0.0s
failed to evaluate path "C:\\Projects\\CadaShop": The system cannot find the path specified.

Shortened docker-compose.yml

services:
  myservice:
    build: .

Steps To Reproduce

1. Clean install of Windows
2. Create a new partition, and instead of assigning a drive letters, use a NTFS mount point
3. Create a docker-compose.yml in your directory on the drive with build: .
4. (Set env COMPOSE_BAKE=true)
5. docker compose up --build

Compose Version

Docker Compose version v2.36.0-desktop.1

Docker Environment

Anything else?

No response

[ndeloof]

Can you please check same issue applies to docker buildx bake -f compose.yaml ?

[valaphee]

Can you please check same issue applies to docker buildx bake -f compose.yaml ?

[+] Building 0.0s (1/1) FINISHED                                                                                                                                                                                              docker:desktop-linux
 => [internal] load local bake definitions                                                                                                                                                                                                    0.0s 
 => => reading docker-compose.yml 545B / 545B                                                                                                                                                                                                 0.0s
ERROR: failed to evaluate path "C:\\Projects\\CadaShop": The system cannot find the path specified.

For completeness I forgot to mention that I already used COMPOSE_BAKE=true; when false the error is

Compose can now delegate builds to bake for better performance.
 To do so, set COMPOSE_BAKE=true.
[+] Building 0.0s (0/0)                                                                                                                                                                                                       docker:desktop-linux
resolve : The system cannot find the path specified.

[ndeloof]

ok, so this issue also applies to Bake, cc @crazy-max

[crazy-max]

this issue also applies to Bake

Yes sounds like it.

@ndeloof Can you transfer the issue to Buildx repo? Does not seem I have perms to do it here.

[crazy-max]

@valaphee Can you show the steps when you create the NTFS mount point? Would like to repro on a Windows machine.

[valaphee]

Basically create a directory on an existing drive, e.g. C:\\Test
and assign a drive to it through Win+R diskmgmt.msc, and either create a new simple volume (as shown in the picture), or just right click on an existing volume and click Change Drive Letter or Paths and set it to C:\\Test for example

[crazy-max]

Was able to repro on Windows VM with Buildx 0.24.0:

I have mounted a volume in the NTFS folder C:\Test and created an HCL definition with a Dockerfile inside this folder:

target "default" {}

FROM busybox
RUN echo hello

When printing the definition it reads the HCL file correctly:

> docker buildx bake --print
[+] Building 0.0s (1/1) FINISHED
 => [internal] load local bake definitions                                                                         0.0s
 => => reading docker-bake.hcl 19B / 19B                                                                           0.0s
{
  "group": {
    "default": {
      "targets": [
        "default"
      ]
    }
  },
  "target": {
    "default": {
      "context": ".",
      "dockerfile": "Dockerfile"
    }
  }
}

But when building it fails to read the Dockerfile with the same error:

> docker buildx bake
[+] Building 0.0s (1/1) FINISHED                                                                   docker:desktop-linux
 => [internal] load local bake definitions                                                                         0.0s
ERROR: failed to evaluate path "Dockerfile": The system cannot find the path specified.
 => => reading docker-bake.hcl 19B / 19B 

The code path for this error is related to fs entitlements when evaluating paths.

If I allow any path for fs entitlements then it fails to open the dockerfile when building:

> docker buildx bake --allow fs=*
[+] Building 0.2s (2/2) FINISHED                                                                   docker:desktop-linux
 => [internal] load local bake definitions                                                                         0.0s
 => => reading docker-bake.hcl 19B / 19B                                                                           0.0s
 => [internal] load build definition from Dockerfile                                                               0.0s
 => => transferring dockerfile: 2B                                                                                 0.0s
ERROR: failed to solve: failed to read dockerfile: open Dockerfile: no such file or directory

If I do a simple build without bake it fails as well:

> docker buildx build .
[+] Building 0.1s (1/1) FINISHED                                                                   docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                               0.0s
 => => transferring dockerfile: 2B                                                                                 0.0s
ERROR: failed to solve: failed to read dockerfile: open Dockerfile: no such file or directory

failed to read dockerfile happens in BuildKit during solve: https://github.com/moby/buildkit/blob/64918db2a77daada87171521970ff2228ffdc193/frontend/dockerui/config.go#L392

And ReadFile is the one returning this error: https://github.com/moby/buildkit/blob/64918db2a77daada87171521970ff2228ffdc193/cache/util/fsutil.go#L51-L84

So we need a fix in fs entitlements for bake and another fix in BuildKit.

I tried to use an UNC path but it fails as well:

C:\> docker buildx build \\?\C:\Test
[+] Building 0.1s (1/1) FINISHED                                                                   docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                               0.0s
 => => transferring dockerfile: 2B                                                                                 0.0s
ERROR: failed to solve: failed to read dockerfile: open Dockerfile: no such file or directory

I will set up an environment to debug this issue. I'm a bit confused that we are able to read the bake definition but it fails to open the file during entitlements check.

@profnandaa Maybe you have an idea?

[crazy-max]

Made some tests and for fs entitlements it fails here

buildx/bake/entitlements.go

Line 609 in d61853b

return "", "", syscall.ENOTDIR

Because file mode on the folder as mounted volume is not a dir but marked as irregular:

func TestFS(t *testing.T) {
	fi, err := os.Lstat(`C:\tmp`)
	require.NoError(t, err)
	t.Logf("File Info mode: %+v", fi.Mode())

	fi, err = os.Lstat(`C:\Test`)
	require.NoError(t, err)
	t.Logf("File Info mode: %+v", fi.Mode())
}

    foo_test.go:32: File Info mode: drwxrwxrwx
    foo_test.go:36: File Info mode: ?rw-rw-rw-

Looking at Go code base it goes through https://github.com/golang/go/blob/cfb4e9bc4ae957dba63cb2ee5e020fcd25d553fd/src/os/types_windows.go#L206 and returns https://github.com/golang/go/blob/cfb4e9bc4ae957dba63cb2ee5e020fcd25d553fd/src/os/types_windows.go#L226-L227

[profnandaa]

@crazy-max -- no idea really, but following...

[crazy-max]

Ah I think I found out why:

• https://github.com/golang/go/blob/cfb4e9bc4ae957dba63cb2ee5e020fcd25d553fd/src/os/stat.go#L21-L23
• https://github.com/golang/go/blob/cfb4e9bc4ae957dba63cb2ee5e020fcd25d553fd/src/os/types_windows.go#L149-L157

To mitigate this issue I guess we could read the reparse data and follow the reparse point using os.Stat?

[vvoland]

One thing that comes to my mind is that with Go 1.23 there were some changes to how some reparse points are treated:

On Windows, the mode bits reported by Lstat and Stat for reparse points changed. Mount points no longer have ModeSymlink set, and reparse points that are not symlinks, Unix sockets, or dedup files now always have ModeIrregular set. This behavior is controlled by the winsymlink setting. For Go 1.23, it defaults to winsymlink=1. Previous versions default to winsymlink=0.
(from https://go.dev/doc/go1.23#ospkgos)

and

On Windows, EvalSymlinks no longer evaluates mount points, which was a source of many inconsistencies and bugs. This behavior is controlled by the winsymlink setting. For Go 1.23, it defaults to winsymlink=1. Previous versions default to winsymlink=0.
(from https://go.dev/doc/go1.23#pathfilepathpkgpathfilepath)

You might try building with winsymlink=0 and see if that solves the issue.