[BUG] include from git repo works with config but not pull, up

[thespad]

Description

As per #10811 we can now include files from a remote git repo, however, while this works with docker compose config, attempting to perform a pull or up operation results in a file not found error.

Steps To Reproduce

Consider a compose.yml containing

include:
  - git@github.com:thespad/somerepo.git#master:myproject/compose.yml

Running docker compose config fetches and parses the remote compose file as expected.
Running docker compose pull or docker compose up -d produces:

open /home/ubuntu/compose/myproject/git@github.com:thespad/somerepo.git#master:myproject/compose.yml: no such file or directory

Compose Version

Docker Compose version v2.21.0

Docker Environment

Client: Docker Engine - Community
 Version:    24.0.6
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /home/ubuntu/.docker/cli-plugins/docker-compose

Server:
 Containers: 8
  Running: 8
  Paused: 0
  Stopped: 0
 Images: 8
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
 runc version: v1.1.8-0-g82f18fe
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-1032-raspi
 Operating System: Ubuntu 22.04.3 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 3.7GiB
 Name: pi1
 ID: 9739e21c-dc5f-4ca5-9136-f94b401a5977
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: true
 Default Address Pools:
   Base: 172.20.0.0/16, Size: 24

Anything else?

No response

[ndeloof]

I can't reproduce. Do you have COMPOSE_EXPERIMENTAL_GIT_REMOTE=true variable set ? Otherwise git remote resource support is disabled

[thespad]

I hadn't (although config was working without it), but even with setting it I'm getting the same behaviour.

If it matters this is on an Pi 4 running Ubuntu 22.04.3, but I've also been able to replicate on an x86_64 host running the same OS/docker/docker compose versions and the same compose yml.

If it's relevant, the repo in this case is private, but git is authed to it, as evidenced by config being able to pull it.

[ndeloof]

Right, in v2.21.0 config was using a distinct configuration path and had git remote resource loader always enabled, while COMPOSE_EXPERIMENTAL_GIT_REMOTE should be required.

but even with setting it I'm getting the same behavior

weird. Can you please past output for running COMPOSE_EXPERIMENTAL_GIT_REMOTE=true docker compose pull?

[thespad]

ubuntu@pi1:~/compose/infra$ COMPOSE_EXPERIMENTAL_GIT_REMOTE=true docker compose pull
open /home/ubuntu/compose/infra/git@github.com:thespad/myrepo.git#master:infra/compose.yml: no such file or directory

[thespad]

Just an FYI, I've upgraded to Docker Compose version v2.22.0 and it's now working on all the hosts I've tried. I didn't see anything in the release notes that looked obviously related, and nothing else has directly changed on the hosts themselves apart from OS updates, so I'm not sure what might have fixed it.

It does also seem that with v2.22 COMPOSE_EXPERIMENTAL_GIT_REMOTE is required even for config so I assume something has been changed in the code around it.

[ndeloof]

This has been fixed as part of #10949