podman-compose(1) not respecting the "networks:" directive ...

[nmvega]

Hello Friends:

First, thank you for this project! =:)

Am I doing something wrong? podman-compose(1) (v0.1.7dev) doesn't seem to respect the networks: directive in docker-compose.yml(5). I tried the below a few ways; first creating the network externally (manually) and specifying that to the services stanzas; as well as allowing it to be created dynamically.

But examining the interactive output of podman-compose -f ./docker-compose.yml up doesn't show --network mynetwork in the podman pod create [ ... ] command.

[ ... snip ... ]
podman pod create --name=acme --share net -p 6379:6379 -p 5432:5432
[ ... snip ... ]

Am I missing something, or is this not implemented? Not being implemented would be a bummer because I have some several very complex docker-compose.yml(5) files that I'd have to convert to manual commands.

Alternatively, is it possible to attach a running pod to networks (not to ports but networks) after the fact? That would help as a workaround (though I doubt that's possible). It would really be cool if podman-compose(1) natively supported this.

See below, and thank you!

---
version: '2'

services:

  postgres:
    image: docker.io/library/postgres:latest
    hostname: postgres
    container_name: postgres
    networks:
      - mynetwork
    ports:
      - "5432:5432"
    environment:
      POSTGRES_PASSWORD: 'LoLOLoLOLoL'

  redis:
    image: docker.io/library/redis:latest
    hostname: redis
    container_name: redis
    networks:
      - mynetwork
    ports:
      - "6379:6379"

#networks:
# default:
#   external:
#     name: mynetwork

[muayyad-alsadi]

Yes. When podman-compose was created, rootless containers did not have the ability to do inter-container communication. So we did a trick we put all containers in the same network namespace and connect everything using localhost.
We have other tricks, you can check them

https://github.com/containers/podman-compose/blob/devel/docs/Mappings.md

Now they support rootless networks, but podman compose is not yet using them.

https://github.com/rootless-containers/slirp4netns

[nmvega]

@muayyad-alsadi Okay. I'll decompose the docker-compose.yml(5) files and build things manually (command by command). Thank you for clarifying this. ☺️

[muayyad-alsadi]

you don't have to. most likely you can go fine by deleting the network section.

here is an example which is a django stack having PostgreSQL, RabbitMQ, Memcached ..etc.

https://github.com/containers/podman-compose/blob/devel/examples/awx3/docker-compose.yml

[nmvega]

@muayyad-alsadi Ah, yes. That scenario does work. But for my case, I need different pods attached to the same podman network because multiple services in containers use the same port, which cause them not to start and cause a degraded pod.

So, for example, I need to have one service (container) in pod-1 publish, say, 8080:8080, and another service (container) in pod-2 publish 8081:8080 (notice the increment). But also have containers cross both pods be able to communicate with one another by placing the pods on the same network.

But I thought of a quick short-cut that will help me build the manual commands:

• Run podman-compose(1) in interactive (non-detached) mode, using my current docker-compose.yml(5) file, and capture the emitted podman(1) commands. Even though this results in a degraded pod, it's okay because I'll adjust the emitted podman(1) commands and add --network mynetwork and/or --pod mypod where necessary. I'll try this on Monday.

Hopefully that makes sense. =:)

[muayyad-alsadi]

does identity mode help
does it work when used with sudo

podman-compose -t identity

[rugk]

AFAIK this can be closed, as podman-compose now uses slirp4netns does not it?

[tgrasl]

The readme states: The main objective of this project is to be able to run docker-compose.yml unmodified and rootless.

To achieve that aim, it must support the networks section. Slirp4netns does not help because pods can not communicate using their name, or expose the same port.

It probably won't work in rootless mode at all, but in rootfull mode I found this workaround:

• create a new network using "podman network create my_net
• For the containers in the compose file, set
  network_mode: my_net
  This because the network_mode is actually passed as --network in podman create
• Start the composition with
  podman-compose -t identity up

[dblitt]

+1 for this issue.

I am okay with running my containers as root, and I just want to use a docker-compose.yml file with podman because new RHEL doesn't support regular docker and docker-compose.

Specifically, my issue is with external networks. I created another podman network that I want to attach to a service within the docker-compose file. However, podman-compose still appends the container-specific prefix.

In my setup, this is my docker-compose.yml:

services:
  nginx-proxy-manager:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    ports:
      - '80:80'
      - '127.0.0.1:81:81'
      - '443:443'
    volumes:
      - ./data:/data:Z
      - ./letsencrypt:/etc/letsencrypt:Z
    networks:
      - nginx-proxy-manager-nw
      - reverseproxy-nw
networks:
  nginx-proxy-manager-nw:
  reverseproxy-nw:
    external: true

Even though reverseproxy-nw is specified as an external network, podman-compose still creates and attaches the following two networks:

• nginx-proxy-manager_reverseproxy-nw
• nginx-proxy-manager_nginx-proxy-manager-nw

[muayyad-alsadi]

@dblitt you have said what happened but you did not specify what was supposed to happen.

would you please specify what podman-compose was supposed to do (what sequence of commands it should have done)

https://github.com/compose-spec/compose-spec/blob/master/spec.md#internal

NOTE: when you run it rootless (without sudo) it won't be able to do 80:80 port mapping because port 80 would require root.

[dblitt]

Yes, I understand that 80 cannot be bound rootless, that is part of the reason I am running rootful - with the other being using external networks in the first place.

When run with docker-compose, the external flagged network is not recreated.

Here is a relevant portion of output from docker-compose --verbose up

compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('nginxproxymanager_nginx-proxy-manager-nw')                                                                                      
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('reverseproxy-nw')                                                                                                               
compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': False,                                                                                                             
 'ConfigFrom': {'Network': ''},                                                                                                                                                                       
 'ConfigOnly': False,                                                                                                                                                                                 
 'Containers': {},                                                                                                                                                                                    
 'Created': '2021-12-12T13:34:22.697758942-06:00',                                                                                                                                                    
 'Driver': 'bridge',                                                                                                                                                                                  
 'EnableIPv6': False,                                                                                                                                                                                 
 'IPAM': {'Config': [{'Gateway': '172.22.0.1', 'Subnet': '172.22.0.0/16'}],                                                                                                                           
          'Driver': 'default',                                                                                                                                                                        
          'Options': {}},                                                                                                                                                                             
...                                                                                                                                                                                                   
compose.cli.verbose_proxy.proxy_callable: docker info <- ()                                                                                                                                           
compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'x86_64',                                                                                                                   
 'BridgeNfIp6tables': True,                                                                                                                                                                           
 'BridgeNfIptables': True,                                                                                                                                                                            
 'CPUSet': True,                                                                                                                                                                                      
 'CPUShares': True,                                                                                                                                                                                   
 'CgroupDriver': 'cgroupfs',                                                                                                                                                                          
 'CgroupVersion': '1',                                                                                                                                                                                
 'ContainerdCommit': {'Expected': 'd71fcd7d8303cbf684402823e425e9dd2e99285d',                                                                                                                         
                      'ID': 'd71fcd7d8303cbf684402823e425e9dd2e99285d'},                                                                                                                              
 'Containers': 7,                                                                                                                                                                                     
...                                                                                                                                                                                                   
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('nginx-proxy-manager_nginx-proxy-manager-nw')                                                                                    
compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True,                                                                                                              
 'ConfigFrom': {'Network': ''},                                                                                                                                                                       
 'ConfigOnly': False,                                                                                                                                                                                 
 'Containers': {},                                                                                                                                                                                    
 'Created': '2021-12-12T13:33:24.86973551-06:00',                                                                                                                                                     
 'Driver': 'bridge',                                                                                                                                                                                  
 'EnableIPv6': False,                                                                                                                                                                                 
 'IPAM': {'Config': [{'Gateway': '172.20.0.1', 'Subnet': '172.20.0.0/16'}],                                                                                                                           
          'Driver': 'default',                                                                                                                                                                        
          'Options': None},                                                                                                                                                                           
...                                                                                                                                                                                                   
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('reverseproxy-nw')
compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': False,
 'ConfigFrom': {'Network': ''},
 'ConfigOnly': False,
 'Containers': {},
 'Created': '2021-12-12T13:34:22.697758942-06:00',
 'Driver': 'bridge',
 'EnableIPv6': False,
 'IPAM': {'Config': [{'Gateway': '172.22.0.1', 'Subnet': '172.22.0.0/16'}],
          'Driver': 'default',
          'Options': {}},
...
compose.network.ensure: Network reverseproxy-nw declared as external. No new network will be created.

And if the network doesn't exist when docker-compose up is called, it spits out the following error:

compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('nginx-proxy-manager_nginx-proxy-manager-nw')                                                                                    
compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True,                                                                                                              
 'ConfigFrom': {'Network': ''},                                                                                                                                                                       
 'ConfigOnly': False,                                                                                                                                                                                 
 'Containers': {},                                                                                                                                                                                    
 'Created': '2021-12-12T13:33:24.86973551-06:00',                                                                                                                                                     
 'Driver': 'bridge',                                                                                                                                                                                  
 'EnableIPv6': False,                                                                                                                                                                                 
 'IPAM': {'Config': [{'Gateway': '172.20.0.1', 'Subnet': '172.20.0.0/16'}],                                                                                                                           
          'Driver': 'default',                                                                                                                                                                        
          'Options': None},                                                                                                                                                                           
...                                                                                                                                                                                                   
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('reverseproxy-nw')                                                                                                               
ERROR: compose.cli.main.exit_with_metrics: Network reverseproxy-nw declared as external, but could not be found. Please create the network manually using `docker network create reverseproxy-nw` and 
try again.

Let me know if I didn't fully answer your question @muayyad-alsadi

[muayyad-alsadi]

When run with docker-compose, the external flagged network is not recreated.

I've just pushed a fix. The given compose will exit

RuntimeError: External network [reverseproxy-nw] does not exists

[dblitt]

Thanks for this fix- it seems to correct the issue as described.

I did run into one edge case with redefining the default network though - with a docker-compose.yml as:

services:
  test-nginx01:
    image: nginx:latest
    volumes:
      - ./html:/usr/share/nginx/html:Z,ro
    ports:
      - 127.0.0.1:8081:80
networks:
  default:
    external:
      name: reverseproxy-nw

podman-compose exits with the following error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 560, in assert_cnt_nets
    try: compose.podman.output([], "network", ["exists", net_name])
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 815, in output
    return subprocess.check_output(cmd_ls)
  File "/usr/lib64/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib64/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['podman', 'network', 'exists', 'default']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/podman-compose", line 11, in <module>
    load_entry_point('podman-compose==1.0.2', 'console_scripts', 'podman-compose')()
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 1744, in main
    podman_compose.run()
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 1015, in run
    cmd(self, args)
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 1239, in wrapped
    return func(*args, **kw)
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 1405, in compose_up
    podman_args = container_to_args(compose, cnt, detached=args.detach)
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 640, in container_to_args
    assert_cnt_nets(compose, cnt)
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 563, in assert_cnt_nets
    raise RuntimeError(f"External network [{net_name}] does not exists")
RuntimeError: External network [default] does not exists

while docker-compose is perfectly happy and outputs the following:

compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('reverseproxy-nw')                                                                                                               
compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': False,                                                                                                             
 'ConfigFrom': {'Network': ''},                                                                                                                                                                       
 'ConfigOnly': False,                                                                                                                                                                                 
 'Containers': {},                                                                                                                                                                                    
 'Created': '2021-12-12T13:34:22.697758942-06:00',                                                                                                                                                    
 'Driver': 'bridge',                                                                                                                                                                                  
 'EnableIPv6': False,                                                                                                                                                                                 
 'IPAM': {'Config': [{'Gateway': '172.22.0.1', 'Subnet': '172.22.0.0/16'}],                                                                                                                           
          'Driver': 'default',                                                                                                                                                                        
          'Options': {}},                                                                                                                                                                             
...                                                                                                                                                                                                   
compose.cli.verbose_proxy.proxy_callable: docker info <- ()                                                                                                                                           
compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'x86_64',                                                                                                                   
 'BridgeNfIp6tables': True,                                                                                                                                                                           
 'BridgeNfIptables': True,                                                                                                                                                                            
 'CPUSet': True,                                                                                                                                                                                      
 'CPUShares': True,                                                                                                                                                                                   
 'CgroupDriver': 'cgroupfs',                                                                                                                                                                          
 'CgroupVersion': '1',                                                                                                                                                                                
 'ContainerdCommit': {'Expected': 'd71fcd7d8303cbf684402823e425e9dd2e99285d',
                      'ID': 'd71fcd7d8303cbf684402823e425e9dd2e99285d'},
 'Containers': 9,
...
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('reverseproxy-nw')
compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': False,
 'ConfigFrom': {'Network': ''},
 'ConfigOnly': False,
 'Containers': {},
 'Created': '2021-12-12T13:34:22.697758942-06:00',
 'Driver': 'bridge',
 'EnableIPv6': False,
 'IPAM': {'Config': [{'Gateway': '172.22.0.1', 'Subnet': '172.22.0.0/16'}],
          'Driver': 'default',
          'Options': {}},
...
compose.network.ensure: Network reverseproxy-nw declared as external. No new network will be created.

I think this is just an edge case with defining the default network (testing a "defaulttt" network just had docker-compose make its own network). After a quick search through the docker-compose reference I couldn't find anything that talks about this, but I got the idea from this page of nginx-proxy-manager documentation: https://nginxproxymanager.com/advanced-config/#best-practice-use-a-docker-network

[muayyad-alsadi]

I hate it when they can't make up their mind how to pass things and what is the type of the thing.

in this case external is not a boolean, it's might be a dictionary, and sometimes name is read from external and sometimes is read from parent

services:
  nginx-proxy-manager:
    image: busybox
    command: httpd -f -p 8080 -h /etc/
    networks:
      - nginx-proxy-manager-nw
      - monkey
      - default
networks:
  default:
    external:
      name: "reverseproxy-nw"
  monkey:
    name: banana
  nginx-proxy-manager-nw:

I've pushed a fix

[dblitt]

Haha I can see why that would be frustrating. Thanks for your work on this!

[dblitt]

Ran into another bug with this:

When I have two networks defined, one external and one not, it seems like something got messed up with creating that non-external network.

My networks section looks like this:

networks:
  internal:
  reverseproxy-nw:
    external: true

And the error is:

['podman', '--version', '']
using podman version: 3.3.1
** excluding:  set()
['podman', 'network', 'exists', 'testpaperless01_internal']
['podman', 'network', 'create', '--label', 'io.podman.compose.project=testpaperless01', '--label', 'com.docker.compose.project=testpaperless01']
['podman', 'network', 'exists', 'testpaperless01_internal']
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 561, in assert_cnt_nets
    try: compose.podman.output([], "network", ["exists", net_name])
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 817, in output
    return subprocess.check_output(cmd_ls)
  File "/usr/lib64/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib64/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['podman', 'network', 'exists', 'testpaperless01_internal']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/podman-compose", line 11, in <module>
    load_entry_point('podman-compose==1.0.2', 'console_scripts', 'podman-compose')()
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 1746, in main
    podman_compose.run()
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 1017, in run
    cmd(self, args)
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 1241, in wrapped
    return func(*args, **kw)
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 1407, in compose_up
    podman_args = container_to_args(compose, cnt, detached=args.detach)
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 642, in container_to_args
    assert_cnt_nets(compose, cnt)
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 577, in assert_cnt_nets
    compose.podman.output([], "network", ["exists", net_name])
  File "/usr/local/lib/python3.6/site-packages/podman_compose.py", line 817, in output
    return subprocess.check_output(cmd_ls)
  File "/usr/lib64/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib64/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['podman', 'network', 'exists', 'testpaperless01_internal']' returned non-zero exit status 1.

If the testpaperless01_internal network is created manually outside of podman-compose, everything works fine.

[muayyad-alsadi]

I was intending to push it down but I deleted it by mistake

-            args.append(net_name)
+            if net_desc.get("internal", None):
+                args.append("--internal")

fixed now