Warning Messages in Portainer Logs Caused by docker.sock not Available in Docker Swarm Setting #8806

pneuschwander · 2023-04-18T13:41:20Z

Bug description
There are warning messages in the Portainer Container log.

Expected behavior
There should not be such a warning message in the Portainer Container log.

Portainer Logs
2023/04/18 01:07PM WRN platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

Steps to reproduce the issue:

Have Portainer for a single node docker swarm using the official manual: https://docs.portainer.io/start/install-ce/server/swarm/linux
Check the Logs of the Portainer Container using Portainer
Notice the warning messages in the Portainer Container Log.

Technical details:

Portainer version: 2.18.1 (CE)
Docker version (managed by Portainer): 23.0.4
Platform (windows/linux): linux

Additional information

The changes introduced by #8095 to DetermineContainerPlatform in api/platform/platform.go (https://github.com/portainer/portainer/pull/8095/files#diff-1de261cf96cf89923c3d864d51e93cb0910700fa0f749bf85ab8167eefdee1b5R37) triggers docker info since that change.
Using the recommended install manual for swarm scenario, the docker.sock will not be mounted into the Portainer container but only to the agent container.
I guess this is causing the warning messages in the Portainer Container Log.

The text was updated successfully, but these errors were encountered:

tamarahenson · 2023-04-18T13:58:25Z

@pneuschwander

Thank you for the information. I am going to further investigate. I will update you as I learn more.

Interim, Is this an upgrade or a brand new install?

Thanks!

tamarahenson · 2023-04-18T14:06:09Z

@pneuschwander

I am able to reproduce the logging messages.

Environment:
[1] Docker Swarm (1) Manager
[2] Docker Swarm (2) Workers
[3] Portainer 2.18.1 CE (new install)

2023/04/18 14:01:21 server: Reverse tunnelling enabled
2023/04/18 14:01:21 server: Fingerprint 50:91:45:a4:37:3a:df:ba:1a:06:35:7c:61:77:cc:21
2023/04/18 14:01:21 server: Listening on 0.0.0.0:8000...
2023/04/18 02:01PM WRN platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/04/18 02:01PM INF portainer/main.go:765 > starting Portainer | build_number=29775 go_version=1.19.4 image_tag=linux-amd64-2.18.1 nodejs_version=18.16.0 version=2.18.1 webpack_version=5.68.0 yarn_version=1.22.19
2023/04/18 02:01PM INF http/server.go:342 > starting HTTPS server | bind_address=:9443
2023/04/18 02:01PM INF http/server.go:327 > starting HTTP server | bind_address=:9000
2023/04/18 07:01AM http: TLS handshake error from 10.0.0.2:56007: remote error: tls: bad certificate 
2023/04/18 02:01PM WRN platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/04/18 02:01PM WRN platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

I am able to login. I am able to navigate Portainer without issue. I believe these may be OK to ignore for now. I am going to forward to the Product team for review. I will update you as I learn more.

Thanks!

tamarahenson · 2023-04-18T14:19:34Z

@pneuschwander

Update: I have a brand new Portainer 2.17.1 install and am seeing the same messages:

2023/04/18 02:17PM WRN platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

Thanks!

pneuschwander · 2023-04-18T14:25:15Z

@tamarahenson Thank you for taking care of this!

Interim, Is this an upgrade or a brand new install?

I use that linux server and portainer since serveral months. So no new install in my case. Only updated the portainer version by executing "docker stack deploy" command from time to time.

I am able to login. I am able to navigate Portainer without issue. I believe these may be OK to ignore for now.

Yes, I can confirm that there seem to be no other big issues or problems caused by this. Portainer seems to work normal.

Update: I have a brand new Portainer 2.17.1 install and am seeing the same messages:

I think the change was introduced by 5cbf523 in December 2022. According to GitHub that commit seems to appear in tag 2.17.0 first. So I guess that log message is there since 2.17.0 or so and I just did not notice it earlier.

MikaelX · 2023-04-24T07:56:08Z

Seeing the same on both 2.17.1 and 2.18.1

2023/04/24 07:45AM INF github.com/portainer/portainer/api/http/server.go:342 > starting HTTPS server | bind_address=:9443
2023/04/24 07:45AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/04/24 07:45AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
{"time":1682322368,"message":"http: proxy error: context canceled"}
2023/04/24 07:46AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/04/24 07:46AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/04/24 07:46AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/04/24 07:46AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

Docker is running though

2023/04/24 08:07AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:517 > proceeding without encryption key |
2023/04/24 08:07AM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/04/24 08:07:55 server: Reverse tunnelling enabled
2023/04/24 08:07:55 server: Fingerprint 06:44:90:20:b6:e7:80:0a:d2:4f:d7:15:f1:55:cb:6a
2023/04/24 08:07:55 server: Listening on 0.0.0.0:8000...
2023/04/24 08:07AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:479 > instance already has defined environments, skipping the environment defined via CLI |
2023/04/24 08:07AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/04/24 08:07AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:765 > starting Portainer | build_number=29775 go_version=1.19.4 image_tag=linux-amd64-2.18.1 nodejs_version=18.16.0 version=2.18.1 webpack_version=5.68.0 yarn_version=1.22.19
2023/04/24 08:07AM INF github.com/portainer/portainer/api/http/server.go:327 > starting HTTP server | bind_address=:9000
2023/04/24 08:07AM INF github.com/portainer/portainer/api/http/server.go:342 > starting HTTPS server | bind_address=:9443
2023/04/24 08:08AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/04/24 08:08AM ERR github.com/portainer/portainer/api/http/client/client.go:94 > unexpected status code | status_code=403
2023/04/24 08:10AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/04/24 08:10AM ERR github.com/portainer/portainer/api/http/client/client.go:94 > unexpected status code | status_code=403```

hhromic · 2023-05-02T15:10:06Z

Just migrated to Portainer 2.18.2 today and started seeing those messages as well. Using with Docker Swarm clusters.

fcrespo82 · 2023-05-03T17:14:57Z

Any workaround for this?

tamarahenson · 2023-05-03T21:59:17Z

@fcrespo82

The only workaround would be to downgrade. An internal issue has been logged to address. I do not yet have the timeline or release. I will update as I learn more.

Thanks!

BillCarsson · 2023-05-20T21:12:21Z

I am facing to the some problem with my 2.18.2 new installation. To which version you recommend to downgrade to ?
Thanks !

fcrespo82 · 2023-05-20T21:21:11Z

The last version that kept working for me was 2.17.0.

hhromic · 2023-05-23T12:09:12Z

Just migrated to Portainer 2.18.2 today and started seeing those messages as well. Using with Docker Swarm clusters.

Just a heads-up that this issue is still present in the newly released 2.18.3 version as well :(

There are periodic error messages in the console logs:

2023/05/23 12:02PM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/05/23 12:02PM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/05/23 12:02PM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/05/23 12:03PM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/05/23 12:03PM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/05/23 12:04PM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/05/23 12:06PM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/05/23 12:06PM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/05/23 12:06PM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

This has the potential to fill up any logger system in place for Portainer, including the default Docker logger driver that does not rotate/limit logs by default.

mrkhachaturov · 2023-06-07T12:19:57Z

Issue is stiil actual for 2.18.3

denispaasch · 2023-06-12T07:53:22Z

The warning comes periodically as well with 2.18.3 when using Docker Standalone > API with a Docker API URL as environment.

cristianourban · 2023-07-13T08:59:16Z

Hi,
probably same issue here, with a small cluster of VMs in swarm mode (manager, worker-1 and worker-2):

OS Rocky Linux 8.8
Docker version: 24.0.4
Portainer CE server and agents version: 2.18.4

[root@manager ~]# docker logs cfbccc2efcf9
2023/07/12 10:53AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:495 > encryption key file not present | filename=portainer
2023/07/12 10:53AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:518 > proceeding without encryption key |
2023/07/12 10:53AM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
2023/07/12 10:53AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:480 > instance already has defined environments, skipping the environment defined via CLI |
2023/07/12 10:53:22 server: Reverse tunnelling enabled
2023/07/12 10:53:22 server: Fingerprint ea:92:7c:35:b7:72:00:cf:7a:b3:86:de:65:c6:d5:85
2023/07/12 10:53:22 server: Listening on 0.0.0.0:8000...
2023/07/12 10:53AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/07/12 10:53AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:766 > starting Portainer | build_number=32198 go_version=1.19.4 image_tag=linux-amd64-2.18.4 nodejs_version=18.16.1 version=2.18.4 webpack_version=5.68.0 yarn_version=1.22.19
2023/07/12 10:53AM INF github.com/portainer/portainer/api/http/server.go:345 > starting HTTPS server | bind_address=:9443
2023/07/12 10:53AM INF github.com/portainer/portainer/api/http/server.go:330 > starting HTTP server | bind_address=:9000
2023/07/12 10:54AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/07/12 10:55AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/07/12 10:57AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/07/12 10:58AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/07/12 11:02AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
{"time":1689160220,"message":"http: proxy error: context canceled"}
2023/07/12 11:12AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
{"time":1689163970,"message":"http: proxy error: context canceled"}
{"time":1689163970,"message":"http: proxy error: context canceled"}
{"time":1689163970,"message":"http: proxy error: context canceled"}
{"time":1689163970,"message":"http: proxy error: context canceled"}
2023/07/12 12:15PM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/07/13 07:16AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/07/13 07:20AM WRN github.com/portainer/portainer/api/platform/platform.go:68 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
[root@manager ~]#

denispaasch · 2023-07-13T09:58:00Z

Hi and I am not very familiar with Go but I looked up the code and I think the issue is that a simple docker client is created here:
https://github.com/portainer/portainer/blob/develop/api/platform/platform.go#L59

In that case the DefaultDockerHost is used which is unix:///var/run/docker.sock.

You can override that with the DOCKER_HOST environment variable but that does not work either because the simple client refers to the DefaultDockerHost. The warning occurs also each time the Info-Endpoint is called via REST which is quite often :)

IMHO this should not be a warning. We use a docker socket proxy in between, so we don't mount the socket directly:
https://github.com/Tecnativa/docker-socket-proxy

Alternatively it would be nice to make that overridable.

neviim · 2023-09-01T18:53:24Z

Hi,
probably same issue here, with a small cluster of CT Docker in swarm mode (manager, worker-01 and worker-02):

- OS Debian 12
- PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
- NAME="Debian GNU/Linux"
- VERSION_ID="12"
- VERSION="12 (bookworm)"
- VERSION_CODENAME=bookworm
- ID=debian
- HOME_URL="https://www.debian.org/"
- SUPPORT_URL="https://www.debian.org/support"
- BUG_REPORT_URL="https://bugs.debian.org/"

Docker version 24.0.5, build ced0996
Portainer-CE Server and Agents version :2.19.0

In maneger:

*docker logs d3e6e3d66e4d*
2023/09/01 05:33PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:359 > encryption key file not present | filename=portainer
2023/09/01 05:33PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:382 > proceeding without encryption key |
2023/09/01 05:33PM INF github.com/portainer/portainer/api/database/boltdb/db.go:125 > loading PortainerDB | filename=portainer.db
2023/09/01 05:33PM INF github.com/portainer/portainer/api/chisel/service.go:197 > Found Chisel private key file on disk | private-key=/data/chisel/private-key.pem
2023/09/01 17:33:49 server: Reverse tunnelling enabled
2023/09/01 17:33:49 server: Fingerprint OVNK17d/Zx8vtoYKk7oUDZf3QGvk0Y5rKqSwLh4suOc=
2023/09/01 17:33:49 server: Listening on http://0.0.0.0:8000
2023/09/01 05:33PM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/09/01 05:33PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:634 > starting Portainer | build_number=34280 go_version=1.20.5 image_tag=linux-amd64-2.19.0 nodejs_version=18.17.1 version=2.19.0 webpack_version=5.88.1 yarn_version=1.22.19
2023/09/01 05:33PM INF github.com/portainer/portainer/api/http/server.go:356 > starting HTTPS server | bind_address=:9443
2023/09/01 05:33PM INF github.com/portainer/portainer/api/http/server.go:340 > starting HTTP server | bind_address=:9000
2023/09/01 05:38PM INF github.com/portainer/portainer/api/adminmonitor/admin_monitor.go:62 > the Portainer instance timed out for security purposes, to re-enable your Portainer instance, you will need to restart Portainer

pavelriha · 2023-09-22T15:30:50Z

I was using portainer-ee:218.2 and did not realize any problem, but after upgrading to 2.19.1
log is flooded with theese two errors:
2023/09/22 03:22PM INF docker/snapshot.go:180 > unable to inspect container in other Swarm nodes | error="Error response from daemon: No such container: d70efd1405298804dc09d3b72e4a8b0a2463799fcbe12478bca093d2dc8a9bc4" container=d70efd1405298804dc09d3b72e4a8b0a2463799fcbe12478bca093d2dc8a9bc4

2023/09/22 03:22PM WRN platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

I can't say it's harmless, sometimes the portainer is not showing the "stats" and "exec console" icons at container.
and then show after refresh

not sure how to downgrade, if I just deploy the 2.18.2 it's not working, I see the stacks, but cant go into the stack (error was something like it can't read compose.yml file)

2 node docker swarm, Docker version 23.0.6

jamescarppe · 2023-09-27T02:12:23Z

Internal Ref: EE-5372

athlonUA · 2023-12-03T10:55:55Z

Hi there, everyone! I'm experiencing the same issue on version 2.19.3-alpine. Do you have any expectations on when it could be fixed?

2023/12/03 10:43AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2023/12/03 10:43AM ERR github.com/portainer/portainer/api/http/client/client.go:92 > unexpected status code | status_code=403

I use Docker version 24.0.7

~# docker --version
Docker version 24.0.7, build afdd53b

FahriDevZ · 2023-12-27T12:27:50Z

same issue on docker swarm use portainer-agent-stack.yml
my test doesn't have any worker, only 1 manager

portainer_agent

portainer_agent.0.qxeee0ajedcv@pallas    | 2023/12/27 11:55:29 http: TLS handshake error from 10.0.2.3:49700: EOF

portainer_portainer

portainer_portainer.1.ql8dowwiqz2n@pallas    | 2023/12/27 12:16PM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
portainer_portainer.1.ql8dowwiqz2n@pallas    | 2023/12/27 12:16PM ERR github.com/portainer/portainer/api/http/client/client.go:92 > unexpected status code | status_code=403

portainer version 2.19.4 (CE)

docker version

Client: Docker Engine - Community
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.20.10
 Git commit:        afdd53b
 Built:             Thu Oct 26 09:07:41 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.7
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.10
  Git commit:       311b9ff
  Built:            Thu Oct 26 09:07:41 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.25
  GitCommit:        d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc:
  Version:          1.1.10
  GitCommit:        v1.1.10-0-g18a0cb0
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

LawF0rge52 · 2023-12-29T19:35:18Z

Not sure if that's helpful and even the fix for the root cause but I came around the issue by adding into the offical portainer-agent-stack.yml yaml:
volumes:
- /var/run/docker.sock:/var/run/docker.sock

Since then the warning disappered and I was able to connect properly to the portainer environement (before just an empty portainer had been presented)

Yanik39 · 2024-01-23T13:37:06Z

Any update on this?

When i add,
volumes:
- /var/run/docker.sock:/var/run/docker.sock
to portainer-agent-stack.yml file, my system also see 2 environments, swarm and local.
swarm comes from the yml file as expected but the added docker.sock creates local environment.

ia-mfriegang · 2024-02-05T18:40:59Z

I'm seeing the same issue on version 2.19. 4

hhromic · 2024-02-05T18:56:33Z

Internal Ref: EE-5372

Thiss issue seems to be under the attention of the Portainer team since a while (Sep 2023), but somehow still unresolved? :(

pneuschwander · 2024-02-05T19:01:04Z

@jamescarppe @tamarahenson Is there any update on this issue or a time schedule regarding a fix? 🙂

medvedevden1s · 2024-02-09T00:51:57Z

in 1.20.5 I see the same issue after shutting down the docker and starting againe

2024-02-09 02:40:46 2024/02/09 12:40AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:369 > encryption key file not present | filename=portainer 2024-02-09 02:40:46 2024/02/09 12:40AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:392 > proceeding without encryption key | 2024-02-09 02:40:46 2024/02/09 12:40AM INF github.com/portainer/portainer/api/database/boltdb/db.go:125 > loading PortainerDB | filename=portainer.db 2024-02-09 02:40:46 2024/02/09 12:40AM INF github.com/portainer/portainer/api/internal/ssl/ssl.go:80 > no cert files found, generating self signed SSL certificates | 2024-02-09 02:40:46 2024/02/09 12:40AM INF github.com/portainer/portainer/api/chisel/service.go:193 > Generated a new Chisel private key file | private-key=/data/chisel/private-key.pem 2024-02-09 02:40:46 2024/02/09 00:40:46 server: Reverse tunnelling enabled 2024-02-09 02:40:46 2024/02/09 00:40:46 server: Fingerprint 5a6Ua/5FZDvsinJvrbiL28cQIQ1RLapTiK2MimnP2oo= 2024-02-09 02:40:46 2024/02/09 00:40:46 server: Listening on http://0.0.0.0:8000 2024-02-09 02:40:46 2024/02/09 12:40AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:649 > starting Portainer | build_number=35428 go_version=1.20.5 image_tag=linux-amd64-2.19.4 nodejs_version=18.19.0 version=2.19.4 webpack_version=5.88.1 yarn_version=1.22.21 2024-02-09 02:40:46 2024/02/09 12:40AM INF github.com/portainer/portainer/api/http/server.go:357 > starting HTTPS server | bind_address=:9443 2024-02-09 02:40:46 2024/02/09 12:40AM INF github.com/portainer/portainer/api/http/server.go:341 > starting HTTP server | bind_address=:9000 2024-02-09 02:43:49 2024/02/09 12:43AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:369 > encryption key file not present | filename=portainer 2024-02-09 02:43:49 2024/02/09 12:43AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:392 > proceeding without encryption key | 2024-02-09 02:43:49 2024/02/09 12:43AM INF github.com/portainer/portainer/api/database/boltdb/db.go:125 > loading PortainerDB | filename=portainer.db 2024-02-09 02:43:49 2024/02/09 12:43AM INF github.com/portainer/portainer/api/chisel/service.go:198 > Found Chisel private key file on disk | private-key=/data/chisel/private-key.pem 2024-02-09 02:43:49 2024/02/09 00:43:49 server: Reverse tunnelling enabled 2024-02-09 02:43:49 2024/02/09 00:43:49 server: Fingerprint 5a6Ua/5FZDvsinJvrbiL28cQIQ1RLapTiK2MimnP2oo= 2024-02-09 02:43:49 2024/02/09 00:43:49 server: Listening on http://0.0.0.0:8000 2024-02-09 02:43:49 2024/02/09 12:43AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?" 2024-02-09 02:43:49 2024/02/09 12:43AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:649 > starting Portainer | build_number=35428 go_version=1.20.5 image_tag=linux-amd64-2.19.4 nodejs_version=18.19.0 version=2.19.4 webpack_version=5.88.1 yarn_version=1.22.21 2024-02-09 02:43:49 2024/02/09 12:43AM INF github.com/portainer/portainer/api/http/server.go:357 > starting HTTPS server | bind_address=:9443 2024-02-09 02:43:49 2024/02/09 12:43AM INF github.com/portainer/portainer/api/http/server.go:341 > starting HTTP server | bind_address=:9000 2024-02-09 02:44:56 2024/02/09 12:44AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:369 > encryption key file not present | filename=portainer 2024-02-09 02:44:56 2024/02/09 12:44AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:392 > proceeding without encryption key | 2024-02-09 02:44:56 2024/02/09 12:44AM INF github.com/portainer/portainer/api/database/boltdb/db.go:125 > loading PortainerDB | filename=portainer.db 2024-02-09 02:44:56 2024/02/09 12:44AM INF github.com/portainer/portainer/api/chisel/service.go:198 > Found Chisel private key file on disk | private-key=/data/chisel/private-key.pem 2024-02-09 02:44:56 2024/02/09 00:44:56 server: Reverse tunnelling enabled 2024-02-09 02:44:56 2024/02/09 00:44:56 server: Fingerprint 5a6Ua/5FZDvsinJvrbiL28cQIQ1RLapTiK2MimnP2oo= 2024-02-09 02:44:56 2024/02/09 00:44:56 server: Listening on http://0.0.0.0:8000 2024-02-09 02:44:56 2024/02/09 12:44AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?" 2024-02-09 02:44:56 2024/02/09 12:44AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:649 > starting Portainer | build_number=35428 go_version=1.20.5 image_tag=linux-amd64-2.19.4 nodejs_version=18.19.0 version=2.19.4 webpack_version=5.88.1 yarn_version=1.22.21 2024-02-09 02:44:56 2024/02/09 12:44AM INF github.com/portainer/portainer/api/http/server.go:357 > starting HTTPS server | bind_address=:9443 2024-02-09 02:44:56 2024/02/09 12:44AM INF github.com/portainer/portainer/api/http/server.go:341 > starting HTTP server | bind_address=:9000 2024-02-09 02:45:16 2024/02/09 12:45AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

hermanoff · 2024-02-21T09:05:56Z

I was looking for solution for quite some time, but since there isn't an actual solution or at least I wasn't able to find one, I decided to mount the docker.sock directly to portainer service with read-only access. This way Portainer isn't able to do anything that is not supposed to do to docker.sock and the errors are finally gone.

volumes:
      - portainer_data:/data
      - /var/run/docker.sock:/var/run/docker.sock:ro

Jognu · 2024-03-06T10:27:00Z

Same problem for me. It works fine with manager + 1 node. But when I add an other node, the UI become "unresponsive" on some pages and I start having some logs :
2024/03/06 10:14AM INF github.com/portainer/portainer/api/docker/snapshot.go:177 > unable to inspect container in other Swarm nodes | error="Error response from daemon: No such container: 39dc92c047bba7ba364df30dd2b85a398f0e889eadccc3ffaf48e4f3a7753f81" container=39dc92c047bba7ba364df30dd2b85a398f0e889eadccc3ffaf48e4f3a7753f81
2024/03/06 10:19AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

If I remove the new node from the swarm, the problem disapear.
I tried the solution from @hermanoff but no changes.
This issue seems similar : #8804

Portainer : 2.19.4
Docker version on the nodes : 25.0.3
OS : Ubuntu 22.04.4 LTS

Jognu · 2024-03-06T16:26:15Z

Sorry for the double post but it may help other people. My problems were ultimately not linked to Portainer but to Docker Swarm, I identified it by observing communication problems on other docker services.
And in the end it all came from a port conflict with VMWare: https://portal.portainer.io/knowledge/known-issues-with-vmware

RickLeite · 2024-03-09T22:48:39Z

same

hhromic · 2024-03-19T11:22:46Z

@jamescarppe Sadly, the recently released 2.20.0 version is still suffering from this issue.

Even when using the standard officially documented installation method for Docker Swarm here:
https://docs.portainer.io/start/install-ce/server/swarm/linux#deployment

Portainer starts to spam the logs frequently (more than once per minute) as seen below:

2024/03/19 10:56AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 10:56AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 10:56AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 10:56AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 10:59AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 10:59AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 10:59AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 10:59AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 10:59AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:00AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:00AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:00AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:00AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:00AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:00AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:01AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:02AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:02AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:02AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:02AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:02AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
2024/03/19 11:03AM WRN github.com/portainer/portainer/api/platform/platform.go:70 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

The problem manifests when the Portainer server container does not have /var/run/docker.sock mounted in it, which is not necessary when using Portainer with a remote Portainer agent, as it goes via network. In fact, even when using Portainer in standalone mode (without containers) it should manifest if the user is running Portainer without Docker running locally.

The problem is in the DetermineContainerPlatform() function here:

portainer/api/platform/platform.go

Lines 39 to 82 in 61642b8

    
           func DetermineContainerPlatform() (ContainerPlatform, error) { 
        
           	podmanModeEnvVar := os.Getenv(PodmanMode) 
        
           	if podmanModeEnvVar == "1" { 
        
           		return PlatformPodman, nil 
        
           	} 
        
           	serviceHostKubernetesEnvVar := os.Getenv(KubernetesServiceHost) 
        
           	if serviceHostKubernetesEnvVar != "" { 
        
           		return PlatformKubernetes, nil 
        
           	} 
        
           	nomadJobName := os.Getenv(NomadJobName) 
        
           	if nomadJobName != "" { 
        
           		return PlatformNomad, nil 
        
           	} 
        
           	if !isRunningInContainer() { 
        
           		return "", nil 
        
           	} 
        
           	dockerCli, err := dockerclient.CreateSimpleClient() 
        
           	if err != nil { 
        
           		return "", errors.WithMessage(err, "failed to create docker client") 
        
           	} 
        
           	defer dockerCli.Close() 
        
           	info, err := dockerCli.Info(context.Background()) 
        
           	if err != nil { 
        
           		if client.IsErrConnectionFailed(err) { 
        
           			log.Warn(). 
        
           				Err(err). 
        
           				Msg("failed to retrieve docker info") 
        
           			return "", nil 
        
           		} 
        
           		return "", errors.WithMessage(err, "failed to retrieve docker info") 
        
           	} 
        
           	if info.Swarm.NodeID == "" { 
        
           		return PlatformDockerStandalone, nil 
        
           	} 
        
           	return PlatformDockerSwarm, nil 
        
           }

As it can be seen, this function tries to create a client to a Docker daemon unconditionally in line 59. I believe that instead, Portainer should check if /var/run/docker.sock (or more generically, DOCKER_HOST) exists before attempting to do this, and skip the rest of the detection if the socket is not available.

I understand the Portainer team is already working on this (Internal Ref: EE-5372) for a while, but it has been a really long while as of now and the log spamming is really an important issue for us (and I think lots of users conscious of log storage space).

We haven't been able to migrate from Portainer 2.14.x due to this (and another) issue.
I hope this can be fixed once and for all sooner than later before the new LTS hits. Thanks!

theanurin · 2024-04-16T06:25:02Z

+1

jamest-portainer · 2024-04-17T00:43:22Z

+1 ...I got this problem (continual error in my logs) too.

ElectricWeasel · 2024-04-22T08:22:33Z

+1

agermalid · 2024-05-02T18:09:44Z

+1

szymek07 · 2024-05-04T11:47:50Z

+1

LawF0rge52 · 2024-05-04T14:18:39Z

Not sure if that's helpful and even the fix for the root cause but I came around the issue by adding into the offical portainer-agent-stack.yml yaml: volumes: - /var/run/docker.sock:/var/run/docker.sock

Since then the warning disappered and I was able to connect properly to the portainer environement (before just an empty portainer had been presented)

@Yanik39 Is right. It's not really resolving the issue. It's rather showing the swarm and local environment.

But I actually found a fix for my setup.
Background: I've deployed the swarm nodes to 3 vmware virtual machines. After initialising the swarm cluster and getting around the issue temporarily with the volumes: - /var/run/docker.sock:/var/run/docker.sock extension in the yaml file, I figured out the ingress network doesn't work as expected. So I digged a bit and found an known issue with vmware virtual nics (I'm using VMXNET3):
Ref: https://portal.portainer.io/knowledge/known-issues-with-vmware

So it came down to an network issue that had to be fixed by switching off the eth offload-tx on the nic interface that serves the ingress network.

nano /etc/network/interfaces
iface ens192 inet dhcp
offload-tx off
iface ens192 inet6 auto
offload-tx off

After reboot the ingress network was working properly and (surprise surprise) portainer was able to finally connect to the swarm agents even without docker.sock extension.

Hope that helps, in case you have a similar setup ;-)

pneuschwander added bug/need-confirmation kind/bug Applied to Bugs labels Apr 18, 2023

tamarahenson self-assigned this Apr 18, 2023

tamarahenson added area/container-logs area/swarm labels Apr 18, 2023

tamarahenson added bug/confirmed and removed bug/need-confirmation labels Apr 18, 2023

tamarahenson added the internal/jira label Apr 19, 2023

jamescarppe assigned jamescarppe and unassigned tamarahenson Sep 27, 2023

Warning Messages in Portainer Logs Caused by docker.sock not Available in Docker Swarm Setting #8806

Warning Messages in Portainer Logs Caused by docker.sock not Available in Docker Swarm Setting #8806

Comments

pneuschwander commented Apr 18, 2023

tamarahenson commented Apr 18, 2023

tamarahenson commented Apr 18, 2023

tamarahenson commented Apr 18, 2023

pneuschwander commented Apr 18, 2023

MikaelX commented Apr 24, 2023 • edited

hhromic commented May 2, 2023 • edited

fcrespo82 commented May 3, 2023

tamarahenson commented May 3, 2023

BillCarsson commented May 20, 2023

fcrespo82 commented May 20, 2023

hhromic commented May 23, 2023

mrkhachaturov commented Jun 7, 2023

denispaasch commented Jun 12, 2023 • edited

cristianourban commented Jul 13, 2023

denispaasch commented Jul 13, 2023

neviim commented Sep 1, 2023 • edited

pavelriha commented Sep 22, 2023 • edited

jamescarppe commented Sep 27, 2023

athlonUA commented Dec 3, 2023 • edited

FahriDevZ commented Dec 27, 2023

LawF0rge52 commented Dec 29, 2023

Yanik39 commented Jan 23, 2024

ia-mfriegang commented Feb 5, 2024

hhromic commented Feb 5, 2024

pneuschwander commented Feb 5, 2024

medvedevden1s commented Feb 9, 2024

hermanoff commented Feb 21, 2024 • edited

Jognu commented Mar 6, 2024

Jognu commented Mar 6, 2024

RickLeite commented Mar 9, 2024

hhromic commented Mar 19, 2024

theanurin commented Apr 16, 2024

jamest-portainer commented Apr 17, 2024

ElectricWeasel commented Apr 22, 2024

agermalid commented May 2, 2024

szymek07 commented May 4, 2024

LawF0rge52 commented May 4, 2024

MikaelX commented Apr 24, 2023 •

edited

hhromic commented May 2, 2023 •

edited

denispaasch commented Jun 12, 2023 •

edited

neviim commented Sep 1, 2023 •

edited

pavelriha commented Sep 22, 2023 •

edited

athlonUA commented Dec 3, 2023 •

edited

hermanoff commented Feb 21, 2024 •

edited