bulk sync to node <some-string> failed

[YuryHrytsuk]

What does <some-string> defines? I was never able to find a node with id equal to <some-string>. How to map <some-string> to an actual node in my cluster?

Example log from journalctl

Feb 12 13:19:47 my-srv01 dockerd[2615989]: time="2024-02-12T13:19:47.878730800Z" level=warning msg="bulk sync to node 993b5ed1c755 failed: failed to send a TCP message during bulk sync

docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.16.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  scan: Docker Scan (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-scan

Server:
 Containers: 10
  Running: 7
  Paused: 0
  Stopped: 3
 Images: 9
 Server Version: 20.10.14
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: gelf
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: gn2rsb5a806r0u3t33b1qei2f
  Is Manager: false
  Node Address: 10.0.3.105
  Manager Addresses:
   10.0.3.45:2377
 Runtimes: nvidia runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: nvidia
 Init Binary: docker-init
 containerd version: 2456e983eb9e37e47538f59ea18f2043c9a73640
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.15.0-1030-aws
 Operating System: Ubuntu 20.04.2 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 62.02GiB
 Name: ip-10-0-3-105
 ID: I5Q6:MIY6:MCQD:4FXN:OBSB:GMUQ:E5ZO:A7WX:VSJK:B7EW:VA4L:33GZ
 Docker Root Dir: /mnt/docker/data
 Debug Mode: true
  File Descriptors: 137
  Goroutines: 195
  System Time: 2024-02-15T13:32:30.121115095Z
  EventsListeners: 3
 Registry: https://index.docker.io/v1/
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 20
   Base: 192.168.0.0/16, Size: 24

moby/moby#47372

[YuryHrytsuk]

This log is probably generated by this line: https://github.com/moby/libnetwork/blob/3797618f9a38372e8107d8c06f6ae199e1133ae8/networkdb/cluster.go#L706

[YuryHrytsuk]

This answer explain how to map <some-string> to a particular swarm cluster Node. It would be nice to have it documented somewhere.

moby/moby#47372 (comment)