Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to initialize libp2p due to conflicting limit configuration #9695

Closed
3 tasks done
thedmdim opened this issue Mar 4, 2023 · 8 comments · Fixed by #9723
Closed
3 tasks done

Unable to initialize libp2p due to conflicting limit configuration #9695

thedmdim opened this issue Mar 4, 2023 · 8 comments · Fixed by #9723
Assignees
Labels
kind/bug A bug in existing code (including security flaws) P1 High: Likely tackled by core team if no one steps up

Comments

@thedmdim
Copy link

thedmdim commented Mar 4, 2023

Checklist

Installation method

built from source

Version

Kubo version: 0.19.0-dev-d1541e1d3
Repo version: 13
System version: arm/linux
Golang version: go1.20.1

Config

{
  "API": {
    "HTTPHeaders": {
      "Access-Control-Allow-Origin": [
        "*"
      ]
    }
  },
  "Addresses": {
    "API": "/ip4/127.0.0.1/tcp/5001",
    "Announce": [],
    "AppendAnnounce": [],
    "Gateway": "/ip4/127.0.0.1/tcp/8080",
    "NoAnnounce": [],
    "Swarm": [
      "/ip4/0.0.0.0/tcp/4001",
      "/ip6/::/tcp/4001",
      "/ip4/0.0.0.0/udp/4001/quic",
      "/ip4/0.0.0.0/udp/4001/quic-v1",
      "/ip4/0.0.0.0/udp/4001/quic-v1/webtransport",
      "/ip6/::/udp/4001/quic",
      "/ip6/::/udp/4001/quic-v1",
      "/ip6/::/udp/4001/quic-v1/webtransport"
    ]
  },
  "AutoNAT": {},
  "Bootstrap": [
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
    "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/ip4/104.131.131.82/udp/4001/quic/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"
  ],
  "DNS": {
    "Resolvers": {}
  },
  "Datastore": {
    "BloomFilterSize": 0,
    "GCPeriod": "1h",
    "HashOnRead": false,
    "Spec": {
      "mounts": [
        {
          "child": {
            "path": "blocks",
            "shardFunc": "/repo/flatfs/shard/v1/next-to-last/2",
            "sync": true,
            "type": "flatfs"
          },
          "mountpoint": "/blocks",
          "prefix": "flatfs.datastore",
          "type": "measure"
        },
        {
          "child": {
            "compression": "none",
            "path": "datastore",
            "type": "levelds"
          },
          "mountpoint": "/",
          "prefix": "leveldb.datastore",
          "type": "measure"
        }
      ],
      "type": "mount"
    },
    "StorageGCWatermark": 90,
    "StorageMax": "10GB"
  },
  "Discovery": {
    "MDNS": {
      "Enabled": true
    }
  },
  "Experimental": {
    "AcceleratedDHTClient": false,
    "FilestoreEnabled": false,
    "GraphsyncEnabled": false,
    "Libp2pStreamMounting": false,
    "P2pHttpProxy": false,
    "StrategicProviding": false,
    "UrlstoreEnabled": false
  },
  "Gateway": {
    "APICommands": [],
    "HTTPHeaders": {
      "Access-Control-Allow-Headers": [
        "X-Requested-With",
        "Range",
        "User-Agent"
      ],
      "Access-Control-Allow-Methods": [
        "GET"
      ],
      "Access-Control-Allow-Origin": [
        "*"
      ]
    },
    "NoDNSLink": false,
    "NoFetch": false,
    "PathPrefixes": [],
    "PublicGateways": null,
    "RootRedirect": "",
    "Writable": false
  },
  "Identity": {
    "PeerID": "12D3KooWHNHgSm1azRN6sawWrPcYTQiWJtL2YmA1Y8VpVY1nJ4WW"
  },
  "Internal": {},
  "Ipns": {
    "RecordLifetime": "",
    "RepublishPeriod": "",
    "ResolveCacheSize": 128
  },
  "Migration": {
    "DownloadSources": [],
    "Keep": ""
  },
  "Mounts": {
    "FuseAllowOther": false,
    "IPFS": "/ipfs",
    "IPNS": "/ipns"
  },
  "Peering": {
    "Peers": null
  },
  "Pinning": {
    "RemoteServices": {}
  },
  "Plugins": {
    "Plugins": null
  },
  "Provider": {
    "Strategy": ""
  },
  "Pubsub": {
    "DisableSigning": false,
    "Router": ""
  },
  "Reprovider": {},
  "Routing": {
    "Methods": null,
    "Routers": null
  },
  "Swarm": {
    "AddrFilters": null,
    "AutoRelay": {
      "Enabled": true
    },
    "ConnMgr": {
      "GracePeriod": "1m",
      "HighWater": 40,
      "LowWater": 20
    },
    "DisableBandwidthMetrics": false,
    "DisableNatPortMap": false,
    "RelayClient": {},
    "RelayService": {},
    "ResourceMgr": {},
    "Transports": {
      "Multiplexers": {},
      "Network": {},
      "Security": {}
    }
  }
}

Description

Hello! I just updated my kubo node from 0.17 to 0.19 at my SBC, but cannot run ipfs daemon, i got:

root@DietPi:~/kubo# ipfs daemon
Initializing daemon...
Kubo version: 0.19.0-dev-d1541e1d3
Repo version: 13
System version: arm/linux
Golang version: go1.20.1

Computing default go-libp2p Resource Manager limits based on:
    - 'Swarm.ResourceMgr.MaxMemory': "259 MB"
    - 'Swarm.ResourceMgr.MaxFileDescriptors': 524288

Applying any user-supplied overrides on top.
Run 'ipfs swarm limit all' to see the resulting limits.

2023-03-04T16:38:03.798Z        ERROR   core    core/builder.go:158     constructing the node: could not build arguments for function "github.com/ipfs/kubo/core/node".PeerWith.func1 (github.com/ipfs/kubo/core/node/peering.go:29): failed to build *peering.PeeringService: could not build arguments for function "github.com/ipfs/kubo/core/node".Peering (github.com/ipfs/kubo/core/node/peering.go:14): failed to build host.Host: could not build arguments for function "github.com/ipfs/kubo/core/node/libp2p".Host (github.com/ipfs/kubo/core/node/libp2p/host.go:40): could not build value group []config.Option[group="libp2p"]: received non-nil error from function "github.com/ipfs/kubo/core/node/libp2p".ResourceManager.func1 (github.com/ipfs/kubo/core/node/libp2p/rcmgr.go:35):
Unable to initialize libp2p due to conflicting limit configuration:
ResourceMgr.Limits.System.StreamsInbound (-1056100900) must be bigger than ConnMgr.HighWater (40)


Error: constructing the node (see log for full detail):
Unable to initialize libp2p due to conflicting limit configuration:
ResourceMgr.Limits.System.StreamsInbound (-1056100900) must be bigger than ConnMgr.HighWater (40)
@thedmdim thedmdim added kind/bug A bug in existing code (including security flaws) need/triage Needs initial labeling and prioritization labels Mar 4, 2023
@BigLep
Copy link
Contributor

BigLep commented Mar 6, 2023

@thedmdim: I don't know how ResourceMgr.Limits.System.StreamsInbound is getting set to -1056100900. That said, there is active changes here in #9650 that are about to land where a lot of the related code is about to get touched. I think you should test once the 0.19 RC is cut (or at least #9680 is merged).

@BigLep
Copy link
Contributor

BigLep commented Mar 6, 2023

That said, I suspect regardless you're going to have issues with only 'Swarm.ResourceMgr.MaxMemory': "259 MB". If you want to unblock yourself, you can just disable the libp2p resource manager/accountant if the protection isn't key for your usecase.

@aschmahmann aschmahmann added P1 High: Likely tackled by core team if no one steps up and removed need/triage Needs initial labeling and prioritization labels Mar 6, 2023
@BigLep
Copy link
Contributor

BigLep commented Mar 8, 2023

@thedmdim : can you please test with the 0.19RC that was released on 2023-03-06?

@BigLep BigLep added the need/author-input Needs input from the original author label Mar 8, 2023
@thedmdim
Copy link
Author

@thedmdim : can you please test with the 0.19RC that was released on 2023-03-06?

yep, i'll test it soon

@thedmdim
Copy link
Author

thedmdim commented Mar 11, 2023

@thedmdim : can you please test with the 0.19RC that was released on 2023-03-06?

build: make build CGO_ENABLED=0

root@DietPi:~/kubo# ipfs version
ipfs version 0.19.0-rc1
root@DietPi:~/kubo# ipfs daemon
Initializing daemon...
Kubo version: 0.19.0-rc1-0f1181d23
Repo version: 13
System version: arm/linux
Golang version: go1.20.1

Computed default go-libp2p Resource Manager limits based on:
    - 'Swarm.ResourceMgr.MaxMemory': "259 MB"
    - 'Swarm.ResourceMgr.MaxFileDescriptors': 524288

Theses can be inspected with 'ipfs swarm resources'.

2023-03-11T15:53:00.612Z        ERROR   core    core/builder.go:158     constructing the node: could not build arguments for function "github.com/ipfs/kubo/core/node".PeerWith.func1 (github.com/ipfs/kubo/core/node/peering.go:29): failed to build *peering.PeeringService: could not build arguments for function "github.com/ipfs/kubo/core/node".Peering (github.com/ipfs/kubo/core/node/peering.go:14): failed to build host.Host: could not build arguments for function "github.com/ipfs/kubo/core/node/libp2p".Host (github.com/ipfs/kubo/core/node/libp2p/host.go:40): could not build value group []config.Option[group="libp2p"]: received non-nil error from function "github.com/ipfs/kubo/core/node/libp2p".ResourceManager.func1 (github.com/ipfs/kubo/core/node/libp2p/rcmgr.go:31):
Unable to initialize libp2p due to conflicting resource manager limit configuration.
resource manager System.StreamsInbound (-3) must be bigger than ConnMgr.HighWater (40)


Error: constructing the node (see log for full detail):
Unable to initialize libp2p due to conflicting resource manager limit configuration.
resource manager System.StreamsInbound (-3) must be bigger than ConnMgr.HighWater (40)

@BigLep
Copy link
Contributor

BigLep commented Mar 12, 2023

ACK, thanks @thedmdim . We should be able to reproduce on our end by setting Swarm.ResourceMgr.MaxMemory to "259 MB". You should be able to unblock yourself by disabling the resource manager: https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmresourcemgrenabled

@Jorropo : two thoughts there:

  1. The odd thing to me is why System.StreamsInbound is -3.
  2. When we print this error message, we should link to https://github.com/ipfs/kubo/blob/master/docs/libp2p-resource-management.md

@BigLep
Copy link
Contributor

BigLep commented Mar 14, 2023

2023-03-14 conversation: @Jorropo will determine by EOD 2023-03-14 whether this should be a 0.19 release blocker or not.

@thedmdim
Copy link
Author

Thanks, new version seems works fine, i tested, uptime 1 day

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug A bug in existing code (including security flaws) P1 High: Likely tackled by core team if no one steps up
Projects
No open projects
Archived in project
Development

Successfully merging a pull request may close this issue.

4 participants