Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Occasionally, the connection will be maintained when /api/v0/add?pin=true&to-files=/xxxxx is called. After forced disconnection, the file content can be found based on the CID, but there is no file in the specified path(MFS). #10224

Open
3 tasks done
daxionga opened this issue Nov 17, 2023 · 4 comments
Labels
kind/bug A bug in existing code (including security flaws) need/triage Needs initial labeling and prioritization

Comments

@daxionga
Copy link

Checklist

Installation method

built from source

Version

ubo version: 0.24.0-e70db65
Repo version: 15
System version: amd64/linux
Golang version: go1.21.4

Config

{
  "API": {
    "HTTPHeaders": {
      "Access-Control-Allow-Methods": [
        "PUT",
        "POST",
        "GET"
      ],
      "Access-Control-Allow-Origin": [
        "*"
      ]
    }
  },
  "Addresses": {
    "API": "/ip4/0.0.0.0/tcp/5001",
    "Announce": null,
    "AppendAnnounce": null,
    "Gateway": "/ip4/0.0.0.0/tcp/8080",
    "NoAnnounce": null,
    "Swarm": [
      "/ip4/0.0.0.0/tcp/4001",
      "/ip6/::/tcp/4001",
      "/ip4/0.0.0.0/udp/4001/quic-v1",
      "/ip4/0.0.0.0/udp/4001/quic-v1/webtransport",
      "/ip6/::/udp/4001/quic-v1",
      "/ip6/::/udp/4001/quic-v1/webtransport"
    ]
  },
  "AutoNAT": {},
  "Bootstrap": [
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
    "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/ip4/104.131.131.82/udp/4001/quic-v1/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
  ],
  "DNS": {
    "Resolvers": {}
  },
  "Datastore": {
    "BloomFilterSize": 0,
    "GCPeriod": "1h",
    "HashOnRead": false,
    "Spec": {
      "mounts": [
        {
          "child": {
            "path": "blocks",
            "shardFunc": "/repo/flatfs/shard/v1/next-to-last/2",
            "sync": true,
            "type": "flatfs"
          },
          "mountpoint": "/blocks",
          "prefix": "flatfs.datastore",
          "type": "measure"
        },
        {
          "child": {
            "compression": "none",
            "path": "datastore",
            "type": "levelds"
          },
          "mountpoint": "/",
          "prefix": "leveldb.datastore",
          "type": "measure"
        }
      ],
      "type": "mount"
    },
    "StorageGCWatermark": 90,
    "StorageMax": "400GB"
  },
  "Discovery": {
    "MDNS": {
      "Enabled": true
    }
  },
  "Experimental": {
    "FilestoreEnabled": false,
    "GraphsyncEnabled": false,
    "Libp2pStreamMounting": false,
    "P2pHttpProxy": false,
    "StrategicProviding": false,
    "UrlstoreEnabled": false
  },
  "Gateway": {
    "APICommands": [],
    "HTTPHeaders": {},
    "NoDNSLink": false,
    "NoFetch": false,
    "PathPrefixes": [],
    "PublicGateways": null,
    "RootRedirect": "",
    "Writable": false
  },
  "Identity": {
    "PeerID": "12D3KooWCJhbS8CGP7ZyYHv121pke3285py4CjgY6B3xo2SaFUxM"
  },
  "Internal": {},
  "Ipns": {
    "RecordLifetime": "",
    "RepublishPeriod": "",
    "ResolveCacheSize": 128
  },
  "Migration": {
    "DownloadSources": [],
    "Keep": ""
  },
  "Mounts": {
    "FuseAllowOther": false,
    "IPFS": "/ipfs",
    "IPNS": "/ipns"
  },
  "Peering": {
    "Peers": [
      {
        "Addrs": [
          "/dnsaddr/node-1.ingress.cloudflare-ipfs.com"
        ],
        "ID": "QmcFf2FH3CEgTNHeMRGhN7HNHU1EXAxoEk6EFuSyXCsvRE"
      },
      {
        "Addrs": [
          "/dnsaddr/node-2.ingress.cloudflare-ipfs.com"
        ],
        "ID": "QmcFmLd5ySfk2WZuJ1mfSWLDjdmHZq7rSAua4GoeSQfs1z"
      },
      {
        "Addrs": [
          "/dnsaddr/node-3.ingress.cloudflare-ipfs.com"
        ],
        "ID": "QmcfFmzSDVbwexQ9Au2pt5YEXHK5xajwgaU6PpkbLWerMa"
      },
      {
        "Addrs": [
          "/dnsaddr/node-4.ingress.cloudflare-ipfs.com"
        ],
        "ID": "QmcfJeB3Js1FG7T8YaZATEiaHqNKVdQfybYYkbT1knUswx"
      },
      {
        "Addrs": [
          "/dnsaddr/node-5.ingress.cloudflare-ipfs.com"
        ],
        "ID": "QmcfVvzK4tMdFmpJjEKDUoqRgP4W9FnmJoziYX5GXJJ8eZ"
      },
      {
        "Addrs": [
          "/dnsaddr/node-6.ingress.cloudflare-ipfs.com"
        ],
        "ID": "QmcfZD3VKrUxyP9BbyUnZDpbqDnT7cQ4WjPP8TRLXaoE7G"
      },
      {
        "Addrs": [
          "/dnsaddr/node-7.ingress.cloudflare-ipfs.com"
        ],
        "ID": "QmcfZP2LuW4jxviTeG8fi28qjnZScACb8PEgHAc17ZEri3"
      },
      {
        "Addrs": [
          "/dnsaddr/node-8.ingress.cloudflare-ipfs.com"
        ],
        "ID": "QmcfgsJsMtx6qJb74akCw1M24X1zFwgGo11h1cuhwQjtJP"
      },
      {
        "Addrs": [
          "/dnsaddr/node-9.ingress.cloudflare-ipfs.com"
        ],
        "ID": "Qmcfr2FC7pFzJbTSDfYaSy1J8Uuy8ccGLeLyqJCKJvTHMi"
      },
      {
        "Addrs": [
          "/dnsaddr/node-10.ingress.cloudflare-ipfs.com"
        ],
        "ID": "QmcfR3V5YAtHBzxVACWCzXTt26SyEkxdwhGJ6875A8BuWx"
      },
      {
        "Addrs": [
          "/dnsaddr/node-11.ingress.cloudflare-ipfs.com"
        ],
        "ID": "Qmcfuo1TM9uUiJp6dTbm915Rf1aTqm3a3dnmCdDQLHgvL5"
      },
      {
        "Addrs": [
          "/dnsaddr/node-12.ingress.cloudflare-ipfs.com"
        ],
        "ID": "QmcfV2sg9zaq7UUHVCGuSvT2M2rnLBAPsiE79vVyK3Cuev"
      },
      {
        "Addrs": [
          "/ip4/139.178.68.217/tcp/6744"
        ],
        "ID": "12D3KooWCVXs8P7iq6ao4XhfAmKWrEeuKFWCJgqe9jGDMTqHYBjw"
      },
      {
        "Addrs": [
          "/ip4/147.75.49.71/tcp/6745"
        ],
        "ID": "12D3KooWGBWx9gyUFTVQcKMTenQMSyE2ad9m7c9fpjS4NMjoDien"
      },
      {
        "Addrs": [
          "/ip4/147.75.86.255/tcp/6745"
        ],
        "ID": "12D3KooWFrnuj5o3tx4fGD2ZVJRyDqTdzGnU3XYXmBbWbc8Hs8Nd"
      },
      {
        "Addrs": [
          "/ip4/3.134.223.177/tcp/6745"
        ],
        "ID": "12D3KooWN8vAoGd6eurUSidcpLYguQiGZwt4eVgDvbgaS7kiGTup"
      },
      {
        "Addrs": [
          "/ip4/35.74.45.12/udp/6746/quic"
        ],
        "ID": "12D3KooWLV128pddyvoG6NBvoZw7sSrgpMTPtjnpu3mSmENqhtL7"
      },
      {
        "Addrs": [
          "/dnsaddr/fra1-1.hostnodes.pinata.cloud"
        ],
        "ID": "QmWaik1eJcGHq1ybTWe7sezRfqKNcDRNkeBaLnGwQJz1Cj"
      },
      {
        "Addrs": [
          "/dnsaddr/fra1-2.hostnodes.pinata.cloud"
        ],
        "ID": "QmNfpLrQQZr5Ns9FAJKpyzgnDL2GgC6xBug1yUZozKFgu4"
      },
      {
        "Addrs": [
          "/dnsaddr/fra1-3.hostnodes.pinata.cloud"
        ],
        "ID": "QmPo1ygpngghu5it8u4Mr3ym6SEU2Wp2wA66Z91Y1S1g29"
      },
      {
        "Addrs": [
          "/dnsaddr/nyc1-1.hostnodes.pinata.cloud"
        ],
        "ID": "QmRjLSisUCHVpFa5ELVvX3qVPfdxajxWJEHs9kN3EcxAW6"
      },
      {
        "Addrs": [
          "/dnsaddr/nyc1-2.hostnodes.pinata.cloud"
        ],
        "ID": "QmPySsdmbczdZYBpbi2oq2WMJ8ErbfxtkG8Mo192UHkfGP"
      },
      {
        "Addrs": [
          "/dnsaddr/nyc1-3.hostnodes.pinata.cloud"
        ],
        "ID": "QmSarArpxemsPESa6FNkmuu9iSE1QWqPX2R3Aw6f5jq4D5"
      },
      {
        "Addrs": [
          "/dns/cluster0.fsn.dwebops.pub"
        ],
        "ID": "QmUEMvxS2e7iDrereVYc5SWPauXPyNwxcy9BXZrC1QTcHE"
      },
      {
        "Addrs": [
          "/dns/cluster1.fsn.dwebops.pub"
        ],
        "ID": "QmNSYxZAiJHeLdkBg38roksAR9So7Y5eojks1yjEcUtZ7i"
      },
      {
        "Addrs": [
          "/dns/cluster2.fsn.dwebops.pub"
        ],
        "ID": "QmUd6zHcbkbcs7SMxwLs48qZVX3vpcM8errYS7xEczwRMA"
      },
      {
        "Addrs": [
          "/dns/cluster3.fsn.dwebops.pub"
        ],
        "ID": "QmbVWZQhCGrS7DhgLqWbgvdmKN7JueKCREVanfnVpgyq8x"
      },
      {
        "Addrs": [
          "/dns/cluster4.fsn.dwebops.pub"
        ],
        "ID": "QmdnXwLrC8p1ueiq2Qya8joNvk3TVVDAut7PrikmZwubtR"
      },
      {
        "Addrs": [
          "/dns4/nft-storage-am6.nft.dwebops.net/tcp/18402"
        ],
        "ID": "12D3KooWCRscMgHgEo3ojm8ovzheydpvTEqsDtq7Vby38cMHrYjt"
      },
      {
        "Addrs": [
          "/dns4/nft-storage-dc13.nft.dwebops.net/tcp/18402"
        ],
        "ID": "12D3KooWQtpvNvUYFzAo1cRYkydgk15JrMSHp6B6oujqgYSnvsVm"
      },
      {
        "Addrs": [
          "/dns4/nft-storage-sv15.nft.dwebops.net/tcp/18402"
        ],
        "ID": "12D3KooWQcgCwNCTYkyLXXQSZuL5ry1TzpM8PRe9dKddfsk1BxXZ"
      },
      {
        "Addrs": [
          "/ip4/5.161.92.43/tcp/4001",
          "/ip4/5.161.92.43/udp/4001/quic",
          "/ip6/2a01:4ff:f0:3b1e::1/tcp/4001",
          "/ip6/2a01:4ff:f0:3b1e::1/udp/4001/quic"
        ],
        "ID": "12D3KooWFFhc8fPYnQXdWBCowxSV21EFYin3rU27p3NVgSMjN41k"
      },
      {
        "Addrs": [
          "/ip4/5.161.55.227/tcp/4001",
          "/ip4/5.161.55.227/udp/4001/quic",
          "/ip6/2a01:4ff:f0:1e5a::1/tcp/4001",
          "/ip6/2a01:4ff:f0:1e5a::1/udp/4001/quic"
        ],
        "ID": "12D3KooWSW4hoHmDXmY5rW7nCi9XmGTy3foFt72u86jNP53LTNBJ"
      },
      {
        "Addrs": [
          "/ip4/5.161.92.36/tcp/4001",
          "/ip4/5.161.92.36/udp/4001/quic",
          "/ip6/2a01:4ff:f0:3764::1/tcp/4001",
          "/ip6/2a01:4ff:f0:3764::1/udp/4001/quic"
        ],
        "ID": "12D3KooWSDj6JM2JmoHwE9AUUwqAFUEg9ndd3pMA8aF2bkYckZfo"
      },
      {
        "Addrs": [
          "/ip4/104.210.43.77"
        ],
        "ID": "QmR69wtWUMm1TWnmuD4JqC1TWLZcc8iR2KrTenfZZbiztd"
      },
      {
        "Addrs": [
          "/ip4/78.46.108.24"
        ],
        "ID": "12D3KooWGASC2jm3pmohEJXUhuStkxDitPgzvs4qMuFPaiD9x1BA"
      },
      {
        "Addrs": [
          "/ip4/65.109.19.136"
        ],
        "ID": "12D3KooWRbWZN3GvLf9CHmozq4vnTzDD4EEoiqtRJxg5FV6Gfjmm"
      }
    ]
  },
  "Pinning": {
    "RemoteServices": {}
  },
  "Plugins": {
    "Plugins": null
  },
  "Provider": {
    "Strategy": ""
  },
  "Pubsub": {
    "DisableSigning": false,
    "Router": ""
  },
  "Reprovider": {
    "Interval": "12h",
    "Strategy": "roots"
  },
  "Routing": {
    "AcceleratedDHTClient": false,
    "Routers": null
  },
  "Swarm": {
    "AddrFilters": null,
    "ConnMgr": {},
    "DisableBandwidthMetrics": false,
    "DisableNatPortMap": false,
    "RelayClient": {},
    "RelayService": {},
    "ResourceMgr": {
      "Limits": {}
    },
    "Transports": {
      "Multiplexers": {},
      "Network": {},
      "Security": {}
    }
  }
}

Description

image
Occasionally, the connection may not be disconnected After returning cid,and cannot be found in the path specified in MFS,But it can be found on the gateway through cid.
image

@daxionga daxionga added kind/bug A bug in existing code (including security flaws) need/triage Needs initial labeling and prioritization labels Nov 17, 2023
@Jorropo
Copy link
Contributor

Jorropo commented Nov 18, 2023

Does curl ever return ? If curl is left hanging we probably don't have commited anything in the database.

@daxionga
Copy link
Author

Does curl ever return ? If curl is left hanging we probably don't have commited anything in the database.

Not only curl, but also java's okhttp SDK. Others may also have this situation, but I didn't test it at the time. This situation only happens occasionally, and it doesn't always happen. Under normal circumstances, curl will not keep hanging.You can use your own IPFS node to test it with curl
image

@Jorropo
Copy link
Contributor

Jorropo commented Nov 18, 2023

I can't copy paste from screenshots can you put text please ?

@daxionga
Copy link
Author

daxionga commented Nov 18, 2023

I can't copy paste from screenshots can you put text please ?

# Create MFS path
curl 'http://xxx.xxx.xxx.xxx:5001/api/v0/files/mkdir?arg=/test0915 -X POST  -H 'content-type:application/json'
# create local file 
echo  'adsafdasfndjfnjksfbkewjbnfkejwnf' > test
# upload
curl 'http://xxx.xxx.xxx.xxx:5001/api/v0/add?pin=true&to-files=/test0915' -X POST -F 'file=@"test";filename="test5"
# create another local file 
echo  'adsafdasfndjfnjksfbkewjbnfkejwnf' > test01
# upload
curl 'http://xxx.xxx.xxx.xxx:5001/api/v0/add?pin=true&to-files=/test0915' -X POST -F 'file=@"test01";filename="test501"
# and then sometime upload will be hung

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug A bug in existing code (including security flaws) need/triage Needs initial labeling and prioritization
Projects
None yet
Development

No branches or pull requests

2 participants