502 Bad Gateway, 0.32.3

[lifeBCE]

• Version:
  0.32.3

• Platform:
  All (Latest FF and Chome on Mac OS X, Latest FF and Chrome on Android)

• Subsystem:

Type:

Bug

Severity:

High

Description:

WebSocket connection to 'wss://lon-1.bootstrap.libp2p.io/' failed: Error during WebSocket handshake: Unexpected response code: 502

I get hundreds of these errors in the console when the node is just running. I also get the CORS issues and the preload stuff as well in this version but I don't think this is the same issue. This happens so often I have had to modify the config of the embedded node on startup and disable certain nodes though the problem changes from node to node so I am regularly re-adding and ignoring different bootstrap nodes.

Steps to reproduce the error:

• Load js-ipfs into browser session.
• Instantiate node and start
• Watch errors flow into console

[lifeBCE]

#1476 I think may need to be reopened. This problem seems to be growing. I cannot access anything on the IPFS network this morning through my app. None of the bootstrap nodes will connect.

[alanshaw]

ping @ipfs/infrastructure any ideas about this?

I know it's not super helpful but if you have a mission critical app you should consider setting up your own bootstrap node(s).

[lifeBCE]

@alanshaw I was considering that but unsure of how well that is supported. Should I remove all default at runtime and add my own or is there a more persisted way of accomplishing this?

I currently have logic such as this in a componentDidUpdate:

      this.props.node.bootstrap.rm('/dns4/nyc-2.bootstrap.libp2p.io/tcp/443/wss/ipfs/QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64');
      this.props.node.bootstrap.rm('/dns4/lon-1.bootstrap.libp2p.io/tcp/443/wss/ipfs/QmSoLMeWqB7YGVLJN3pNLQpmmEk35v6wYtsMGLzSr5QBU3');

Should I just expand on this and manually remove all default and then manually add my won each time?

[kyledrake]

I believe I have fixed the servers with the 502 errors and have a mitigation to prevent this particular issue from happening again. I'm going to close this as resolved, but feel free to keep commenting if you need to relay more information.

[kyledrake]

Actually, I'm still seeing it on some of the machines, reopened for now.

[kyledrake]

Alright, we should be good here. Re-open if you see this again.

[lifeBCE]

Still see the following (and no connectivity):

GET https://node0.preload.ipfs.io/api/v0/refs?r=true&arg=QmQMXh5gVvfa41Wvbbjdx2A8KG6reNaNj41KzTvxHBYez6 502 (Bad Gateway)

Access to fetch at 'https://node0.preload.ipfs.io/api/v0/refs?r=true&arg=QmQMXh5gVvfa41Wvbbjdx2A8KG6reNaNj41KzTvxHBYez6' from origin 'http://localhost:3000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

I am using version 0.32.3 with default bootstrap config

After your reply here, I cleared browser cache, refreshed a few times and continue to see these errors and cannot access any data. Until this morning, I have always seen these errors but have stil been able to connect somehow.

[kyledrake]

@lifeBCE Can you run curl "https://node0.preload.ipfs.io/api/v0/refs?r=true&arg=QmQMXh5gVvfa41Wvbbjdx2A8KG6reNaNj41KzTvxHBYez6" and tell me if it says 502 bad gateway? You should not be getting a 502 for a CORS issue. That curl command works fine for me.

[kyledrake]

Ah, I see that it's failing sporadically. I'm taking a look.

[lifeBCE]

@kyledrake Running that curl yields the response list of refs without error. I have checked my production deployment and it is working great again, no errors. My local dev (localhost:3000) still has the intermittent issues but it will at least get some blocks through now. Looks like node0 will fail and node1 won't and then vise versa. Sometimes both still but all of this seems to only be in local dev.

UPDATE:
Looks like everything is good again in local dev as well. Thanks for looking into this.

[alanshaw]

Thanks for your help @kyledrake

[scammi]

Hi, I seem to have the same issue
"https://node0.preload.ipfs.io/api/v0/refs?r=true&arg=QmX6UbMVw33h37YfN6uhLLKUfdgZggceydYX2pzbTKKqu7 with CURL"

I am working with js-ipfs, creating a node and using add to push to the network.
What seems weird is that it some times goes throgh and I am able add. I been looking around with no luck.

[lifeBCE]

To anyone experiencing these issues still I would echo @alanshaw above in pointing your app at your own node for hash references. The default config for the ipfs-go node has the proper config for CORS support out of the box.

import IPFS from 'ipfs';

const options = {
  preload: {
    enabled: true,
    addresses: [ '/dnsaddr/ipfs.domain.com/https' ],
  }
}

const IPFSNode = new IPFS(options);

This is how I accomplished this. I could not find it covered very well in any docs so I went to the code and found that you can pass the constructor an options hash with preload config. Your node's API interface (default 5001) will need to be accessible.

Also, any time you make any changes to the IPFS package/instance config, blow away your browser cache. This includes adding or removing bootstrap nodes as well.

[scammi]

I implemented your idea of passing options to the constructor, but allowing CORS:

 const IPFS = require('ipfs')

    const options = {
     "API": {
        "HTTPHeaders": {
            "Access-Control-Allow-Origin": [
                "*"
            ],
            "Access-Control-Allow-Methods": [
                "GET",
                "POST"
            ],
            "Access-Control-Allow-Headers": [
                "Authorization"
            ],
            "Access-Control-Expose-Headers": [
                "Location"
            ],
            "Access-Control-Allow-Credentials": [
                "true"
            ]
        }
     }
    }
    const node = new IPFS(options);

Thank you for helping me, I am very new and found it very satisfiying to fix the issue.

[RotonEvan]

To anyone experiencing these issues still I would echo @alanshaw above in pointing your app at your own node for hash references. The default config for the ipfs-go node has the proper config for CORS support out of the box.

import IPFS from 'ipfs';

const options = {
  preload: {
    enabled: true,
    addresses: [ '/dnsaddr/ipfs.domain.com/https' ],
  }
}

const IPFSNode = new IPFS(options);

This is how I accomplished this. I could not find it covered very well in any docs so I went to the code and found that you can pass the constructor an options hash with preload config. Your node's API interface (default 5001) will need to be accessible.

Also, any time you make any changes to the IPFS package/instance config, blow away your browser cache. This includes adding or removing bootstrap nodes as well.

Are there any free or paid preload node services available ? We are building some prototype and need a quick solution