IPFS API rejects HTTP-queries with User-Agent: Mozilla but accepts User-Agent: IE and other random string

[dogada]

Checklist

• This is a bug report, not a question. Ask questions on discuss.ipfs.io.
• I have searched on the issue tracker for my bug.
• I am running the latest go-ipfs version or have an issue updating.

Installation method

third-party binary

Version

go-ipfs version: 0.10.0-64b532f
Repo version: 11
System version: amd64/linux
Golang version: go1.16.7

Config

{
  "API": {
    "HTTPHeaders": {}
  },
  "Addresses": {
    "API": "/ip4/0.0.0.0/tcp/5001",
    "Announce": [],
    "Gateway": "/ip4/0.0.0.0/tcp/8080",
    "NoAnnounce": [],
    "Swarm": [
      "/ip4/0.0.0.0/tcp/4001",
      "/ip6/::/tcp/4001",
      "/ip4/0.0.0.0/udp/4001/quic",
      "/ip6/::/udp/4001/quic"
    ]
  },
  "AutoNAT": {},
  "Bootstrap": [
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
    "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/ip4/104.131.131.82/udp/4001/quic/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
  ],
  "DNS": {
    "Resolvers": {}
  },
  "Datastore": {
    "BloomFilterSize": 0,
    "GCPeriod": "1h",
    "HashOnRead": false,
    "Spec": {
      "mounts": [
        {
          "child": {
            "path": "blocks",
            "shardFunc": "/repo/flatfs/shard/v1/next-to-last/2",
            "sync": true,
            "type": "flatfs"
          },
          "mountpoint": "/blocks",
          "prefix": "flatfs.datastore",
          "type": "measure"
        },
        {
          "child": {
            "compression": "none",
            "path": "datastore",
            "type": "levelds"
          },
          "mountpoint": "/",
          "prefix": "leveldb.datastore",
          "type": "measure"
        }
      ],
      "type": "mount"
    },
    "StorageGCWatermark": 90,
    "StorageMax": "10GB"
  },
  "Discovery": {
    "MDNS": {
      "Enabled": true,
      "Interval": 10
    }
  },
  "Experimental": {
    "AcceleratedDHTClient": false,
    "FilestoreEnabled": false,
    "GraphsyncEnabled": false,
    "Libp2pStreamMounting": false,
    "P2pHttpProxy": false,
    "ShardingEnabled": false,
    "StrategicProviding": false,
    "UrlstoreEnabled": false
  },
  "Gateway": {
    "APICommands": [],
    "HTTPHeaders": {
      "Access-Control-Allow-Headers": [
        "X-Requested-With",
        "Range",
        "User-Agent"
      ],
      "Access-Control-Allow-Methods": [
        "GET"
      ],
      "Access-Control-Allow-Origin": [
        "*"
      ]
    },
    "NoDNSLink": false,
    "NoFetch": false,
    "PathPrefixes": [],
    "PublicGateways": null,
    "RootRedirect": "",
    "Writable": false
  },
  "Identity": {
    "PeerID": "12D3KooWG7aHXhaD7Ns9aDEmmekabokzwAwK6AB4Y8LMrQgmTtZz"
  },
  "Internal": {},
  "Ipns": {
    "RecordLifetime": "",
    "RepublishPeriod": "",
    "ResolveCacheSize": 128
  },
  "Migration": {
    "DownloadSources": [],
    "Keep": ""
  },
  "Mounts": {
    "FuseAllowOther": false,
    "IPFS": "/ipfs",
    "IPNS": "/ipns"
  },
  "Peering": {
    "Peers": null
  },
  "Pinning": {
    "RemoteServices": {}
  },
  "Plugins": {
    "Plugins": null
  },
  "Provider": {
    "Strategy": ""
  },
  "Pubsub": {
    "DisableSigning": false,
    "Router": ""
  },
  "Reprovider": {
    "Interval": "12h",
    "Strategy": "all"
  },
  "Routing": {
    "Type": "dht"
  },
  "Swarm": {
    "AddrFilters": null,
    "ConnMgr": {
      "GracePeriod": "20s",
      "HighWater": 900,
      "LowWater": 600,
      "Type": "basic"
    },
    "DisableBandwidthMetrics": false,
    "DisableNatPortMap": false,
    "EnableAutoRelay": false,
    "EnableRelayHop": false,
    "Transports": {
      "Multiplexers": {},
      "Network": {},
      "Security": {}
    }
  }
}

Description

I try to connect to IPFS daemon from Chrome browser and I found that default IPFS installation rejects queries when user agent is set to 'Mozilla' but accepts if user-agent is IE or other random string? I understand when requests are blocked by CORS policy, but what the reason to block Mozilla?

Do you have any example of config that allows to accept XHR-queries from browser?

$ curl 'http://localhost:5001/api/v0/version'   -X 'POST'  -H 'User-Agent: Mozilla'
403 - Forbidden

$ curl 'http://localhost:5001/api/v0/version'   -X 'POST'  -H 'User-Agent: IE ()'
{"Version":"0.10.0","Commit":"64b532f","Repo":"11","System":"amd64/linux","Golang":"go1.16.7"}

$ curl 'http://localhost:5001/api/v0/version'   -X 'POST'  -H 'User-Agent: IE random string'
{"Version":"0.10.0","Commit":"64b532f","Repo":"11","System":"amd64/linux","Golang":"go1.16.7"}

[dogada]

Just in case bellow is full query exported from Chrome:

curl 'http://localhost:5001/api/v0/id' \
  -X 'POST' \
  -H 'Connection: keep-alive' \
  -H 'Content-Length: 0' \
  -H 'sec-ch-ua: ";Not A Brand";v="99", "Chromium";v="94"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36' \
  -H 'sec-ch-ua-platform: "Linux"' \
  -H 'Accept: */*' \
  -H 'Origin: http://localhost:3000' \
  -H 'Sec-Fetch-Site: same-site' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Referer: http://localhost:3000/' \
  -H 'Accept-Language: en-US,en;q=0.9,ru;q=0.8,uk;q=0.7' \
  --compressed

[Jorropo]

@dogada
If you are faking Firefox you should properly impersonate it :

curl 'http://localhost:5001/api/v0/version' -X 'POST'  -H 'User-Agent: Mozilla' -H "Origin: http://127.0.0.1:5001"

(see sending the Origin header)

The usage of Origin allows to filter which websites are allowed to access the API, if this wasn't in place CSRF would be trivial (you could load any random website that would execute POST requests using fetch on your IPFS API and doing bad stuff to your node).

I belive the reason to filter like that is that we don't want to force everyone (like python-ipfs-http-api or the go cli) to send Origin headers too.

If there is a bug here, it's that IE doesn't have that applied, but anyway you shouldn't use IE and I'm not actually confident that IE correctly sends Origin headers.

[Jorropo]

@dogada

Just in case bellow is full query exported from Chrome:

Origin: http://localhost:3000 isn't a trusted source for the API (note this would have been fine for the gateway with the default config)

Try with origin being the API instead :
Origin: http://localhost:5001 that will work (obviously that only fixes curl)

Add this to your config (in the root object) and restart your node, that will allows access to your API by all websites (note that unsafe so please on do that for testing, ideally you whitelist the websites you trust) :

  "API": {
    "HTTPHeaders": {
      "Access-Control-Allow-Origin": [
        "*"
      ]
    }
  },

[lidel]

I understand when requests are blocked by CORS policy, but what the reason to block Mozilla?

This is a generic test. Due to SNAFU^Wlegacy reasons, all the major browsers send requests with User-Agent string that starts with Mozilla. Even the old versions of IE do this. 🙈 When a request is sent from a browser it follows Origin-based security model, and go-ipfs-cmds expects Origin header to be present. You get 403 when it is missing.

As @Jorropo noted, when it comes to Origin isolation, the HTTP RPC API refuses requests coming from Origins other than the HTTP RPC API port. One can change this behavior by adjusting the API.HTTPHeaders.Access-Control-Allow-Origin safelist.

I'm closing this, as this works as expected.

Opened ipfs/ipfs-docs#959 to document the way HTTP RPC API uses Origin-based security in browser contexts.