Improve performance of servePretty404IfPresent in absence of a pretty 404

[iand]

Checklist

• My issue is specific & actionable.
• I am not suggesting a protocol enhancement.
• I have searched on the issue tracker for my issue.

Description

The function servePretty404IfPresent recurses a UnixFS looking for a file named ipfs-404.html at each level.

When the file does not exist, the function walks up the DAG looking for the file at each level. The default timeout (from go-path) is 1 minute for each traversal.

As an example, fetching the following request failed after 5m since it failed to fetch QmZFP53gibtpCjJAMmhAf7ugPS8hvhQiZ8b8ttMkhdqWGY/1/6/6/1661/1661-h.zip
after 1m and then spent a further 1m for each of:

• /ipfs/QmZFP53gibtpCjJAMmhAf7ugPS8hvhQiZ8b8ttMkhdqWGY/1/6/6/1661/1661-h.zip/ipfs-404.html
• /ipfs/QmZFP53gibtpCjJAMmhAf7ugPS8hvhQiZ8b8ttMkhdqWGY/1/6/6/1661/ipfs-404.html
• /ipfs/QmZFP53gibtpCjJAMmhAf7ugPS8hvhQiZ8b8ttMkhdqWGY/1/6/6/ipfs-404.html
• /ipfs/QmZFP53gibtpCjJAMmhAf7ugPS8hvhQiZ8b8ttMkhdqWGY/1/6/ipfs-404.html

All failed trying to get block QmSpnerMwvUS4ZxZGw2Dx66E63amgYAKgehxnAP8rc7fLE. But the following failed fast because (presumably) the relevant blocks were present in the local blockstore:

• /ipfs/QmZFP53gibtpCjJAMmhAf7ugPS8hvhQiZ8b8ttMkhdqWGY/1/ipfs-404.html
• /ipfs/QmZFP53gibtpCjJAMmhAf7ugPS8hvhQiZ8b8ttMkhdqWGY/ipfs-404.html

servePretty404IfPresent should remember that a block was unreachable and not include that in the recursion and should set an overall timeout for the entire traversal, not for each lookup.

[lidel]

We should revisit this after #8890 lands.