Define skip-able IPNI advertisement error signal from indexer to provider software

[LexLuthr]

Problem

Some IPNI advertisements can be bad in a way that prevents an IPNI indexer from syncing past them. Retrying does not help if the advertisement itself is malformed or references invalid data.

Today there is no standardized way for an IPNI indexer to tell provider software that a specific advertisement is permanently bad and should no longer be served. Current operational notes already call out the need for IPNI-to-Curio error signaling so Curio can return 404 for known-bad advertisements and allow problematic advertisements to be skipped in sync chains.

Goal

Define a standard signal from the IPNI indexer to provider software for advertisements that are permanently unsyncable.

For Curio, receiving this signal would mean marking that advertisement as skipped and returning 404 if the indexer requests it again.

Scope

This issue is specifically about:

• Defining which advertisement errors are safe to classify as skip-able.
• Defining how the IPNI indexer should signal that error to provider software.
• Defining the minimum information that must be included in the signal.
• Candidate skip-able errors

The IPNI team should define the authoritative list.

Possible examples:

• Advertisement CID validation failure.
• CID mismatch during reconstruction.
• Missing entries referenced by the advertisement.
• Advertisement parse failure.
• Entry/data hash mismatch.
• Malformed advertisement structure that cannot become valid through retry.
• Non-goals

This should not cover transient failures such as:

• Network timeout.
• Temporary provider unavailability.
• Slow provider response.
• Bandwidth limitation.
• Temporary HTTP failure.
• Indexer-side overload.

Those should remain retryable and must not cause an advertisement to be skipped.

Signal format to define

The spec should define the signaling mechanism. Possible choices include:

1. HTTP status code with a json with adCid and maybe what the error is.
2. Reserved error string/code prefix for LastError along with bad ad CID.

Open questions

• What exact errors are permanent and safe to skip?
• What signal format should be used?

[willscott]

there are two options for how to do this already in the current flow:

1. the request will be re-made multiple times. if you give back data that does not match the hash / cid you are asked for, it will be re-tried. You can either checksum that you are in-fact sending data that hashes to the correct thing, or can notice that htis is the n'th request by the same index for this advertisement in the last seconds and use those to identify that the ad is bad and should be skipped with the 404 error code on the next request.

2. the status is reported on https://cid.contact/providers/ as LastError - that will identify block mismatch / failure / etc.

So is the goal "make the format / metadata around LastError more precise" or is there a reason one of these two existing options is insufficient / what is the additional un-met constraint

[BigLep]

@LexLuthr : will let you respond on the above from the curio perspective when you're able.