Missing files don't result in a manifest being considered invalid

[job]

In a MITM scenario where an attacker intercepts and manipulates the rsync channel (for example strategically withholding certain .roa files from the view of the validator being attacked), the resulting set of VRPs will be incomplete and can cause severe operational issues. (Note that a MITM can trivially downgrade RPKI fetches from RRDP to rsync)

When a manifest is valid (manifest is parsable, CRL exists is valid (also not expired), and manifest is signed with keys not revoked by the CRL), and references files which do not exist in the repository at hand, the publication point should be considered compromised.

So in the case of APNIC where an End User (self-hosted) RPKI publication point misses a few .roa files, the validator can proceed to consider all data from all RPs it could reach eligeble for further validation, except any data from the publication point where files were missing.

In other words: if one or a few files are missing from the repository, the repository should be considered 'down', and no attempt should be made to start guessing what can be salvaged and what not.

[job]

To offer an example:

In manifest 1-tcQDnftkRnWbiMhu2cR1-dgmCs.mft, a number of ROAs and a CRL are referenced:

1-tcQDnftkRnWbiMhu2cR1-dgmCs.crl
IM3xMiMU1QChuWMOGgnbDwYolrU.roa
LkKeUPYrfgzjsOIejLjsHGk44cU.roa
LkKeUPYrfgzjsOIejLjsHGk44cU.roa
fIeCcC8KpdJQd-olU2APdxNZkyA.roa
r5vtD4isKhTzGXNGulSnZ7XCS04.roa
r7TSyWn_GbYPjNWvt4r5ewSNAsk.roa

If an attacker withholds all but LkKeUPYrfgzjsOIejLjsHGk44cU.roa, so the MITM does the equivalent of:

cd repository/DEFAULT/3e/01d411-d915-4277-8fe2-76b0dda2bf3e/1/
rm LkKeUPYrfgzjsOIejLjsHGk44cU.roa \
    fIeCcC8KpdJQd-olU2APdxNZkyA.roa \
    IM3xMiMU1QChuWMOGgnbDwYolrU.roa \
    NpI_Uj3OZb6jvwfUTX0-eOk9QV4.roa

and the validator does not consider 1-tcQDnftkRnWbiMhu2cR1-dgmCs.mft in its entirety invalid due to one or more missing .roa or .crl files, in this specific example the remaining ROA will render all BGP announcements equal to or covered under 80.128.0.0/11 invalid.

If the manifest (which references a missing .roa file in its entirety is ignored or considered invalid), the remaining r7TSyWn_GbYPjNWvt4r5ewSNAsk.roa ROA which would've been transformed into VRP 80.128.0.0/11 AS 0 is also ignored. This renders all BGP announcements under that /11 not-found rather than invalid, which is a preferable situation.

rpki-client now has a patch where if a file is referenced the manifest, but is missing (either due to CA operational error, or because of a MITM attack - we can't differentiate the two), that specific manifest is considered invalid. It will depend on the structure of how CA Certificate trees whether this measure is sufficient or whether a single missing file has to result in the entire publication point being considered invalid.

But until we have more information about other attack vectors, I recommend that a manifest as a whole is considered invalid, when it references a non-existing or wrongly checksummed file. The onus is on the CA publication points to publish correct valid RPKI data, the validator's can't be expected to compensate for CA operator errors (or MITM attacks)

[job]

An additional check is needed: if one of the .roa files a manifest references is corrupt (the MITM didn't delete strategically file, but filled some .roa files with garbage), the fix no longer works:

rpki-client:  ...trace: error:0DFFF08E:asn1 encoding routines:CRYPTO_internal:not enough data
rpki-client: rpki.ripe.net/repository/DEFAULT/3e/01d411-d915-4277-8fe2-76b0dda2bf3e/1/fIeCcC8KpdJQd-olU2APdxNZkyA.roa: RFC 6488: failed CMS parse

corrupting that one .roa file results in an incomplete (operationally problematic) set of VRPs:

job@anton ~$ grepcidr 80.128.0.0/11 /var/db/rpki-client/openbgpd
	80.128.0.0/12 source-as 3320
	80.128.0.0/11 source-as 0
	80.128.0.0/11 source-as 3320
	80.144.0.0/13 source-as 3320
	80.152.0.0/14 source-as 3320
	80.156.0.0/16 source-as 3320
	80.157.0.0/16 source-as 3320
	80.157.8.0/21 source-as 3320
	80.157.16.0/20 source-as 3320
	80.158.0.0/17 maxlen 24 source-as 34086
	80.159.224.0/19 maxlen 24 source-as 2792

In the above eaxmple there should be ~ 19 VRPs in total, fIeCcC8KpdJQd-olU2APdxNZkyA.roa contained 8 VRPs.

In summary:

If a manifests references a non-existing file, or if there is a checksum mismatch between the hash described in the manifest and the hash as derived from the .roa file, the RP MUST consider the whole manifest invalid and not produce VRPs with the remaining .roa files.

[mahtin]

In the case of a non-existing file in the manifest: What about race conditions caused by rsync/rrdp transfers that are ill-timed? Does some hysteresis need to be coded into the system? Or maybe at-least a quick retry (once!) from the source just in case a file is missed/not-ready.

[job]

What I describe here is about the set of files that goes into the RP's validation process. This is after potential re-tries a RP fetcher thread may attempt to overcome connectivity issues. Example: connection didn't work on IPv4, retry on IPv6. That's fine.

The onus is on on the CA operator to publish a coherent valid manifest for a given set of files. From the way the RPKI data is structured CA software will first generate the .roa files and then generate the manifest (often inside the same process?), so this wouldn't be too much to require.

The use of rsync in RPKI has a few downsides such as the lack of directory-wide atomicity. RRDP's structure addresses some of those shortcomings. In both cases we can't differentiate between a broken CA and a MITM attack though, so whatever way is used to fetch the data (with or without retries), the validation should probably follow this decision tree:

1. are all files referenced in a given manifest present?
2. do the hashes as listed inside the manifest match the contents of the referenced files?
3. is the CRL present and valid?
4. are all of the relevant signatures from not-revoked keys?

If the answer to any of the above questions is 'no', I think the leaf of the tree at hand should be considered invalid to prevent tainting the RFC 6811 validation procedure with incomplete data.

[lspgn]

Just an update: per discussion: as it was now .
Will implement the behavior: if a file is missing/invalid from the manifest: drop the repository+path. If missing CRL: consider the certificate invalid.

Idea from rpki-client: make sure only the files validated are present in the "folder" (delete from rsync).

[job]

This was fixed through f34f046

[mahtin]

@job - Thank you very much for updating http://sobornost.net/~job/manifest_handling_issue.txt to reflect this.

@lspgn - Do you need to EOL the 1.x.x releases on https://github.com/cloudflare/cfrpki/releases ?