WIP: add new version of intoto type

[bobcallaway]

• move tests out of e2e dir and into pkg-specific version
• add test coverage

Summary

This adds v0.0.3 of the intoto type, aiming to address a few issues in v0.0.2:

• Signatures in DSSE envelopes for intoto/v0.0.2 are double encoded #1139
• 502 error when posting intoto 0.0.2 entry w/o hash #1164
• ensure that index keys and digests are computed only off of validated inputs
• more cleanly separating the schema between what should be proposed VS what is returned from the log

Release Note

Adds new default version of intoto type

FYI @pxp928 @bdehamer

[asraa]

sorry, quick comment before I dive in: wdyt about renaming to a dsse type? i was talking with @laurentsimon and had a huge amount of confusion myself about what content/payloadHash referred to, and if it's a dsse type it could be better named as predicate/envelopeHash

I think this came up in an issue before but I can't find a reference.

[bobcallaway]

sorry, quick comment before I dive in: wdyt about renaming to a dsse type? i was talking with @laurentsimon and had a huge amount of confusion myself about what content/payloadHash referred to, and if it's a dsse type it could be better named as predicate/envelopeHash

I think this came up in an issue before but I can't find a reference.

In

rekor/pkg/types/intoto/v0.0.3/intoto_v0_0_3_schema.json

Line 12 in 224afca

"description": "DSSE envelope specified as a stringified JSON object; payloadType in the envelope MUST be set to 'application/vnd.in-toto+json'",

I tried to make this clear that this would only accept DSSE envelopes that are signed intoto attestations.

I would presume that a more generically named dsse type could refer to any payload, but this type has been specific to intoto attestations?

[kommendorkapten]

My two cents:

I think it would be a bit cleaner to have the type referring to the outer envelope, e.g. dsse. Then after the envelope is parsed, we can extract certain fields based on a set of payload types Rekor recognizes, such as in-toto statements (and the predicate type for build provenances like it is today). This is e.g. how cose does it:

rekor/pkg/types/cose/v0.0.1/entry.go

Line 115 in fd8e054

if ok && contentType == in_toto.PayloadType {

.

The risk with this is of course that there can be a combinatorial explosion of "subtypes" in the dsse type, but with proper structure of the code I think it should be fine.
I guess with this way, it would be easier for a consumer to use.

To start with we may only recognize in-toto statements, but any dsse enevelope could be uploaded.

[asraa]

Just some very minor comments, but overall this is so nicely organized, multiple signatures are covered, and so on.

[asraa]

Because this is incoming proposed content with canonicalized fields, shouldn't verifyEnvelope still apply here for robustness?

What if a user proposed a ProposedEntry with pre-canonicalized fields, let's say, because it wants to hide the Envelope content? It should still be verifiable, assuming verification over a hash.

But if it's clear that users MUST be publishing full envelope content to the log, fine with me.

[asraa]

Only hesitation here is that I am wondering if it's possible for a client to be setting the EnvelopeHash, PayloadHash, and Signatures of a bogus signed envelope and bypassing validation (not through the CLI, but through setting a proposed entry in the createentry handler)