Incorporate status checking into verification algorithm

[msporny]

This PR incorporates status checking into the verification algorithm as requested by @jandrieu. This PR is being provided for discussion. I'm on the fence wrt. whether the PR helps or harms as it is wandering into the realm of "VC validation", which we've not been specifying normatively, to date. That said, the algorithm is generic enough that I presume any production verifier implementation has to perform these steps at some point.

---

Preview | Diff

[iherman]

A minor editorial(?) issue: there should be a proper reference to the status list specification in the document. At present, there is a link, but the document does not appear in the references' section.

The algorithm refers to the status list document as a "for example", so maybe, but only maybe, it is all right to refer to it informatively. (But I am not sure.)

[msporny]

A minor editorial(?) issue: there should be a proper reference to the status list specification in the document. At present, there is a link, but the document does not appear in the references' section.

We have gotten objections from @selfissued and @OR13 about referencing those specifications in any way in the algorithms section. I believe @jyasskin wants us to reference specifications that are concrete implementations of the rest of the algorithms we expect implementers to implement here.

IOW, we have conflicting desires on referencing other specifications that the WG is working on from VC Data Model.

The algorithm refers to the status list document as a "for example", so maybe, but only maybe, it is all right to refer to it informatively. (But I am not sure.)

The algorithm refers to the "Status" section in the VC-SPECS directory as a "for example" :) -- and I did this to avoid objections from @selfissued and @OR13. I expect that if we remove all references to external specifications that @jyasskin will become (rightly) concerned.

My position as an Editor is that it's perfectly fine to do "For example"s, and then provide non-normative references to any specification that the WG is delivering, including VC-SPECS directory. It's also fine to normatively refer to specifications that this WG is working on. IOW, I don't see a strong basis for the objections from @OR13 and @selfissued wrt. referencing specifications that the WG is working from VCDM. Both of them just need to make it clear what they can live with.

[iherman]

A minor editorial(?) issue: there should be a proper reference to the status list specification in the document. At present, there is a link, but the document does not appear in the references' section.

We have gotten objections from @selfissued and @OR13 about referencing those specifications in any way in the algorithms section. I believe @jyasskin wants us to reference specifications that are concrete implementations of the rest of the algorithms we expect implementers to implement here.

IOW, we have conflicting desires on referencing other specifications that the WG is working on from VC Data Model.

The algorithm refers to the status list document as a "for example", so maybe, but only maybe, it is all right to refer to it informatively. (But I am not sure.)

The algorithm refers to the "Status" section in the VC-SPECS directory as a "for example" :) -- and I did this to avoid objections from @selfissued and @OR13. I expect that if we remove all references to external specifications that @jyasskin will become (rightly) concerned.

It is fine to keep at as informative. But then it should be properly referenced, through a [....] reference and an entry in the references' section. At the moment, we only have a reference to the document through a link in the text itself.

My position as an Editor is that it's perfectly fine to do "For example"s, and then provide non-normative references to any specification that the WG is delivering, including VC-SPECS directory.

I agree.

It's also fine to normatively refer to specifications that this WG is working on. IOW, I don't see a strong basis for the objections from @OR13 and @selfissued wrt. referencing specifications that the WG is working from VCDM. Both of them just need to make it clear what they can live with.

[andresuribe87]

I'm of the opinion that we should update the definition of verfication in the terminology section by removing the "and current" descriptor, so that it becomes the following: "The evaluation of whether a verifiable credential or verifiable presentation is an authentic statement of the issuer or presenter, respectively."

This is for a couple of reasons:

1. The term "cryptographic verification" has been around for a long time, and to my knowledge it never includes "currency". I believe most people interpret the "verifiable" part in the term "verifiable credentials" as being around authenticity via cryptography.
2. Our definition of "verification" is adding an additional property. This will cause misuse in the industry. At best, the currency checks will be performed twice (during verification and then during validation). At worst, a consumer using a non-conformant implementation will assume that verification checks for currency, which may not happen at all.

@jandrieu has mentioned that it's not what has been communicated broadly in the use-cases document. I think we'd be better off correcting existing documents to reflect a mental model where we meet people where they are at.

[OR13]

Prefer not to have this reference here, unless W3C is planning to curate the registered status mechanisms.

[jyasskin]

I believe the IETF's experience has been that it's valuable to use a registry to get people to list all the terms they use, and that it's good to reduce the amount of curation the registry requires in order to make that happen. If VC-SPECS becomes a registry you can use either the IETF's Specification Required policy or their First Come First Served policy, which don't require the custodian to make any decisions about how "good" one of the specs is.

(This PR probably isn't the place to discuss exactly how the registry is set up, but I hope to convince @OR13 that it's ok to cite a list of status types without the group committing to curating the list.)

[OR13]

do we need "statusPurpose" or is "type" what gives that predicate meanings?

[OR13]

status checking is "validation" not "verification"... status checking happens after verification in safe apis.

[longpd]

I support the usefulness of referencing other specs from the VCDM as per Manu’s comment:

My position as an Editor is that it's perfectly fine to do "For example"s, and then provide non-normative references to any specification that the WG is delivering, including VC-SPECS directory. It's also fine to normatively refer to specifications that this WG is working on. IOW, I don't see a strong basis for the objections from @OR13 and @selfissued wrt. referencing specifications that the WG is working from VCDM. Both of them just need to make it clear what they can live with.

I also concur with Or13 that

status checking is "validation" not "verification"... status checking happens after verification in safe apis.

Validation is the about the business logic that a verifier cares about not the syntactical conformance of the expressions in the data model, IMHO.

[longpd]

Manu: I prefer that checks of things that involve business logic are separate from things that involve conformance with syntax specifications. But that may be a minority view here. Hence, I have concerns about referring to credentials being “verified as current”. It has been my understanding that including an expiry, for example in a VC, should have result in the verification checking for syntactic conformance and leave the credential interpretation of the semantic meaning of expiry (is it current or has expired) left for the validation process. I’ve commented as such in #1369. I could be convinced otherwise but at present I don’t see status checks as a role verification plays. And since validation is and should be done by relying parties, I see nothing that weakens the credential for a validation to determine status. Am I missing something?PhilPhillip D. Long, Ph.D. pers e: ***@***.***: http://www.linkedin.com/in/longpdCharlottesville, VA 22902On Dec 3, 2023, at 2:51 PM, Manu Sporny ***@***.***> wrote: @msporny requested your review on: #1369 Incorporate status checking into verification algorithm. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because your review was requested.Message ID: ***@***.***>

[dlongley]

We might be able to get to some consensus around "status checking" during verification if it only includes "determining the authentic, current status of the credential" -- but not making a decision about what to do with that information, just returning it.

[TallTed]

"determining the authentic, current status of the credential" blurs those lines. These should be distinct:

1. Determine authenticity
2. Check currency/validity-dates

I strongly believe authenticity (e.g., { ex:joe ex:issued ex:vc ; ex:vc ex:holds ex:claims }) is distinct from expiration/currency (e.g., { ex:vc ex:expires "YYYYMMDD"^^xsd:date }).

The former is what VCs are all about — undeniable, irrefutable, THEY said THIS about THAT.

The latter may be important to some verifiers, but exactly what it means to them (content of the VC is rejected? content of the VC is usable for X but not for Y? etc.) absolutely falls into their business logic.

[dlongley]

@TallTed,

"determining the authentic, current status of the credential" blurs those lines.

I hope this helps (but can't be sure), but what I meant was:

"Obtaining the authentic status information asserted by the issuer."

This information is expected, in some cases (certainly for "BitStringStatusList"), to be externalized from the VC itself, and requires an additional step to go fetch and ensure it is authentic. The act of fetching and checking the authenticity of the returned information is what I meant could be considered part of "verification". What is done with that information would be "validation".

[iherman]

The issue was discussed in a meeting on 2023-12-05

• no resolutions were taken

View the transcript

1.2. Incorporate status checking into verification algorithm (pr vc-data-model#1369)

See github pull request vc-data-model#1369.

Brent Zundel: incorporate status checking into the verification algorithm.
… this was in part in response to a threatened FO from Google.

Manu Sporny: I think Joe requested this.
… Andres also had some good things to say.
… It's an extension to the existing algorithm, but there is quite a bit of conversation.
… maybe some of this should be under verification.
… It does pull us into validation land, more-so than other algorithms.

Brent Zundel: this one is not as clear cut. one approval one request for changes.
… please indicate your approval or commentary.

Manu Sporny: I will note that Orie is objecting because this is about validation not verification.
… but elsewhere Orie has said that verifiers are doing validation, so the waters have gotten muddied.
… It's safe to say that some of the things we are doing is debatable as validation.
… If you're reading "validation" it may not mean what we've traditionally meant.
… That makes it a bit challenging to read.
… Likely closed in the week without further support.
… Andres, could you vocalize your comment?

Manu Sporny: #1369 (comment).

Andres Uribe: We have two terms we are defining, verification (securing and currency).
… Does the issuer stand by what they said?
… I think we should modify and update that term so that it does not include currency.
… because cryptographic verification is well understood in the industry and does not include currency.
… So, I think it would be better to remove currency from verification.

Manu Sporny: FWIW, I agree with andres' take on what we could do to make this cleaner in the spec.

Ted Thibodeau Jr.: business logic!

Joe Andrieu: It's a compelling argument, we are not just doing cryptographic securing, we're doing semantic securing. If we were just cryptographically securing, we'd just use a JWT. We are doing more here... knowing issuer stands by statement is in nature of whether or not statement is attributable to the author. That is different from timeliness about expiry. That the issuer is committed to this is what we're doing that's different than other cryptographic based mechanisms for verification.

Ted Thibodeau Jr.: VCs are defined to say anything about the issuer standing by the statement.
… rather "they said this about Pat".
… forcing them together is not going to help.

Brent Zundel: TallTed's definition matches my definition.
… that the VC is something said at some point.
… the status property might not even be there.
… so that doesn't seem wholly involved so far.
… otherwise, wouldn't we have made status a require.

Joe Andrieu: What VCs say are not about "that they said it" that "they DO say it" -- present tense -- that's important. Verifier is in present tense, am I going to rely on the information, but if issuer no longer stands by it, why would I rely on it?

Ted Thibodeau Jr.: "said" is not in the present "say" tense in any of our documents, to date.

Manu Sporny: knowing someone had a valid driver's license, even if they don't stand by it right now, might be useful.
… so there are use cases where you would ignore the currency.
… but I think the main point is that verifiers should apply their business logic, without restriction.
… so if you don't care about currency, then we're saying that you can't use VCs?
… verifiers will check if its important, I don't think we need to instruct them.
… maybe a separate validation algorithm in the future.

Dave Longley: i think, when acting as a verifier, "now" is itself an input parameter to some algorithm.

Andres Uribe: answering Joe's point about semantic verification. That's a blurry line that is difficult to define. Who is to say that status is part of that or not?
… if you care about it, you do it.
… We should make verification as clear as possible.
… These need to be secure. So we need clear verification.

Dave Longley: I'm sympathetic with tightening up what verification means. Modulo status check, there are other things that we do during verification, such as formedness.
… the real question we've used is about business rules.
… There are certain things that need to be done to gather the information needed to validate.
… So, some part of verification requires you to go out an check a list. Then you should do that.
… If we say it's just checking cryptography, that's not going to work.
… The business rules get to are we going to accept something.

Brent Zundel: this conversation will go into the PR.
… this one is not as cut and dried.

[jyasskin]

Without wading into the question of whether this is verification or validation...

[jyasskin]

Should "associated with" be "that is a [=value=] of", or is there some other way that mechanisms can be associated with this property?

[jyasskin]

How can implementations use type and statusPurpose for this? Are Status specifications expected to provide an algorithm that returns an expectedStatus?

Consider pushing this step down into the status checking algorithm or even to the individual Status specifications.

[jyasskin]

You should make this a [=link=] to a <dfn> instead of a link to a header, so that the <dfn> can provide backlinks to the places that refer to it. E.g. click on https://w3c.github.io/vc-data-model/#dfn-claims.

(Sorry for forgetting this comment for all the algorithm cross-links, but it's a good idea for them too.)

[jyasskin]

I keep forgetting about this, but Respec lets you attach a variable's type to the variable itself: https://respec.org/docs/#giving-variables-data-types

Suggested change

	a status (boolean <var>status</var>)
	a status (boolean |status:boolean|)

[jyasskin]

I believe the IETF's experience has been that it's valuable to use a registry to get people to list all the terms they use, and that it's good to reduce the amount of curation the registry requires in order to make that happen. If VC-SPECS becomes a registry you can use either the IETF's Specification Required policy or their First Come First Served policy, which don't require the custodian to make any decisions about how "good" one of the specs is.

(This PR probably isn't the place to discuss exactly how the registry is set up, but I hope to convince @OR13 that it's ok to cite a list of status types without the group committing to curating the list.)

[jyasskin]

To support the [=status checking algorithm=] link suggested above:

Suggested change

	The algorithm in this section can be used to check one or more status
	The <dfn>status checking algorithm</dfn> can be used to check one or more status

[jyasskin]

If the statusMechanism is given an @id, a JSON-LD processor might discover more properties on it than a JSON processor would. Which set of properties gets used here?

Looking at https://www.w3.org/TR/vc-bitstring-status-list/#bitstringstatuslistentry, none of the properties are optional, so I think mischief is limited to providing extra copies of statusListIndex or statusListCredential. https://www.w3.org/TR/vc-bitstring-status-list/#validate-algorithm doesn't consider that there might be extra copies with different values ... or even that the required fields might be missing. I assume you'll want status checking to declare that the VC as a whole is invalid when any of these requirements are violated?

[jyasskin]

I don't think it's a good idea to leave the "function-specific options" for implementations to guess at. Either they should be specified by the particular type's spec, or they should be part of the standard interface for all Status types, and specified by this algorithm.

[msporny]

Based on the feedback on the last call, this PR is not going to make it. I'm marking it pending close.

[OR13]

I agree this PR probably won't make it, but I think a simpler approach would, and that it will be necessary to address some of the other verification issues.

[iherman]

The issue was discussed in a meeting on 2023-12-12

• no resolutions were taken

View the transcript

2.2. Incorporate status checking into verification algorithm (pr vc-data-model#1369)

See github pull request vc-data-model#1369.

Brent Zundel: moving on to next PR.

Joe Andrieu: I've had a shift in this whole verification thing... I haven't made it through Manu's edits w/ his new PRs and verification/validation... shift, which aligns w/ Andres, Verification defined "not cryptographically", but steps that can be done generically w/o business rules... so service provider can do that for you.

Dmitri Zagidulin: expiry is business rules, though.

Dmitri Zagidulin: but I like this phrasing.

Joe Andrieu: It could check status, expiry, well-formed-ness, so I want to queue up Manu... how does that float into latest language? At least for me, I have a lot more movement on this topic.

Dave Longley: +1 to the framing being closer to what i think works / people can agree with.

Manu Sporny: good news is that many people are granting leeway than before. that's good.
… I think there is strong push back on putting validation in scope.
… I think Orie has wanted us to say we are both verifying and validating. but once we put one validation thing in there, it's a slippery slope.
… I think Orie is aligned with an idea from Dave: these calls to validating any of the extension points, you just get a yes/no answer.
… that seems aligned with what Joe was saying.
… I think there's some language we could get to.
… we could make another attempt at this, but I'd rather not do it.
… we probably don't need this defined.
… I can try another variation.
… but this PR should die.

Brent Zundel: if anyone is opposed to this PR dying, speak up.

Andres Uribe: If its not in the spec, we should agree to use three different words for what we mean.
… "verification", "validation", but there's a third.
… One is cryptographic securing mechanism.
… Second is what Joe just alluded to, the status checking.
… Third is business validation.
… if we start using a different term that could help.

Dmitri Zagidulin: what is the current consensus re where well-formedness (required fields, etc) belongs? verification?

Joe Andrieu: One thing I wanted to note, in different PR, shift about "validation" that we don't validate what the roles do... even though it's expected that a verifier validates... there isn't software that we can test to be conformant. We say what conformant implementations do, not what conformant roles do (there still exists some sloppy language).
… Manu, you said verification should be 'yes/no' w/ subjects. I want to make sure metadata is returned so validation can be done by validation. Everything should bubble back up so business rules can be done/provided.

Manu Sporny: yes. that's the slippery slope I'm worried about.
… if we could define all that... it's not just expiry, it's also schema.
… So, depending on the detail that Jeffrey is asking for.
… For example, the interface for securing mechanisms took six weeks. At best, 3 weeks.
… that means we don't get to JOSE-COSE and unlinkability.
… Unless that interface can be handwavey...
… What you want Joe, is great, but we are out of time.
… unless we can solve it in the next week or two.
… Bikeshedding, etc. is a recipe for us just not completing.

Brent Zundel: thank you, Manu. I agree with your concerns.
… what about this PR.

Dave Longley: I think we might not need 3 terms.
… we just need to split these up better.
… Verification is getting you the information you need to make a decision.
… Validation is making that decision.

Joe Andrieu: I get the time thing.
… My concern is not whether or not we do this, I appreciate the time concern and that we want to do this, I think Jeffrey is asking for an answer and we need to figure out the level of detail to assuage his concerns. If we don't define it in this way, what way could we meet it?

Dmitri Zagidulin: I want to get a sense where the current consensus is wrt conforming documents and well-formedness.
… does checking that an object has required field, that timestamp is proper format, do those belong in verification or validation.

Dave Longley: i think we're only defining the securing mechanism portion of verification and we can say that other interfaces can be used for other extension points to get secured information from them -- and then after all that, validation (business rules) can be applied.

Dmitri Zagidulin: @manu - what were you going to say?

[msporny]

PR failed to achieve consensus, marked "pending close" for over a week, closing.