Introduce new query language (Approach 2) #260

danielfett · 2024-09-11T15:38:35Z

This is a new attempt to design a new query language, incorporating a lot of the feedback received on the old PR.

Fixes #178 and #157.

This PR replaces #220.

Open issues:

Define queries for matching parameters
Introduce transformed claims?
Determine which format-specific parameters we need
Define interaction with existing query lang (see co-existence of multiple query languages (to PE, or not to PE) #211)
Define new format for the VP Token
Maybe re-introduce purpose?

Co-authored-by: Ted Thibodeau Jr <tthibodeau@openlinksw.com> Co-authored-by: Oliver Terbu <o.terbu@gmail.com> Co-authored-by: Christian Bormann <8774236+c2bo@users.noreply.github.com>

This reverts commit 7d71960.

Co-authored-by: Michael B. Jones <michael_b_jones@hotmail.com>

Co-authored-by: Christian Bormann <8774236+c2bo@users.noreply.github.com>

examples/query_lang/multi_credentials.json

Co-authored-by: Timo Glastra <timo@animo.id>

martijnharing · 2024-09-12T14:44:51Z

openid-4-verifiable-presentations-1_0.md

+A valid VP Query is defined as a JSON-encoded object with the following
+top-level property:
+
+`expect_credentials`:


I'm not sure if I fully understand the logic for requesting more than one credential. But to support the different use cases like "A or B", "A and B and C", "A or (B and C)", we should have effectively first an OR array of document sets that should be return and within that array an AND where you have to return all the documents.

That can be done through a mechanism using alternative documents, but the OR(AND) order should still hold.

I think it's now the other way round, meaning "give me A or (B and C)" cannot be done.

@martijnharing You can have either DNF or CNF and you need both to cater for all possible combinations of credential request. "give me A or (B and C)" can be done, if you have DNF.

The syntax proposed here is sufficient to express what you want. But I also tried to come up with an option that combines some of the elements of this proposal and a CNF in a new PR: #266 - please review!

martijnharing · 2024-09-12T14:46:09Z

openid-4-verifiable-presentations-1_0.md

+`format`:
+: REQUIRED. A string that specifies the format of the requested
+Verifiable Credential. Valid Credential format identifier values are defined in
+Appendix A of [@!OpenID.VCI]. The value of this property MUST be one of the


We shouldn't reference OpenID.VCI for the request logic format definitions. At least for mdocs the format definition contains a number of items (like algorithms, curves, hash algorithms) that aren't used in requests.

martijnharing · 2024-09-12T14:55:40Z

openid-4-verifiable-presentations-1_0.md

+The value MUST be a string that specifies the name of the claim within the provided namespace
+in the Verifiable Credential, e.g., `first_name`.
+
+`value`:


Matching of values is a tricky subject. Some of the issues:

Matching on values can be complicated for wallet / credential selection. Because you may not have access to the value of the element at the selection time.

Even after wallet selection, having access to the values of the data elements may still only be available after user consent because of security reasons (decryption etc). Meaning that the presentation may fail only after user consent.

Values of data elements are not just strings, integers or booleans. Depending on the credential format they can be maps, arrays, byte strings or other items.

So besides requiring a stricter definition, mostly it's a mechanism that may just not be available because of security or ux reasons, so I would not be in favor of having it as an option since it's not a mechanism RPs can rely on.

It is a tricky subject, but the working group did spent a significant amount of time analysing what is needed from a query language for various use cases, and the conclusion was that matching of values is something that must be supported by a query language: #144

It's also something supported by OID4VP today in presentation exchange, so something that is likely to be required if we want to move back to having only one query language in the future.

I acknowledge the difficulties, but I think we need to focus on how it can be done (and not focus on whether it should be done or not). Whether it's something that's mandatory to implement or not is I think something that's more likely to be dictated by the ecosystems. I understand some of the mandated EU use cases will require matching.

We have syntax for other types of matching in the Advanced Syntax for Claims specification which we could re-use here. There is at least one implementation of that spec, so we're not starting from scratch.

Please see a proposal here: #266 (comment)

TimoGlastra · 2024-09-14T09:48:01Z

openid-4-verifiable-presentations-1_0.md

+
+By default, each element within `expect_credentials` or `expect_claims` MUST be returned, unless:
+
+- The element contains an `alternative_claims` or `alternative_credentials`


To me it feels a bit weird that alternative_claims is added to an expected claim. As this is a two way relation (you could also add it to the other claim).

I think separating the requirements from the credentials is one of the things that really make sense to me in PEX, as it gives a nice structure:

define which credentials can be presented.

define which combination of credentials are allowed.

I feel this syntax intertwines it more and loses readability over it, while also making it quite hard to spot invalid configurations (as a claim/cred referenced as alternative can't reference alternatives itself). Would be interesting to see the complexity of implementing such an approach (hard to wrap my head around at the moment)

TimoGlastra · 2024-09-14T09:52:18Z

openid-4-verifiable-presentations-1_0.md

+  - To address an element within an array, append the index to the array (as a
+    non-negative, 0-based integer).
+  - To address all elements within an array, append the string `*` to the array.


With PEX it's possible to query for a specific item in an array based on value. Would that be possible?

I.e. i need to prove my german nationality, but my other nationalities are not important to disclose. This is something SD-JWT disclosures can support, but may be complex to convey in a vp query

That would be possible by using elements from the "Transformed Claims" defined in the Advanced Syntax for Claims specification. We could re-use that here.

Please see a proposal here: #266 (comment)

peppelinux · 2024-09-19T13:50:27Z

examples/query_lang/claims_alternatives.json

@@ -0,0 +1,35 @@
+{
+  "expect_credentials": [


Suggested change

"expect_credentials": [

"requested_credentials": [

I prefer requested, it sounds more clear and direct than "expect" to me.

peppelinux · 2024-09-19T13:50:50Z

examples/query_lang/claims_alternatives.json

+        "credential_signing_alg_values": ["ES256", "ES384"],
+        "sd_alg_values": ["SHA-256"]
+      },
+      "expect_claims": [


Suggested change

"expect_claims": [

"requested_claims": [

peppelinux · 2024-09-19T13:51:48Z

openid-4-verifiable-presentations-1_0.md

+: A string containing an HTTPS URL pointing to a resource where a Presentation Definition JSON object can be retrieved. See (#request_presentation_definition_uri) for more details. Exactly one of `vp_query` or `presentation_definition` or `presentation_definition_uri` MUST be present in the Authorizat
+ion Request.
+


Suggested change

: A string containing an HTTPS URL pointing to a resource where a Presentation Definition JSON object can be retrieved. See (#request_presentation_definition_uri) for more details. Exactly one of `vp_query` or `presentation_definition` or `presentation_definition_uri` MUST be present in the Authorizat

ion Request.

: A string containing an HTTPS URL pointing to a resource where a Presentation Definition JSON object can be retrieved. See (#request_presentation_definition_uri) for more details. Exactly one of `vp_query` or `presentation_definition` or `presentation_definition_uri` MUST be present in the Authorization Request.

peppelinux · 2024-09-19T13:52:35Z

openid-4-verifiable-presentations-1_0.md

+The Verifiable Presentation Query Language (VP Query) is a JSON-encoded query
+language that allows the Verifier to request the Wallet to present Verifiable
+Credentials that match the query. The query consists of a set of constraints
+that the requested Verifiable Credentials must satisfy. The Wallet evaluates the


Suggested change

that the requested Verifiable Credentials must satisfy. The Wallet evaluates the

that the requested Verifiable Credentials MUST satisfy. The Wallet evaluates the

peppelinux · 2024-09-19T13:53:13Z

openid-4-verifiable-presentations-1_0.md

+## Credential Query {#credential_query}
+
+A Credential Query is an object representing a request for one specific
+credential. It contains the following properties:


Suggested change

credential. It contains the following properties:

Credential. It contains the following properties:

peppelinux · 2024-09-19T13:53:23Z

openid-4-verifiable-presentations-1_0.md

+credential. It contains the following properties:
+
+`group`:
+: REQUIRED. A string identifying the credential in the response, or, if multiple


Suggested change

: REQUIRED. A string identifying the credential in the response, or, if multiple

: REQUIRED. A string identifying the Credential in the response, or, if multiple

peppelinux · 2024-09-19T13:53:33Z

openid-4-verifiable-presentations-1_0.md

+
+`group`:
+: REQUIRED. A string identifying the credential in the response, or, if multiple
+credentials carry the same group identifier, the group of credentials that the


Suggested change

credentials carry the same group identifier, the group of credentials that the

Credentials carry the same group identifier, the group of Credentials that the

peppelinux · 2024-09-19T13:53:42Z

openid-4-verifiable-presentations-1_0.md

+`group`:
+: REQUIRED. A string identifying the credential in the response, or, if multiple
+credentials carry the same group identifier, the group of credentials that the
+requested credential belongs to.


Suggested change

requested credential belongs to.

requested Credential belongs to.

peppelinux · 2024-09-19T13:54:13Z

openid-4-verifiable-presentations-1_0.md

+`expect_meta`: 
+: OPTIONAL. An object defining additional properties requested by the Verifier that
+apply to the metadata and validity data of the credential. The properties of
+this object are defined per format in (#format_specific_properties). If omitted,
+no specific restrictions are placed on the metadata or validity of the requested
+Credential.


Suggested change

`expect_meta`:

: OPTIONAL. An object defining additional properties requested by the Verifier that

apply to the metadata and validity data of the credential. The properties of

this object are defined per format in (#format_specific_properties). If omitted,

no specific restrictions are placed on the metadata or validity of the requested

Credential.

`expect_meta`:

: OPTIONAL. An object defining additional properties requested by the Verifier that

apply to the metadata and validity data of the Credential. The properties of

this object are defined per format in (#format_specific_properties). If omitted,

no specific restrictions are placed on the metadata or validity of the requested

Credential.

peppelinux · 2024-09-19T13:54:59Z

openid-4-verifiable-presentations-1_0.md

+
+`expect_claims`:
+: OPTIONAL. An array that specifies the claims that the requested
+Verifiable Credential must contain. Each entry in the array corresponds to exactly


Suggested change

Verifiable Credential must contain. Each entry in the array corresponds to exactly

Verifiable Credential MUST contain. Each entry in the array corresponds to exactly

I hope that requested_claims sounds better to you too and that we might evaluate this change as well.

peppelinux · 2024-09-19T13:55:11Z

openid-4-verifiable-presentations-1_0.md

+`expect_claims`:
+: OPTIONAL. An array that specifies the claims that the requested
+Verifiable Credential must contain. Each entry in the array corresponds to exactly
+one claim that is to be returned by the Wallet to fulfill the credential


Suggested change

one claim that is to be returned by the Wallet to fulfill the credential

one claim that is to be returned by the Wallet to fulfill the Credential

peppelinux · 2024-09-19T13:55:17Z

openid-4-verifiable-presentations-1_0.md

+request. 
+
+`alternative_credentials`:
+: OPTIONAL. An array of credential group identifiers that specify alternative


Suggested change

: OPTIONAL. An array of credential group identifiers that specify alternative

: OPTIONAL. An array of Credential group identifiers that specify alternative

peppelinux · 2024-09-19T13:55:29Z

openid-4-verifiable-presentations-1_0.md

+
+`alternative_credentials`:
+: OPTIONAL. An array of credential group identifiers that specify alternative
+groups of credentials that MAY be returned instead of the credential. Any credential


Suggested change

groups of credentials that MAY be returned instead of the credential. Any credential

groups of Credentials that MAY be returned instead of the Credential. Any Credential

peppelinux · 2024-09-19T13:55:45Z

openid-4-verifiable-presentations-1_0.md

+identifier for claims is not reflected in the response.
+
+`path`:
+: REQUIRED if the credential format uses a JSON-based claims structure; MUST NOT


Suggested change

: REQUIRED if the credential format uses a JSON-based claims structure; MUST NOT

: REQUIRED if the Credential format uses a JSON-based claims structure; MUST NOT

peppelinux · 2024-09-19T13:55:57Z

openid-4-verifiable-presentations-1_0.md

+within the Verifiable Credential, as defined in (#claims_path_pointer).
+
+`namespace`:
+: REQUIRED if the credential format is based on ISO 18013-5; MUST NOT be present otherwise. 


Suggested change

: REQUIRED if the credential format is based on ISO 18013-5; MUST NOT be present otherwise.

: REQUIRED if the Credential format is based on ISO 18013-5; MUST NOT be present otherwise.

peppelinux · 2024-09-19T13:56:09Z

openid-4-verifiable-presentations-1_0.md

+within the Verifiable Credential, e.g., `org.iso.18013.5.1`.
+
+`claim_name`:
+: REQUIRED if the credential format is based on ISO 18013-5; MUST NOT be present otherwise. 


Suggested change

: REQUIRED if the credential format is based on ISO 18013-5; MUST NOT be present otherwise.

: REQUIRED if the Credential format is based on ISO 18013-5; MUST NOT be present otherwise.

peppelinux · 2024-09-19T13:57:13Z

openid-4-verifiable-presentations-1_0.md

+: OPTIONAL. An array of strings that specifies allowed values for
+the type of the requested Verifiable Credential. All elements in the array MUST
+be valid type identifiers as defined in [@!I-D.ietf-oauth-sd-jwt-vc]. The Wallet
+MAY return credentials that inherit from any of the specified types, following


Suggested change

MAY return credentials that inherit from any of the specified types, following

MAY return Credentials that inherit from any of the specified types, following

peppelinux · 2024-09-19T13:57:30Z

openid-4-verifiable-presentations-1_0.md

+`kbjwt_signing_alg_values`:
+: OPTIONAL. An array of strings that specifies the
+allowed algorithms for signing the Key-Binding JWT (KB-JWT) (i.e., the part signed
+by the holder).


Suggested change

by the holder).

by the Holder).

peppelinux · 2024-09-19T14:00:00Z

openid-4-verifiable-presentations-1_0.md

+
+### Format `mso_mdoc` {#format_mso_mdoc}
+
+TBD


I got a very bad reaction from @bc-pi against the use of TBD in the specs, even if only draft. This is educating me.

peppelinux · 2024-09-19T14:02:23Z

openid-4-verifiable-presentations-1_0.md

+
+In detail, the array is processed by the Wallet from left to right as follows:
+
+ 1. Select the root element of the credential, i.e., the top-level JSON object.


Suggested change

1. Select the root element of the credential, i.e., the top-level JSON object.

1. Select the root element of the Credential, i.e., the top-level JSON object.

peppelinux · 2024-09-19T14:03:12Z

openid-4-verifiable-presentations-1_0.md

+<{{examples/query_lang/claims_alternatives.json}}
+
+The following is a non-normative example of a VP Query that requests multiple
+Verifiable Credentials; all of them must be returned:


Suggested change

Verifiable Credentials; all of them must be returned:

Verifiable Credentials; all of them MUST be returned:

peppelinux · 2024-09-19T14:03:22Z

openid-4-verifiable-presentations-1_0.md

+<{{examples/query_lang/multi_credentials.json}}
+
+The following shows a complex query where the Wallet can either deliver
+the first credential (here called `pid`, containing all data the Verifier


Suggested change

the first credential (here called `pid`, containing all data the Verifier

the first Credential (here called `pid`, containing all data the Verifier

peppelinux · 2024-09-19T14:03:31Z

openid-4-verifiable-presentations-1_0.md

+
+The following shows a complex query where the Wallet can either deliver
+the first credential (here called `pid`, containing all data the Verifier
+needs), or a second credential (here called `other_pid`), or present two


Suggested change

needs), or a second credential (here called `other_pid`), or present two

needs), or a second Credential (here called `other_pid`), or present two

peppelinux · 2024-09-19T14:03:41Z

openid-4-verifiable-presentations-1_0.md

+The following shows a complex query where the Wallet can either deliver
+the first credential (here called `pid`, containing all data the Verifier
+needs), or a second credential (here called `other_pid`), or present two
+credentials that together contain all the data the Verifier needs:


Suggested change

credentials that together contain all the data the Verifier needs:

Credentials that together contain all the data the Verifier needs:

peppelinux · 2024-09-19T14:03:51Z

openid-4-verifiable-presentations-1_0.md

+<{{examples/query_lang/credentials_alternatives.json}}
+
+The following example shows a query that uses the `value` and `values` restrictions
+to request a credential with specific values for the `last_name` and `postal_code` claims:


Suggested change

to request a credential with specific values for the `last_name` and `postal_code` claims:

to request a Credential with specific values for the `last_name` and `postal_code` claims:

peppelinux · 2024-09-19T14:04:35Z

It looks really promising!

danielfett · 2024-10-09T06:25:58Z

I'll close this PR as it seems that there is consensus that #266 is the better approach.

danielfett and others added 18 commits July 25, 2024 14:03

First draft of new query language

665eda8

Merge branch 'main' into danielfett/vp-query-language

7562b69

Fix compile errors

157e696

Apply suggestions from code review

e4bd784

Co-authored-by: Ted Thibodeau Jr <tthibodeau@openlinksw.com> Co-authored-by: Oliver Terbu <o.terbu@gmail.com> Co-authored-by: Christian Bormann <8774236+c2bo@users.noreply.github.com>

Add back accidentally deleted whitespace

bf0ee77

Fix anchor name

7d71960

Revert "Fix anchor name"

50b1721

This reverts commit 7d71960.

Fix anchor name

2005364

Apply suggestions from Mike's code review

ba440ba

Co-authored-by: Michael B. Jones <michael_b_jones@hotmail.com>

Update openid-4-verifiable-presentations-1_0.md

24aa273

Co-authored-by: Christian Bormann <8774236+c2bo@users.noreply.github.com>

Merge branch 'main' into danielfett/vp-query-language

8db4ddb

Remove purpose from credential query language proposal

acdfd88

Add missing vp_query definition back into the PR

110714c

Remove another purpose mention

da38019

New format in examples

da854b3

Merge branch 'main' into danielfett/vp-query-language-2

ef77fbf

Update spec text

d21e8e9

Add value matching

1263648

danielfett mentioned this pull request Sep 11, 2024

Introduce new query language (Approach 1) #220

Closed

4 tasks

TimoGlastra reviewed Sep 11, 2024

View reviewed changes

examples/query_lang/multi_credentials.json Outdated Show resolved Hide resolved

Apply suggestions from code review

df46111

Co-authored-by: Timo Glastra <timo@animo.id>

martijnharing reviewed Sep 12, 2024

View reviewed changes

Add comments

fc66dd1

TimoGlastra reviewed Sep 14, 2024

View reviewed changes

danielfett changed the title ~~Introduce new query language (new approach)~~ Introduce new query language (Approach 2) Sep 18, 2024

Merge branch 'main' into danielfett/vp-query-language-2

a5d69a4

peppelinux reviewed Sep 19, 2024

View reviewed changes

danielfett closed this Oct 9, 2024

jogu mentioned this pull request Oct 16, 2024

Introduce new query language (Approach 3) #266

Merged

6 tasks


		By default, each element within `expect_credentials` or `expect_claims` MUST be returned, unless:

		- The element contains an `alternative_claims` or `alternative_credentials`

		: A string containing an HTTPS URL pointing to a resource where a Presentation Definition JSON object can be retrieved. See (#request_presentation_definition_uri) for more details. Exactly one of `vp_query` or `presentation_definition` or `presentation_definition_uri` MUST be present in the Authorizat
		ion Request.

	that the requested Verifiable Credentials must satisfy. The Wallet evaluates the
	that the requested Verifiable Credentials MUST satisfy. The Wallet evaluates the

	credential. It contains the following properties:
	Credential. It contains the following properties:

	: REQUIRED. A string identifying the credential in the response, or, if multiple
	: REQUIRED. A string identifying the Credential in the response, or, if multiple

	credentials carry the same group identifier, the group of credentials that the
	Credentials carry the same group identifier, the group of Credentials that the

	requested credential belongs to.
	requested Credential belongs to.

	Verifiable Credential must contain. Each entry in the array corresponds to exactly
	Verifiable Credential MUST contain. Each entry in the array corresponds to exactly

	one claim that is to be returned by the Wallet to fulfill the credential
	one claim that is to be returned by the Wallet to fulfill the Credential

	: OPTIONAL. An array of credential group identifiers that specify alternative
	: OPTIONAL. An array of Credential group identifiers that specify alternative

	groups of credentials that MAY be returned instead of the credential. Any credential
	groups of Credentials that MAY be returned instead of the Credential. Any Credential

	: REQUIRED if the credential format uses a JSON-based claims structure; MUST NOT
	: REQUIRED if the Credential format uses a JSON-based claims structure; MUST NOT

	: REQUIRED if the credential format is based on ISO 18013-5; MUST NOT be present otherwise.
	: REQUIRED if the Credential format is based on ISO 18013-5; MUST NOT be present otherwise.

	MAY return credentials that inherit from any of the specified types, following
	MAY return Credentials that inherit from any of the specified types, following


		In detail, the array is processed by the Wallet from left to right as follows:

		1. Select the root element of the credential, i.e., the top-level JSON object.

	Verifiable Credentials; all of them must be returned:
	Verifiable Credentials; all of them MUST be returned:

	the first credential (here called `pid`, containing all data the Verifier
	the first Credential (here called `pid`, containing all data the Verifier

	needs), or a second credential (here called `other_pid`), or present two
	needs), or a second Credential (here called `other_pid`), or present two

	credentials that together contain all the data the Verifier needs:
	Credentials that together contain all the data the Verifier needs:

	to request a credential with specific values for the `last_name` and `postal_code` claims:
	to request a Credential with specific values for the `last_name` and `postal_code` claims:

Introduce new query language (Approach 2) #260

Introduce new query language (Approach 2) #260

Uh oh!

Conversation

danielfett commented Sep 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danielfett commented Sep 11, 2024 •

edited

Loading