Support for JOSE (JSON Signature and Encryption) Standards

[cyberphone]

I'm not currently a user of OpenAPI but a follower of standards initiatives like OpenBanking/FAPI where members claim that OpenAPI currently does not support JOSE (JSON Signature and Encryption) standards forcing them to use various workarounds.

I wonder if there is anybody out there with knowledge of the OpenAPI platform who could be interested in working with me to integrate the missing support?

There are also enhanced versions of JOSE JWS and JWE in the workings (through the IETF), providing Clear Text support which should be a nice fit for information centric systems, here illustrated by a minute JWS-CT sample:

{
     "@context": "https://example.com/paymentStandard/pay",
     "amount": "255.00",
     "currency": "USD",
     "signature": {
         "alg": "ES256",
         "jwk": {
             "kty": "EC",
             "crv": "P-256",
             "x": "PxlJQu9Q6dOvM4LKoZUh2XIe9-pdcLkvKfBfQk11Sb0",
             "y": "6IDquxrbdq5ABe4-HQ78_dhM6eEBUbvDtdqK31YfRP8"
         },
         "val": "RSLmFihg8QmXxM .... N0lGIdSEYvMMLTL8hEaYV9kW6A"
     }
}

[MikeRalphson]

There was an earlier discussion touching on this here: #550 (comment)

[isamauny]

We are looking at a proposal to the OpenAPI foundation for this exact issue. The idea is to describe in the OpenAPI Specification all the information needed at the consumer side to either create encrypted/signed contents or validate such contents. It should also be descriptive enough that the generation of the JOSE contents can be automated.

Our approach is trust no one. So while you could decrypt and validate contents from the contents of the message you receive, you must have the information of what was the contract in order to validate properly. In other words, you could receive contents which have been tempered with via a MITM attack and are perfectly valid in terms of structure, but not corresponding at all with it should have been.

Clear Text support as described above is very much inline with our thinking.

So yes happy to work with the foundation and Anders on this via FAPI.

[cyberphone]

Would it be possible getting a "second opinion" on an issue where the conveners currently are somewhat divided?

Current (draft) proposal:
  {
      "app-specific-property-1": ...,
      "app-specific-property-2": ...,

      "__cleartext_signature": {
           Signature Object (Cleartext JWS)
      }
  }

Main benefit/argument: By using a fixed name interoperability is easier to achieve. This may be a requirement for pulling through the IETF process. The unusual name eliminates clashes with applications.

 
Alternative solution:
  {
      "app-specific-property-1": ...,
      "app-specific-property-2": ...,

      "app-specific-property-n": {
           Signature Object (Cleartext JWS)
      }
  }

Main benefit/argument: Schema-driven designs do not generally benefit by "hard coded" properties. A signature property could equally well be expressed as "attestation", "authorization", "requestSignature", etc. depending on the application context.

[cyberphone]

@isamauny @MikeRalphson @timburks @danhorst @elazar Should I interpret the silence that a hard coded __cleartext_signature property as in the current draft is your preference, rather than a Signature Object which can be featured under an application specific name?

This is to be decided now, which is why there is of some urgency getting your input!

Thanx,
Anders

[MikeRalphson]

@cyberphone could you link to the current draft?

[cyberphone]

https://xml2rfc.tools.ietf.org/cgi-bin/cat.cgi/.txt?input=4d0c0e0897622dc8cbc21cb706de607c634b2c9689fb1a5ce86337-1518163603

3.  The __cleartext_signature Member

   The "__cleartext_signature" member that is added to the data contains
   the JWS header parameters and the signature value.

   When putting the header parameters into the data to be signed, they
   MUST be labeled with the key "__cleartext_signature", i.e., the
   object to be signed cannot have its own root item that is named
   "__cleartext_signature".

[cyberphone]

Pardon me, the link above apparently expires after a while...
https://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi?url=https%3A%2F%2Fraw.githubusercontent.com%2Ferdtman%2FCleartext-JOSE%2Fmaster%2Fdraft-erdtman-jose-cleartext-jws.xml

[darrelmiller]

@cyberphone As I understand it, this issue is related to a reserved property name within an object that is a JWT. As such, I don't think either option you describe would have any impact OpenAPI descriptions. If the structure of the JWT token were to be described in an OpenAPI description, then we would likely use JSON schema which could describe both approaches.

[cyberphone]

@darrelmiller The new scheme is called Cleartext JWS and is not a JWT but that doesn't really matter; the question I have is more related to how "hard coded" properties fit the needs and habits of the OpenAPI development community in general.

My take on this is regarding a Cleartext JWS as a Derived Data Type, like dateTime. At least this is how I use it in my own designs. I indeed started with a fixed property but later found out that this was unnecessary and could equally well be addressed by having a suitable "default" at the API level.

However, in the actual draft a default does not make sense since it describes a data structure, not an API.

[cyberphone]

@darrelmiller After reading your blog, I came up with a highly related question:: How do "restafarians" envision signed REST requests and particularly in an OpenAPI context?

[cyberphone]

A free-standing Signature Object can also be used for signing JSON arrays:

["Hi there!",2003,{
  "alg": "ES256",
  "jwk": {
    "kty": "EC",
    "crv": "P-256",
    "x": "censDzcMEkgiePz6DXB7cDuwFemshAFR90UNVQFCg8Q",
    "y": "xq8rze6ewG0-eVcSF72J77gKiD0IHnzpwHaU7t6nVeY"
  },
  "val": "gOIohCbkhQOftFjqHgqRuRG1qqROSzwTiW8C7FUAQzojtqtpVoOz7BOYYNRQ7e09EfDlejz7jHumAvlAlQ6txw"
}]

[pleothaud]

Hi there,

I'm Philippe Leothaud a colleague of @isamauny at 42Crunch, and I'm working atm on a proposal to explore how we could extend OAS so that it can describe JOSE-signed and JOSE-encrypted content.

With "regular" Jose artefacts, it happens that signed and/or encrypted content have a specific content-type, namely :
- the "application/jose" media type which can be used to indicate that the content is a JWS or JWE using the JWS Compact Serialization or the JWE Compact Serialization.
- the "application/jose+json" media type which can be used to indicate that the content is a JWS or JWE using the JWS JSON Serialization or the JWE JSON Serialization (general or flattened)

The JWT RFC (7519) also defines a specific content-type for JWTs, "application/jwt".

For these media types, we could therefore extend the Open API mediaType Object so that it can contain a cryptographicDefinitionsUrl field that will allow for the retrieval of the cryptographic contraints that must be fulfilled to consume the API (authorized algorithms, key to be used for encryption, authorised key type and key reference mechanisms for signature) as well as the description of the cryptographic operations computed on the API responses so that the client will know how to decrypt/verify the signature of the responses.

Another way to use signatures is to use the unencoded payload option for JWS (signatures) described by RFC 7797, leave the body of the request/response unchanged (aplication/json media type) and then pass the signature as a detached signature in a specific HTTP header (x-jws-signature is at least the name used by UK's OpenBanking). This means that you remove the payload part from the signature and the rest of the base64url encoded structure in such a header.

Here also, extending the header Open API object with the same cryptographicDefinitionsUrl field should be enough to describe the cryptographic contract.

As far as I understand, cleartext JOSE will not change content-types from regular JOSE so it should fit in the proposal.

Also, I'm to meet (hopefully) @cyberphone next week to understand better the Cleartext JOSE spec.

Last, I should have finished my proposal document by tomorrow, I will then pass it to @isamauny so that she can introduce it to the OAS group.

Thanks,

Philippe

PS: sorry for all the crypto mumbo-jumbo, all will be properly explained in my doc :-)

[cyberphone]

Hi Philippe e.t al.
Open Banking UK is a relevant application for this discussion. Here is a scaled down version of a payment operation:

POST /payments HTTP/1.1
x-jws-signature: TGlmZSdzIGEgam91cm5 ... leSBhg6fttg6gh88bfxmlf5bdDrA3
Content-Type: application/json

{
  "amount": "259.99",
  "currency": "GBP"
}

This scheme has (in my opinion NB) a number of shortcomings like:

• Ties signatures to HTTP
• Does not permit signed objects to be embedded in other objects
• Does not easily facilitate serialization of signed data
• Shrouds header data in Base64Url

A Cleartext JWS counterpart would address all the issues above:

POST /payments HTTP/1.1
Content-Type: application/json

{
  "amount": "259.99",
  "currency": "GBP",
  "signature": {
    "alg": "RS256",
    "kid": "#45233",
    "val": "Mtqlw2PPUoZSO4DAHNzZ9gk_ ... z8HsK3fE1jux6jFVrrhwSZiRaW6M"
  }
}

Remaining Problem
Strictly speaking none of those schemes properly represent a signed REST request since requests like above are only fully characterized by the HTTP Body, URL, and HTTP Verb. There are essentially three different ways of dealing with this issue:

• Current solution: ignore it. Well, Amazon do use something similar to the "right" way below
• The "right" way: https://tools.ietf.org/html/draft-cavage-http-signatures-09
• The "heretic" way: https://cyberphone.github.io/doc/web/yasmin.html

[pleothaud]

Hi Anders,

I propose we talk face to face about that next week, as discussing in this thread might be bothering to other OAS people (like comparing advantages and disadvantages of detached, enveloping and enveloped signatures...).

Just wanted to add that supporting the http-signatures standard could just be accomplished in a similar manner than I propose to support the detached signature/HTTP header OpenBanking is using: by extending the header object so that you can refer to the description of requirements to the to-be-used Signature header draft-cavage-http-signatures defines and relies on.

Thanks,

Philippe

[ainthek]

@cyberphone: amazing discussion, thanx:
can anyone point me to some documents about this:
"here are also enhanced versions of JOSE JWS and JWE in the workings (through the IETF), providing Clear Text support which should be a nice fit for information centric systems, here illustrated by a minute JWS-CT sample"

[cyberphone]

@ainthek The WG has not yet been created but you can follow the work on the link provided earlier in this thread. https://github.com/cyberphone/openkeystore holds a fully usable precursor to this work.

[darrelmiller]

@cyberphone @pleothaud I appreciate your explanations above, I am now in a much better place to have this conversation.

Here's my perspective on this. OpenAPI is a description of an HTTP API. In order to describe how these JOSE concepts can be described in OpenAPI, we must first identify how they will be described in a HTTP message and then the answer will fall out of that.

There are generally just three places you can put data in a HTTP request. The URL, headers or the payload. Let's ignore the URL for moment based on the assumption that there is probably too much metadata to put into the URL cleanly.

HTTP headers are intended to contain metadata about request or response. Putting signature and encoding information in the HTTP headers would seem like the obvious choice. I do hope it is not too late to remove that x- prefix from the x-jws-signature from the header as that has been discouraged for some time now RFC6648.

I don't think placing the signature in a HTTP header ties the signature to the header. The signature can be removed from the HTTP header and is still fully functional. However, it is true that signed payload is not self-contained. I can see some value to having both the content and the signature in a single blob of data.

Which brings me to the HTTP Content-Encoding header. This header is designed to allow a message to still use the appropriate media type to describe the semantics of the un-encrypted message and then layer some kind of encoding on top of that message. The header is currently used for compression mechanisms like gzip, deflate and more recently brotli. The types of content encoding available is an HTTP extension point and therefore there is an IANA registry for listing the available encodings.

Interestingly, when I looked at the content encoding registry I noticed the aes128gcm encoding which is described in the related spec Encrypted Content-Encoding for HTTP. This spec appears to describe very similar capabilities to what we seem to be trying to achieve with the JOSE standards. I see no reason why a similar approach would not work for the JOSE case also.

[cyberphone]

For people like me who want to stick to JSON encoding as well maybe only having an encrypted sub object, Cleartext JWE will bring something very similar to this:

{
  "enc": "A128CBC-HS256",
  "alg": "ECDH-ES+A256KW",
  "kid": "example.com:p256",
  "epk": {
    "kty": "EC",
    "crv": "P-256",
    "x": "Fyzfe-04AJLJUObu2DVgL6PTmEckU2CNeDDpoMsGPto",
    "y": "k3bd9ccy6r7Lnd5o116_BD9sawwKGJDYUw7vhOTR6Rc"
  },
  "encrypted_key": "WIezNZ2eebtGoe0u13S8ye5dOHIF41iqRGXq2NdpQTulOdB0aQp3IQ",
  "iv": "vZoQqmLylkXhOeinqWa6og",
  "tag": "7SLHaUp1xzjOCVhek-meaA",
  "ciphertext": "jgNLUWMWyhfxMm-addN13GqQcbM_q6xcxRir0oW-WxA"
}

Cleartext JOSE versions could essentially be regarded as derived data types living in a JSON/JavaScript world.

[darrelmiller]

@cyberphone If you want to keep all of your security related information in a JSON payload, then OpenAPI is quite capable of describing that. The Schema Object can describe the structure of your payload including all the additional security metadata. I am not aware of anything that needs to be changed in OpenAPI to allow this.

[cyberphone]

First public submission:
https://tools.ietf.org/id/draft-erdtman-jose-cleartext-jws-00.html
https://tools.ietf.org/id/draft-erdtman-jose-cleartext-jwe-00.html

[cyberphone]

Just "for fun" there's now an entirely different proposition on the table 😂
Documentation: https://github.com/cyberphone/jws-jcs#combining-detached-jws-with-jcs-json-canonicalization-scheme
On-line demo: https://mobilepki.org/jws-jcs/home

[MikeRalphson]

Is there anything currently acting as a roadblock in the OAS to prevent use of JOSE signatures today? With the possible addition of a format value in the proposed registry, I can't see any particular problems with having a signature or __cleartext_signature property in schema objects.

[cyberphone]

The main (only?) problem is for implementations supporting signature schemes, at least this is what the Open Banking folks claim.

[MikeRalphson]

The main (only?) problem is for implementations supporting signature schemes,

Thanks - a concrete example in OAS terms would be enlightening.

[cyberphone]

@pleothaud is working on that

[ioggstream]

@cyberphone I made some investigations on header-based signatures here https://forum.italia.it/t/non-repudiation-and-rsa-considered-legacy/5152

There are a couple of issues, eg. whether the signature is part of the application logic or not.
In the first case, it could be useful to attach it to the data (eg. jcs) though you are going to lose endpoint informations.

[cyberphone]

@ioggstream You mention endpoint informations. Here I believe there are considerable problems causing Amazon AWS taking an entirely different route:
https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
This seems like a more correct method for REST than used by Open Banking:

Request = Operation + Data

I have dealt with this issue in a different (and much simpler) way:

{
  "requestUrl": "https://example.com/addData etc etc",
  "data-1..." : ..,
        .
   "data-n": ...,
   "signature": "jws-header..jws-signature"
}

[ioggstream]

@cyberphone both draft-cavage and signed-exchanges serialize endpoint informations (though in different ways - signed-exchanges takes into account a lot of security improvements implemented in TLS1.3).

Embedding data in payload is ok, as long as the application on the other side is aware of the signing mechanism (eg. it's not delegated to a gateway or an api-manager).

[ioggstream]

@darrelmiller your idea of a application/jose content-encoding seems very reasonable. #1464 (comment) Do you know of any attempt to register it with the IANA?

[whitlockjc]

I've been advocating for supporting JWT for authn/authz, and I think we can implement both of these at one time. Ideally, here's how we could do it:

1. Make JWK's an OAS component
2. Using JSON References, use these keys where necessary
   a. Security Schemes (JWT-based Security Scheme)
   b. Signed/Encrypted requests/responses (this issue)

I'm going to read this more thoroughly and will comment here. I will eventually suggest we break this out into three different features/PRs:

1. The ability to support JSON Web Keys in OAS via #/components/keys
2. Signed requests/responses
3. JWT-based Security Scheme and Security Requirements

[pleothaud]

Hi @whitlockjc

I'm Philippe Leohaud, working with @isamauny at 42Crunch.

I've been working with the TSC to define security extensions to support JOSE, JWT and HTTP SIgnature for the last year or so.

This is still a WIP but could you please have a look at https://docs.google.com/document/d/13THkz7867blJ464ohpz9MUrhMQp-28Cx77JvDHlyurc from OAI Google Drive and let me know what you think?

What I've done is obviously too complex and it would be nice to work with you to simplify it!

Many thanks in advance,

Phil

[cyberphone]

@pleothaud I could not open the document because I had no access right to it. Would it be possible giving it public read access?

[pleothaud]

@cyberphone @whitlockjc : document shared :-)

Eagerly waiting for your feedback

[cyberphone]

@pleothaud @darrelmiller @whitlockjc @MikeRalphson I'm not an expert on OAI but I have a comment about the document's description of Clear Text JOSE. The original idea as well as the draft has effectively been shelved. The signature version is now powered by an (almost) traditional canonicalization scheme which currently runs flawlessly on 5 quite different programming platforms. The new version also uses JWS exactly as described in RFC7515. There were indeed interest at the IETF-104, I eventually ended up with 100 minutes of meeting time but the transformation of this into an IETF standard is yet to be seen.

Playground: https://mobilepki.org/jws-jcs

With respect to OAI there is though a snag: The OAI architects insist that it is more important using the "full power" of the JSON RFC than to be compatible with Browsers/JavaScript/Node.js. Personally I don't see the logic behind that since the "crippled" JSON mode supported by JS (IEEE double precision Number) do not in any way hamper applications including usage of the recent JS addition BigInt. It seems like a pretty high price for saving a measly two characters on the wire. As far as I know nobody uses JSON Number for monetary data either. Pardon the rant...

[pleothaud]

I created another issue (#1953) explaining how you could describe message level security, Feedback welcome!

[mlainez]

Hi everyone, I wanted to give a bit more context linked to what @cyberphone mentioned in his comment about OpenBanking UK: #1464 (comment). Since September 2019, the banking sector in Europe has had to adapt to a new Payment Service Directive, PSD2. It forces banks to open up APIs to connect to payment accounts and to issue payments, and it applies to banks in 30 countries. In that context, two API security schemes have been inforced:

• The use of Qualified Web Application Certificates (QWAC) to connect to the banks, using mutualTLS (from what I see this will be in OAS3.1) at the transport layer.
• The use of Qualified Seal Certificates (QSealC) to sign requests between application layers of the consuming party and the banks.

This introduction has brought a lot of complexity in the banking sector and several banks are exposing their API spec using OAS but since the spec doesn't offer support for mutualTLS nor HTTP signature, it makes it impossible to benefit from all the advantages of using OAS. For instance, you can't generate wrappers that work out of the box.

Support in OAS would have a major impact on the financial sector in EU and be an enabler for quite a lot of people using it.

[handrews]

@pleothaud

The JWT RFC (7519) also defines a specific content-type for JWTs, "application/jwt".

For these media types, we could therefore extend the Open API mediaType Object so that it can contain a cryptographicDefinitionsUrl field that will allow for the retrieval of the cryptographic contraints that must be fulfilled to consume the API (authorized algorithms, key to be used for encryption, authorised key type and key reference mechanisms for signature) as well as the description of the cryptographic operations computed on the API responses so that the client will know how to decrypt/verify the signature of the responses.

For OAS 3.1 and later, the JSON Schema draft 2020-12 specification is capable of modeling JWTs already, and includes an example of how to do it.

I would assume that a similar approach could be taken to other JOSE formats.

What needs to be in the OAS itself? Perhaps a note that tooling SHOULD (or MUST) support these JSON Schema techniques? (also pinging @cyberphone ) We could only add a MUST in 3.2, but we might be able to add a weaker requirement as a "clarification" in 3.1.1 since the JSON schema spec that is referenced clearly shows how to do it.

@whitlockjc do these things really need to be their own components (which would require an OAS 3.2) or is the ability to model them in OAS 3.1 Schema Objects sufficient?

@mlainez perhaps I'm missing something (we're well outside my area of expertise), but it seems like QWAC and QSealC are different (or additional?) technologies and probably deserve their own issue.

[cyberphone]

This is a really old issue; lots of things have happened since I created it.
Most important is probably the HTTP Message Signatures standard: https://www.rfc-editor.org/rfc/rfc9421.html

Personally, I have given up on JSON in favor of CBOR.
Rationale: https://github.com/cyberphone/cbor-everywhere
Related rationale: #1517
Even works in JavaScript: https://github.com/cyberphone/CBOR.js#cborjs

[handrews]

@cyberphone it sounds to me like you're suggesting closing this issue as no longer relevant. Is that the case? (if there's something that needs to be done with HTTP Message Signatures that should get its own issue, and I'm aware of the CBOR request- I personally hope we can support CBOR and CDDL in OAS 4 "Moonwalk" but that's a different topic).

[handrews]

(BTW I'd still like to hear from @pleothaud @whitlockjc and @mlainez even if @cyberphone thinks this should be closed- I'll keep it open for a while to give folks a chance to reply).

[mlainez]

@handrews you can indeed close this one as far as I'm concerned. Adding mutual TLS support to open api is still a relevant topic imo but probably outside of the scope of this specific issue.

[handrews]

@mlainez thanks! MutualTLS was added as a Security Scheme in 3.1. If there's more work to be done for it, yeah that should get a separate issue.

[SensibleWood]

@handrews I think the merit of the issue depends on scope.

If it's "just" describing a JWT accurately using a Schema Object then we are probably golden (other than guidance about serialization/deserialization, which would probably be handy for consumers).

However, the title says support for JOSE signature and encryption and there's probably a bunch of stuff we could either implement or more probably point to - validation of alg property, correlating kid in a JWKS, etc. I think this stuff is probably well covered elsewhere, but it could do with nice pointers from OpenAPI a la what we do with the OpenID Connect URL i.e. providing the pointer and (limited) guidance, but leaving the meat of it to OpenID itself.

Obvs this relies on some investigation to assertain the right "stuff" to include. I think JOSE is popular enough to merit some time well spent looking at it (I'd be happy to help on that if is prioritised)

[cyberphone]

@handrews Yes, for signing HTTP requests and responses, RFC 9421 is probably a better choice than the JOSE stack. Individual elements may of course be expressed as JWS, JWT, and JWE tokens, but that should (hopefully) not affect Open API.

For my own work I stick to enveloped signature schemes that began with JSON canonicalization (RFC 8785). Nowadays I build on deterministically encoded CBOR which eliminates both wrapping data-to-be-signed and canonicalization, not to mention having a fully standard 64-bit integer representation. MSFT and Oracle did never agree on the latter for JSON. The ability to serialize a signed HTTP request enabled by enveloped signatures is a highly desired feature in my designs.

[mlainez]

@handrews My apologies, I didn't know mutual TLS was added as a security scheme in the meantime. The truth is that since I worked on that issue, I had to move on to other things that didn't involve Open API anymore... But thanks for following up!

[handrews]

@mlainez no worries, I don't expect folks to even still be paying attention 2.5 years after their last comment, so thanks for replying at all! I think when you last commented, 3.1 wasn't widely supported so it might not have been relevant enough to day-to-day work to notice.