PROPOSAL: A new document structure for this API

[adrianhopebailie]

There are a number of issues raised and comments made about the need to explicitly define the role of payment mediator and user agent. I propose that we separate the current API spec into a spec defining the conformance criteria for a payment mediator and a spec defining conformance criteria for a user agent in it's role as a payment mediator but also in its role as data collection service to optimize the checkout flow.

Proposal

1. Create a new payment mediator specification that defines the conformance criteria for any mediator (not just a user agent in this role).
2. Migrate the appropriate elements of the API specification into this new specification.
3. Make the new specification a normative dependency of the API specification.
4. Update the API specification so that it describes the conformance criteria of a user agent in assuming two roles:
   1. Payment mediator (as defined in the payment mediator spec)
   2. Data collection mediator for collection of data to optimize the checkout flow.

[ianbjacobs]

@adrianhopebailie,

+1 at a high level to the proposal. I think it will also involve moving some material out of the payment app specification (and that one should target payment app developers and will also refer to the mediator spec)

Ian

[adrianhopebailie]

I think it will also involve moving some material out of the payment app specification (and that one should target payment app developers and will also refer to the mediator spec)

+1 - I didn't mention that yet because the payment apps spec is still just a proposal itself

[mattsaxon]

+1

[msporny]

Create a new payment mediator specification that defines the conformance criteria for any mediator (not just a user agent in this role).

Generally +1, but I'm struggling to understand exactly what language and how the conformance criteria would work for such a meta spec. For example, the algorithm used to select payment apps being placed in a payment mediator spec makes sense to me and I know what that would look like.

How a payment request is handed off to a Web-based payment app via HTTP is not clear to me and the only language I could think of putting in such a spec would be a bit toothless/generic. Maybe that's fine, but a more concrete spec would help me understand this a bit better.

Migrate the appropriate elements of the API specification into this new specification.

What are the appropriate elements? Which sections would move to this new payment mediator spec? Also - whatever happened to "payment agent", I find that wording better than "payment mediator" as 'mediator' tends to imply that there is conflict.

Make the new specification a normative dependency of the API specification.

+1 if we can figure out the bits above.

Update the API specification so that it describes the conformance criteria of a user agent in assuming two roles:
Payment mediator (as defined in the payment mediator spec)

+1, again if we can get the wording right, which I'm concerned about. We'd want to make the HTTP API dependent on the mediator spec as well.

Data collection mediator for collection of data to optimize the checkout flow.

-0.5 - we're not chartered to build a generalized data collection mechanism. I get that doing that is the best way to design this shipping address collection mechanism that Google/Microsoft are proposing, and maybe that should be another sign that what we're attempting to do there is problematic. We should either commit to doing a generalized user-provided data collection API (and spin up a separate WG to do that), hand this over to the Verifiable Claims group, or keep this API highly specific and very narrow in the payment request API.

[msporny]

@dlongley and I had a discussion about the various specs that this group is thinking of publishing and how they all fit together. After a bit of back and forth, we arrived at this organizational structure:

The identified goals are:

• Create a cohesive set of specifications that reflect the current direction of the group while also ensuring the proper separation of concerns.
• Ensure it's easy to write new extensions to the specifications.
• Don't publish so many specifications that it's hard for people to get their mind wrapped around the architecture.

Here are the specs and what they do:

Web Payments Architecture

Details the Web Payments ecosystem by describing payment apps, payment mediator/agents, payment messages, and how messages are routed from payee to payer and back. Demonstrates that the architecture is flexible and doesn't define just one protocol for routing messages (HTTP /and/ browser are supported, for example).

Core Messages

Describes the payment request and payment response messages. Integrates the payment method identifiers spec. Details how message validation and extensibility is achieved.

Payment Method Specific Specs

Credit Card Messages and SEPA CT/DD Messages specs detail how payment request and response in Core Messages specs are augmented to provide credit card or SEPA functionality. There will be many of these specs, one for every different type of payment method (eventually).

Payment Apps

Describes the conformance criteria for a payment app. Basically, that a payment app receives a payment request and produces a payment response.

We assert (and this may be controversial) that a payment mediator / payment agent is just a specialized subclass of payment app. A payment mediator receives a payment request and produces a payment response by routing that request elsewhere (that's the definition of a payment app, btw). The difference between the foundational definition of a payment app and a payment mediator/agent is that the latter can keep track of multiple payment apps that can process a payment and thus has registration functionality inherent in it.

So, the beginning of the payment apps spec would talk about payment apps generically and the bottom of the spec would define a new subclass called the payment mediator / payment agent, which would have the algorithms for registration and payment app selection.

HTTP API

The HTTP API would define how a payment mediator/agent accepts a payment request over HTTP and responds via HTTP. It would also define how payment app registration is done over HTTP.

Browser API

The browser API would define how a payment mediator/agent accepts a payment request via a browser API call and responds in the same way. It would define how payment app registration is done via a browser. It can also have all of the user data collection bits that Google/Microsoft have proposed.

Alternatives

To be clear, we /could/ split out payment mediator/ payment agent from the payment apps spec. We could split registration out from the HTTP API or the Browser API. I don't think we should do that until it's clear that we are going to hold up CR by doing so.

I'd like the opportunity to present this during a telecon so that the group can get their mind wrapped around what's being proposed as explaining this in an issue tracker is not ideal.

[dlongley]

The HTTP API would define how a payment mediator/agent accepts a payment request over HTTP and responds via HTTP. It would also define how payment app registration is done over HTTP.

To clarify, this spec would define how to send payment requests over HTTP and how to respond (this is applicable to any Payment App). It would also define how Payment App registration is done over HTTP -- for those who are also implementing the specialized class of Payment App called a Payment Agent/Mediator.

[adrianhopebailie]

Having recently done a more thorough review of the architecture document I have a slightly different take on this. I don't see enough content for that document if we describe the core components in other specs.

I propose that the architecture spec should define the roles of the payment app and the payment mediator in the form of conformance criteria for those roles. It should describe payment methods and how these are used to provide extinsibility and then refer to another spec (currently the Payment Method Identifiers spec) which provides details on how to define a payment method and the minimum requirements for a new payment method specific spec.

The HTTP and browser API specs will depend on these two and define the implementation specifics of implementing:

1. A payment mediator in a user agent
2. A payment app as a web app
3. A payment mediator as a web service

We could have a stand alone messages spec which defines the base schema of the messages and the rules for encoding these in different forms but this could also be encapsulated in the two API specs.

I am working on a concrete proposal following this approach and will share it in the next few days.

[ianbjacobs]

Thanks Manu and Dave, diagrams are helpful. I've attached an alternative view.

Here's a text description of the relationships as I conceive of them:

• Applications will enable users to initiate payments. In a browser context, Web applications will use
  the paymentRequestAPI. In other contexts, applications will use the HTTP API.
• Payment requests will either be routed through a Mediator to enable the user to choose a Payment App. I can also imagine that in some contexts people may wish to route requests directly to a Payment App. The information that is sent to the Payment App (directly or through the Mediator) will depend on the payment method.
• Payment methods will be identified with Payment Method Identifiers. Payment Method specifications will describe the input and output data that will flow from the payee to the payer and back.

Thus:

• All the specs will depend on the Payment Method Identifier spec
• Both of the API specs (paymentRequest, HTTP) will rely on the Mediator and App specs for handing off payment requests and receiving responses.
• The paymentRequest API and HTTP API specifications could end up sharing a parameter definition spec; I hope they do but I'd like to see each one mature before we factor that out.

Ian

[msporny]

@ianbjacobs wrote:

Here's a text description of the relationships as I conceive of them

@dlongley and I have read over your first three bullet points multiple times and agree with them. So, I think we're agreed on those 3 bullet points (did you think we didn't agree on those bullet points?).

Only one minor nit on your conclusion (agree w/ the other bullet points not mentioned):

All the specs will depend on the Payment Method Identifier spec

I assume you mean that there will be a fairly substantial rework/modification of the content of this spec (to the point that it's a new spec). The current PMI spec is only about the identifiers. I suggest (integrating some of @adrianhopebailie's ideas):

• Renaming the spec to "Payment Methods"
• Add content to the spec that provides details on how to define a payment method and the minimum requirements for a new payment method specific spec

I'll note that we keep deviating from hard lessons learned in ISO20022 and IETF. Transactional systems (like payment systems) are typically best modeled as messages + protocols (many systems fit well into that model). A Payment Method is just another type of message in the system that expands on the core payment request / response.

[msporny]

@adrianhopebailie wrote:

We could have a stand alone messages spec which defines the base schema of the messages and the rules for encoding these in different forms but this could also be encapsulated in the two API specs.

@ianbjacobs wrote:

The paymentRequest API and HTTP API specifications could end up sharing a parameter definition spec; I hope they do but I'd like to see each one mature before we factor that out.

I haven't heard a good reason to not have a common format for the payment request and payment response messages. Both the Browser API and the HTTP API are expected to communicate with Payment Apps by sending them a payment request and getting back a payment response. Why are we saying that the message that a browser sends might be different from the one that a non-browser sends? Seems like an interoperability failure waiting to happen.

[adrianhopebailie]

Why are we saying that the message that a browser sends might be different from the one that a non-browser sends?

My point, put differently, is this; the messages (encoded as JSON) could simply be defined in the HTTP API spec and then in the browser API spec all we require are rules for translating between these two forms.

I am loathe to produce too many separate documents unless it is really necessary.

[msporny]

I am loathe to produce too many separate documents unless it is really necessary.

Agreed.

all we require are rules for translating between these two forms

Having tried to come up with a generalized algorithm to do this a long time ago (while working on JSON-LD), I don't think this is as easy as it sounds. The data model for JSON and WebIDL are not perfectly aligned. The same goes for differences between JSON and JSON-LD and JSON and XML. There are no simple rules for translating between one and the other and/or back, you end up writing complex, object+syntax-specific algorithms. You can punt by calling .toString(), but even then there are issues - like how you deal w/ numbers.

The counter-proposals, which is what the Core Message spec does, is define a generic base data model and then provides pointers to how you express the message in each leaf syntax.

[adrianhopebailie]

Having tried to come up with a generalized algorithm to do this a long time ago, I don't think this is as easy as it sounds. There are no simple rules for translating between JSON and WebIDL and/or back, you end up writing complex, object-specific algorithms.

+1 - I don't think we want to do that.

We need SOME place to describe the translation for these two objects: payment request and payment response. I propose we do it in the browser API spec because that's where we define the WebIDL.

[msporny]

I propose we do it in the browser API spec because that's where we define the WebIDL.

If we do that, the HTTP API spec will be normatively dependent on the browser API spec. That would be a bad thing for at least two reasons:

1. There should be no dependency between the HTTP API spec and the Browser API spec - they shouldn't be bound together at all because they're for completely different execution environments.
2. The HTTP API messages would have a core dependency on a WebIDL definition and I don't know of any other HTTP API that specifies the core messages based on a translation of a WebIDL definition.

[adrianhopebailie]

If we do that, the HTTP API spec will be normatively dependent on the browser API spec.

An visa versa. Is that a problem?

I think that the HTTP API, browser API, payment method and architecture specs should all be rec track and normatively inter-dependent 😄

They come as a package

[msporny]

An visa versa. Is that a problem?

Yes, that's a big problem. :)

I think that the HTTP API, browser API, payment method and architecture specs should all be rec track and normatively inter-dependent.

We're miscommunicating.

All W3C specs have a set of dependencies, and those dependencies are directed acyclic graphs.

When an implementer sits down to try and implement something, they pick a leaf spec and end up pulling in all of that's leaf spec's MUST language... and it's dependencies MUST language... all the way up the spec chain.

As a spec editor, you try to minimize the amount of MUST language an implementer pulls in (the less MUST language, the easier the implementation and the easier it is to do conformance testing). The less specs the developer has to read, the easier the implementation.

What you're saying sounds like someone implementing a simple HTTP API payment agent/mediator has to pick up all of the specs including the Browser API one to implement.

What I'm proposing is that they shouldn't have to read anything in the Browser API to do so (because they're not implementing any portion of the Browser API and the messages they have to implement are clearly defined elsewhere - in the Core Messages spec).

[ianbjacobs]

@msporny,

@dlongley and I have read over your first three bullet points multiple times and agree with them. So, I think we're agreed on those 3 bullet points (did you think we didn't agree on those bullet points?).

I am glad to hear there is agreement. My thoughts are still evolving, but glad to hear there's a chance for us to move in the same direction.

[Regarding the payment method identifier spec] I assume you mean that there will be a fairly substantial rework/modification of the content of this spec (to the point that it's a new spec). The current PMI spec is only about the identifiers.

Add content to the spec that provides details on how to define a payment method and the minimum requirements for a new payment method specific spec

I agree we "want" both things: payment method identifiers and instructions for how to write a payment method identifier spec.

It is not clear to me (yet) that both of those belong in the same spec because the audiences are different:

• The primary consumers of the payment method identifier spec will be Web app developers, payment app developers, and those writing implementations of the APIs.
• The primary consumers of the "how to write a payment method spec" will be people who design payment methods.

So while the topics are indeed related (and we see them come together in each "payment method spec"), the audiences may be different enough to split the material into two resources.

Furthermore, I think we may have several resources that are not really normative (for implementations) but rather good practices, and those resources should be able to evolve on an ongoing basis (e.g., in the group's github repo). These include:

• How to write a payment app (showing some examples of how to use different payment methods)
• How to write a Web app to use the paymentRequest API
• How to write a payment method specification

Ian

[adrianhopebailie]

Yes, that's a big problem. :)

Sorry I should have said, actually visa versa. I can see a configuration of the documents where the JSON encoding of a payment request and response and the mechanism for submitting these to a web based payment app are defined in the HTTP API.

The browser API would be dependent on the HTTP API and would describe how a paymentRequest is converted to the JSON format defined in the HTTP spec and then submitted to a web based payment app (as described in the HTTP API).

[msporny]

I should have said, actually visa versa.

I knew what you meant :), and I think that the reverse would not be good either (having the Browser API spec depend on the HTTP API). I don't know of a W3C spec that takes this approach and that might be a signal that this approach is on the wrong path. That said, we need some concrete proposals on the table before discussing further, so I await your next set of proposals @adrianhopebailie.

I'm also concerned about the notion of a "web based payment app" - as one could argue that the Browser API is a perfectly valid way to talk to a "web based payment app"... and that the HTTP API is a perfectly valid way to talk to a "web based payment app". I can see both arguments and am worried that we're headed down a path where we have two mechanisms of talking to a web based payment app.

Instead, we may want to say that a payment app can support two different protocols - an HTTP API based protocol, or a browser API based protocol. The first is non-interactive, the second is interactive. In any case, just pointing out that we still have a bit to go wrt. understanding how this entire ecosystem fits together (but are making progress).

Can we have a call where we hash this stuff out? Trying to do this on an issue thread is taking forever and leading to a good bit of miscommunication and (possibly) selective hearing.

[msporny]

The primary consumers of the payment method identifier spec will be Web app developers

I don't think we should count on web app developers reading any W3C spec. I don't think they're the audience for W3C specs - implementers are. I'm going to assert (arguably) that Web app developers go to tutorial sites and blog posts to learn about W3C technologies... they don't (as a general rule) read dense W3C specs.

payment app developers, and those writing implementations of the APIs. The primary consumers of the "how to write a payment method spec" will be people who design payment methods.

I think all of these people have roughly the same skill level and will read the W3C specs. I do agree that "How to write a payment app" and "how to write a payment method spec" are two different audiences.

How to write a payment app (showing some examples of how to use different payment methods)
How to write a Web app to use the paymentRequest API
How to write a payment method specification

I agree that these are good tutorials to have out there. I doubt that this WG will be able to produce and maintain those documents. I know of very few W3C tutorials like this that have kept pace w/ the quality of the tutorials created by web authors out in the wild. In other words, I don't think the group should spend time on it because others will do a better job than we can.

[ianbjacobs]

Here's another graphic (with FPWD work highlighted). AdrianHB is making the case to me that the payment app and mediator specs should be merged. I still think that they address different audiences, although I agree they are closely related.

Ian

[mattsaxon]

I believe they should be le speedster as they address different concerns as well as having a different audience.

[mountainhippo]

@mattsaxon - I think autocorrect has mangled your last comment
@ianbjacobs - i like this articulation of structure in your graphic because it highlights something that is troubling me currently, which is that we have gone to FPWD with components in at the top and the bottom of the stack but not in the middle. Notwithstanding @zkoch 's very helpful priorities list for experimental implementations, I think wearing my chair's hat that getting proposals for payment mediator and payment app are now super important and I will be looking for volunteers to push this forward on the call today.

[adrianhopebailie]

@mattsaxon I have put a proposal together that shows what an architecture document looks like that would combine descriptions of the roles of payment mediator and payment app.

https://w3c.github.io/webpayments/proposals/architecture/

I don't agree that they will target different audiences as it's quite possible that in many implementations both roles will be assumed by the same system (e.g. In a browser or in a head-less IoT system where the payer is a device that implements both roles using HTTP APIs).

My proposal is that the payment apps document (https://w3c.github.io/webpayments/proposals/paymentapps/payment-apps.html) is now split into three parts:

1. Some elements of it will be incorporated into the new architecture document (where similar content is not already there)
2. The user agent specific elements regarding registration and passing payment messages between the user agent and apps should be in-corproated into the browser API
3. The manifest specific elements should be submitted for inclusion into the manifest spec (I have already started engaging the editors on this).

What we will have are two base specs, the Architecture and the Payment Method Identifiers which provide the foundation for all implementations.

On top of these we will have specs that define how the roles are implemented in different ways (in browser or as HTTP client/server) and specs that define the specifics of how a payment method works.

I realize that this has now dropped the information about what specs we have produced and how they relate. This COULD be included in the architecture document or in a stand-alone note from the WG.

@mountainhippo said:

getting proposals for payment mediator and payment app are now super important and I will be looking for volunteers to push this forward on the call today.

I'm hoping that the WG will build on the new arch proposal I have made and that other editors will step up to flesh this document out to meet our needs.

[dret]

as mentioned in #148, it seems to me that there is little benefit in having a separate "identifier" spec. coming from an IETF background where the "registry pattern" is used a lot, i think it makes it easier to define the registry contents and operations in the same spec as where the registry values are used. after all, the spec using the registry values and the registry spec itself both normatively depend on each other, correct?

[ianbjacobs]

@dret,

As I commented [1], it's not clear yet whether we will have a registry.

If we do, my first instinct is that registry information should be part of the identifier
spec.

Ian

[1] #148 (comment)

[dret]

On 2016-04-21 14:19, ianbjacobs wrote:

As I commented [1], it's not clear yet whether we will have a registry.
If we do, my first instinct is that registry information should be part
of the identifier spec.

my apologies for not being clear enough. absolutely the registry
definition should be part of the identifier definition. what i wanted to
say is that those two should not be a separate spec to begin with, but
part of the API spec that uses and needs the identifiers.

[adrianhopebailie]

Migrated to w3c/webpayments#134