New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Canonical transaction paths #227

Open
netmilk opened this Issue Jun 22, 2015 · 10 comments

Comments

Projects
None yet
5 participants
@netmilk
Copy link
Contributor

netmilk commented Jun 22, 2015

I propose to start using new syntax for addressing transactions compiled from the blueprint in hooks instead of transaction names. Transaction names will be supported for backward compatibility.

Transaction names were historically intended only as bread crubs — a human readable location of the specific transaction in the blueprint. Transaction name was never meant as a canonical transaction identifier and that's the reason for its terrible indeterminism.

Format will be a concatenation/serialization of the origin object:

  • Use colon : character as a delimiter
  • Colon character in API Name, Resource Name, Resource Group Name, Action Name or Example Name will be escaed with backslash
  • Examples are identified by string "Example " + its index in array starting from 1 (not 0)

Possible questions:

  • Omit leading "Example " string and use zero indexed position?
  • Use different delimiter?
  • Use delimiter string at the beginning of the path?
  • Does term "Canonical transaction path" make sense?

Examples

1. Full notation with multiple request-response pairs

# Some API Name

## Group Some Group Name

### Some Resource Name [/resource]

#### Some Action [GET]

+ Request (application/json)
+ Response 200(application/json)

+ Request (application/xml)
+ Response 200 (application/xml)

Transaction origin object:

{
  "apiName": "Some API Name",
  "resourceGroupName": "Some Group Name",
  "resourceName": "Some Resource Name",
  "actionName": "Some Action Name",
  "exampleName": "Example 2"
}

Compiled canonical path:

Some API Name:Some Group Name:Some Resource Name:Some Action Name:Example 2

2. Full notation without group

# Some API Name

### Some Resource Name [/resource]

#### Some Action [GET]

+ Request (application/json)
+ Response 200 (application/json)

Transaction origin object:

{
  "apiName": "Some API Name",
  "resourceGroupName": "",
  "resourceName": "Some Resource Name",
  "actionName": "Some Action Name",
  "exampleName": "Example 1"
}

Compiled canonical path:

Some API Name::Some Resource Name:Some Action Name:Example 1

3. Full notation without group and API name

### Some Resource Name [/resource]

#### Some Action [GET]

+ Request (application/json)
+ Response 200 (application/json)

Transaction origin object:

{
  "apiName": "",
  "resourceGroupName": "",
  "resourceName": "Some Resource Name",
  "actionName": "Some Action Name",
  "exampleName": "Example 1"
}

Compiled canonical path:

::Some Resource Name:Some Action Name:Example 1

4. Simplified notation

# GET /message
+ Response 200 (text/plain)

      Hello World

Transaction origin object:

{
  "apiName": "",
  "resourceGroupName": "",
  "resourceName": "/message",
  "actionName": "GET",
  "exampleName": "Example 1"
}

Compiled canonical path:

::/message:GET:Example 1
@netmilk

This comment has been minimized.

Copy link
Contributor

netmilk commented Jul 14, 2015

Implemented here apiaryio/blueprint-transactions#1. To be implemented in Dredd's hook handler.

@honzajavorek

This comment has been minimized.

Copy link
Member

honzajavorek commented Jan 25, 2016

I am proposing a slightly different approach.

Introduction

The proposal below attempts to address following issues:

  • Transaction Examples are hard to identify (just by enumeration: Example 1, Example 2) and because they're implicit, people do not know their concept well and confuse them with Requests. I removed them from the Transaction Path. (apiaryio/blueprint-transactions#3)

  • Since Transaction Examples and Request-Response pairs are different things, the original specification above doesn't allow addressing complex combinations, such as multiple Requests with multiple Responses. I included identifiers for Requests and Responses as components of the Transaction Path. (#25, #78, #294)

  • Resulting Transaction Path should be much more descriptive and unique. It still won't catch all scenarios though. Example of an edge case:

    + Request
    + Response 200 (application/json)
        + Body ...
    + Response 200 (application/json)
        + Body ...

    However, while such thing is valid API Blueprint syntax, it is ambiguous and should be treated as such. Transaction Path won't distinguish between the two Responses and Dredd will refuse to consume such blueprint. That would be considered as correct and expected behavior, because Dredd can't possibly guess what to test.

Transaction Path (string)

Canonical, deterministic way how to address a single HTTP Transaction (single Request-Response pair). It can serve as a unique identifier of the HTTP Transaction.

The Transaction Path string is a serialization of the pathOrigin object according to following rules:

  • Colon : character as a delimiter.
  • Colon character, which happens to be part of any component of the path, is escaped with backslash character \.
  • No other characters than colon : are escaped.

Components

  • If certain component isn't available in the source data, it is present in both the pathOrigin object and the Transaction Path as an empty string.
  • The requestName component is defined as name of the request followed by parentheses containing effective value of the Content-Type header. Any of those two parts can be omitted. If both parts are present, they're separated by a single space character.
  • The responseName component is defined as HTTP code of the response followed by parentheses containing effective value of the Content-Type header. The Content-Type part can be omitted. If both parts are present, they're separated by a single space character.

Examples

Full Notation with Multiple Request-Response Pairs

# Sample API Name

## Group Sample Group Name

### Sample Resource Name [/resource]

#### Sample Action Name [POST]

+ Request (application/json)
+ Response 200 (application/json)

+ Request (application/xml)
+ Response 200 (application/xml)

Transaction Path Origin:

{
  "apiName": "Sample API Name",
  "resourceGroupName": "Sample Group Name",
  "resourceName": "Sample Resource Name",
  "actionName": "Sample Action Name",
  "requestName": "(application/xml)",
  "responseName": "200 (application/xml)",
}

Transaction Path:

Sample API Name:Sample Group Name:Sample Resource Name:Sample Action Name:(application/xml):200 (application/xml)

Full Notation with Implicit Request

# Sample API Name

## Group Sample Group Name

### Sample Resource Name [/resource]

#### Sample Action Name [POST]

+ Response 200

Transaction Path Origin:

{
  "apiName": "Sample API Name",
  "resourceGroupName": "Sample Group Name",
  "resourceName": "Sample Resource Name",
  "actionName": "Sample Action Name",
  "requestName": "",
  "responseName": "200",
}

Transaction Path:

Sample API Name:Sample Group Name:Sample Resource Name:Sample Action Name::200

Full Notation with Multiple Requests within One Transaction Example

# Sample API Name

## Group Sample Group Name

### Sample Resource Name [/resource]

#### Sample Action Name [POST]

+ Response 200 (text/plain)

+ Request Sample Request Name (application/json)
+ Request Another Sample Request Name (application/json)
+ Request (application/json)
+ Response 200 (application/json)
+ Response 401 (application/json)

Transaction Path Origin:

{
  "apiName": "Sample API Name",
  "resourceGroupName": "Sample Group Name",
  "resourceName": "Sample Resource Name",
  "actionName": "Sample Action Name",
  "requestName": "Another Sample Request Name (application/json)",
  "responseName": "401 (application/json)",
}

Transaction Path:

Sample API Name:Sample Group Name:Sample Resource Name:Sample Action Name:Another Sample Request Name (application/json):401 (application/json)

Full Notation with Multiple Requests Each Having Multiple Responses

# Sample API Name

## Group Sample Group Name

### Sample Resource Name [/resource]

#### Sample Action Name [POST]

+ Response 200 (application/json)
+ Response 401 (application/json)
+ Response 500 (application/json)

+ Request (application/xml)
+ Response 200 (application/xml)
+ Response 500 (application/xml)

Transaction Path Origin:

{
  "apiName": "Sample API Name",
  "resourceGroupName": "Sample Group Name",
  "resourceName": "Sample Resource Name",
  "actionName": "Sample Action Name",
  "requestName": "",
  "responseName": "401 (application/json)",
}

Transaction Path:

Sample API Name:Sample Group Name:Sample Resource Name:Sample Action Name::200 (application/json)

Full Notation with Resolution of the Effective Content-Type Value

# Sample API Name

## Group Sample Group Name

### Sample Resource Name [/resource]

#### Sample Action Name [POST]

+ Request
    + Headers

            X-Request-ID: 30f14c6c1fc85cba12bfd093aa8f90e3
            Content-Type: application/hal+json

+ Response 200

Transaction Path Origin:

{
  "apiName": "Sample API Name",
  "resourceGroupName": "Sample Group Name",
  "resourceName": "Sample Resource Name",
  "actionName": "Sample Action Name",
  "requestName": "(application/hal+json)",
  "responseName": "200",
}

Transaction Path:

Sample API Name:Sample Group Name:Sample Resource Name:Sample Action Name::200 (application/json)

Full Notation without Group

# Sample API Name

### Sample Resource Name [/resource]

#### Sample Action Name [POST]

+ Request (application/json)
+ Response 200 (application/json)

Transaction Path Origin:

{
  "apiName": "Sample API Name",
  "resourceGroupName": "",
  "resourceName": "Sample Resource Name",
  "actionName": "Sample Action Name",
  "requestName": "(application/json)",
  "responseName": "200 (application/json)"
}

Transaction Path:

Sample API Name::Sample Resource Name:Sample Action Name:(application/json):200 (application/json)

Full Notation without Group and API Name

### Sample Resource Name [/resource]

#### Sample Action Name [POST]

+ Request (application/json)
+ Response 200 (application/json)

Transaction Path Origin:

{
  "apiName": "",
  "resourceGroupName": "",
  "resourceName": "Sample Resource Name",
  "actionName": "Sample Action Name",
  "requestName": "(application/json)",
  "responseName": "200 (application/json)"
}

Transaction Path:

::Sample Resource Name:Sample Action Name:(application/json):200 (application/json)

Full Notation without Group and with API Name Containing a Colon

# My API: Revamp

### Sample Resource Name [/resource]

#### Sample Action Name [POST]

+ Request (application/json)
+ Response 200 (application/json)

Transaction Path Origin:

{
  "apiName": "My API: Revamp",
  "resourceGroupName": "",
  "resourceName": "Sample Resource Name",
  "actionName": "Sample Action Name",
  "requestName": "(application/json)",
  "responseName": "200 (application/json)"
}

Transaction Path:

My API\: Revamp::Sample Resource Name:Sample Action Name:(application/json):200 (application/json)

Simplified Notation

# GET /message
+ Response 200 (text/plain)

      Hello World

Transaction Path Origin:

{
  "apiName": "",
  "resourceGroupName": "",
  "resourceName": "/message",
  "actionName": "GET",
  "requestName": "",
  "responseName": "200 (text/plain)"
}

Transaction Path:

::/message:GET::200 (text/plain)
@netmilk

This comment has been minimized.

Copy link
Contributor

netmilk commented Feb 12, 2016

Thanks @honzajavorek so much for this proposal incorporating solution for identifying multiple request and responses addressing #25 #294 #361 and others

My $0.50 would be:

  • Dredd should throw a description error and exit with non zero status code if some path isn't unique across all API documents loaded to the runtime
  • Please provide Transaction Path Origin and Path Examples for all Request-Response pairs in the "Full Notation with Multiple Requests Each Having Multiple Responses" example
  • When you are referring to some Transactionorigin object, please distinguish between the legacy Transaction Name Origin used for the Transaction Names or the actual Transaction Path Origin used for these Canonical Transaction Paths
  • Please stress the fact this change isn't braking the backward compatibility for integrations with the Transaction Name Origin object in the transaction object being passed to hook functions and hooks using Transaction Names.

The Transaction Name Origin and Transaction Name will be still supported but it will be considered as deprecated. The name compilation will be maintained in the reporters not in the transaction compiler in the future.

Regarding the Multiple Request and Responses support:

  • For keeping the backward compatibility I propose Dredd to test only the first Transaction (first request and first response in the Example) by default.
  • All other transactions from the Example will be passed to the runner as skipped (transaction.skip = true)
  • Any skipped transaction can be enabled by "unskipping" it in hooks (transaction.skip = false)
  • Introduce JS and CLI option for enabling all transactions (request-response pairs) to be tested by default e.g. --unskip-all or --all-pairs

honzajavorek added a commit that referenced this issue May 20, 2016

Removing redundant file.
Canonical paths are already documented enough in https://github.com/apiaryio/dredd-transactions and #227. And this file isn't linked from anywhere.

honzajavorek added a commit that referenced this issue Aug 24, 2016

fix: Use just the 1st req-res pair within APIB transaction examples
This change tests and fixes a problem introduced with migration to API Elements
in order to support Swagger in Dredd. Original implementation always selected
the first request-response pair from each transaction example. This wasn't
re-implemented correctly on top of API Elements. Instead, all specified responses
are appearing, which breaks Dredd's behavior in many ways. Respective test was
ported, but unfortunately with the same mistake. This commit fixes the situation.

Some early adopters discovered the issue and considered it to be a new
feature, but it really breaks how Dredd should work at the moment and needs
to be removed. It leads to duplicate transaction names and other undefined
behavior.

In order to implement #25 and #78,
which many believed happened when they discovered the bug, much more work needs
to be done. Namely designing and adopting a new way of addressing transactions
in Dredd #227.

Closes #615

BREAKING CHANGE

honzajavorek added a commit to apiaryio/dredd-transactions that referenced this issue Aug 24, 2016

fix: Use just the 1st req-res pair within APIB transaction examples
This change fixes a problem introduced with migration to API Elements in order
to support Swagger in Dredd. Original implementation always selected the first
request-response pair from each transaction example. This wasn't re-implemented
correctly on top of API Elements. Instead, all specified responses are appearing,
which breaks Dredd's behavior in many ways. Respective test was ported, but
unfortunately with the same mistake.

Some early adopters discovered the issue and considered it to be a new
feature, but it really breaks how Dredd should work at the moment and needs
to be removed. It leads to duplicate transaction names and other undefined
behavior.

In order to implement apiaryio/dredd#25 and apiaryio/dredd#78,
which many believed happened when they discovered the bug, much more work needs
to be done. Namely designing and adopting a new way of addressing transactions
in Dredd apiaryio/dredd#227.

BREAKING CHANGE

honzajavorek added a commit that referenced this issue Aug 24, 2016

fix: Use just the 1st req-res pair within APIB transaction examples
This change tests and fixes a problem introduced with migration to API Elements
in order to support Swagger in Dredd. Original implementation always selected
the first request-response pair from each transaction example. This wasn't
re-implemented correctly on top of API Elements. Instead, all specified responses
are appearing, which breaks Dredd's behavior in many ways. Respective test was
ported, but unfortunately with the same mistake. This commit fixes the situation.

Some early adopters discovered the issue and considered it to be a new
feature, but it really breaks how Dredd should work at the moment and needs
to be removed. It leads to duplicate transaction names and other undefined
behavior.

In order to implement #25 and #78,
which many believed happened when they discovered the bug, much more work needs
to be done. Namely designing and adopting a new way of addressing transactions
in Dredd #227.

Closes #615

BREAKING CHANGE

honzajavorek added a commit that referenced this issue Aug 24, 2016

fix: Use just the 1st req-res pair within APIB transaction examples
BREAKING CHANGE: This change tests and fixes a problem introduced with migration to API Elements
in order to support Swagger in Dredd. Original implementation always selected
the first request-response pair from each transaction example. This wasn't
re-implemented correctly on top of API Elements. Instead, all specified responses
are appearing, which breaks Dredd's behavior in many ways. Respective test was
ported, but unfortunately with the same mistake. This commit fixes the situation.

Some early adopters discovered the issue and considered it to be a new
feature, but it really breaks how Dredd should work at the moment and needs
to be removed. It leads to duplicate transaction names and other undefined
behavior.

In order to implement #25 and #78,
which many believed happened when they discovered the bug, much more work needs
to be done. Namely designing and adopting a new way of addressing transactions
in Dredd #227.

Closes #615
@kaikun213

This comment has been minimized.

Copy link

kaikun213 commented Mar 4, 2017

Is there a feature to see the transaction origins? E.g. as the transaction names with dredd --names a command dredd --origins. Otherwise I would suggest such feature.

@honzajavorek

This comment has been minimized.

Copy link
Member

honzajavorek commented Mar 4, 2017

@kaikun213 There is no such feature right now and it wasn't planned. Could you share more about why it would be useful for you?

@kaikun213

This comment has been minimized.

Copy link

kaikun213 commented Mar 4, 2017

@honzajavorek I am quite new to dredd and at the moment debugging my apib and hooks. Somehow some requests get not invoked and others get unwillingly invoked twice. So it would be useful to fastly list the origins and convert the used transaction names to origins.

@honzajavorek

This comment has been minimized.

Copy link
Member

honzajavorek commented Mar 8, 2017

@kaikun213 I still don't quite understand how origins are more useful then the transaction names themselves. The transaction names are just origin object properties joined together. You can list origins and all the other info in the hooks just by inspecting the transaction object.

Introducing the transaction paths suggested in this issue would probably solve your problems though, because from what you write it seems to me you're experiencing some of the non-determinism of transaction names.

@honzajavorek

This comment has been minimized.

Copy link
Member

honzajavorek commented Mar 6, 2018

Note: We should drop any structure from the transaction path. Its only purpose should be to "hash" individual requests present in the API description into a single human-readable declarative string, in a deterministic way. We should forget about XPath / JSON Pointer. This is mapping, not breadcrumbs.

For cases when this couldn't be deterministic, user should always be able to give the request/response a name in the API description to distinguish them and to address them.

@halfvector

This comment has been minimized.

Copy link

halfvector commented Aug 14, 2018

@honzajavorek Is there any movement on this? I'm very interested in clearly defining various request/response pairs for the same 200 status code for contract testing purposes.

@honzajavorek

This comment has been minimized.

Copy link
Member

honzajavorek commented Aug 14, 2018

@halfvector We had some design sessions around this in April and per my last comment I have a better idea on how it should work. Currently there's no ETA though. It needs finalization of the design before anyone can implement it. I recognize it as one of the largest Dredd pain points, but as such it also requires me to carve out a significant block of uninterrupted time to tackle it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment