feat: introduces unified validation result structure

BREAKING CHANGE: Gavel validation result structure includes the next changes: * Validity of an HTTP message and its fields is marked in `valid` property: ```diff -result.isValid -result.fields.body.isValid +result.valid +result.fields.body.valid ``` * HTTP message fields may include expected/actual values that represent end compared values: ```js result.fields.body.values.expected result.fields.body.values.actual ``` * Each HTTP message field contains the "kind" enum property of values "text" and "json" representing the end value type: ```js result.fields.body.kind // "json" ``` * HTTP message fields no longer include the next properties: * `validator` * `expectedType` * `realType* * `rawData`
apiaryio · Jul 3, 2019 · 884f80e · 884f80e
1 parent cdddbb3
commit 884f80e
Show file tree

Hide file tree

Showing 13 changed files with 117 additions and 102 deletions.
diff --git a/features/cli.feature b/features/cli.feature
@@ -36,6 +36,4 @@ Feature: Command-line interface
     cat real_without_cors | curl-trace-parser | gavel expected
     """
 
-    Then exit status is 1 
-
-
+    Then exit status is 1
diff --git a/features/expectations/body_json_example.feature b/features/expectations/body_json_example.feature
@@ -33,8 +33,8 @@ Feature: Body - JSON example
       "string": "Hello World"
     }
     """
-    Then field "body" is NOT valid
-    And Request or Response is NOT valid
+    Then field "body" MUST NOT be valid
+    And Request or Response MUST NOT be valid
 
   Scenario: Extra key in real JSON body
     When real HTTP body is following:
@@ -53,8 +53,8 @@ Feature: Body - JSON example
       "boolean": true
     }
     """
-    Then field "body" is valid
-    And Request or Response is valid
+    Then field "body" MUST be valid
+    And Request or Response MUST be valid
 
   Scenario: Different values in real JSON body
     When real HTTP body is following:
@@ -73,8 +73,8 @@ Feature: Body - JSON example
       "boolean": false
     }
     """
-    Then field "body" is valid
-    And Request or Response is valid
+    Then field "body" MUST be valid
+    And Request or Response MUST be valid
 
   Scenario: Extra array member in real JSON body
     When real HTTP body is following:
@@ -93,6 +93,5 @@ Feature: Body - JSON example
       "string": "Foo bar"
     }
     """
-    Then field "body" is valid
-    And Request or Response is valid
-
+    Then field "body" MUST be valid
+    And Request or Response MUST be valid
diff --git a/features/expectations/body_json_schema.feature b/features/expectations/body_json_schema.feature
@@ -44,8 +44,8 @@ Feature: Body - JSON schema draft v4
       "string": "Hello World"
     }
     """
-    Then field "body" is valid
-    And Request or Response is valid
+    Then field "body" MUST be valid
+    And Request or Response MUST be valid
 
   Scenario: Payload body not valid against schema
     When real HTTP body is following:
@@ -58,5 +58,5 @@ Feature: Body - JSON schema draft v4
       "string": "Hello World"
     }
     """
-    Then field "body" is NOT valid
-    And Request or Response is NOT valid
+    Then field "body" MUST NOT be valid
+    And Request or Response MUST NOT be valid
diff --git a/features/expectations/body_json_schemav3.feature b/features/expectations/body_json_schemav3.feature
@@ -47,8 +47,8 @@ Feature: Body - JSON schema draft v3
       "string": "Hello World"
     }
     """
-    Then field "body" is valid
-    And Request or Response is valid
+    Then field "body" MUST be valid
+    And Request or Response MUST be valid
 
   Scenario: Payload body not valid against schema
     When real HTTP body is following:
@@ -61,5 +61,5 @@ Feature: Body - JSON schema draft v3
       "string": "Hello World"
     }
     """
-    Then field "body" is NOT valid
-    And Request or Response is NOT valid
+    Then field "body" MUST NOT valid
+    And Request or Response MUST valid
diff --git a/features/expectations/body_text_example.feature b/features/expectations/body_text_example.feature
@@ -15,8 +15,8 @@ Feature: Body - text example
       One, two, three, four.
       Orange, strawberry, banana?
       """
-    Then field "body" is NOT valid
-    And Request or Response is NOT valid
+    Then field "body" MUST NOT be valid
+    And Request or Response MUST NOT be valid
 
   Scenario: Extra line in real payload textual body
     When real HTTP body is following:
@@ -26,8 +26,8 @@ Feature: Body - text example
       Orange, strawberry, banana?
       Dog, cat, mouse!
       """
-    Then field "body" is NOT valid
-    And Request or Response is NOT valid
+    Then field "body" MUST NOT be valid
+    And Request or Response MUST NOT be valid
 
   Scenario: Line is changed in real textual body
     When real HTTP body is following:
@@ -36,8 +36,8 @@ Feature: Body - text example
       Orange, strawberry, banana?
       Dog, cat, mouse!
       """
-    Then field "body" is NOT valid
-    And Request or Response is NOT valid
+    Then field "body" MUST NOT be valid
+    And Request or Response MUST NOT be valid
 
   Scenario: Text in body equals defined example
     When real HTTP body is following:
@@ -46,6 +46,5 @@ Feature: Body - text example
       Orange, strawberry, banana?
       Dog, cat, mouse!
       """
-    Then field "body" is valid
-    And Request or Response is valid
-
+    Then field "body" MUST valid
+    And Request or Response MUST valid
diff --git a/features/expectations/headers.feature b/features/expectations/headers.feature
@@ -17,8 +17,8 @@ Feature: Headers
       """
       Content-Type: text/plain
       """
-    Then field "headers" is NOT valid
-    And Request or Response is NOT valid
+    Then field "headers" MUST NOT be valid
+    And Request or Response MUST NOT be valid
 
   @stable
   Scenario: Extra real header in real payload
@@ -30,8 +30,8 @@ Feature: Headers
       Location: /here
       ETag: 68b329da9893e34099c7d8ad5cb9c940
       """
-    Then field "headers" is valid
-    And Request or Response is valid
+    Then field "headers" MUST be valid
+    And Request or Response MUST be valid
 
   @stable
   Scenario: Content nogotiation significant header value is different in real payload
@@ -42,8 +42,8 @@ Feature: Headers
       Location: /here
       ETag: 68b329da9893e34099c7d8ad5cb9c940
       """
-    Then field "headers" is NOT valid
-    And Request or Response is NOT valid
+    Then field "headers" MUST NOT be valid
+    And Request or Response MUST NOT be valid
 
   @stable
   Scenario: Content negotiation not significant header value is different in real payload
@@ -54,7 +54,5 @@ Feature: Headers
       Location: /there
       ETag: something-completely-different
       """
-    Then field "headers" is valid
-    And Request or Response is valid
-
-
+    Then field "headers" MUST be valid
+    And Request or Response MUST be valid
diff --git a/features/expectations/method.feature b/features/expectations/method.feature
@@ -6,10 +6,10 @@ Feature: Method
 
   Scenario: HTTP message method match
     When real HTTP message method is "POST"
-    Then field "method" is valid
-    And Request or Response is valid
+    Then field "method" MUST be valid
+    And Request or Response MUST be valid
 
   Scenario: Different HTTP message method
     When real HTTP message method is "GET"
-    Then field "method" is NOT valid
-    And Request or Response is NOT valid
+    Then field "method" MUST NOT be valid
+    And Request or Response MUST NOT be valid
diff --git a/features/expectations/status_code.feature b/features/expectations/status_code.feature
@@ -6,10 +6,10 @@ Feature: Status code
 
   Scenario: Response status code match
     When real status code is "200"
-    Then field "statusCode" is valid
-    And Request or Response is valid
+    Then field "statusCode" MUST be valid
+    And Request or Response MUST be valid
 
   Scenario: Different real response status
     When real status code is "500"
-    Then field "statusCode" is NOT valid
-    And Request or Response is NOT valid
+    Then field "statusCode" MUST NOT be valid
+    And Request or Response MUST NOT be valid
diff --git a/features/expectations/uri.feature b/features/expectations/uri.feature
@@ -4,20 +4,20 @@ Feature: URI
   Scenario: Matching HTTP message URI
     Given you expect HTTP message URI "/animals?type=cats"
     When real HTTP message URI is "/animals?type=cats"
-    Then field "uri" is valid
-    And Request or Response is valid
+    Then field "uri" MUST be valid
+    And Request or Response MUST be valid
 
   Scenario: Non-matching HTTP message URI
     Given you expect HTTP message URI "/animals?type=cats"
     When real HTTP message URI is "/animals?type=dogs"
-    Then field "uri" is NOT valid
-    And Request or Response is NOT valid
+    Then field "uri" MUST NOT be valid
+    And Request or Response MUST NOT be valid
 
   Scenario: URI with the same parameters in different order
     Given you expect HTTP message URI "/animals?type=cats&count=2"
     When real HTTP message URI is "/animals?count=2&type=cats"
-    Then field "uri" is valid
-    And Request or Response is valid
+    Then field "uri" MUST be valid
+    And Request or Response MUST be valid
 
   Scenario Outline: URI with parameter having multiple values
     Given you expect HTTP message URI "/animals?type=cats&type=dogs"
@@ -33,5 +33,5 @@ Feature: URI
   Scenario: Case-sensitive URI comparison
     Given you expect HTTP message URI "/animals?type=cats"
     When real HTTP message URI is "/animals?tYpe=cAtS"
-    Then field "uri" is NOT valid
-    And Request or Response is NOT valid
+    Then field "uri" MUST NOT be valid
+    And Request or Response MUST NOT be valid
diff --git a/features/javascript/README.md b/features/javascript/README.md
@@ -31,7 +31,7 @@ const expectedResponse = {
   'body': '{\n  "origin": "94.113.241.2"\n}'
 };
 
-const realResponse = {
+const actualResponse = {
   'statusCode': '200',
   'headers': {
     'content-type': 'application/json',
@@ -43,7 +43,7 @@ const realResponse = {
   'body': '{\n  "origin": "94.113.241.2"\n}'
 };
 
-const result = gavel.validate(expectedResponse, realResponse);
+const result = gavel.validate(expectedResponse, actualResponse);
 ```
 
 ### Command-line interface

diff --git a/features/javascript/request.feature b/features/javascript/request.feature
@@ -6,11 +6,9 @@ Feature: Request validation
     """
     const gavel = require('gavel');
     """
-    And you define following HTTP request object:
+    And you define following expected HTTP request object:
     """
-    const real = {
-      "method": "GET",
-      "uri": "/ip",
+    const expected = {
       "headers": {
         "user-agent": "curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5",
         "host": "httpbin.org",
@@ -19,9 +17,11 @@ Feature: Request validation
       "body": ""
     };
     """
-    And you define following expected HTTP request object:
+    And you define following HTTP request object:
     """
-    const expected = {
+    const actual = {
+      "method": "GET",
+      "uri": "/ip",
       "headers": {
         "user-agent": "curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5",
         "host": "httpbin.org",
@@ -35,28 +35,38 @@ Feature: Request validation
   Scenario: validate
     When you call:
     """
-    gavel.validate(expected, real);
+    gavel.validate(expected, actual);
     """
     Then it will return:
     """
     {
-      isValid: true,
+      valid: true,
       fields: {
         headers: {
-          isValid: true,
+          valid: true,
+          kind: 'json',
+          values: {
+            expected: {
+              user-agent: 'curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5',
+              host: 'httpbin.org',
+              accept: '*/*'
+            },
+            actual: {
+              user-agent: 'curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5',
+              host: 'httpbin.org',
+              accept: '*/*'
+            },
+          },
           errors: [],
-          realType: 'application/vnd.apiary.http-headers+json',
-          expectedType: 'application/vnd.apiary.http-headers+json',
-          validator: 'HeadersJsonExample',
-          rawData: { length: 0 }
         },
         body: {
-          isValid: true,
+          valid: true,
+          kind: 'text',
+          values: {
+            expected: '',
+            actual: ''
+          },
           errors: [],
-          realType: 'text/plain',
-          expectedType: 'text/plain',
-          validator: 'TextDiff',
-          rawData: ''
         }
       }
     }