Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update error messages #30

Merged
merged 13 commits into from
Apr 4, 2024
Merged

feat: update error messages #30

merged 13 commits into from
Apr 4, 2024

Conversation

danut-t
Copy link
Collaborator

@danut-t danut-t commented Mar 28, 2024

Currently, the error messages make the debugging hard, as it is difficult to figure out where are there issues (in the request or the documentation), for which request it is happening and what's the exact problem and the expectation.

This PR aims to fix that (partially) by changing a few things:

  1. Change how the reference is created:
    The existing reference contained minimal information and it also had a bug where it would continuously attach fields to it, even when it should have pointed to only one field. The new format will contain informations about the request path and type, if it's the request or response, and the drilldowned path to the key with the issues (if it's the case)

The new format will look like this: '<request_type> <request_path> > <request/response> > <status_code(for responses)> > path > to > key'

and in practice it could look like this: GET /api/pets > response > 200 > name > id

  1. Add whenever possible the actual request/response body and the schema section for easy comparison

Update most of the messages to include, besides the new reference, a copy of the actual body and the schema section, along with (hopefully) clearer language as to what went wrong. The combination of these and the new reference should provide enough information to quickly investigate the issue without have to take the request and the documentation and scan for differences side by side.

Examples of how it would look like:

missing property from the request data

E               openapi_tester.exceptions.DocumentationError: The following property was found in the schema definition, but is missing from the request data: "name"
E               
E               Reference:
E               
E               POST /api/pets > request > name
E               
E               Request body:
E                 {
E                   "tag": "dog"
E               }
E               Schema section:
E                 {
E                   "name": {
E                       "type": "string"
E                   },
E                   "tag": {
E                       "type": "string"
E                   }
E               }
E               
E               Hint: Remove the key from your OpenAPI docs, or include it in your API request

missing property from the response data

E               openapi_tester.exceptions.DocumentationError: The following property was found in the schema definition, but is missing from the response data: "width"
E               
E               Reference:
E               
E               GET /api/v1/cars/incorrect > response > 200 > width
E               
E               Response body:
E                 {
E                   "name": "Saab",
E                   "color": "Yellow",
E                   "height": "Medium height"
E               }
E               Schema section:
E                 {
E                   "name": {
E                       "type": "string",
E                       "maxLength": 254
E                   },
E                   "color": {
E                       "type": "string",
E                       "maxLength": 254
E                   },
E                   "height": {
E                       "type": "string",
E                       "maxLength": 254
E                   },
E                   "width": {
E                       "type": "string",
E                       "maxLength": 254
E                   },
E                   "length": {
E                       "type": "string",
E                       "maxLength": 254
E                   }
E               }
E               
E               Hint: Remove the key from your OpenAPI docs, or include it in your API response

Besides all of these, another minor change is to use by default json.dumps on data whenever the content is application/json in order to reduce verbosity in tests

Additionally Part 3: added support for OAS 3.1 lists of types e.g.:

    NewPet:
      type: object
      properties:
        name:
          type:
            - string
            - 'null'

@danut-t danut-t self-assigned this Mar 28, 2024
@danut-t danut-t marked this pull request as ready for review April 2, 2024 13:33
@maticardenas
Copy link
Owner

@danut-t thanks for this change, IMO this is a huge improvement to the main goal of this package, which is informing these inconsistencies. Just left a comment about the minor change of the json serialization on this package's side.

@danut-t
Copy link
Collaborator Author

danut-t commented Apr 3, 2024

@maticardenas Hmm I don't see the comment, could you link it to me please?

@maticardenas
Copy link
Owner

@danut-t it should be visible now, I had forgotten to “submit the review” and it was only visible to me

@kazantsevaanna
Copy link
Collaborator

So nice! These error messages are so useful 😻

@danut-t danut-t merged commit f00ae02 into master Apr 4, 2024
14 checks passed
@danut-t danut-t deleted the 28-improve-error-messages branch April 4, 2024 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve error messages Add json.dumps for data by default when content type is json
3 participants