How to log more info on 422 unprocessable entity

[nickgieschen]

I get this a lot in development and can usually pretty quickly figure out what is wrong with my json payload. However, I'd love a way to see what exactly the validation error is when I get this error. Is there a way I can log what the specific schema validation error is on the server side?

[Garito]

Just the other day I had a pretty absurd issue about exactly that

My gui was doing a fetch request and thanks to the apple keyword crap, I set the content-type to applications/json (notice the applications instead of application)

The 422 error indicated that value wasn't a dict

I spend too much time till I noticed the additional s

This would be avoided with better 422 error or a way to debug it

[panla]

@nickgieschen
hello

catch exception

def register_exception(app: FastAPI):
    @app.exception_handler(RequestValidationError)
    async def validation_exception_handler(request: Request, exc: RequestValidationError):

        exc_str = f'{exc}'.replace('\n', ' ').replace('   ', ' ')
        # or logger.error(f'{exc}')
        logger.error(request, exc_str)
        content = {'status_code': 10422, 'message': exc_str, 'data': None}
        return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

after you can log error and get response

and you can see issue 394 @ruslankrivoshein

[nickgieschen]

@nickgieschen
hello

catch exception

def register_exception(app: FastAPI):
    @app.exception_handler(RequestValidationError)
    async def validation_exception_handler(request: Request, exc: RequestValidationError):

        exc_str = f'{exc}'.replace('\n', ' ').replace('   ', ' ')
        # or logger.error(f'{exc}')
        logger.error(request, exc_str)
        content = {'status_code': 10422, 'message': exc_str, 'data': None}
        return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

after you can log error and get response

and you can see issue 394 @ruslankrivoshein

Thanks. Can you tell me what logger that is that you can pass request as the first argument?

[panla]

def log_message(request: Request, e):

    logger.error('start error'.center(60, '*'))
    logger.error(f'{request.method} {request.url}')
    logger.error(f'error is {e}')
    logger.error('end error'.center(60, '*'))

log_message(request, exc_str)

or

logger.error(exc_str)

[Torxed]

The above examples are excerpts of handling-errors documentation.
For the sake of clarity to anyone ending up here after having the same issues as OP, here's the full code @panla scissored in:

import logging
from fastapi import FastAPI, Request, status
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse

app = FastAPI()

@app.exception_handler(RequestValidationError)
async def validation_exception_handler(request: Request, exc: RequestValidationError):
	exc_str = f'{exc}'.replace('\n', ' ').replace('   ', ' ')
	logging.error(f"{request}: {exc_str}")
	content = {'status_code': 10422, 'message': exc_str, 'data': None}
	return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

This will produce something along the lines of:

ERROR:root:<starlette.requests.Request object at 0x7fba45db99d0>: 3 validation errors for Request body -> X none is not an allowed value (type=type_error.none.not_allowed) body -> Y value is not a valid list (type=type_error.list) body -> Z value is not a valid list (type=type_error.list)

This error is not only bound to the header, but also if the data submitted does not match the BaseModel supplied to the API endpoint.

[zwessels]

content = {'status_code': 10422, 'message': exc_str, 'data': None

presume it should be 422?

[dancaugherty]

What is the significance of 10422 ?

[VBoB13]

The above examples are excerpts of handling-errors documentation. For the sake of clarity to anyone ending up here after having the same issues as OP, here's the full code @panla scissored in:

import logging
from fastapi import FastAPI, Request, status
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse

app = FastAPI()

@app.exception_handler(RequestValidationError)
async def validation_exception_handler(request: Request, exc: RequestValidationError):
	exc_str = f'{exc}'.replace('\n', ' ').replace('   ', ' ')
	logging.error(f"{request}: {exc_str}")
	content = {'status_code': 10422, 'message': exc_str, 'data': None}
	return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

This will produce something along the lines of:

ERROR:root:<starlette.requests.Request object at 0x7fba45db99d0>: 3 validation errors for Request body -> X none is not an allowed value (type=type_error.none.not_allowed) body -> Y value is not a valid list (type=type_error.list) body -> Z value is not a valid list (type=type_error.list)

This error is not only bound to the header, but also if the data submitted does not match the BaseModel supplied to the API endpoint.

This should be the standard implementation IMHO... a beginner like me had to do quite a lot of digging simply because the error wasn't expressive enough :P

[JustinGuese]

wouldn't this catch all errors instead of only 422? and what about the 10422 like @dancaugherty mentioned?

[Torxed]

This should be the standard implementation IMHO..

At least if --debug is given or something.
It takes a toll if you have a lot of errors being catched this way, but yea, it's a rabbit hole before you end up here and just copy paste it in. Which is problematic in it's own right.

[JustinGuese]

yeah i agree, especially because there's no real way to log it properly if you'll get a 422

[JustinGuese]

or it would be cool to define when creating the pydantic model...
like

class Person(BaseModel):
   name: str
   
   def on_422_error(error):
       logging.log etc

[panla]

@nickgieschen @Torxed @zwessels @dancaugherty @VBoB13 @JustinGuese

please see

codes

extensions/exceptions.py

from typing import Optional, Any, Dict

from fastapi import status
from starlette.exceptions import HTTPException

from conf.const import StatusCode
from .schema import SchemaMixin


class BaseHTTPException(HTTPException):
    MESSAGE = None
    STATUS_CODE = status.HTTP_400_BAD_REQUEST
    CODE = 40000

    def __init__(
            self,
            message: Any = None,
            code: int = None,
            headers: Optional[Dict[str, Any]] = None
    ) -> None:
        self.message = message or self.MESSAGE
        self.status_code = self.STATUS_CODE
        self.code = code or self.CODE
        self.detail = self.message
        self.headers = headers

    def __repr__(self) -> str:
        class_name = self.__class__.__name__
        return f"{class_name}(status_code={self.status_code!r}, code={self.code}, msg={self.message!r})"

    def response(self):
        return SchemaMixin(code=self.code, message=self.message, data=None).dict()


class BadRequest(BaseHTTPException):
    STATUS_CODE = status.HTTP_400_BAD_REQUEST
    CODE = StatusCode.bad_request


class Unauthorized(BaseHTTPException):
    STATUS_CODE = status.HTTP_401_UNAUTHORIZED
    CODE = StatusCode.unauthorized


class Forbidden(BaseHTTPException):
    STATUS_CODE = status.HTTP_403_FORBIDDEN
    CODE = StatusCode.forbidden


class NotFound(BaseHTTPException):
    STATUS_CODE = status.HTTP_404_NOT_FOUND
    CODE = StatusCode.not_found


class MethodNotAllowed(BaseHTTPException):
    STATUS_CODE = status.HTTP_405_METHOD_NOT_ALLOWED
    CODE = StatusCode.method_not_allowed


class Locked(BaseHTTPException):
    STATUS_CODE = status.HTTP_423_LOCKED
    CODE = StatusCode.locked

extensions/exceptions.py

from typing import Optional, Any

from fastapi import Query
from pydantic import BaseModel
from pydantic.typing import NoneType

from conf.const import StatusCode, PaginateConst


class BadRequestSchema(BaseModel):
    code: int = StatusCode.bad_request
    message: str = ''
    data: NoneType = "null"


class UnauthorizedSchema(BaseModel):
    code: int = StatusCode.unauthorized
    message: str = ''
    data: NoneType = "null"


class ForbiddenSchema(BaseModel):
    code: int = StatusCode.forbidden
    message: str = ''
    data: NoneType = "null"


class NotFoundSchema(BaseModel):
    code: int = StatusCode.not_found
    message: str = ''
    data: NoneType = "null"


class ValidatorErrorSchema(BaseModel):
    code: int = StatusCode.validator_error
    message: str = ''
    data: NoneType = "null"


ErrorSchema = {
    400: {
        'model': BadRequestSchema,
        'description': 'bad_request'
    },
    401: {
        'model': UnauthorizedSchema,
        'description': 'unauthorized'
    },
    403: {
        'model': ForbiddenSchema,
        'description': 'forbidden'
    },
    404: {
        'model': NotFoundSchema,
        'description': 'not_found'
    },
    422: {
        'model': ValidatorErrorSchema,
        'description': 'request parameters validator'
    }
}


class SchemaMixin(BaseModel):
    code: int = 10000
    message: str = ''
    data: Optional[Any]


class NormalSchema(SchemaMixin):
    """"""

    data: Optional[str] = 'success'

conf/const.py

class StatusCode(object):
    success = 10000

    bad_request = 40000
    unauthorized = 40100
    forbidden = 40300
    not_found = 40400
    method_not_allowed = 40500
    not_acceptable = 40600
    request_timeout = 40800
    length_required = 41100
    entity_too_large = 41300
    request_uri_too_long = 41400
    validator_error = 42200
    locked = 42300
    header_fields_too_large = 43100

    server_error = 45000
    unknown_error = 45001

apps/libs/exceptions.py

import traceback
from typing import Union, Any

from fastapi import FastAPI, Request, status
from fastapi.exceptions import HTTPException, RequestValidationError
from fastapi.responses import JSONResponse
from starlette.datastructures import URL
from tortoise.validators import ValidationError

from conf.const import StatusCode
from extensions.log import logger
from extensions.exceptions import BaseHTTPException


def log_message(method: str, url: Union[str, URL], message: Any):
    """log message when catch exception"""

    logger.error('start error, this is'.center(60, '*'))
    logger.error(f'{method} {url}')
    logger.error(message)
    logger.error('end error'.center(60, '*'))


def register_exception(app: FastAPI):
    @app.exception_handler(BaseHTTPException)
    async def catch_c_http_exception(request: Request, exc: BaseHTTPException):
        """catch custom exception"""

        log_message(request.method, request.url, exc.message)
        content = exc.response()
        return JSONResponse(content=content, status_code=exc.status_code, headers=exc.headers)

    @app.exception_handler(HTTPException)
    async def http_exception_handler(request: Request, exc: HTTPException):
        """catch FastAPI HTTPException"""

        log_message(request.method, request.url, exc.detail)
        content = {'code': StatusCode.bad_request, 'message': exc.detail, 'data': None}
        return JSONResponse(content=content, status_code=exc.status_code, headers=exc.headers)

    @app.exception_handler(AssertionError)
    async def assert_exception_handle(request: Request, exc: AssertionError):
        """catch Python AssertError"""

        exc_str = ' '.join(exc.args)
        log_message(request.method, request.url, exc_str)
        content = {'code': StatusCode.validator_error, 'message': exc_str, 'data': None}
        return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

    @app.exception_handler(ValidationError)
    async def db_validation_exception_handle(request: Request, exc: ValidationError):
        """catch tortoise-orm ValidatorError"""

        exc_str = '|'.join(exc.args)
        log_message(request.method, request.url, exc.args)
        content = {'code': StatusCode.validator_error, 'message': exc_str, 'data': None}
        return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

    @app.exception_handler(RequestValidationError)
    async def validation_exception_handler(request: Request, exc: RequestValidationError):
        """catch FastAPI RequestValidationError"""

        exc_str = f'{exc}'.replace('\n', ' ').replace('   ', ' ')
        log_message(request.method, request.url, exc)
        # content = exc.errors()
        content = {'code': StatusCode.validator_error, 'message': exc_str, 'data': None}
        return JSONResponse(content=content, status_code=status.HTTP_422_UNPROCESSABLE_ENTITY)

    @app.exception_handler(Exception)
    async def exception_handle(request: Request, exc: Exception):
        """catch other exception"""

        log_message(request.method, request.url, traceback.format_exc())
        content = {'code': StatusCode.server_error, 'message': str(exc), 'data': None}
        return JSONResponse(content=content, status_code=status.HTTP_500_INTERNAL_SERVER_ERROR)

desc

success

http status code 200 or 201 or 204

response

{
    "code": 10000,
    "message": "success",
    "data": "how are you?"
}

422 error

http status code 422

response

{
    "code": 42200,
    "message": "validator error",
    "data": null
}

other response

use exception

from extensions.exceptions import NotFound, BadRequest

raise NotFound(message=f'there is no this user {cellphone}')
raise BadRequest(message='Request too fast!')

raise CustomException(message="", code=1000000000000, headers=None)

last

Don't pay too much attention to some definitions.

This is just a demo

Defined according to your actual situation