Flask 1.1.2 accidentally ships with itsdangerous v2, hence ignores simplejson

[lukasjuhrich]

Description

A deployment of our app suddenly stopped being able to (de)serialize Decimals.
An installation of Flask~=1.1.2 suddenly ships with itsdangerous version 2, which has been released yesterday.
However, this introduces the breaking change of not using the simplejson anymore, even if it's present.

This is most likely not intended, because the v1.1.2 docs of flask.json.dumps clearly dictates that simplejson will be used if present:

>>> from flask.json import dumps                                                                                                                                                                                                               
>>> help(dumps)                                                                                                                                                                                                                            
Help on function dumps in module flask.json:                                                                                                                                                                                                   
                                                                                                                       
dumps(obj, app=None, **kwargs)                                                                                                                                                                                                                 
    Serialize ``obj`` to a JSON-formatted string. If there is an                                                                                                                                                                               
    app context pushed, use the current app's configured encoder                                                                                                                                                                               
    (:attr:`~flask.Flask.json_encoder`), or fall back to the default                                                                                                                                                                           
    :class:`JSONEncoder`.                                                                                                                                                                                                                      
                                                                                                                                                                                                                                               
    Takes the same arguments as the built-in :func:`json.dumps`, and                                                                                                                                                                           
    does some extra configuration based on the application. If the                                                                                                                                                                             
    simplejson package is installed, it is preferred.                                                                                                                                                                                          
                                                                                                                                                                                                                                               
    :param obj: Object to serialize to JSON.                                                                                                                                                                                                   
    :param app: App instance to use to configure the JSON encoder.                                                                                                                                                                             
        Uses ``current_app`` if not given, and falls back to the default                                                                                                                                                                       
        encoder when not in an app context.                                                                                                                                                                                                    
    :param kwargs: Extra arguments passed to :func:`json.dumps`.                                                                                                                                                                               
                                                                                                                                                                                                                                               
    .. versionchanged:: 1.0.3                                                                                                                                                                                                                  
                                                                                                                                                                                                                                               
        ``app`` can be passed directly, rather than requiring an app                                                                                                                                                                           
        context for configuration. 

Replication

1. install Flask~=1.1.2 and simplejson~=3.11.1 (or any other version, really)
2. Try to deserialize a Decimal:

>>> from flask import Flask, jsonify
>>> from decimal import Decimal
>>> with Flask('test').app_context():
...     print(jsonify(foo=Decimal('20.00')))

Environment

• Python version: 3.9.5
• Flask version: 1.1.2
• simplejson~=3.11.1 in requirements.txt

Fix

Change itsdangerous>=0.24 to itsdangerous>=0.24, <2.0 in the setup.py.

[ThiefMaster]

You should pin transitive requirements in your application. Tools like pip-compile make this task rather easy.

[lukasjuhrich]

Sorry about that, I forgot that requirements.txt was meant to be the output of pip freeze. I didn't know about pip-compile.

However, the fact remains that the v1.12 docstring of flask.json.dumps now contains incorrect information.
If the itsdangerous requirement shall remains to be a lower bound, then the docstring should be fixed, shouldn't it?

[lukasjuhrich]

Also, may I ask whether there is a specific reason against adding the upper bound? Just because I did something suboptimally does not mean it would be a bad thing for flask to assist me with slightly more conservative requirements.

If users of Flask~=1.12 for whatever reason desperately miss the improvements of itsdangerous~=2.0, then they can always upgrade to Flask 2.

I do realize it's additional work to commit things and add a patch release though, I'm just wondering whether that's the only reason.

Thanks for the quick response so far.

[ThiefMaster]

I do realize it's additional work to commit things and add a patch release though, I'm just wondering whether that's the only reason.

IIRC that's the main reason

[davidism]

This is the type of incompatibility you could get from any library upgrade, potentially. Which is why applications need to pin their transitive dependencies in general.

[Roguelazer]

As with #4027, this means that all Flask users need to either upgrade to the newest major release within a few hours or know your internal policies on which transitive dependencies work correctly when they're following the (very important!) best practice of routinely updating their dependencies.

This level of implementation detail knowledge is probably unwise to rely on

It would be really great to get a Flask 1.1.3 that sets upper bounds on all of its dependencies.

[davidism]

#4043