Use tomllib/tomli/tomli-w instead of unmaintained toml [WIP]

[mgorny]

Replace the toml dependency that is unmaintained (last release in 2020) and does not implement TOML 1.0 with more modern libraries. Use the built-in tomllib module to load .toml files on Python 3.11, and its drop-in replacement tomli package on older versions of Python. For writing .toml files, use the tomli-w package.

[mgorny]

There's one problem with this: right now it fails because it attempts to serialize None that has no valid representation in TOML. Apparently the buggy toml module either ignored keys with None as value or serialized it as string "None" (sic!).

[beliaev-maksim]

we will add support for 3.11 once it is released
please remove it so far

does tomli include type annotations as inlines?

[mgorny]

we will add support for 3.11 once it is released please remove it so far

That sounds quite pointless, given it's at rc2 now and the API is frozen already but sure.

does tomli include type annotations as inlines?

Yes.

[beliaev-maksim]

makes sense to switch. Thanks for the PR

at the time of writing the feature I chose toml because of the wider usage by the community. However, if it is unmaintained there is no reason not to go for tomli

will also resolve #595

[mgorny]

at the time of writing the feature I chose toml because of the wider usage by the community.

I'm surprised at that but I suppose there's still work to be done there. I think most of the major projects (black, flit, mypy, pytest, setuptools, tox to list just a few) have switched to tomli already. At least a quick glance it seems that tomli has more revdeps in Gentoo than toml.

[beliaev-maksim]

@mgorny
please run tox until all are passed and ensure 100% code coverage

https://github.com/getsentry/responses#tests-and-code-quality-validation

[mgorny]

See #596 (comment). The problem is that the input is broken right now, and I'm looking for guidance what fix would you prefer for it.

[beliaev-maksim]

Apparently the buggy toml module either ignored keys with None

it is not buggy, but done on purpose
I just went over discussions on toml-lang/toml#921
also, read threads in tomli-w and toml

according to toml-lang the value with null should not be present in the file, thus, it should be filtered out in languages where it is present eg python. So, we need to filter all nulls out.

def remove_nones(d):
    if isinstance(d, dict):
        return {k: remove_nones(v) for k, v in d.items() if v is not None}
    return d

and then dump the result

[mgorny]

Thanks for the snippet. Done that now and the tests seem to pass.

[codecov]

Codecov Report

Base: 100.00% // Head: 99.92% // Decreases project coverage by -0.07% ⚠️

Coverage data is based on head (dd205bb) compared to base (27cb0c7).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files

@@             Coverage Diff             @@
##            master     #596      +/-   ##
===========================================
- Coverage   100.00%   99.92%   -0.08%     
===========================================
  Files            9        9              
  Lines         2775     2788      +13     
===========================================
+ Hits          2775     2786      +11     
- Misses           0        2       +2     

Impacted Files	Coverage Δ
responses/__init__.py	99.60% <100.00%> (-0.40%)	⬇️
responses/_recorder.py	100.00% <100.00%> (ø)
responses/tests/test_recorder.py	100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[markstory]

Thank you 🎉

[mgorny]

Thanks!

[mgorny]

we will add support for 3.11 once it is released please remove it so far

Python 3.11.0 has been released on Oct 24th.