Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace toml with tomli for TOML v1 support #43

Merged
merged 4 commits into from
Feb 9, 2022
Merged

Replace toml with tomli for TOML v1 support #43

merged 4 commits into from
Feb 9, 2022

Conversation

eugenetriguba
Copy link
Contributor

@eugenetriguba eugenetriguba commented Feb 6, 2022

In order to make taskipy works for newer toml syntax, this replaces the toml library which only supports v0.5 with tomli for v1 support.

Example from toml spec (https://toml.io/en/v1.0.0#array):

test.toml

integers = [ 1, 2, 3 ]
colors = [ "red", "yellow", "green" ]
nested_arrays_of_ints = [ [ 1, 2 ], [3, 4, 5] ]
nested_mixed_array = [ [ 1, 2 ], ["a", "b", "c"] ]
string_array = [ "all", 'strings', """are the same""", '''type''' ]

# Mixed-type arrays are allowed
numbers = [ 0.1, 0.2, 0.5, 1, 2, 5 ]
contributors = [
  "Foo Bar <foo@example.com>",
  { name = "Baz Qux", email = "bazqux@example.com", url = "https://example.com/bazqux" }
]
>>> import toml
>>> import tomli
>>> from pathlib import Path
>>> toml.loads(Path("test.toml").read_text())
Traceback (most recent call last):
  File "/home/eugene/.cache/pypoetry/virtualenvs/taskipy-4wak4GVC-py3.9/lib/python3.9/site-packages/toml/decoder.py", line 511, in loads
    ret = decoder.load_line(line, currentlevel, multikey,
  File "/home/eugene/.cache/pypoetry/virtualenvs/taskipy-4wak4GVC-py3.9/lib/python3.9/site-packages/toml/decoder.py", line 778, in load_line
    value, vtype = self.load_value(pair[1], strictly_valid)
  File "/home/eugene/.cache/pypoetry/virtualenvs/taskipy-4wak4GVC-py3.9/lib/python3.9/site-packages/toml/decoder.py", line 880, in load_value
    return (self.load_array(v), "array")
  File "/home/eugene/.cache/pypoetry/virtualenvs/taskipy-4wak4GVC-py3.9/lib/python3.9/site-packages/toml/decoder.py", line 1029, in load_array
    raise ValueError("Not a homogeneous array")
ValueError: Not a homogeneous array

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/eugene/.cache/pypoetry/virtualenvs/taskipy-4wak4GVC-py3.9/lib/python3.9/site-packages/toml/decoder.py", line 514, in loads
    raise TomlDecodeError(str(err), original, pos)
toml.decoder.TomlDecodeError: Not a homogeneous array (line 8 column 1 char 261)
>>> tomli.loads(Path("test.toml").read_text())
{'integers': [1, 2, 3], 'colors': ['red', 'yellow', 'green'], 'nested_arrays_of_ints': [[1, 2], [3, 4, 5]], 'nested_mixed_array': [[1, 2], ['a', 'b', 'c']], 'string_array': ['all', 'strings', 'are the same', 'type'], 'numbers': [0.1, 0.2, 0.5, 1, 2, 5], 'contributors': ['Foo Bar <foo@example.com>', {'name': 'Baz Qux', 'email': 'bazqux@example.com', 'url': 'https://example.com/bazqux'}]}

The choice for tomli comes down to the fact we only need read-only support for the toml file. We're never writing to the toml file. So in that case, it makes sense to use a smaller and simpler library.

In the future, we could see support for this in the standard library and not need a third party library: https://www.python.org/dev/peps/pep-0680/

@illBeRoy
Copy link
Collaborator

illBeRoy commented Feb 8, 2022

What version of toml is used as standard in PEP? We should stick to that. If minimal version for PEP is 0.5, then we cannot use 1.0 (even if 1.0 is a superset of 0.5, we shouldn't allow anything that is not allowed by PEP).

I tried looking it up (mainly in PEP-621) but wasn't able to find any specification...

@eugenetriguba
Copy link
Contributor Author

eugenetriguba commented Feb 9, 2022

All PEP 518 mentions is that it should be "in the TOML format" (https://www.python.org/dev/peps/pep-0518/#file-format). This has caused issues: psf/black#2280

There is a discussion on updating the PEP to specify the version, which I would assume would be v1:

pip used to use toml and switched to tomli for v1 support.

The PEP for a tomllib proposes incorporating tomli:

Many psf tools have moved to tomli, as the PEP mentions and as pip has moved to toml v1. See flit, pip, black, etc. and other psf projects. I would say the "standard" right now is whatever pip does, which is v1 since July of 2021.

@illBeRoy
Copy link
Collaborator

illBeRoy commented Feb 9, 2022

Looks legit, thanks for the clarification :)

Copy link
Collaborator

@illBeRoy illBeRoy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! I had one minor comment regarding the tomli version. LMK when you fix that and I'l merge.

pyproject.toml Outdated Show resolved Hide resolved
@illBeRoy illBeRoy merged commit e1adc6c into taskipy:master Feb 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants