Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validation using dataclasses #52

Merged
merged 59 commits into from
Jan 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
d6182e8
Migrate sources to dataclass
oz123 Aug 30, 2023
bf8492e
Migrate pacakges to dataclass
oz123 Sep 1, 2023
d21b424
Fix model and tests for Package
oz123 Sep 2, 2023
777cbdf
Add cli interface to validate Lockfile
oz123 Sep 2, 2023
bd4c50c
Continue migration to dataclasses
oz123 Sep 2, 2023
acbb854
Update Hash class - use dataclasses
oz123 Sep 3, 2023
c2f6343
Fix some linter errors
oz123 Sep 30, 2023
eee4989
Migrate scripts to dataclass
oz123 Sep 30, 2023
38f4b17
Disable some pylint errors, remove unused import
oz123 Oct 22, 2023
b462444
Disable some pylint errors, remove unused import
oz123 Oct 22, 2023
6b8dccf
Disable some pylint errors, remove unused import
oz123 Oct 22, 2023
c5e2404
Add pylintrc
oz123 Oct 22, 2023
b9f25f1
Migrate sections.py to dataclasses
oz123 Oct 22, 2023
100fd3d
WIP: convert lockfiles to dataclasses
oz123 Oct 22, 2023
5a0aac2
Add pipfile example with extras in the package spec
oz123 Oct 22, 2023
536fdd4
WIP: lockfiles progress
oz123 Oct 22, 2023
b8b23b7
Add missing return value in SourceCollection validator
oz123 Nov 6, 2023
aff82f7
Continue work on lockfile parsing
oz123 Nov 6, 2023
e8f8b13
Fix hasheh.py
oz123 Nov 6, 2023
6d6895b
Fix tests for packages.py
oz123 Nov 6, 2023
91d322d
Update pylint rules
oz123 Nov 11, 2023
31e4466
Fix another lockfiles test
oz123 Nov 11, 2023
229dcff
Remove tests for sources from test_models.py
oz123 Nov 11, 2023
bed6cba
Move all tests for package into tests/test_models_packages.py
oz123 Nov 11, 2023
faebcee
Extract tests for meta
oz123 Nov 11, 2023
f84d401
Extract tests for hashes
oz123 Nov 11, 2023
56fa7d6
Extract tests for models.Requires
oz123 Nov 12, 2023
e8aa17b
Restore iterable behaviour for SourceCollection
oz123 Nov 18, 2023
09859c6
Fix tests for models.SourceCollection
oz123 Nov 18, 2023
37617ce
Remove tests for SourceCollection from test_models
oz123 Nov 18, 2023
00eac23
Remove cerberus from test_scripts.py
oz123 Nov 18, 2023
b49bb3a
Fix test for validation error
oz123 Nov 19, 2023
9eccc54
WIP: slowly getting lockfiles properly work ...
oz123 Nov 19, 2023
f675b1c
Remove break point tasks/__init__.py
oz123 Nov 19, 2023
0c03c77
Fix one more tests for lockfiles
oz123 Nov 19, 2023
6d6bbfe
Remove breakpoint from lockfiles.py
oz123 Nov 19, 2023
13028a3
All tests for lockfiles pass
oz123 Nov 22, 2023
63a16c2
2 more tests for pipfiles pass now
oz123 Nov 22, 2023
172cd1e
All pipefile tests work except for dump
oz123 Nov 23, 2023
0c71b9d
All tests pass
oz123 Nov 23, 2023
da05a0d
Fix validation test for allow_prereleases
oz123 Nov 23, 2023
093019e
Fix all tests for lockfiles
oz123 Nov 24, 2023
c7be254
All tests for pipfile now pass
oz123 Nov 24, 2023
246420b
All tests pass!
oz123 Nov 25, 2023
f8690db
Merge all models into one file
oz123 Nov 28, 2023
3cb15c6
Address some linting issues
oz123 Dec 10, 2023
c025e85
Update the docs
oz123 Dec 10, 2023
ba0a7c7
Remove support for python3.7 add python3.12
oz123 Dec 10, 2023
72eca46
Remove duplicate method
oz123 Dec 10, 2023
3a1f17b
lockfiles: address some pylint issues
oz123 Dec 10, 2023
2b0767d
Fix bug introduced because of wrong validation
oz123 Dec 10, 2023
58bb217
Inherit Pipefile class from BaseModel
oz123 Dec 10, 2023
54419ad
Remove redundant comment
oz123 Dec 10, 2023
1693b41
Remove redundant code in lockfiles
oz123 Dec 10, 2023
08b8545
Remove unused field from all functions
oz123 Dec 10, 2023
b60109d
Address some pylint warnings
oz123 Dec 10, 2023
35cf04b
Remove unused argument field
oz123 Dec 10, 2023
9fd0118
Remove another usage of field in signature
oz123 Dec 10, 2023
e493f6a
Remove duplicate function
oz123 Dec 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
python-version: [3.7, 3.8, 3.9, "3.10", 3.11-dev]
python-version: [3.8, 3.9, "3.10", "3.11", "3.12"]
os: [ubuntu-latest]

steps:
Expand Down
67 changes: 33 additions & 34 deletions docs/nested.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,46 +12,45 @@ Pipfile and Pipfile.lock, such as individual entries in ``[packages]``,
``[dev-packages]``, and ``lockfile['_meta']``.


The Data View
=============
Base Model
===========

Every non-scalar value you get from Plette (e.g. sequence, mapping) is
represented as a `DataView`, or one of its subclasses. This class is simply a
wrapper around the basic collection class, and you can access the underlying
data structure via the ``_data`` attribute::
represented inherits from `models.BaseModel`, which is a Python `dataclass`::

>>> import plette.models
>>> source = plette.models.Source({
>>> source = plette.models.Source(**{
... 'name': 'pypi',
... 'url': 'https://pypi.org/simple',
... 'verify_ssl': True,
... })
...
>>> source._data
{'name': 'pypi', 'url': 'https://pypi.org/simple', 'verify_ssl': True}


Data View Collections
=====================

There are two special collection classes, ``DataViewMapping`` and
``DataViewSequence``, that hold homogeneous ``DataView`` members. They are
also simply wrappers to ``dict`` and ``list``, respectively, but have specially
implemented magic methods to automatically coerce contained data into a
``DataView`` subclass::

>>> sources = plette.models.SourceCollection([source._data])
>>> sources._data
[{'name': 'pypi', 'url': 'https://pypi.org/simple', 'verify_ssl': True}]
>>> type(sources[0])
<class 'plette.models.sources.Source'>
>>> sources[0] == source
True
>>> sources[0] = {
... 'name': 'devpi',
... 'url': 'http://localhost/simple',
... 'verify_ssl': True,
... }
...
>>> sources._data
[{'name': 'devpi', 'url': 'http://localhost/simple', 'verify_ssl': True}]
>>> source
Source(name='pypi', verify_ssl=True, url='https://pypi.org/simple')


Collections
===========

There a few special collection classes, which can be I dentified by the
suffix ``Collection`` or ``Specifiers``.
They group attributes and behave like ``list`` or ``mappings``.
These classes accept a list of dictionaries as input,
and convert them to the correct object type::

>>> SourceCollection([{'name': 'r-pi', 'url': '192.168.1.129:8000', 'verify_ssl': False}, {'name': 'pypi', 'url': 'https://pypi.org/simple', 'verify_ssl': True}])
SourceCollection(sources=[Source(name='r-pi', verify_ssl=False, url='192.168.1.129:8000'), Source(name='pypi', verify_ssl=True, url='https://pypi.org/simple')])


In addition, they can also accept a list of items of the correct type::

>>> rpi = models.Source(**{'name': 'r-pi', 'url': '192.168.1.129:8000', 'verify_ssl': False})
>>> pypi = models.Source(**{'name': 'pypi', 'url': 'https://pypi.org/simple', 'verify_ssl': True})
>>> SourceCollection([rpi, pypi])
SourceCollection(sources=[Source(name='r-pi', verify_ssl=False, url='192.168.1.129:8000'), Source(name='pypi', verify_ssl=True, url='https://pypi.org/simple')])

They can also be indexed by name, and can be iterated over::

>>> sc = SourceCollection([{'name': 'r-pi', 'url': '192.168.1.129:8000', 'verify_ssl': False}, {'name': 'pypi', 'url': 'https://pypi.org/simple', 'verify_ssl': True}])
>>> sc[0]
Source(name='r-pi', verify_ssl=False, url='192.168.1.129:8000')
54 changes: 1 addition & 53 deletions docs/validation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,56 +2,4 @@
Validating Data
===============

Plette provides optional validation for input data. This chapter discusses
how validation works.


Setting up Validation
=====================

Validation is provided by the Cerberus_ library. You can install it along with
Plette manually, or by specifying the “validation” extra when installing
Plette:

.. code-block:: none

pip install plette[validation]

Plette automatically enables validation when Cerberus is available.

.. _Cerberus: http://docs.python-cerberus.org/


Validating Data
===============

Data is validated on input (or when a model is loaded). ``ValidationError`` is
raised when validation fails::

>>> plette.models.Source({})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "plette/models/base.py", line 37, in __init__
self.validate(data)
File "plette/models/base.py", line 67, in validate
return validate(cls, data)
File "plette/models/base.py", line 27, in validate
raise ValidationError(data, v)
plette.models.base.ValidationError: {}

This exception class has a ``validator`` member to allow you to access the
underlying Cerberus validator, so you can know what exactly went wrong::

>>> try:
... plette.models.Source({'verify_ssl': True})
... except plette.models.ValidationError as e:
... for error in e.validator._errors:
... print(error.schema_path)
...
('name', 'required')
('url', 'required')

See `Ceberus’s error handling documentation`_ to know how the errors are
represented and reported.

.. _`Ceberus’s error handling documentation`: http://docs.python-cerberus.org/en/stable/errors.html
Validation is no longer an optional feature. All data models are validated.
2 changes: 1 addition & 1 deletion examples/Pipfile.invalid.extras-list
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# package specifiers should be a dict
# package extras should be a list
[packages]
msal = {version= "==1.20.0", extras = "broker"}

2 changes: 2 additions & 0 deletions examples/Pipfile.invalid.with-categories
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# this file contains a section with one package with wrong extras
# extras should be passed as a list
[packages]
plette = { path = '.', extras = ['validation'], editable = true }

Expand Down
6 changes: 6 additions & 0 deletions examples/extras-list/Pipfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@

# extras should be a list
[packages]
msal = {version="==1.20.0", extras=["broker"]}
parver = '*'

Loading