Skip to content
This repository has been archived by the owner on Sep 12, 2023. It is now read-only.

Commit

Permalink
feat(dto) - rework of dto pattern (#139)
Browse files Browse the repository at this point in the history
* ✨ feat(dto): decorator based pydantic models gen with validation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* ♻️ refactor: fix mypy

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* ♻️ refactor: fix pylint

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* feat(dto): refactor dto.factory.

Moves the core changes from dto.dto() into dto.factory() to centralize
logic.

* Update src/starlite_saqlalchemy/dto.py

* feat(dto): more iteration.

- Adds `dto.Attrib.pydantic_type` and `dto.Attrib.validators` where
`pydantic_type` allows the dto field type to be explicitly set, and
`validators` allows arbitrary single argument callables to be specified
as dto validators for any model attribute.
- `factory()` should now handle `MappedAsDataclass` sqla models
- `decorator()` is for decorating pydantic models, and uses the
decorated class as the base class for the model.
- adds some tests.

* feat(dto): more iteration.

Further development of the decorator pattern.

* feat!(dto): refactor of DTOs.. again.

Have done away with both `factory()` and `decorator()`, as neither
plays nicely with static type checkers.

Using `FromMapped[Annotated[Author, dto.config("read")]]` has the same
result as the old `dto.factory("DTO", Author, dto.Purpose.READ)`.

In `FromMapped.__class_getitem__()` we create a new class, assign the
model to it, and return it inplace of `FromMapped`.

This new class can be subclassed in order to define pydantic validators
and override types, just as any other pydantic model would be.

* feat!(dto): more dto iteration.

This PR refreshes DTO docs, gets test cov up to 100%, and adds some
ergonomic improvements to the dto declaration syntax.

Co-authored-by: gazorby <gazorby@pm.me>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Matthieu MN <10926130+gazorby@users.noreply.github.com>
  • Loading branch information
4 people committed Nov 28, 2022
1 parent e929713 commit e503447
Show file tree
Hide file tree
Showing 28 changed files with 881 additions and 475 deletions.
2 changes: 2 additions & 0 deletions .flake8
Expand Up @@ -4,6 +4,8 @@ max-line-length = 100
ignore = E,W,B008
type-checking-exempt-modules = from sqlalchemy.orm
per-file-ignores =
examples/dto/*:T201,TC
examples/tests/*:SCS108,PT013
src/starlite_saqlalchemy/dependencies.py:TC
src/starlite_saqlalchemy/health.py:TC
src/starlite_saqlalchemy/repository/filters.py:TC
Expand Down
4 changes: 2 additions & 2 deletions CHANGELOG.md
Expand Up @@ -85,14 +85,14 @@

## v0.3.0 (2022-11-07)
### Feature
* **dto:** Dto.Attrib and dto.Mark. ([`ca47028`](https://github.com/topsport-com-au/starlite-saqlalchemy/commit/ca47028f674b696493564d07379b589756433cc1))
* **dto:** Dto.DTOField and dto.Mark. ([`ca47028`](https://github.com/topsport-com-au/starlite-saqlalchemy/commit/ca47028f674b696493564d07379b589756433cc1))

### Fix
* **log:** Fix dev config write to bytes logger. ([`2d8c69e`](https://github.com/topsport-com-au/starlite-saqlalchemy/commit/2d8c69ec93083d1d6dc42ebceb8e43b02cde9408))
* Removes static files config. ([`bdf5ea5`](https://github.com/topsport-com-au/starlite-saqlalchemy/commit/bdf5ea56d4f04f6fa7b907492d305417e48be9f1))

### Breaking
* dto.Attrib and dto.Mark. ([`ca47028`](https://github.com/topsport-com-au/starlite-saqlalchemy/commit/ca47028f674b696493564d07379b589756433cc1))
* dto.DTOField and dto.Mark. ([`ca47028`](https://github.com/topsport-com-au/starlite-saqlalchemy/commit/ca47028f674b696493564d07379b589756433cc1))
* removes static files config. ([`bdf5ea5`](https://github.com/topsport-com-au/starlite-saqlalchemy/commit/bdf5ea56d4f04f6fa7b907492d305417e48be9f1))

### Documentation
Expand Down
216 changes: 131 additions & 85 deletions docs/dto.md
Expand Up @@ -5,127 +5,173 @@
DTO stands for "Data Transfer Object". They are the filter through which data is accepted into, and
output from the application.

## DTO Factory
## Why DTOs?

`starlite-saqlalchemy` includes
[`dto.factory()`](../reference/starlite_saqlalchemy/dto/#starlite_saqlalchemy.dto.factory)
which automatically creates [pydantic](https://pydantic-docs.helpmanual.io/) models from
[SQLAlchemy 2.0 ORM](https://docs.sqlalchemy.org/en/20/orm/) models.
Data that is modifiable by clients, and that should be read by clients is often only a subset of the
attributes that make up a domain object.

### Creating a SQLAlchemy ORM model
For example, lets say we have an internal representation of an `Author` that looks like this:

If you are new to SQLAlchemy, I cannot recommend their docs enough. Start at
[the beginning](https://docs.sqlalchemy.org/en/20/orm/quickstart.html), and follow along until you
are comfortable. I won't even try to compete with the quality and depth of information that can be
found there - a credit to everyone who has contributed to that project over the years.
```json
{
"id": "97108ac1-ffcb-411d-8b1e-d9183399f63b",
"name": "Agatha Christie",
"dob": "1890-9-15",
"created": "2022-11-27T01:58:00",
"updated": "2022-11-27T01:59:00"
}
```

### Configuring generated DTOs
Of those attributes, values for "id", "created" and "updated" are internally generated, and should
not be available to be modified by clients of our application.

#### dto.Purpose
This is where a DTO comes in. We create a type to validate user input that will only allow values
for "name" and "dob" from clients, for example:

The `dto.Purpose` enum tells the factory if the purpose of the DTO is to parse data submitted by the
client for updating or "writing" to a resource, or if it is to serialize data to be transmitted back
to, or "read" by the client.
```json
{
"name": "Agatha Christie",
"dob": "1890-9-15"
}
```

The DTO objects generated by the factory may differ due to the intended purpose of the DTO. Here's
an example of setting the DTO purpose:
## dto.FromMapped

```python
from starlite_saqlalchemy import dto
- Generate pydantic models from SQLAlchemy ORM models.
- Mark fields as "read-only" or "private" to control inclusion of fields on DTO models.
- Automatically infer defaults, and default factories from the SQLAlchemy column definitions.

from domain.users import User
The [`dto.FromMapped`](../reference/starlite_saqlalchemy/dto/#starlite_saqlalchemy.dto.FromMapped)
type allows us to use our domain models, which are defined as SQLAlchemy ORM types, to generate
DTOs.

Here's a quick example.

ReadDTO = dto.factory("UserReadDTO", model=User, purpose=dto.Purpose.READ)
WriteDTO = dto.factory("UserWriteDTO", model=User, purpose=dto.Purpose.WRITE)
```py title="Simple Example"
--8<-- "examples/dto/minimal.py"
```

#### dto.Attrib
Read the comments in the example for a description of everything that is going on, however notice
that the fields on our DTO type include "id", "created", and "updated" - fields that should not be
modifiable by clients.

The `dto.Attrib` object is a container for configuring how the generated DTO object should reflect
the SQLAlchemy model field.
Let's have another go:

Define this in the SQLAlchemy `info` parameter to `mapped_column()`, for example,
`mapped_column(info={"dto": dto.Attrib()})`.
```py title="Simple Example with Read Only Fields"
--8<-- "examples/dto/minimal_configure_fields.py"
```

The DTO Attrib object has two values that can be set:
That's better! Now, we'll only parse "name" and "dob" fields out of client input.

- `dto.Attrib.mark`: a value of the enum [`dto.Mark`](.. /reference/starlite_saqlalchemy/dto/#Mark).
- `dto.Attrib.pydantic_field`: return value of the pydantic
[`Field`](https://pydantic-docs.helpmanual.io/usage/schema/#field-customization) function that
will be used to construct the pydantic model field for the model attribute.
## Configuring generated DTOs

#### dto.Mark
Th two main factors that influence how a DTO is generated for a given domain model are:

We use the `info` parameter to `mapped_column()` to guide `dto.factory()`.
1. The modifiability and privacy of the individual attributes of the domain model.
2. The purpose of the DTO, is it to be used to parse and validate inbound client data, or to
serialize outbound data.

The [dto.Mark](../reference/starlite_saqlalchemy/dto/#Mark) enumeration is used to indicate on the
whether properties on the SQLAlchemy model should always be private, or read-only.
## Configuring DTO Fields

For example:
### dto.DTOField

```python
from datetime import datetime
This is the object that we use to configure DTO fields. To use the
[`dto.DTOField`][starlite_saqlalchemy.dto.types.DTOField] object assign a dict to the `mapped_column()` or
`relationship` `info` parameter, with the `"dto"` key and an instance of `dto.DTOField` as value, for
example `col: Mapped[str] = mapped_column(info={"dto": dto.DTOField(...)})`.

from sqlalchemy.orm import mapped_column
from starlite_saqlalchemy import dto, orm
The `dto.DTOField` object supports marking fields as `"read-only"` or `"private"`, setting an explicit
pydantic `FieldInfo` and type, and setting validators for the field.

The easiest way to configure a DTO field is through the
[`dto.field()`][starlite_saqlalchemy.dto.utils.field] function.

class User(orm.Base):
name: str
password_hash: str = mapped_column(info={"dto": dto.Attrib(mark=dto.Mark.SKIP)})
updated_at: datetime = mapped_column(
info={"dto": dto.Attrib(mark=dto.Mark.READ_ONLY)}
)
### dto.field()

The [`dto.field()`][starlite_saqlalchemy.dto.utils.field] function creates an `info` dict for us,
setting values on an `dto.DTOField` instance as appropriate. For example, the following are identical:

ReadDTO = dto.factory("UserReadDTO", model=User, purpose=dto.Purpose.READ)
WriteDTO = dto.factory("UserWriteDTO", model=User, purpose=dto.Purpose.WRITE)
```
- `col: Mapped[str] = mapped_column(info={"dto": dto.DTOField(mark=dto.Mark.PRIVATE)})`
- `col: Mapped[str] = mapped_column(info=dto.field("private"))`

Both `ReadDTO` and `WriteDTO` are pydantic models that have a `name` attribute.
`field()` supports the same arguments as `DTOField`, however it will also coerce string values for mark
to the appropriate enum.

Neither `ReadDTO` or `WriteDTO` have a `password_hash` attribute - this is the side effect of
marking the column with `dto.Mark.SKIP`. Skipped columns are never included in any generated DTO
model, meaning that they are unable to be read or modified by the client.
### dto.Mark

`ReadDTO` has an `updated_at` field, while `WriteDTO` does not. This is the side effect of marking
the column with `dto.Mark.READ_ONLY` - these fields will only be included in DTOs generated for
`dto.Purpose.READ` and make sense for fields that have internally generated values.
Fields on our domain models can take one of three states.

The following class is pretty much the same as
[`orm.Base`](../reference/starlite_saqlalchemy/orm/#starlite_saqlalchemy.orm.Base) - the bundled
SQLAlchemy base class that comes with `starlite-saqlalchemy`.
1. Normal - field can be written to, and read by clients, this is the state of unmarked fields.
2. Read-only - field can be read by clients, but not modified.
3. Private - field can not be read or updated by client.

```python
from datetime import datetime
from uuid import UUID, uuid4
The [`dto.Mark`][starlite_saqlalchemy.dto.types.Mark] enumeration lets us express these states on
our domain models.

from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
`dto.field()` will accept the mark values as either the explicit enum, or its string representation,
e.g., `dto.field(dto.Mark.PRIVATE)` and `dto.field("private")` are equivalent.

from starlite_saqlalchemy import dto
### Example

The following example demonstrates all field configurations available via the `field()` function.

class Base(DeclarativeBase):
id: Mapped[UUID] = mapped_column(
default=uuid4,
primary_key=True,
info={"dto": dto.Attrib(mark=dto.Mark.READ_ONLY)},
)
"""Primary key column."""
created: Mapped[datetime] = mapped_column(
default=datetime.now, info={"dto": dto.Attrib(mark=dto.Mark.READ_ONLY)}
)
"""Date/time of instance creation."""
updated: Mapped[datetime] = mapped_column(
default=datetime.now, info={"dto": dto.Attrib(mark=dto.Mark.READ_ONLY)}
)
```py title="DTOField Configuration Example"
--8<-- "examples/dto/complete_field_example.py"
```

Notice that all these fields are marked as `dto.Mark.READ_ONLY`. This means that they are unable to
be modified by clients, even if they include values for them in the payloads to `POST`/`PUT`/`PATCH`
routes.
### SQLAlchemy info dictionary

SQLAlchemy [`Column`][sqlalchemy.schema.Column] and [`relationship`][sqlalchemy.orm.relationship]
accept an `info` parameter, which allows us to store data alongside the columns and relationships of
our model definitions. This is what we use to configure our DTOs at the model level.

### Info dict namespace key

The key that is used to namespace our DTO configuration in the `info` dict is configurable via
environment. By default, this is `"dto"`, however it can be changed to anything you like by setting
the `API_DTO_INFO_KEY` environment variable.

## Configuring DTO Objects

### dto.DTOConfig

This is the object that controls the generated DTO, and should be passed as the first argument to
`Annotated` when declaring the DTO. For example, to create a "read" purposed DTO that excludes the
"id" field:

`ReadDTO = dto.FromMapped[Annotated[Author, dto.DTOConfig(purpose=dto.Purpose.READ, exclude={"id"})]]`

The [`dto.config()`][starlite_saqlalchemy.dto.utils.config] function allows for more compact
expression of DTO configuration.

### dto.config()

Factory function for creating `DTOConfig` instances, and handles coercing the literal strings "read"
and "write" to their `dto.Purpose` enum counterpart.

For example to create a write purposed DTO using the `dto.config()` function:

`WriteDTO = dto.FromMapped[Annotated[Author, dto.config("write")]]`

Which is equivalent to:

`WriteDTO = dto.FromMapped[Annotated[Author, dto.DTOConfig(purpose=dto.Purpose.WRITE)]]`

### Annotated positional arguments

The first argument to [`Annotated`][typing.Annotated] must always be the SQLAlchemy ORM type.
We inspect a single additional positional argument after that, which can either be the string name
of a `dto.Purpose` enum, or a `dto.DTOConfig` object.

For example, these three definitions are equivalent:

- `WriteDTO = dto.FromMapped[Annotated[Author, dto.DTOConfig(purpose=dto.Purpose.WRITE)]]`
- `WriteDTO = dto.FromMapped[Annotated[Author, dto.config("write")]]`
- `WriteDTO = dto.FromMapped[Annotated[Author, "write"]]`

### dto.Purpose

You can inherit from `orm.Base` to create your SQLAlchemy models, but you don't have to. You can
choose to subclass `orm.Base` or roll your own base class altogether. `dto.factory()` will still
work as advertised.
`dto.Purpose` has two values, `dto.Purpose.READ` and `dto.Purpose.WRITE`. These are used to tell
the factory if the purpose of the DTO is to parse data submitted by the client for updating or
"writing" to a resource, or if it is to serialize data to be transmitted back to, or "read" by the
client.
Empty file added examples/dto/__init__.py
Empty file.
34 changes: 34 additions & 0 deletions examples/dto/complete_field_example.py
@@ -0,0 +1,34 @@
from __future__ import annotations

from pydantic import Field, constr
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column

from starlite_saqlalchemy import dto


def check_email(email: str) -> str:
"""Validate an email."""
if "@" not in email:
raise ValueError("Invalid email!")
return email


class Base(DeclarativeBase):
"""Our ORM base class."""


class Thing(Base):
"""Something in our domain."""

__tablename__ = "things"
# demonstrates marking a field as "read-only" and overriding the generated pydantic `FieldInfo`
# for the DTO field.
id = mapped_column(
primary_key=True, info=dto.field("read-only", pydantic_field=Field(alias="identifier"))
)
# demonstrates overriding the type assigned to the field in generated DTO
always_upper: Mapped[str] = mapped_column(info=dto.field(pydantic_type=constr(to_upper=True)))
# demonstrates setting a field as "private"
private: Mapped[str] = mapped_column(info=dto.field("private"))
# demonstrates setting a validator for the field
email: Mapped[str] = mapped_column(info=dto.field(validators=[check_email]))
51 changes: 51 additions & 0 deletions examples/dto/minimal.py
@@ -0,0 +1,51 @@
from __future__ import annotations

from datetime import date, datetime
from typing import Annotated

from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column

from starlite_saqlalchemy import dto


class Base(DeclarativeBase):
"""ORM base class.
All SQLAlchemy ORM models must inherit from DeclarativeBase. We also
define some common columns that we want to be present on every model
in our domain.
"""

id: Mapped[int] = mapped_column(primary_key=True)
created: Mapped[datetime]
updated: Mapped[datetime]


class Author(Base):
"""A domain model.
In addition to the columns defined on `Base` we have "name" and
"dob".
"""

__tablename__ = "authors"
name: Mapped[str]
dob: Mapped[date]


# This creates a DTO, which is simply a Pydantic model that inherits
# from our special `FromMapped` subclass. We call it "WriteDTO" as it
# is the model that we'll use to parse client data as they try to
# "write" to (that is, create or update) authors in our domain.
WriteDTO = dto.FromMapped[Annotated[Author, "write"]]

# we can inspect the fields that are available on the DTO
print(WriteDTO.__fields__)

# {
# "id": ModelField(name="id", type=int, required=True),
# "created": ModelField(name="created", type=datetime, required=True),
# "updated": ModelField(name="updated", type=datetime, required=True),
# "name": ModelField(name="name", type=str, required=True),
# "dob": ModelField(name="dob", type=date, required=True),
# }

0 comments on commit e503447

Please sign in to comment.