Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Upgrade to support both Pydantic v1 and v2 #699

Closed
wants to merge 29 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
d0ae3e8
migration to sqlalchemy 2.0
farahats9 Mar 7, 2023
8a27655
fix some linting errors
farahats9 Mar 24, 2023
a2b3c14
reflecting python 3.6 deprecation in docs and tests
farahats9 Mar 31, 2023
4760dbd
Update sqlmodel/sql/expression.py
farahats9 Mar 31, 2023
ad76a88
resolving @sbor23 comments
farahats9 Mar 31, 2023
96e44e5
add the new Subquery class
farahats9 Apr 30, 2023
9e72750
fix jinja2 template
farahats9 Apr 30, 2023
b19a709
`Result` expects a type `Tuple[_T]`
peterlandry Jul 26, 2023
ae369ed
Remove unused type ignore
peterlandry Jul 26, 2023
2ff42db
Result seems well enough typed in SqlAlchemy now we can simply shim over
peterlandry Jul 26, 2023
b6bd94f
_Select expects a `Tuple[Any, ...]`
peterlandry Jul 26, 2023
1dbce4d
Use Dict type instead of Mapping for SqlAlchemy compat
peterlandry Jul 26, 2023
c2d310e
Execution options are not Optional in SA
peterlandry Jul 26, 2023
05a1352
Another instance of non-optional execution_options
peterlandry Jul 26, 2023
bd00a2b
Fix Tuple in jinja template as well
peterlandry Jul 26, 2023
c65f018
Use ForUpdateArg from sqlalchemy
peterlandry Jul 26, 2023
4e29e00
Fix signature for `Session.get`
peterlandry Jul 26, 2023
fd85d02
formatting and remove unused type
farahats9 Jul 27, 2023
d556059
Upgrade to Pydantic 2
AntonDeMeester Jul 31, 2023
e92a52e
Formatting
AntonDeMeester Jul 31, 2023
61d5d8d
Linting
AntonDeMeester Jul 31, 2023
cb494b7
Make all tests but fastapi work
AntonDeMeester Aug 1, 2023
f590548
Get all tests except for openapi working
AntonDeMeester Aug 1, 2023
7ecbc38
Write for pydantic v1 and v2 compat
AntonDeMeester Nov 13, 2023
c21ff69
Make pydantic v1 work again
AntonDeMeester Nov 14, 2023
254fb13
Liniting
AntonDeMeester Nov 14, 2023
ab07514
Linter
AntonDeMeester Nov 14, 2023
4a4161a
Move lint to after tests to see tests
AntonDeMeester Nov 14, 2023
e2d4d1f
Make most tests succeed
AntonDeMeester Nov 16, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
17 changes: 10 additions & 7 deletions .github/workflows/test.yml
Expand Up @@ -20,11 +20,8 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version:
- "3.7"
- "3.8"
- "3.9"
- "3.10"
python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
pydantic-version: ["pydantic-v1", "pydantic-v2"]
fail-fast: false

steps:
Expand Down Expand Up @@ -55,8 +52,12 @@ jobs:
- name: Install Dependencies
if: steps.cache.outputs.cache-hit != 'true'
run: python -m poetry install
- name: Lint
run: python -m poetry run bash scripts/lint.sh
- name: Install Pydantic v1
if: matrix.pydantic-version == 'pydantic-v1'
run: pip install "pydantic>=1.10.0,<2.0.0"
- name: Install Pydantic v2
if: matrix.pydantic-version == 'pydantic-v2'
run: pip install "pydantic>=2.0.2,<3.0.0"
- run: mkdir coverage
- name: Test
run: python -m poetry run bash scripts/test.sh
Expand All @@ -68,6 +69,8 @@ jobs:
with:
name: coverage
path: coverage
- name: Lint
run: python -m poetry run bash scripts/lint.sh
coverage-combine:
needs:
- test
Expand Down
6 changes: 3 additions & 3 deletions docs/tutorial/fastapi/multiple-models.md
Expand Up @@ -174,13 +174,13 @@ Now we use the type annotation `HeroCreate` for the request JSON data in the `he
# Code below omitted 👇
```

Then we create a new `Hero` (this is the actual **table** model that saves things to the database) using `Hero.from_orm()`.
Then we create a new `Hero` (this is the actual **table** model that saves things to the database) using `Hero.model_validate()`.

The method `.from_orm()` reads data from another object with attributes and creates a new instance of this class, in this case `Hero`.
The method `.model_validate()` reads data from another object with attributes and creates a new instance of this class, in this case `Hero`.

The alternative is `Hero.parse_obj()` that reads data from a dictionary.

But as in this case, we have a `HeroCreate` instance in the `hero` variable. This is an object with attributes, so we use `.from_orm()` to read those attributes.
But as in this case, we have a `HeroCreate` instance in the `hero` variable. This is an object with attributes, so we use `.model_validate()` to read those attributes.

With this, we create a new `Hero` instance (the one for the database) and put it in the variable `db_hero` from the data in the `hero` variable that is the `HeroCreate` instance we received from the request.

Expand Down
4 changes: 4 additions & 0 deletions docs/tutorial/index.md
Expand Up @@ -64,6 +64,8 @@ $ cd sqlmodel-tutorial

Make sure you have an officially supported version of Python.

Currently it is **Python 3.7** and above (Python 3.6 was already deprecated).

You can check which version you have with:

<div class="termy">
Expand All @@ -79,9 +81,11 @@ There's a chance that you have multiple Python versions installed.

You might want to try with the specific versions, for example with:

* `python3.11`
* `python3.10`
* `python3.9`
* `python3.8`
* `python3.7`

The code would look like this:

Expand Down
6 changes: 5 additions & 1 deletion docs_src/tutorial/fastapi/app_testing/tutorial001/main.py
Expand Up @@ -2,6 +2,7 @@

from fastapi import Depends, FastAPI, HTTPException, Query
from sqlmodel import Field, Session, SQLModel, create_engine, select
from sqlmodel.compat import IS_PYDANTIC_V2


class HeroBase(SQLModel):
Expand Down Expand Up @@ -54,7 +55,10 @@ def on_startup():

@app.post("/heroes/", response_model=HeroRead)
def create_hero(*, session: Session = Depends(get_session), hero: HeroCreate):
db_hero = Hero.from_orm(hero)
if IS_PYDANTIC_V2:
db_hero = Hero.model_validate(hero)
else:
db_hero = Hero.from_orm(hero)
session.add(db_hero)
session.commit()
session.refresh(db_hero)
Expand Down
6 changes: 5 additions & 1 deletion docs_src/tutorial/fastapi/delete/tutorial001.py
Expand Up @@ -2,6 +2,7 @@

from fastapi import FastAPI, HTTPException, Query
from sqlmodel import Field, Session, SQLModel, create_engine, select
from sqlmodel.compat import IS_PYDANTIC_V2


class HeroBase(SQLModel):
Expand Down Expand Up @@ -50,7 +51,10 @@ def on_startup():
@app.post("/heroes/", response_model=HeroRead)
def create_hero(hero: HeroCreate):
with Session(engine) as session:
db_hero = Hero.from_orm(hero)
if IS_PYDANTIC_V2:
db_hero = Hero.model_validate(hero)
else:
db_hero = Hero.from_orm(hero)
session.add(db_hero)
session.commit()
session.refresh(db_hero)
Expand Down
6 changes: 5 additions & 1 deletion docs_src/tutorial/fastapi/limit_and_offset/tutorial001.py
Expand Up @@ -2,6 +2,7 @@

from fastapi import FastAPI, HTTPException, Query
from sqlmodel import Field, Session, SQLModel, create_engine, select
from sqlmodel.compat import IS_PYDANTIC_V2


class HeroBase(SQLModel):
Expand Down Expand Up @@ -44,7 +45,10 @@ def on_startup():
@app.post("/heroes/", response_model=HeroRead)
def create_hero(hero: HeroCreate):
with Session(engine) as session:
db_hero = Hero.from_orm(hero)
if IS_PYDANTIC_V2:
db_hero = Hero.model_validate(hero)
else:
db_hero = Hero.from_orm(hero)
session.add(db_hero)
session.commit()
session.refresh(db_hero)
Expand Down
6 changes: 5 additions & 1 deletion docs_src/tutorial/fastapi/multiple_models/tutorial001.py
Expand Up @@ -2,6 +2,7 @@

from fastapi import FastAPI
from sqlmodel import Field, Session, SQLModel, create_engine, select
from sqlmodel.compat import IS_PYDANTIC_V2


class Hero(SQLModel, table=True):
Expand Down Expand Up @@ -46,7 +47,10 @@ def on_startup():
@app.post("/heroes/", response_model=HeroRead)
def create_hero(hero: HeroCreate):
with Session(engine) as session:
db_hero = Hero.from_orm(hero)
if IS_PYDANTIC_V2:
db_hero = Hero.model_validate(hero)
else:
db_hero = Hero.from_orm(hero)
session.add(db_hero)
session.commit()
session.refresh(db_hero)
Expand Down
6 changes: 5 additions & 1 deletion docs_src/tutorial/fastapi/multiple_models/tutorial002.py
Expand Up @@ -2,6 +2,7 @@

from fastapi import FastAPI
from sqlmodel import Field, Session, SQLModel, create_engine, select
from sqlmodel.compat import IS_PYDANTIC_V2


class HeroBase(SQLModel):
Expand Down Expand Up @@ -44,7 +45,10 @@ def on_startup():
@app.post("/heroes/", response_model=HeroRead)
def create_hero(hero: HeroCreate):
with Session(engine) as session:
db_hero = Hero.from_orm(hero)
if IS_PYDANTIC_V2:
db_hero = Hero.model_validate(hero)
else:
db_hero = Hero.from_orm(hero)
session.add(db_hero)
session.commit()
session.refresh(db_hero)
Expand Down
6 changes: 5 additions & 1 deletion docs_src/tutorial/fastapi/read_one/tutorial001.py
Expand Up @@ -2,6 +2,7 @@

from fastapi import FastAPI, HTTPException
from sqlmodel import Field, Session, SQLModel, create_engine, select
from sqlmodel.compat import IS_PYDANTIC_V2


class HeroBase(SQLModel):
Expand Down Expand Up @@ -44,7 +45,10 @@ def on_startup():
@app.post("/heroes/", response_model=HeroRead)
def create_hero(hero: HeroCreate):
with Session(engine) as session:
db_hero = Hero.from_orm(hero)
if IS_PYDANTIC_V2:
db_hero = Hero.model_validate(hero)
else:
db_hero = Hero.from_orm(hero)
session.add(db_hero)
session.commit()
session.refresh(db_hero)
Expand Down
11 changes: 9 additions & 2 deletions docs_src/tutorial/fastapi/relationships/tutorial001.py
Expand Up @@ -2,6 +2,7 @@

from fastapi import Depends, FastAPI, HTTPException, Query
from sqlmodel import Field, Relationship, Session, SQLModel, create_engine, select
from sqlmodel.compat import IS_PYDANTIC_V2


class TeamBase(SQLModel):
Expand Down Expand Up @@ -92,7 +93,10 @@ def on_startup():

@app.post("/heroes/", response_model=HeroRead)
def create_hero(*, session: Session = Depends(get_session), hero: HeroCreate):
db_hero = Hero.from_orm(hero)
if IS_PYDANTIC_V2:
db_hero = Hero.model_validate(hero)
else:
db_hero = Hero.from_orm(hero)
session.add(db_hero)
session.commit()
session.refresh(db_hero)
Expand Down Expand Up @@ -146,7 +150,10 @@ def delete_hero(*, session: Session = Depends(get_session), hero_id: int):

@app.post("/teams/", response_model=TeamRead)
def create_team(*, session: Session = Depends(get_session), team: TeamCreate):
db_team = Team.from_orm(team)
if IS_PYDANTIC_V2:
db_team = Team.model_validate(team)
else:
db_team = Team.from_orm(team)
session.add(db_team)
session.commit()
session.refresh(db_team)
Expand Down
Expand Up @@ -2,6 +2,7 @@

from fastapi import Depends, FastAPI, HTTPException, Query
from sqlmodel import Field, Session, SQLModel, create_engine, select
from sqlmodel.compat import IS_PYDANTIC_V2


class HeroBase(SQLModel):
Expand Down Expand Up @@ -54,7 +55,10 @@ def on_startup():

@app.post("/heroes/", response_model=HeroRead)
def create_hero(*, session: Session = Depends(get_session), hero: HeroCreate):
db_hero = Hero.from_orm(hero)
if IS_PYDANTIC_V2:
db_hero = Hero.model_validate(hero)
else:
db_hero = Hero.from_orm(hero)
session.add(db_hero)
session.commit()
session.refresh(db_hero)
Expand Down
11 changes: 9 additions & 2 deletions docs_src/tutorial/fastapi/teams/tutorial001.py
Expand Up @@ -2,6 +2,7 @@

from fastapi import Depends, FastAPI, HTTPException, Query
from sqlmodel import Field, Relationship, Session, SQLModel, create_engine, select
from sqlmodel.compat import IS_PYDANTIC_V2


class TeamBase(SQLModel):
Expand Down Expand Up @@ -83,7 +84,10 @@ def on_startup():

@app.post("/heroes/", response_model=HeroRead)
def create_hero(*, session: Session = Depends(get_session), hero: HeroCreate):
db_hero = Hero.from_orm(hero)
if IS_PYDANTIC_V2:
db_hero = Hero.model_validate(hero)
else:
db_hero = Hero.from_orm(hero)
session.add(db_hero)
session.commit()
session.refresh(db_hero)
Expand Down Expand Up @@ -137,7 +141,10 @@ def delete_hero(*, session: Session = Depends(get_session), hero_id: int):

@app.post("/teams/", response_model=TeamRead)
def create_team(*, session: Session = Depends(get_session), team: TeamCreate):
db_team = Team.from_orm(team)
if IS_PYDANTIC_V2:
db_team = Team.model_validate(team)
else:
db_team = Team.from_orm(team)
session.add(db_team)
session.commit()
session.refresh(db_team)
Expand Down
6 changes: 5 additions & 1 deletion docs_src/tutorial/fastapi/update/tutorial001.py
Expand Up @@ -2,6 +2,7 @@

from fastapi import FastAPI, HTTPException, Query
from sqlmodel import Field, Session, SQLModel, create_engine, select
from sqlmodel.compat import IS_PYDANTIC_V2


class HeroBase(SQLModel):
Expand Down Expand Up @@ -50,7 +51,10 @@ def on_startup():
@app.post("/heroes/", response_model=HeroRead)
def create_hero(hero: HeroCreate):
with Session(engine) as session:
db_hero = Hero.from_orm(hero)
if IS_PYDANTIC_V2:
db_hero = Hero.model_validate(hero)
else:
db_hero = Hero.from_orm(hero)
session.add(db_hero)
session.commit()
session.refresh(db_hero)
Expand Down
30 changes: 15 additions & 15 deletions docs_src/tutorial/many_to_many/tutorial003.py
Expand Up @@ -3,25 +3,12 @@
from sqlmodel import Field, Relationship, Session, SQLModel, create_engine, select


class HeroTeamLink(SQLModel, table=True):
team_id: Optional[int] = Field(
default=None, foreign_key="team.id", primary_key=True
)
hero_id: Optional[int] = Field(
default=None, foreign_key="hero.id", primary_key=True
)
is_training: bool = False

team: "Team" = Relationship(back_populates="hero_links")
hero: "Hero" = Relationship(back_populates="team_links")


class Team(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str = Field(index=True)
headquarters: str

hero_links: List[HeroTeamLink] = Relationship(back_populates="team")
hero_links: List["HeroTeamLink"] = Relationship(back_populates="team")


class Hero(SQLModel, table=True):
Expand All @@ -30,7 +17,20 @@ class Hero(SQLModel, table=True):
secret_name: str
age: Optional[int] = Field(default=None, index=True)

team_links: List[HeroTeamLink] = Relationship(back_populates="hero")
team_links: List["HeroTeamLink"] = Relationship(back_populates="hero")


class HeroTeamLink(SQLModel, table=True):
team_id: Optional[int] = Field(
default=None, foreign_key="team.id", primary_key=True
)
hero_id: Optional[int] = Field(
default=None, foreign_key="hero.id", primary_key=True
)
is_training: bool = False

team: "Team" = Relationship(back_populates="hero_links")
hero: "Hero" = Relationship(back_populates="team_links")


sqlite_file_name = "database.db"
Expand Down
Expand Up @@ -3,6 +3,21 @@
from sqlmodel import Field, Relationship, SQLModel, create_engine


class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str = Field(index=True)
secret_name: str
age: Optional[int] = Field(default=None, index=True)

team_id: Optional[int] = Field(default=None, foreign_key="team.id")
team: Optional["Team"] = Relationship(back_populates="heroes")

weapon_id: Optional[int] = Field(default=None, foreign_key="weapon.id")
weapon: Optional["Weapon"] = Relationship(back_populates="hero")

powers: List["Power"] = Relationship(back_populates="hero")


class Weapon(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str = Field(index=True)
Expand All @@ -26,21 +41,6 @@ class Team(SQLModel, table=True):
heroes: List["Hero"] = Relationship(back_populates="team")


class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str = Field(index=True)
secret_name: str
age: Optional[int] = Field(default=None, index=True)

team_id: Optional[int] = Field(default=None, foreign_key="team.id")
team: Optional[Team] = Relationship(back_populates="heroes")

weapon_id: Optional[int] = Field(default=None, foreign_key="weapon.id")
weapon: Optional[Weapon] = Relationship(back_populates="hero")

powers: List[Power] = Relationship(back_populates="hero")


sqlite_file_name = "database.db"
sqlite_url = f"sqlite:///{sqlite_file_name}"

Expand Down