-
-
Notifications
You must be signed in to change notification settings - Fork 608
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SQLModel doesn't raise ValidationError #52
Comments
I was just going through this aswell! If the model as table=True then validations are disabled, and it assume SQLalchemy will pick up any issues. It then basically drops those values. I haven't figured out the best way to handle this. |
I just checked and I experience the same behaviour with |
May it have connection with pydantic's |
Ok so I have been looking at work-arounds for this - as I don't want to double up on models just to do validation. If the model is defined with From the init of the SQL Model Class: # Only raise errors if not a SQLModel model
if (
not getattr(__pydantic_self__.__config__, "table", False)
and validation_error
):
raise validation_error You can manually validate when importing the data using the 'Model.validate()` class method for assignment, but for that we need to pass in a dict with all the values, but at least at this point in will validate the data. from typing import Optional
from sqlmodel import SQLModel, Field, Column, VARCHAR
from pydantic import EmailStr
class UserExample(SQLModel, table=True):
id: Optional[int] = Field(primary_key=True)
username: EmailStr = Field(sa_column=Column("username", VARCHAR, unique=True))
full_name: Optional[str] = None
disabled: Optional[bool] = None
# Create and instance
test_user1 = UserExample(
username="jdoe", # oops this should be an email address
full_name="John Doe",
disabled=False,
)
print("Test User 1:", test_user1) Which gives us
This is bad, since we have silently dropped our required email address field - which also needs to be unique. If I write this to the database it will fail, which is fine but it would be nice to catch this at the assignment point rather than trying to write it to the DB. Also if this was an optional field, it would silently drop the data, and we wouldn't know until we tried to read it and it wasn't there ( it wouldn't break anything since it is optional, but it would be data loss) so now if I do it this way: from typing import Optional
from sqlmodel import SQLModel, Field, Column, VARCHAR
from pydantic import EmailStr
class UserExample(SQLModel, table=True):
id: Optional[int] = Field(primary_key=True)
username: EmailStr = Field(sa_column=Column("username", VARCHAR, unique=True))
full_name: Optional[str] = None
disabled: Optional[bool] = None
# This time create the instance and pass in a dict of the values by call .validate
test_user2 = UserExample.validate(
{"username": "jdoe", "full_name": "John Doe", "disabled": False}
)
print("Test User 2:", test_user2) I get this:
So this works and gives me my validation error. So it can be used as a workaround for those times where you are getting data ready to put in the DB, though it does mean that data needs to be in a dict format - but most likely that wont be that much of a problem. Hopefully this helps as a workaround. |
My god @obassett, I've been pulling my hair out for hours trying to figure out why my validators were running, but doing nothing.
Much thanks |
One caveat of using The |
In the documentation, it recommends to create a base model and inherit from it for the database model (table=True). |
This is really sad. A prime reason to use SQLModel is to be able to continue to use your Pydantic models, just with some added SQL goodness. But if all validation is ignored, the models are hardly useful as Pydantic models anymore. Working around this limitation either by doubling the number of classes or passing all data as dictionaries is not particularly appealing. Really hoping for a simpler way to enable the built-in validation. Recently chipped in a small sponsorship hoping this gets some well-deserved attention by @tiangolo! 😞 |
Any updates on this issue? I found a workaround that raises the validation error while also having from sqlmodel import SQLModel, Field
class BaseSQLModel(SQLModel):
def __init__(self, **kwargs):
self.__config__.table = False
super().__init__(**kwargs)
self.__config__.table = True
class Config:
validate_assignment = True
class MyTable1(BaseSQLModel, table=True):
a: int = Field(primary_key=True)
b: int
class MyTable2(BaseSQLModel, table=True):
a: int = Field(primary_key=True)
b: float
t1 = MyTable1(a=1, b=2) # ok
t2 = MyTable2(b="text") # Raises ValidationError
# pydantic.error_wrappers.ValidationError: 2 validation errors for MyTable2
# a
# field required (type=value_error.missing)
# b
# value is not a valid float (type=type_error.float) By creating a base class and defining the With this approach, creating multiple models for different classes to inherit or calling the Edit: Added Config class to BaseSQLModel with the |
@andremmori This works excellently and solves this long standing problem. Thank you! |
I personnaly don't feel this way. It's more of a hack, and actually doesn't even enable to validate fields on update (not at initialization of the model). And defining a BaseModel, from which the SQLModel should inherit, seems a bit over-complexifying things to me... I'd be glad if there was an actual clean and easy way to validate a SQLModel, the same way it is proposed for a BaseModel. |
I don't think it's a good solution, but it seems to work for now. from typing import Tuple, Dict, Any, Union, Type
from traceback import format_exception_only
from sqlmodel.main import (
SQLModelMetaclass,
__dataclass_transform__,
Field,
FieldInfo,
default_registry,
)
from sqlmodel import SQLModel
@__dataclass_transform__(kw_only_default=True, field_descriptors=(Field, FieldInfo))
class ValidSQLModelMetaclass(SQLModelMetaclass):
def __new__(
cls,
name: str,
bases: Tuple[Type[Any], ...],
class_dict: Dict[str, Any],
**kwargs: Any,
) -> Any:
valid: bool = kwargs.pop("valid", False)
new = super().__new__(cls, name, bases, class_dict, **kwargs)
if valid and kwargs.pop("table", False):
setattr(
new.__config__,
"_pydantic_model",
SQLModelMetaclass.__new__(cls, name, bases, class_dict, **kwargs),
)
else:
setattr(new.__config__, "_pydantic_model", new)
setattr(new.__config__, "_valid_sqlmodel", valid)
return new
class ValidSQLModel(
SQLModel, metaclass=ValidSQLModelMetaclass, registry=default_registry
):
def __new__(cls, *args: Any, **kwargs: Any) -> Any:
new = super().__new__(cls, *args, **kwargs)
if getattr(new.__config__, "_valid_sqlmodel", False) and getattr(
new.__config__, "table", False
):
getattr(new.__config__, "_pydantic_model").validate(kwargs)
return new
class TestBase(ValidSQLModel):
id: Union[int, None] = Field(default=None, primary_key=True)
a: int = Field(gt=100)
b: float = Field(gt=-1, lt=1)
c: str = Field(min_length=1, max_length=10)
class InvalidateTest(TestBase, table=True):
...
class ValidateTest(TestBase, table=True, valid=True):
...
def test(target: Type[TestBase], *, a: int, b: float, c: str, end: str):
print(f"{target=}", end=":: ")
try:
print(target(a=a, b=b, c=c), end="\n" + end)
except ValueError as exc:
print("".join(format_exception_only(type(exc), exc)), end=end + "\n")
for value in [
{"a": 1, "b": 1, "c": ""},
{"a": 1, "b": 0.5, "c": ""},
{"a": 1, "b": 0.5, "c": "1"},
{"a": 101, "b": 0.5, "c": "1"},
{"a": 101, "b": 1, "c": "100_000_000"},
]:
print(f"{value=}")
test(InvalidateTest, **value, end="")
test(ValidateTest, **value, end="=" * 50)
"""
value={'a': 1, 'b': 1, 'c': ''}
target=<class '__main__.InvalidateTest'>:: id=None
target=<class '__main__.ValidateTest'>:: pydantic.error_wrappers.ValidationError: 3 validation errors for ValidateTest
a
ensure this value is greater than 100 (type=value_error.number.not_gt; limit_value=100)
b
ensure this value is less than 1 (type=value_error.number.not_lt; limit_value=1)
c
ensure this value has at least 1 characters (type=value_error.any_str.min_length; limit_value=1)
==================================================
value={'a': 1, 'b': 0.5, 'c': ''}
target=<class '__main__.InvalidateTest'>:: id=None b=0.5
target=<class '__main__.ValidateTest'>:: pydantic.error_wrappers.ValidationError: 2 validation errors for ValidateTest
a
ensure this value is greater than 100 (type=value_error.number.not_gt; limit_value=100)
c
ensure this value has at least 1 characters (type=value_error.any_str.min_length; limit_value=1)
==================================================
value={'a': 1, 'b': 0.5, 'c': '1'}
target=<class '__main__.InvalidateTest'>:: id=None b=0.5 c='1'
target=<class '__main__.ValidateTest'>:: pydantic.error_wrappers.ValidationError: 1 validation error for ValidateTest
a
ensure this value is greater than 100 (type=value_error.number.not_gt; limit_value=100)
==================================================
value={'a': 101, 'b': 0.5, 'c': '1'}
target=<class '__main__.InvalidateTest'>:: id=None a=101 b=0.5 c='1'
target=<class '__main__.ValidateTest'>:: id=None a=101 b=0.5 c='1'
==================================================value={'a': 101, 'b': 1, 'c': '100_000_000'}
target=<class '__main__.InvalidateTest'>:: id=None a=101
target=<class '__main__.ValidateTest'>:: pydantic.error_wrappers.ValidationError: 2 validation errors for ValidateTest
b
ensure this value is less than 1 (type=value_error.number.not_lt; limit_value=1)
c
ensure this value has at most 10 characters (type=value_error.any_str.max_length; limit_value=10)
==================================================
""" |
My previous comment is just a workaround while this issue isn't fixed on SQLModel's side. Thanks for pointing out that it doesn't validate fields on updates; I edited my last comment with a fix for this. It is (strangely) Pydantic's BaseModel default behavior not to validate assignments. To do so, it requires a Config class on the model with the Unfortunately, we still have to wait for this to be addressed in a future release to have a proper solution. PR #227 was already created to fix this but hasn't gotten the attention it deserves. |
I feel that the proposed solution by @andremmori is quite hackish. I don't know SQLModel well enough (yet) to feel confident about setting I find this workaround safer:
|
Hey all! Please read the docs: https://sqlmodel.tiangolo.com/tutorial/fastapi/multiple-models/ The model with If you need to validate data, declare a base model with Validate the data with the data model and copy the data to the table model after validation. And using inheritance you avoid duplicating code. |
@tiangolo thanks for clarifying. Closing the issue now. |
Thanks for reporting back and closing the issue 👍 |
@tiangolo Thank you for clarifying the validation process in https://sqlmodel.tiangolo.com/tutorial/fastapi/multiple-models/ |
thank you @andremmori , using your workaround as it seems the best to me to achieve table declaration and validation at the same time, which is why I'm using SQLModel in the first place I found myself removing class Config:
validate_assignment = True as it was validating twice |
TODO: check why instantiating an ExerciseType does not validate it! See: tiangolo/sqlmodel#52
@tiangolo That approach is fine (and often necessary) from a data creation point of view, but that's only one side of the coin. Often you simply need to do some transformation/validation on the data you get from the database (e.g. when dealing with potentially incorrectly formatted, legacy data, or turning a naive datetime to UTC, etc.) and it is extremely inconvenient to declare yet another model and manually do the conversion just to get Pydantic validation working. I see two ways for improvement without breaking the existing functionality: adding an extra class argument to If one of these proposals is acceptable, I'd be happy to start working on a PR. |
Sorry, that I have to kindly express quite some criticism on the status quo and support @volfpeter . Also, I just recently started to dive into SQLmodel, so sorry in advance, if I overlooked something. This comes with all due respect to the authors of this package. Thanks for your work, highly appreciated! This package promises, to fuse validation and database definition model into one thing. Unfortunately, this issue seems to defeat this main goal to quite some extent. Reading #52 (comment) : Unfortunately I still don't see, why it shouldn't be possible to execute at least the most common pydantic validations. Checking for ordinary constraints, range, size, no extra fields and so on. The number of different issues for this topic alone (#324, #148, #225, #406, #453 and #134) shows, how counter-intuitive the current behavior is for users. For For a user the current situation means:
When validation would just work for all SQLModel derived classes, also About the boiler-plate in the example with the auto-incremented ID: This is such a common behavior, that I'd wish I wouldn't have to create three separate classes for it. If all the multiple models should really be necessary, it could all happen under the hood, e.g. by offering these different model classes automatically created as It might be argued, that these different classes are "more correct". Yes, maybe. But if defining |
@tiangolo I know it was considered a closed issue and maybe considered finally settled. But as you see above, I don't seem to be the only one rather unhappy with the current status quo. Would you mind to comment on this again and maybe consider one of the suggestions above? |
With v0.0.16 @andremmori workaround is raising from sqlmodel import SQLModel, Field
from pydantic import ConfigDict
# class BaseSQLModel(SQLModel):
# def __init__(self, **kwargs):
# self.__config__.table = False # AttributeError: 'MyTable1' object has no attribute '__config__'
# super().__init__(**kwargs)
# self.__config__.table = True
# class Config:
# validate_assignment = True
class BaseSQLModel(SQLModel):
model_config = ConfigDict(validate_assignment=True)
class MyTable1(BaseSQLModel, table=True):
a: int = Field(primary_key=True)
b: int
class MyTable2(BaseSQLModel, table=True):
a: int = Field(primary_key=True)
b: float
t1 = MyTable1(a=1, b=2) # ok
t2 = MyTable2(b="text") # Raises ValidationError Input should be a valid number, unable to parse string as a number [type=float_parsing, input_value='text', input_type=str] P.S. below solution suggested by @tiangolo doesn't work
|
@armujahid What versions are you using? I'm trying to upgrade to sqlmodel 0.0.16 and sqlalchemy 2.0.28, and neither the old nor your new workaround now works. With your workaround I get:
|
v0.0.16 as mentioned in my comment. |
Thanks a lot for this workaround, I wasn't aware that |
I agree with most of the comment of @ml31415 #52 (comment) I think it would be great to have computed fields in the class I was also very surprised when validation was not being applied. |
class BaseSQLModel(SQLModel):
model_config = ConfigDict(validate_assignment=True)
class Personage(BaseSQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str = Field(max_length=20, min_length=2)
age: PositiveInt @copdips thanks, with this subclass, It works exactly as I expected the library to work. I have tested to modify the lib code like this : It work too ... a new version with this update should be cool if it's ok ! |
Do I understand correctly, that this also makes it impossible to use values transformation on SQLModel class init? Or don't I understand? For example I would like to decrypt column value and make it available as new attribute. # Decrypt config on class init
def __init__(self, **data):
# THIS IS NEVER CALLED
super().__init__(**data)
if self.config:
self._config_obj = self.decrypt_config(self.config)
... |
First Check
Commit to Help
Example Code
Description
The pydantic model raises an ValidationError whereas the SQLModel doesn't raise a ValidationError even though an required argument is not provided. I would expect a ValidationError if not all required arguments is provided.
Operating System
macOS
Operating System Details
No response
SQLModel Version
0.0.4
Python Version
Python 3.8.5
Additional Context
No response
The text was updated successfully, but these errors were encountered: