support dataclass defaults that don't affect subsequent ORM operations

[CaselIT]

The use case for this is mainly dataclass defaults, so that we can use mapped_column(default=<new-constrant-here>) and that's equivalent to not setting the field at all.

This should also work for relationships:

Some considerations from other discussions:

i think one area that we have a choice to make is if you have an object with state and you say, myobject.foo = NO_VALUE - does that expire the attribute or just do nothing? I vote for the latter
#12165 (reply in thread)

I guess the behaviour wanted is

thing = session.get(Thing, 1)
assert thing.foo == 'bar'
thing.foo = NO_VALUE
assert thing.foo == 'bar'

and if foo is expired it stays expired, so

thing = session.get(Thing, 1)
session.expire(thing, 'foo')
assert 'foo' in sa.inspect(x).expired_attributes
thing.foo = NO_VALUE
assert 'foo' in sa.inspect(x).expired_attributes

In the above example either behaviour is fine.
Not sure if before assert thing.foo == 'bar' in the first example 'foo' should be in the expired_attributes or not. I trust your insight here if you prefer for not.
#12165 (reply in thread)

Connected discussions:

• MappedAsDataclass uses standard values ​​as explicit... why? #12165
• SQLAlchemy2.0 + mapped_as_dataclass - how do I init only foreign key or link property? #9383
• How to directly set the value of a ForeignKey in a Dataclass mixin #9856

Target is 2.0.x but may be moved to 2.1 depending on the changes

[CaselIT]

I prefer UNSET compared with NO_VALUE as a word, but overall they mean probably the same thing

[CaselIT]

myobject.foo = NO_VALUE - does that expire the attribute or just do nothing

Another case:

thing = session.get(Thing, 1)
assert thing.foo == 'bar'
thing.foo = 'new-value'
thing.foo = NO_VALUE
# assert thing.foo == 'bar' # this would again pass
session.commit()  # nothing to do here. No update is done, new-value is lost

Here I think the correct behaviour would be to ignore the set of new-value, so I would go for expire, since I'm not sure if it can be done without expiring the field. If we can do that that I'm back ad no preference

[zzzeek]

as mentioned elsewhere there's not a great way to type this

if you say

field: Mapped[str | None] = mapped_column(default=DONTSET)

That's going to fail typing, would have to be like

field: Mapped[str | None | DontSetType] = mapped_column(default=DONTSET)

unless if we typed DONTSET as None .....

[zzzeek]

so that typing looks like:

if TYPE_CHECKING:
    DONTSET = None
else:
    DONTSET = object()

class UserMappedModel(MappedAsDataclass, Base):
    __tablename__ = "user"

    id: Mapped[int] = mapped_column(primary_key=True)
    name: Mapped[str]
    status: Mapped[str | None] = mapped_column(default=DONTSET)

we also have a way to customize that default "scalar get" value which is little used : https://docs.sqlalchemy.org/en/20/orm/events.html#sqlalchemy.orm.AttributeEvents.init_scalar

to use that, someone would need to make another DONTSET of their own, like:

if TYPE_CHECKING:
    DONTSET_MY_DEFAULT = "default"
else:
    DONTSET_MY_DEFAULT = DONTSET

class UserMappedModel(MappedAsDataclass, Base):
    __tablename__ = "user"

    id: Mapped[int] = mapped_column(primary_key=True)
    name: Mapped[str]
    status: Mapped[str] = mapped_column(default=DONTSET_MY_DEFAULT)

[zzzeek]

maybe we have a bunch of them canned for typical defaults

if TYPE_CHECKING:
    DONTSET_NONE = None
    DONTSET_STR = "string"
    DONTSET_INT = 0
    # ... etc

else:
    _DONTSET = object()
    DONTSET_NONE = _DONTSET
    DONTSET_STR = _DONTSET
    DONTSET_INT = _DONTSET
    # ...

[zzzeek]

this does the basic thing

diff --git a/lib/sqlalchemy/orm/attributes.py b/lib/sqlalchemy/orm/attributes.py
index d65597238..9c47d9553 100644
--- a/lib/sqlalchemy/orm/attributes.py
+++ b/lib/sqlalchemy/orm/attributes.py
@@ -45,6 +45,7 @@ from .base import ATTR_EMPTY
 from .base import ATTR_WAS_SET
 from .base import CALLABLES_OK
 from .base import DEFERRED_HISTORY_LOAD
+from .base import DONT_SET
 from .base import INCLUDE_PENDING_MUTATIONS  # noqa
 from .base import INIT_OK
 from .base import instance_dict as instance_dict
@@ -1268,6 +1269,9 @@ class _ScalarAttributeImpl(_AttributeImpl):
         check_old: Optional[object] = None,
         pop: bool = False,
     ) -> None:
+        if value is DONT_SET:
+            return
+
         if self.dispatch._active_history:
             old = self.get(state, dict_, PASSIVE_RETURN_NO_VALUE)
         else:
@@ -1434,6 +1438,9 @@ class _ScalarObjectAttributeImpl(_ScalarAttributeImpl):
     ) -> None:
         """Set a value on the given InstanceState."""
 
+        if value is DONT_SET:
+            return
+
         if self.dispatch._active_history:
             old = self.get(
                 state,
diff --git a/lib/sqlalchemy/orm/base.py b/lib/sqlalchemy/orm/base.py
index b5f7dbbaf..948ed0bed 100644
--- a/lib/sqlalchemy/orm/base.py
+++ b/lib/sqlalchemy/orm/base.py
@@ -97,6 +97,7 @@ class LoaderCallableStatus(Enum):
 
     """
 
+    DONT_SET = 5
 
 (
     PASSIVE_NO_RESULT,
@@ -104,6 +105,7 @@ class LoaderCallableStatus(Enum):
     ATTR_WAS_SET,
     ATTR_EMPTY,
     NO_VALUE,
+    DONT_SET,
 ) = tuple(LoaderCallableStatus)
 
 NEVER_SET = NO_VALUE

demo

from __future__ import annotations

from sqlalchemy import create_engine
from sqlalchemy import ForeignKey
from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.orm import Mapped
from sqlalchemy.orm import mapped_column
from sqlalchemy.orm import relationship
from sqlalchemy.orm import Session, MappedAsDataclass
from sqlalchemy.orm.attributes import DONT_SET
from typing import TYPE_CHECKING

class Base(DeclarativeBase):
    pass

if TYPE_CHECKING:
    DONTSET_NONE = None

else:
    DONTSET_NONE = DONT_SET

class UserMappedModel(MappedAsDataclass, Base):
    __tablename__ = "user"

    id: Mapped[int] = mapped_column(primary_key=True)
    name: Mapped[str]
    status: Mapped[str | None] = mapped_column(default=DONTSET_NONE)

e = create_engine("sqlite://")
Base.metadata.create_all(e)

session = Session(e)

user = UserMappedModel(id=1, name="Jon", status="active")
session.merge(user)
session.commit()


user = UserMappedModel(id=1, name="John")
assert user.status is None
new_user = session.merge(user)
session.commit()

print(new_user)

there can be the possibility of doing more with the init_scalar event and having this None be more customizable

[CaselIT]

as mentioned elsewhere there's not a great way to type this

We can just type this as Any. That eliminates any typing issue

[zzzeek]

@n-leonovets I can see doing the above pattern, integrating init_scalar, and having the two flags such as:

class UserMappedModel(MappedAsDataclass, Base):
    __tablename__ = "user"

    id: Mapped[int] = mapped_column(primary_key=True)
    name: Mapped[str]
    status: Mapped[str | None] = mapped_column(default=None, stealth_default=True)

what it means is that making a UserMappedModel will not actually populate None into __dict__ if not passed, that is, it still is using DONT_SET above, you just dont see it.

stealth_default could set up an event to support other kinds of defaults too other than None, this is what init_scalar supports

here is that

from __future__ import annotations

from sqlalchemy import create_engine
from sqlalchemy import event
from sqlalchemy import ForeignKey
from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.orm import Mapped
from sqlalchemy.orm import mapped_column
from sqlalchemy.orm import relationship
from sqlalchemy.orm import Session, MappedAsDataclass
from sqlalchemy.orm.attributes import DONT_SET
from typing import TYPE_CHECKING

class Base(DeclarativeBase):
    pass


class UserMappedModel(MappedAsDataclass, Base):
    __tablename__ = "user"

    id: Mapped[int] = mapped_column(primary_key=True)
    name: Mapped[str]

    if TYPE_CHECKING:
        # what the API would be
        status: Mapped[str] = mapped_column(default="default_status", stealth_default=True)
    else:
        # what actually happens, part I
        status: Mapped[str] = mapped_column(default=DONT_SET)

# what actually happens, part II
@event.listens_for(UserMappedModel.status, "init_scalar", retval=True)
def receive_init_scalar(target, value, dict_):
    return "default_status"

e = create_engine("sqlite://")
Base.metadata.create_all(e)

session = Session(e)

user = UserMappedModel(id=1, name="Jon", status="active")
session.merge(user)
session.commit()


user = UserMappedModel(id=1, name="John")
assert user.status == "default_status"

new_user = session.merge(user)
session.commit()

print(new_user)

[CaselIT]

I'm -1 on doing that under the hood, but that new argument may make sense to add. But I think it's better to just pass to it the value while keeping default as is.

So something like mapped_column(default=UNSET, new_argument="default_status")