Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed #373 -- Added CompositeField-based Meta.primary_key. #18031

Closed
Closed
Show file tree
Hide file tree
Changes from 57 commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
cc9293f
Add PrimaryKeyConstraint
csirmazbendeguz Mar 11, 2024
1efbcda
Add composite foreign keys
csirmazbendeguz Mar 14, 2024
ad708a0
FIX
csirmazbendeguz Mar 19, 2024
9b206a8
Merge branch 'main' into ticket_373_pk_meta
csirmazbendeguz Mar 23, 2024
fdd944d
FIXES
csirmazbendeguz Mar 23, 2024
11d30e7
FIX
csirmazbendeguz Mar 23, 2024
4335f3a
FIXES
csirmazbendeguz Mar 23, 2024
1916bbc
FIXES
csirmazbendeguz Mar 23, 2024
392249a
FIX
csirmazbendeguz Mar 23, 2024
f9e866e
FIX
csirmazbendeguz Mar 23, 2024
6387cac
FIX
csirmazbendeguz Mar 23, 2024
ff2bdfc
FIX
csirmazbendeguz Mar 23, 2024
66484e1
FIX
csirmazbendeguz Mar 23, 2024
2e7bff3
FIX
csirmazbendeguz Mar 23, 2024
d44c80f
FIX
csirmazbendeguz Mar 23, 2024
811a072
Move tests
csirmazbendeguz Mar 23, 2024
2b1349b
Add test
csirmazbendeguz Mar 23, 2024
fb8aa35
Add CompositeField-based Meta.primary_key
csirmazbendeguz Mar 27, 2024
1ebec79
Remove prints
csirmazbendeguz Mar 27, 2024
25c311e
Fix
csirmazbendeguz Mar 27, 2024
4f17cad
Adding tests
csirmazbendeguz Mar 27, 2024
8359ddc
Add tests
csirmazbendeguz Mar 28, 2024
d22a791
Add tests
csirmazbendeguz Mar 28, 2024
f495ad7
Add test
csirmazbendeguz Mar 28, 2024
de9dbe6
Add doc
csirmazbendeguz Mar 28, 2024
bbc4bba
Add test
csirmazbendeguz Mar 28, 2024
c8d0958
Fix
csirmazbendeguz Mar 28, 2024
0ad6949
Add tests
csirmazbendeguz Mar 28, 2024
a47a5e0
Add tests
csirmazbendeguz Mar 28, 2024
5d83f76
Fixes
csirmazbendeguz Mar 28, 2024
0f014dc
Add test case
csirmazbendeguz Mar 28, 2024
bdae908
Add fix
csirmazbendeguz Mar 28, 2024
985d275
Fixes
csirmazbendeguz Mar 28, 2024
5b8b069
Fixes
csirmazbendeguz Mar 28, 2024
8065666
Fix
csirmazbendeguz Mar 28, 2024
54acc3b
Add tests
csirmazbendeguz Mar 28, 2024
e883f25
isort
csirmazbendeguz Mar 28, 2024
310d604
Fix
csirmazbendeguz Mar 28, 2024
db146f6
Reverting
csirmazbendeguz Mar 28, 2024
1e3bccc
Fix
csirmazbendeguz Mar 28, 2024
3491fc7
Fix imports
csirmazbendeguz Mar 28, 2024
9d9454b
Fix
csirmazbendeguz Mar 28, 2024
fc7812c
Fix
csirmazbendeguz Mar 28, 2024
299ee09
Fix formattin
csirmazbendeguz Mar 28, 2024
076267f
Fix
csirmazbendeguz Mar 28, 2024
7a5e80f
Fix imports
csirmazbendeguz Mar 28, 2024
01968ec
Fix
csirmazbendeguz Mar 28, 2024
cc8d4cc
Fix test
csirmazbendeguz Mar 28, 2024
e503c99
Add test
csirmazbendeguz Mar 28, 2024
1f0bcb3
Fix
csirmazbendeguz Mar 28, 2024
a5eec5b
Fix imports
csirmazbendeguz Mar 28, 2024
5481fb6
Fix
csirmazbendeguz Mar 28, 2024
3dc1b21
Add
csirmazbendeguz Mar 28, 2024
bbfd2c8
Add test
csirmazbendeguz Mar 28, 2024
49466df
Add bulk updates
csirmazbendeguz Mar 28, 2024
8a99d05
Fix
csirmazbendeguz Mar 28, 2024
9467349
Refactor
csirmazbendeguz Mar 28, 2024
f3d3eac
Fix tests
csirmazbendeguz Mar 29, 2024
656df05
Fix tests
csirmazbendeguz Mar 29, 2024
3d0f576
Add test
csirmazbendeguz Mar 29, 2024
65df6ef
Split tests, add order by
csirmazbendeguz Mar 29, 2024
38bb96e
Add test
csirmazbendeguz Mar 29, 2024
8ebeb64
Refactor tests
csirmazbendeguz Mar 29, 2024
b134640
Add .only() support
csirmazbendeguz Mar 29, 2024
c144611
Fix import
csirmazbendeguz Mar 29, 2024
eb205b4
Refactor
csirmazbendeguz Mar 29, 2024
632fd17
Add test for get_or_create()
csirmazbendeguz Mar 29, 2024
4e21293
Fix
csirmazbendeguz Mar 29, 2024
3556851
Add test for update-or_create()
csirmazbendeguz Mar 29, 2024
b33dd6b
Add test
csirmazbendeguz Mar 29, 2024
ef6a288
Add in_bulk test
csirmazbendeguz Mar 29, 2024
8809ffe
Add lookup errors
csirmazbendeguz Mar 29, 2024
154411c
Add tests
csirmazbendeguz Mar 29, 2024
df5c7c1
Add more asserts
csirmazbendeguz Mar 29, 2024
3d71c8e
Add more tests
csirmazbendeguz Mar 29, 2024
190bb79
Add exclude tests
csirmazbendeguz Mar 29, 2024
943cc5b
Add fix
csirmazbendeguz Mar 29, 2024
56f9b44
Add test for contains()
csirmazbendeguz Mar 29, 2024
de4c434
Rename auto field
csirmazbendeguz Mar 29, 2024
0ffeaf3
Simplify
csirmazbendeguz Mar 29, 2024
61f6984
Fix
csirmazbendeguz Mar 29, 2024
d08ecb3
Delete line
csirmazbendeguz Mar 29, 2024
6f92808
Refactor
csirmazbendeguz Mar 29, 2024
fb6acc1
Fix
csirmazbendeguz Mar 29, 2024
78a7ce9
Simplify
csirmazbendeguz Mar 29, 2024
1c84d1a
Make CompositeField private
csirmazbendeguz Mar 30, 2024
6783f9e
Fix
csirmazbendeguz Mar 30, 2024
ef014a6
Fix
csirmazbendeguz Mar 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
13 changes: 13 additions & 0 deletions django/db/backends/base/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ class BaseDatabaseSchemaEditor:
sql_check_constraint = "CHECK (%(check)s)"
sql_delete_constraint = "ALTER TABLE %(table)s DROP CONSTRAINT %(name)s"
sql_constraint = "CONSTRAINT %(name)s %(constraint)s"
sql_pk_constraint = "PRIMARY KEY (%(columns)s)"

sql_create_check = "ALTER TABLE %(table)s ADD CONSTRAINT %(name)s CHECK (%(check)s)"
sql_delete_check = sql_delete_constraint
Expand Down Expand Up @@ -268,6 +269,13 @@ def table_sql(self, model):
constraint.constraint_sql(model, self)
for constraint in model._meta.constraints
]

# If the model defines Meta.primary_key, add the primary key constraint
# to the table definition.
# It's expected primary_key=True isn't set on any fields (see E042).
if model._meta.primary_key:
constraints.append(self._pk_constraint_sql(model._meta.primary_key))

sql = self.sql_create_table % {
"table": self.quote_name(model._meta.db_table),
"definition": ", ".join(
Expand Down Expand Up @@ -1962,6 +1970,11 @@ def _constraint_names(
result.append(name)
return result

def _pk_constraint_sql(self, fields):
return self.sql_pk_constraint % {
"columns": ", ".join(self.quote_name(field) for field in fields)
}

def _delete_primary_key(self, model, strict=False):
constraint_names = self._constraint_names(model, primary_key=True)
if strict and len(constraint_names) != 1:
Expand Down
31 changes: 29 additions & 2 deletions django/db/models/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
pre_init,
pre_save,
)
from django.db.models.utils import AltersData, make_model_tuple
from django.db.models.utils import AltersData, is_pk_set, make_model_tuple
from django.utils.deprecation import RemovedInDjango60Warning
from django.utils.encoding import force_str
from django.utils.hashable import make_hashable
Expand Down Expand Up @@ -1076,7 +1076,7 @@ def _save_table(
if pk_val is None:
pk_val = meta.pk.get_pk_value_on_save(self)
setattr(self, meta.pk.attname, pk_val)
pk_set = pk_val is not None
pk_set = is_pk_set(pk_val)
if not pk_set and (force_update or update_fields):
raise ValueError("Cannot force an update in save() with no primary key.")
updated = False
Expand Down Expand Up @@ -1682,6 +1682,7 @@ def check(cls, **kwargs):
*cls._check_constraints(databases),
*cls._check_default_pk(),
*cls._check_db_table_comment(databases),
*cls._check_composite_pk(),
]

return errors
Expand All @@ -1690,6 +1691,9 @@ def check(cls, **kwargs):
def _check_default_pk(cls):
if (
not cls._meta.abstract
# If the model defines Meta.primary_key, the check should be skipped,
# since there's no default primary key.
and not cls._meta.primary_key
and cls._meta.pk.auto_created
and
# Inherited PKs are checked in parents models.
Expand Down Expand Up @@ -1718,6 +1722,24 @@ def _check_default_pk(cls):
]
return []

@classmethod
def _check_composite_pk(cls):
errors = []

if cls._meta.primary_key and any(
field for field in cls._meta.fields if field.primary_key
):
errors.append(
checks.Error(
"primary_key=True must not be set if Meta.primary_key "
"is defined.",
obj=cls,
id="models.E042",
)
)

return errors

@classmethod
def _check_db_table_comment(cls, databases):
if not cls._meta.db_table_comment:
Expand Down Expand Up @@ -1838,6 +1860,11 @@ def _check_m2m_through_same_relationship(cls):
@classmethod
def _check_id_field(cls):
"""Check if `id` field is a primary key."""
# If the model defines Meta.primary_key, the check should be skipped,
# since primary_key=True can't be set on any fields (including `id`).
if cls._meta.primary_key:
return []

fields = [
f for f in cls._meta.local_fields if f.name == "id" and f != cls._meta.pk
]
Expand Down
9 changes: 8 additions & 1 deletion django/db/models/deletion.py
Original file line number Diff line number Diff line change
Expand Up @@ -514,5 +514,12 @@ def delete(self):

for model, instances in self.data.items():
for instance in instances:
setattr(instance, model._meta.pk.attname, None)
try:
fields = iter(model._meta.pk)
except TypeError:
fields = (model._meta.pk,)

for field in fields:
setattr(instance, field.attname, None)

return sum(deleted_counter.values()), dict(deleted_counter)
7 changes: 6 additions & 1 deletion django/db/models/fields/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2794,6 +2794,11 @@ def check(self, **kwargs):
]

def _check_primary_key(self):
# If the model defines Meta.primary_key, primary_key=True can't be set on
# any field (including AutoFields).
if self.model._meta.primary_key:
return []

if not self.primary_key:
return [
checks.Error(
Expand All @@ -2808,7 +2813,7 @@ def _check_primary_key(self):
def deconstruct(self):
name, path, args, kwargs = super().deconstruct()
del kwargs["blank"]
kwargs["primary_key"] = True
kwargs["primary_key"] = self.primary_key
return name, path, args, kwargs

def validate(self, value, model_instance):
Expand Down
81 changes: 81 additions & 0 deletions django/db/models/fields/composite.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
from django.db.models import Field
from django.db.models.expressions import Col, Expression
from django.db.models.lookups import TupleExact, TupleIn
from django.db.models.signals import class_prepared
from django.utils.functional import cached_property


class Cols(Expression):
def __init__(self, alias, targets, output_field):
super().__init__(output_field=output_field)
self.alias, self.targets = alias, targets

def get_source_expressions(self):
return [Col(self.alias, target) for target in self.targets]

def set_source_expressions(self, exprs):
assert all(isinstance(expr, Col) for expr in exprs)
assert len(exprs) == len(self.targets)

def __iter__(self):
return iter(self.get_source_expressions())


class CompositeAttribute:
def __init__(self, field):
self.field = field

def __get__(self, instance, cls=None):
return tuple(
getattr(instance, field_name) for field_name in self.field.field_names
)

def __set__(self, instance, values):
for field_name, value in zip(self.field.field_names, values):
setattr(instance, field_name, value)


class CompositeField(Field):
descriptor_class = CompositeAttribute

def __init__(self, *args, **kwargs):
kwargs["db_column"] = None
kwargs["editable"] = False
super().__init__(**kwargs)
self.field_names = args
self.fields = None

def contribute_to_class(self, cls, name, private_only=False):
super().contribute_to_class(cls, name, private_only)
cls._meta.pk = self
setattr(cls, self.attname, self.descriptor_class(self))

def get_attname_column(self):
return self.get_attname(), self.db_column

def __iter__(self):
return iter(self.fields)

@cached_property
def cached_col(self):
return Cols(self.model._meta.db_table, self.fields, self)

def get_lookup(self, lookup_name):
if lookup_name == "exact":
return TupleExact
elif lookup_name == "in":
return TupleIn

return super().get_lookup(lookup_name)


def resolve_columns(*args, **kwargs):
cls = kwargs.pop("sender")
for field in cls._meta.local_fields:
if isinstance(field, CompositeField) and field.fields is None:
field.fields = tuple(
cls._meta.get_field(name) for name in field.field_names
)


class_prepared.connect(resolve_columns)
10 changes: 10 additions & 0 deletions django/db/models/fields/related.py
Original file line number Diff line number Diff line change
Expand Up @@ -615,6 +615,16 @@ def _check_unique_target(self):
if not self.foreign_related_fields:
return []

# If a model defines Meta.primary_key and a foreign key refers to it,
# the check should be skipped (since primary keys are unique).
pk = self.remote_field.model._meta.primary_key
if pk:
pk = set(pk)
if pk == {f.attname for f in self.foreign_related_fields}:
return []
elif pk == {f.name for f in self.foreign_related_fields}:
return []

has_unique_constraint = any(
rel_field.unique for rel_field in self.foreign_related_fields
)
Expand Down
45 changes: 45 additions & 0 deletions django/db/models/lookups.py
Original file line number Diff line number Diff line change
Expand Up @@ -385,6 +385,18 @@ def as_sql(self, compiler, connection):
return super().as_sql(compiler, connection)


class TupleExact(Exact):
def as_sql(self, compiler, connection):
from django.db.models.sql.where import AND, WhereNode

exprs = [
Exact(lhs, rhs)
for lhs, rhs in zip(self.lhs.get_source_expressions(), self.rhs)
]

return compiler.compile(WhereNode(exprs, connector=AND))


@Field.register_lookup
class IExact(BuiltinLookup):
lookup_name = "iexact"
Expand Down Expand Up @@ -558,6 +570,39 @@ def split_parameter_list_as_sql(self, compiler, connection):
return "".join(in_clause_elements), params


class TupleIn(In):
def get_prep_lookup(self):
try:
lhs = list(self.lhs)
except TypeError:
raise ValueError(f"lhs={self.lhs} is not iterable")
try:
rhs = list(self.rhs)
except TypeError:
raise ValueError(f"rhs={self.rhs} is not iterable")

if not all(len(values) == len(lhs) for values in rhs):
raise ValueError(f"rhs={rhs} must match lhs={lhs}")

return super().get_prep_lookup()

def as_sql(self, compiler, connection):
from django.db.models.sql.where import AND, OR, WhereNode

exprs = []
lhs = self.lhs.get_source_expressions()

for rhs in self.rhs:
exprs.append(
WhereNode(
[Exact(col, value) for col, value in zip(lhs, rhs)],
connector=AND,
)
)

return compiler.compile(WhereNode(exprs, connector=OR))


class PatternLookup(BuiltinLookup):
param_pattern = "%%%s%%"
prepare_rhs = False
Expand Down
8 changes: 7 additions & 1 deletion django/db/models/options.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from django.core.exceptions import FieldDoesNotExist, ImproperlyConfigured
from django.db import connections
from django.db.models import AutoField, Manager, OrderWrt, UniqueConstraint
from django.db.models.fields.composite import CompositeField
from django.db.models.query_utils import PathInfo
from django.utils.datastructures import ImmutableList, OrderedSet
from django.utils.functional import cached_property
Expand All @@ -24,6 +25,7 @@
)

DEFAULT_NAMES = (
"primary_key",
"verbose_name",
"verbose_name_plural",
"db_table",
Expand Down Expand Up @@ -106,6 +108,7 @@ def __init__(self, meta, app_label=None):
self.base_manager_name = None
self.default_manager_name = None
self.model_name = None
self.primary_key = None
self.verbose_name = None
self.verbose_name_plural = None
self.db_table = ""
Expand Down Expand Up @@ -296,7 +299,10 @@ def _prepare(self, model):
self.order_with_respect_to = None

if self.pk is None:
if self.parents:
if self.primary_key:
pk = CompositeField(*self.primary_key)
model.add_to_class("composite_pk", pk)
elif self.parents:
# Promote the first parent link in lieu of adding yet another
# field.
field = next(iter(self.parents.values()))
Expand Down
5 changes: 4 additions & 1 deletion django/db/models/query.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
from django.db.models.utils import (
AltersData,
create_namedtuple_class,
is_pk_set,
resolve_callables,
)
from django.utils import timezone
Expand Down Expand Up @@ -813,7 +814,9 @@ def bulk_create(
objs = list(objs)
self._prepare_for_bulk_create(objs)
with transaction.atomic(using=self.db, savepoint=False):
objs_with_pk, objs_without_pk = partition(lambda o: o.pk is None, objs)
objs_with_pk, objs_without_pk = partition(
lambda o: not is_pk_set(o.pk), objs
)
if objs_with_pk:
returned_columns = self._batched_insert(
objs_with_pk,
Expand Down
20 changes: 20 additions & 0 deletions django/db/models/utils.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import functools
from collections import namedtuple
from collections.abc import Iterable


def make_model_tuple(model):
Expand Down Expand Up @@ -67,3 +68,22 @@ def __init_subclass__(cls, **kwargs):
break

super().__init_subclass__(**kwargs)


def is_pk_set(pk):
"""
>>> is_pk_set(1)
True
>>> is_pk_set(None)
False
>>> is_pk_set((1, 1))
True
>>> is_pk_set((1, None))
False
"""
if pk is None:
return False
if isinstance(pk, Iterable):
return not any(value is None for value in pk)

return True
2 changes: 2 additions & 0 deletions docs/ref/checks.txt
Original file line number Diff line number Diff line change
Expand Up @@ -416,6 +416,8 @@ Models
* **models.W040**: ``<database>`` does not support indexes with non-key
columns.
* **models.E041**: ``constraints`` refers to the joined field ``<field name>``.
* **models.E042**: primary_key=True must not be set if Meta.primary_key
is defined.
* **models.W042**: Auto-created primary key used when not defining a primary
key type, by default ``django.db.models.AutoField``.
* **models.W043**: ``<database>`` does not support indexes on expressions.
Expand Down
Empty file added tests/composite_pk/__init__.py
Empty file.