Skip to content

Commit

Permalink
support django 5.0 (#61)
Browse files Browse the repository at this point in the history
[Django
5.0](https://docs.djangoproject.com/en/5.0/releases/5.0/#database-computed-default-values)
New Feature (db related):
1. Database-computed default values, TiDB has limited support for
default value expressions, see
[docs](https://docs.pingcap.com/tidb/dev/data-type-default-values#specify-expressions-as-default-values).
2. Database generated model field, TiDB also has some
[limitations](https://docs.pingcap.com/tidb/stable/generated-columns).

This PR also dropped support for TiDB 4.x as it has already reached its
[EOL](https://www.pingcap.com/tidb-release-support-policy/).
  • Loading branch information
wd0517 committed Apr 8, 2024
1 parent 137c475 commit 113d1e9
Show file tree
Hide file tree
Showing 13 changed files with 368 additions and 25 deletions.
18 changes: 8 additions & 10 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,25 +30,23 @@ jobs:
fail-fast: false
matrix:
python-version:
- '3.8'
- '3.9'
- '3.10'
- '3.11'
# blocked by https://github.com/lericson/pylibmc/issues/288
# - '3.12'
django-version:
- '4.2.5'
- '5.0.4'
tidb-version:
- 'v7.1.1'
- 'v6.5.3'
- 'v7.5.1'
- 'v7.1.4'
- 'v6.5.8'
- 'v5.4.3'
- 'v4.0.15'
exclude:
# Django introduced the `debug_transaction` feature in version 4.2.x,
# but it does not consider databases that do not support savepoints(TiDB < 6.2.0),
# as a result, all `assertNumQueries` in test cases failed.
# https://github.com/django/django/commit/798e38c2b9c46ab72e2ee8c33dc822f01b194b1e
- django-version: '4.2.5'
tidb-version: 'v4.0.15'
- django-version: '4.2.5'
- django-version: '5.0.4'
tidb-version: 'v5.4.3'

name: py${{ matrix.python-version }}_tidb${{ matrix.tidb-version }}_django${{ matrix.django-version }}
Expand Down Expand Up @@ -88,7 +86,7 @@ jobs:
python-version:
- '3.11'
django-version:
- '4.2.5'
- '5.0.4'

name: vector-py${{ matrix.python-version }}_django${{ matrix.django-version }}
runs-on: ubuntu-latest
Expand Down
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ To install django-tidb, you need to select the version that corresponds with you
|django|django-tidb|install command|
|:----:|:---------:|:-------------:|
|v5.0.x|v5.0.x|`pip install 'django-tidb>=5.0.0,<5.1.0'`|
|v4.2.x|v4.2.x|`pip install 'django-tidb>=4.2.0,<4.3.0'`|
|v4.1.x|v4.1.x|`pip install 'django-tidb>=4.1.0,<4.2.0'`|
|v3.2.x|v3.2.x|`pip install 'django-tidb>=3.2.0,<3.3.0'`|
Expand Down Expand Up @@ -188,8 +189,8 @@ Test.objects.alias(distance=CosineDistance('embedding', [3, 1, 2])).filter(dista

## Supported versions

- TiDB 4.0 and newer
- Django 3.2, 4.1 and 4.2
- TiDB 5.0 and newer
- Django 3.2, 4.1, 4.2 and 5.0
- Python 3.6 and newer(must match Django's Python version requirement)

## Test
Expand All @@ -213,10 +214,10 @@ $ DJANGO_VERSION=3.2.12 python run_testing_worker.py

Releases on PyPi before 3.0.0 are published from repository https://github.com/blacktear23/django_tidb. This repository is a new implementation and released under versions from 3.0.0. No backwards compatibility is ensured. The most significant points are:

- Only Django 3.2 and 4.0 are tested and supported.
- Engine name is `django_tidb` instead of `django_tidb.tidb`.

## Known issues

- TiDB before v6.6.0 does not support FOREIGN KEY constraints([#18209](https://github.com/pingcap/tidb/issues/18209)).
- TiDB before v6.2.0 does not support SAVEPOINT([#6840](https://github.com/pingcap/tidb/issues/6840)).
- TiDB has limited support for default value expressions, please refer to the [documentation](https://docs.pingcap.com/tidb/dev/data-type-default-values#specify-expressions-as-default-values).
2 changes: 1 addition & 1 deletion django_test_apps.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
tidb
tidb_field_defaults
admin_changelist
admin_custom_urls
admin_docs
Expand Down Expand Up @@ -40,7 +41,6 @@ empty
expressions_case
expressions_window
extra_regress
field_defaults
field_subclassing
file_storage
file_uploads
Expand Down
1 change: 1 addition & 0 deletions django_test_suite.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ git clone --depth 1 --branch $DJANGO_VERSION https://github.com/django/django.g
cp tidb_settings.py $DJANGO_TESTS_DIR/django/tidb_settings.py
cp tidb_settings.py $DJANGO_TESTS_DIR/django/tests/tidb_settings.py
cp -r ./tests/tidb/ $DJANGO_TESTS_DIR/django/tests/tidb/
cp -r ./tests/tidb_field_defaults/ $DJANGO_TESTS_DIR/django/tests/tidb_field_defaults/

cd $DJANGO_TESTS_DIR/django && pip3 install -e . && pip3 install -r tests/requirements/py3.txt && pip3 install -r tests/requirements/mysql.txt; cd ../../
cd $DJANGO_TESTS_DIR/django/tests
Expand Down
2 changes: 1 addition & 1 deletion django_tidb/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

from .patch import monkey_patch

__version__ = "4.2.3"
__version__ = "5.0.0"


monkey_patch()
32 changes: 31 additions & 1 deletion django_tidb/features.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@

class DatabaseFeatures(MysqlDatabaseFeatures):
has_select_for_update = True
has_native_uuid_field = False
atomic_transactions = False
supports_atomic_references_rename = False
can_clone_databases = False
Expand All @@ -34,9 +35,10 @@ class DatabaseFeatures(MysqlDatabaseFeatures):
test_collations = {
"ci": "utf8mb4_general_ci",
"non_default": "utf8mb4_bin",
"virtual": "utf8mb4_general_ci",
}

minimum_database_version = (4,)
minimum_database_version = (5,)

@cached_property
def supports_foreign_keys(self):
Expand Down Expand Up @@ -75,6 +77,8 @@ def django_test_skips(self):
"This doesn't work on MySQL.": {
"db_functions.comparison.test_greatest.GreatestTests.test_coalesce_workaround",
"db_functions.comparison.test_least.LeastTests.test_coalesce_workaround",
# UPDATE ... ORDER BY syntax on MySQL/MariaDB does not support ordering by related fields
"update.tests.AdvancedTests.test_update_ordered_by_m2m_annotation_desc",
},
"MySQL doesn't support functional indexes on a function that "
"returns JSON": {
Expand Down Expand Up @@ -135,6 +139,7 @@ def django_test_skips(self):
# IntegrityError not raised
"constraints.tests.CheckConstraintTests.test_database_constraint",
"constraints.tests.CheckConstraintTests.test_database_constraint_unicode",
# Result of function ROUND(x, d) is different from MySQL
# https://github.com/pingcap/tidb/issues/26993
"db_functions.math.test_round.RoundTests.test_integer_with_negative_precision",
"db_functions.text.test_chr.ChrTests.test_transform",
Expand All @@ -160,8 +165,27 @@ def django_test_skips(self):
"migrations.test_operations.OperationTests.test_alter_field_pk_mti_and_fk_to_base",
"migrations.test_operations.OperationTests.test_alter_field_pk_mti_fk",
"migrations.test_operations.OperationTests.test_create_model_with_boolean_expression_in_check_constraint",
# Unsupported adding a stored generated column through ALTER TABLE
"migrations.test_operations.OperationTests.test_add_field_after_generated_field",
"migrations.test_operations.OperationTests.test_add_generated_field_stored",
"migrations.test_operations.OperationTests.test_invalid_generated_field_changes_stored",
"migrations.test_operations.OperationTests.test_invalid_generated_field_persistency_change",
"migrations.test_operations.OperationTests.test_remove_generated_field_stored",
"schema.tests.SchemaTests.test_add_generated_field_contains",
# Failed to modify column's default value when has expression index
# https://github.com/pingcap/tidb/issues/52355
"migrations.test_operations.OperationTests.test_alter_field_with_func_index",
# TiDB has limited support for default value expressions
# https://docs.pingcap.com/tidb/dev/data-type-default-values#specify-expressions-as-default-values
"migrations.test_operations.OperationTests.test_add_field_database_default_function",
"schema.tests.SchemaTests.test_add_text_field_with_db_default",
"schema.tests.SchemaTests.test_db_default_equivalent_sql_noop",
"schema.tests.SchemaTests.test_db_default_output_field_resolving",
# about Pessimistic/Optimistic Transaction Model
"select_for_update.tests.SelectForUpdateTests.test_raw_lock_not_available",
# Wrong referenced_table_schema in information_schema.key_column_usage
# https://github.com/pingcap/tidb/issues/52350
"backends.mysql.test_introspection.TestCrossDatabaseRelations.test_omit_cross_database_relations",
},
}
if self.connection.tidb_version < (5,):
Expand Down Expand Up @@ -269,6 +293,12 @@ def supports_over_clause(self):
def supports_column_check_constraints(self):
return True

@cached_property
def supports_expression_defaults(self):
# TiDB has limited support for default value expressions
# https://docs.pingcap.com/tidb/dev/data-type-default-values#specify-expressions-as-default-values
return True

supports_table_check_constraints = property(
operator.attrgetter("supports_column_check_constraints")
)
Expand Down
4 changes: 3 additions & 1 deletion django_tidb/introspection.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@

FieldInfo = namedtuple(
"FieldInfo",
BaseFieldInfo._fields + ("extra", "is_unsigned", "has_json_constraint", "comment"),
BaseFieldInfo._fields
+ ("extra", "is_unsigned", "has_json_constraint", "comment", "data_type"),
)
InfoLine = namedtuple(
"InfoLine",
Expand Down Expand Up @@ -115,6 +116,7 @@ def to_int(i):
info.is_unsigned,
line[0] in json_constraints,
info.comment,
info.data_type,
)
)
return fields
Expand Down
9 changes: 4 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,18 @@ authors = [
]
description = "Django backend for TiDB"
readme = "README.md"
requires-python = ">=3.8"
requires-python = ">=3.10"
classifiers = [
"Development Status :: 5 - Production/Stable",
"Framework :: Django",
"Framework :: Django :: 4.2",
"Framework :: Django :: 5.0",
"License :: OSI Approved :: Apache Software License",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11"
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12"
]
dynamic = ["version"]

Expand Down
3 changes: 3 additions & 0 deletions tests/tidb_field_defaults/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# About

This test is copied from the Django [field_defaults](https://github.com/django/django/tree/main/tests/field_defaults), as TiDB has some [limitations](https://docs.pingcap.com/tidb/dev/data-type-default-values#specify-expressions-as-default-values) on the default expression of the field, it does not support such many expressions as MySQL.
Empty file.
75 changes: 75 additions & 0 deletions tests/tidb_field_defaults/models.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
"""
Callable defaults
You can pass callable objects as the ``default`` parameter to a field. When
the object is created without an explicit value passed in, Django will call
the method to determine the default value.
This example uses ``datetime.datetime.now`` as the default for the ``pub_date``
field.
"""

from datetime import datetime
from decimal import Decimal

from django.db import models
from django.db.models.functions import Random, Now


class Article(models.Model):
headline = models.CharField(max_length=100, default="Default headline")
pub_date = models.DateTimeField(default=datetime.now)

def __str__(self):
return self.headline


class DBArticle(models.Model):
"""
Values or expressions can be passed as the db_default parameter to a field.
When the object is created without an explicit value passed in, the
database will insert the default value automatically.
"""

headline = models.CharField(max_length=100, db_default="Default headline")
pub_date = models.DateTimeField(db_default=Now())
cost = models.DecimalField(
max_digits=3, decimal_places=2, db_default=Decimal("3.33")
)

class Meta:
required_db_features = {"supports_expression_defaults"}


class DBDefaults(models.Model):
both = models.IntegerField(default=1, db_default=2)
null = models.FloatField(null=True, db_default=1.1)


# This model has too many db_default expressions that TiDB does not support
# class DBDefaultsFunction(models.Model):
# number = models.FloatField(db_default=Pi())
# year = models.IntegerField(db_default=ExtractYear(Now()))
# added = models.FloatField(db_default=Pi() + 4.5)
# multiple_subfunctions = models.FloatField(db_default=Coalesce(4.5, Pi()))
# case_when = models.IntegerField(
# db_default=models.Case(models.When(GreaterThan(2, 1), then=3), default=4)
# )

# class Meta:
# required_db_features = {"supports_expression_defaults"}


class TiDBDefaultsFunction(models.Model):
number = models.DecimalField(max_digits=3, decimal_places=2, db_default=Random())
created_at = models.DateTimeField(db_default=Now())


class DBDefaultsPK(models.Model):
language_code = models.CharField(primary_key=True, max_length=2, db_default="en")


class DBDefaultsFK(models.Model):
language_code = models.ForeignKey(
DBDefaultsPK, db_default="fr", on_delete=models.CASCADE
)
Loading

0 comments on commit 113d1e9

Please sign in to comment.