Skip to content

Commit

Permalink
feat: add CLICKHOUSE_ENABLE_UPDATE_ROWCOUNT django setting
Browse files Browse the repository at this point in the history
  • Loading branch information
jayvynl committed Jan 4, 2024
1 parent 4b066d4 commit a89eebf
Show file tree
Hide file tree
Showing 7 changed files with 54 additions and 2 deletions.
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
### 1.1.6

- add `CLICKHOUSE_ENABLE_UPDATE_ROWCOUNT` django setting.

### 1.1.5
- refactor: refactor uniq aggregate function.
- feat: add some ClickHouse tuple and hash functions.
Expand Down
2 changes: 1 addition & 1 deletion clickhouse_backend/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from clickhouse_backend.utils.version import get_version

VERSION = (1, 1, 5, "final", 0)
VERSION = (1, 1, 6, "final", 0)

__version__ = get_version(VERSION)
5 changes: 4 additions & 1 deletion clickhouse_backend/driver/connection.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
from clickhouse_driver.dbapi import connection as dbapi_connection
from clickhouse_driver.dbapi import cursor, errors
from clickhouse_pool.pool import ChPoolError
from django.conf import settings

from .escape import escape_params
from .pool import ClickhousePool
Expand Down Expand Up @@ -93,7 +94,9 @@ def __del__(self):

def execute(self, operation, parameters=None):
"""fix https://github.com/jayvynl/django-clickhouse-backend/issues/9"""
if update_pattern.match(operation):
if getattr(
settings, "CLICKHOUSE_ENABLE_UPDATE_ROWCOUNT", True
) and update_pattern.match(operation):
query = self._client.substitute_params(
operation, parameters, self._client.connection.context
)
Expand Down
17 changes: 17 additions & 0 deletions docs/Configurations.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,3 +73,20 @@ If data insertions happen on multiple datacenter, server, process or thread, you
Because work_id and datacenter_id are 5 bits, they should be an integer between 0 and 31. CLICKHOUSE_WORKER_ID default to 0, CLICKHOUSE_DATACENTER_ID will be generated randomly if not provided.

`clickhouse.idworker.snowflake.SnowflakeIDWorker` is not thread safe. You could inherit `clickhouse.idworker.base.BaseIDWorker` and implement one, then set `CLICKHOUSE_ID_WORKER` in `settings.py` to doted import path of your IDWorker instance.

### Other

#### CLICKHOUSE_ENABLE_UPDATE_ROWCOUNT

*New in 1.1.6*

Whether returning rowcount when executing `update` query, default is `True`.

Django rely on [rowcount](https://peps.python.org/pep-0249/#rowcount) of [Python DBAPI 2.0](https://peps.python.org/pep-0249/) to get number of rows deleted or updated.
However, clickhouse_driver always have rowcount equals -1 when updating and deleting.

Updating of Django builtin model instances will [work abnormally](https://github.com/jayvynl/django-clickhouse-backend/issues/9) when this feature is missing.
To fix this issue, ClickHouse backend use a modified version of clickhouse_driver which execute count aggregation and set rowcount before updating.

However, this will cause another problem, count aggregation will be extremely slow sometimes, because it scans all affected rows.
Set `CLICKHOUSE_ENABLE_UPDATE_ROWCOUNT` to `False` when this feature is not needed.
Empty file added tests/queries/__init__.py
Empty file.
6 changes: 6 additions & 0 deletions tests/queries/models.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
from django.db import models


class Author(models.Model):
name = models.CharField(max_length=10)
num = models.IntegerField()
22 changes: 22 additions & 0 deletions tests/queries/tests.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
from django.test import TestCase

from . import models


class QueriesTests(TestCase):
@classmethod
def setUpTestData(cls):
cls.a1 = models.Author.objects.create(name="a1", num=1001)

def test_update(self):
with self.settings(CLICKHOUSE_ENABLE_UPDATE_ROWCOUNT=True):
self.assertEqual(models.Author.objects.update(name="a11"), 1)
with self.assertNumQueries(1):
self.a1.save()
self.a1.refresh_from_db()
with self.settings(CLICKHOUSE_ENABLE_UPDATE_ROWCOUNT=False):
self.assertEqual(models.Author.objects.update(name="a11"), -1)
with self.assertNumQueries(2):
self.a1.save()
with self.assertRaises(models.Author.MultipleObjectsReturned):
self.a1.refresh_from_db()

0 comments on commit a89eebf

Please sign in to comment.