v2.0.0

Breaking

• JSONField maps to the JSON type instead of Object('json'), which ClickHouse deprecated and removed in 25.11. It needs ClickHouse 24.8, is production ready since 25.3, and needs allow_experimental_json_type = 1 on 24.8 to 25.2 instead of allow_experimental_object_type. No migration is generated for an existing table, its column keeps the old type until you convert it yourself. QuerySet.update() and Model.save() of an existing row need 25.6.3, which allowed ALTER TABLE ... UPDATE on a column holding dynamic sub-columns.
• key transforms compile to JSON sub-column accessors, json.a.b and json.^a.b, instead of tupleElement(). A value is no longer padded into a uniform schema, so {"b": [{"c": 1}, {"d": 2}]} reads back as it was written; a key which is absent reads as None instead of failing the query; and a value is compared in its JSON type, so filter(json__c=('e',)) and filter(json__c={'any_key': 'e'}) no longer match {"c": {"d": "e"}}. See Fields for the details.
• clickhouse_backend.driver.JSON is removed. A JSON value now travels as the text JSONField serializes it to, which ClickHouse casts to JSON itself; the CAST a comparison needs is compiled by the lookups of JSONField instead.
• escape_param() and escape_params() lose their for_server argument, together with the clickhouse_driver.connection.Connection.send_query monkey patch added for #14. clickhouse-driver has guarded that code with server_side_params itself since 0.2.7, so the patch no longer changed anything here, while still replacing upstream's escaping process wide — costing any code that uses clickhouse-driver directly the list and tuple escaping fix in 0.2.11.
• DatabaseWrapper.ch_version caches the version tuple, such as (22, 9, 3, 18), instead of the "22.9.3.18" string, so get_database_version() no longer reparses it on every call.
• toStartOfDay() now returns timezone aware DateTime values, see the timezone fix below.

Features

• JSONField gains: null=True, mapping to Nullable(JSON); the JSON type hints max_dynamic_types, max_dynamic_paths, typed_paths, skip_paths and skip_regexps, which inspectdb reads back; the has_key, has_keys and has_any_keys lookups; the contains and contained_by lookups, with the semantics of the postgres operators django names them after; and negative array indexes, values('json__a__-1') being the last element of a. Each has limits that follow from how ClickHouse stores JSON, documented in Fields.
• database functions for the JSON type: JSONAllPaths() and JSONAllPathsWithTypes() for the paths a value holds, JSONSharedDataPaths() for the ones that spilled past max_dynamic_paths, JSONAllValues() (ClickHouse 26.4) for every value as text, and dynamicType() for the type a path holds in each row. JSONAllPaths() and JSONAllValues() are also the expressions a data skipping index is built on. Plus toJSONString(), isValidJSON(), JSONLength() and JSONMergePatch() for JSON held in a string.
• django's JSONObject() and JSONArray() (django 5.2 and later) build a JSON value out of their arguments, which ClickHouse has no function of its own for, and a key path can be compared with one: filter(json__c=JSONObject(d=Value('e'))) and filter(json__c=Value({'d': 'e'}, JSONField())). Such a comparison used to match nothing.
• support django 6.1, which the test suite now runs against too. It needs python 3.12 or later.
• #135 support SAMPLE BY in MergeTree family engines and the SAMPLE clause via QuerySet.sample(), based on an idea from @asantoni.
• #135 add database functions toStartOfDay, toStartOfWeek, toStartOfMonth, toStartOfQuarter and toStartOfYear.
• add the ULIDStringToDateTime database function, contributed by @caitriona-cloudsmith.
• #166 the startswith and istartswith lookups now compile to ClickHouse's native prefix functions instead of LIKE/ILIKE, based on a PR from @AhmedIbrahim226. istartswith uses startsWithCaseInsensitiveUTF8() on ClickHouse 25.10 and later and startsWith(lowerUTF8(...), lowerUTF8(...)) on older servers. Both match what ILIKE matched, but a literal prefix no longer needs its % and _ escaped.

Fixes

• every connection now sends output_format_json_quote_64bit_integers = 0, which is the default from ClickHouse 25.8 on only. A JSON value is read through toJSONString(), so without it an Int64 came back as its own text: {'a': 1} read back as {'a': '1'} on 25.7 and below. Set it back to 1 in the settings of a connection's OPTIONS if some other query of yours needs the quoting.
• JSONField compiles an expression saved into it, create(json=JSONObject(a=1)), instead of passing it to the driver as a value, and reads a value with its decoder, which it used to ignore.
• Client no longer adds use_client_time_zone to the settings dict of a connection's OPTIONS in place. DatabaseWrapper.get_new_connection() compares connection parameters to decide whether a connection can be shared, and only the aliasing of that one dict kept the comparison true.
• escape_param() only implements the types clickhouse-driver escapes differently or not at all — datetime, the collections, IPv4Address, IPv6Address, Enum and bytes — and delegates the rest to it.
• toStartOfMinute and the functions inheriting from it used django's own DateTimeField as output_field instead of the ClickHouse one, which does not truncate the microseconds that DateTime cannot hold.
• toStartOfDay, toStartOfWeek, toStartOfMonth, toStartOfQuarter, toStartOfYear and toYearWeek now always pass a timezone, defaulting to the current one like toYYYYMM already did. Without it ClickHouse truncates in the server timezone, so results depended on server configuration and were wrong whenever the server and the client were in different timezones, or when the offset was not a whole hour. The four functions returning a Date leave the timezone out for a Date or a Date32 argument, for which ClickHouse rejects it instead of ignoring it.
• QuerySet.union(all=True) generated UNION ALL ALL, which ClickHouse rejects as a syntax error.
• #137 two EXPLAIN problems reported by @Azmisov: QUERY TREE was not an allowed type, and QuerySet.explain(format=...) rejected every output format whose ClickHouse spelling is not all upper case, such as TabSeparated.
• #154 the migration recorder only runs ALTER TABLE django_migrations ADD COLUMN deleted when that column is actually missing, instead of on every migrate, contributed by @michalpleszczynski.
• #161 the connection pool logs a client that fails to disconnect() instead of dropping the exception on the floor, and keeps closing the remaining clients.
• #167 test database cloning skipped migrations entirely for a database whose TEST settings set managed to False. Such an alias must not run CREATE DATABASE for a clone, because the managed alias already created it ON CLUSTER, but it does have to migrate: tables that are not ON CLUSTER only exist on the node the migration ran against. Clones were therefore missing every local table, such as django_content_type, on the unmanaged alias' node, which broke manage.py test --parallel against a cluster.
• the second argument of generateSerialID(), the start value of a new series, needs ClickHouse 25.10, which added it. Its test used to run on every version that has the function at all, failing with Number of arguments for function generateSerialID doesn't match on 25.1 to 25.9.
• #172 update clickhouse-driver to 0.2.11 on python 3.9 and later, reported by @khvn26. Python 3.7 and 3.8 stay on 0.2.9.

Known issues

• on ClickHouse 26.1 to at least 26.7, Min() and Max() over a DateTimeField return a wrong row for values before 1970, once the query has run often enough to be JIT compiled (three times by default). ClickHouse compares the DateTime64 as unsigned there, so a value before the epoch comes out as the largest one, reported as ClickHouse#113942. Add "compile_aggregate_expressions": 0 to the settings of the database OPTIONS if you aggregate datetimes before 1970.