sqlite3 test CheckTraceCallbackContent fails for sqlite v3.7.3 through 3.7.14.1

[erlend-aasland]

BPO	40810
Nosy	@gvanrossum, @tiran, @berkerpeksag, @csabella, @corona10, @miss-islington, @brandtbucher, @erlend-aasland
PRs	bpo-40744: Drop support for SQLite3 pre v3.7.3 #20330 bpo-40810: Fix CheckTraceCallbackContent for SQLite pre 3.7.15 #20530 [3.9] bpo-40810: Fix CheckTraceCallbackContent for SQLite pre 3.7.15 (GH-20530) #24104 [3.8] bpo-40810: Fix CheckTraceCallbackContent for SQLite pre 3.7.15 (GH-20530) #24105 bpo-40810: Require SQLite 3.7.15 #24106 bpo-40810: Fix broken NEWS markup #24110 [3.9] bpo-40810: Fix broken NEWS markup (GH-24110) #24111 [3.8] bpo-40810: Fix broken NEWS markup (GH-24110) #24112 bpo-32631: IDLE: Enable zzdummy extension module #14491

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-01-06.00:04:50.476>
created_at = <Date 2020-05-28.17:47:38.365>
labels = ['3.8', 'type-feature', 'library', '3.9', '3.10']
title = 'sqlite3 test CheckTraceCallbackContent fails for sqlite v3.7.3 through 3.7.14.1'
updated_at = <Date 2021-01-06.00:05:08.010>
user = 'https://github.com/erlend-aasland'

bugs.python.org fields:

activity = <Date 2021-01-06.00:05:08.010>
actor = 'berker.peksag'
assignee = 'none'
closed = True
closed_date = <Date 2021-01-06.00:04:50.476>
closer = 'berker.peksag'
components = ['Library (Lib)']
creation = <Date 2020-05-28.17:47:38.365>
creator = 'erlendaasland'
dependencies = []
files = []
hgrepos = []
issue_num = 40810
keywords = ['patch']
message_count = 15.0
messages = ['370256', '370263', '370264', '370266', '370271', '370278', '370304', '384355', '384356', '384357', '384358', '384359', '384369', '384445', '384446']
nosy_count = 8.0
nosy_names = ['gvanrossum', 'christian.heimes', 'berker.peksag', 'cheryl.sabella', 'corona10', 'miss-islington', 'brandtbucher', 'erlendaasland']
pr_nums = ['20330', '20530', '24104', '24105', '24106', '24110', '24111', '24112', '14491']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40810'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[erlend-aasland]

CheckTraceCallbackContent() in [Lib/sqlite3/test/hooks.py](https://github.com/python/cpython/blob/main/Lib/sqlite3/test/hooks.py) fails for SQLite versions 3.7.3 through 3.7.14.1, apparently because of an SQLite bug that was fixed in 3.7.15 (2012-12-12). Extract from the changelog https://sqlite.org/changes.html:

Avoid invoking the sqlite3_trace() callback multiple times when a statement is automatically reprepared due to SQLITE_SCHEMA errors.

Either we fix tests for SQLite < 3.7.15, or we drop support for SQLite < 3.7.15. (I'm +1 for the latter.)

NB: Versions pre 3.7.3 does not build at all, because of sqlite3_create_function_v2(). See bpo-40744 and #64529.

[erlend-aasland]

Also related to bpo-26187

[tiran]

Could you please check https://distrowatch.com/ and verify that all major Linux distributions have a sufficient version of sqlite. Please also check LTS like Debian oldstable, Ubuntu, and CentOS.

[erlend-aasland]

Debian oldstable (jessie) has sqlite 3.8.7.1
Ubuntu 14.04 LTS (trusty) has sqlite 3.8.2
CentOS 7 and 8 has sqlite versions > 3.7.15

The following distributions include sqlite version _less than_ 3.7.15:
• Porteus (release v2.1 and older)
• OLPC OS
• HardenedBSD
• 0Linux / FreePBX (release v10 and older)
• ToOpPy Linux
• Baruwa Enterprise Edition

The other hits were inactive distros.

https://distrowatch.com/search.php?pkg=sqlite&relation=less&pkgver=3.7.15&distrorange=InLatest#pkgsearch

[berkerpeksag]

+1 for dropping support for < 3.7.15 in master. We should fix or skip tests in maintenance branches.

[erlend-aasland]

All right, will do, Berker Persag.

[erlend-aasland]

FYI, the #if in line 1563 in [Modules/_sqlite/connection.c](https://github.com/python/cpython/blob/main/Modules/_sqlite/connection.c) is wrong, but the comment five lines below is correct:
https://github.com/python/cpython/blob/e56ce3ce865cb5539cdedbeae54eaa11ec563922/Modules/_sqlite/connection.c#L1563-L1589

sqlite3_errstr() was introduced in 3.7.15, so the #if comparison operator should have been '>=', not '>'.

[miss-islington]

New changeset f7f0ed5 by Erlend Egeberg Aasland in branch 'master':
bpo-40810: Fix CheckTraceCallbackContent for SQLite pre 3.7.15 (GH-20530)
f7f0ed5

[gvanrossum]

Please ping this issue if the backports don't auto-merge after the tests pass, or if the tests fail.

[erlend-aasland]

+1 for dropping support for < 3.7.15 in master. We should fix or skip tests in maintenance branches.

PR's for fixing the tests in 3.9 and 3.8 are now awaiting merge. PR for dropping support for SQLite pre 3.7.15 in master is opened.

[miss-islington]

New changeset 0ccac5f by Miss Islington (bot) in branch '3.8':
bpo-40810: Fix CheckTraceCallbackContent for SQLite pre 3.7.15 (GH-20530)
0ccac5f

[miss-islington]

New changeset 80e5732 by Miss Islington (bot) in branch '3.9':
bpo-40810: Fix CheckTraceCallbackContent for SQLite pre 3.7.15 (GH-20530)
80e5732

[brandtbucher]

It looks like the markup in the NEWS entry broke Travis on master. I guess that's one downside of Travis not being a required job anymore.

[berkerpeksag]

New changeset cf0b239 by Erlend Egeberg Aasland in branch 'master':
bpo-40810: Require SQLite 3.7.15 (GH-24106)
cf0b239

[berkerpeksag]

I think this is now done. Thank you, Erlend!