test_datetime leaks memory on Windows

[vstinner]

BPO	35322
Nosy	@vstinner, @izbyshev
PRs	bpo-34523, bpo-35322: Fix unicode_encode_locale() #10759 [3.7] bpo-34523, bpo-35322: Fix unicode_encode_locale() (GH-10759) #10761
Files	datetimetester.py datetimetester.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-11-28.11:47:37.161>
created_at = <Date 2018-11-27.00:20:52.694>
labels = ['3.8', '3.7', 'library']
title = 'test_datetime leaks memory on Windows'
updated_at = <Date 2018-11-28.11:47:37.160>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2018-11-28.11:47:37.160>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-11-28.11:47:37.161>
closer = 'vstinner'
components = ['Library (Lib)']
creation = <Date 2018-11-27.00:20:52.694>
creator = 'vstinner'
dependencies = []
files = ['47948', '47949']
hgrepos = []
issue_num = 35322
keywords = ['patch']
message_count = 9.0
messages = ['330482', '330483', '330534', '330536', '330571', '330587', '330589', '330592', '330593']
nosy_count = 2.0
nosy_names = ['vstinner', 'izbyshev']
pr_nums = ['10759', '10761']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue35322'
versions = ['Python 3.7', 'Python 3.8']

[vstinner]

test_datetime currently leaks memory blocks:

test_datetime leaked [3, 4, 3] memory blocks, sum=10

Attached patch should be added to Lib/test/ to only run a minimum set of tests which reproduce the bug.

[vstinner]

C:\vstinner\python\master>python -m test -R 3:20 -u all test_datetime
Running Debug|x64 interpreter...
Run tests sequentially
0:00:00 [1/1] test_datetime
beginning 23 repetitions
12345678901234567890123
.......................
test_datetime leaked [1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] memory blocks, sum=21
test_datetime failed

== Tests result: FAILURE ==

1 test failed:
test_datetime

Total duration: 562 ms
Tests result: FAILURE

[izbyshev]

This is because of a leak of 'wstr' at

cpython/Objects/unicodeobject.c

Line 3476 in 1005c84

return NULL;

.

There is another leak and usage of uninitialized 'str' because the following "else if" clause doesn't have a 'return' (probably due to copy-paste from unicode_decode_locale() in 3d4226a).

[izbyshev]

Correction: the fall-through in "else if (res == -3)" clause doesn't cause a memory leak, but still results in usage of uninitialized 'str'.

[vstinner]

Oh, I see. unicode_encode_locale() has multiple bugs in the code handling errors :-(

I don't think that my commit 3d4226a introduced the memory leak triggered by test_datetime, it seems older. Python 3.7 also has the bug in unicode_encode_locale(). The memory leak has been introduced by commit 7ed7aea, also written by me :-)

[vstinner]

New changeset bde9d6b by Victor Stinner in branch 'master':
bpo-34523, bpo-35322: Fix unicode_encode_locale() (GH-10759)
bde9d6b

[vstinner]

bpo-34482 added the test in test_datetime which triggered the bug.

[vstinner]

New changeset 85ab974 by Victor Stinner in branch '3.7':
bpo-34523, bpo-35322: Fix unicode_encode_locale() (GH-10759) (GH-10761)
85ab974

[vstinner]

I tested manually that "python -m test test_datetime -R 3:3" pass on 3.7 and master branches.

Note: PyUnicode_DecodeLocale() is still untested, only _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx() are tested.