Gentoo Refleaks 3.7: test_smtplib has dangling threads

[pablogsal]

BPO	34246
Nosy	@warsaw, @vstinner, @bitdancer, @serhiy-storchaka, @scorphus, @pablogsal, @miss-islington
PRs	bpo-34246: Use no mutable default args in smtplib #8554 bpo-34246: Make sure test_smtplib always cleans resources when finished #9108 [3.7] bpo-34246: Use no mutable default args in smtplib (GH-8554) #9111 [3.6] bpo-34246: Use no mutable default args in smtplib (GH-8554) #9112

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-09-08.01:22:13.079>
created_at = <Date 2018-07-27.10:02:13.149>
labels = ['3.8', 'type-bug', '3.7', 'expert-email']
title = 'Gentoo Refleaks 3.7: test_smtplib has dangling threads'
updated_at = <Date 2018-09-08.01:22:13.078>
user = 'https://github.com/pablogsal'

bugs.python.org fields:

activity = <Date 2018-09-08.01:22:13.078>
actor = 'pablogsal'
assignee = 'none'
closed = True
closed_date = <Date 2018-09-08.01:22:13.079>
closer = 'pablogsal'
components = ['email']
creation = <Date 2018-07-27.10:02:13.149>
creator = 'pablogsal'
dependencies = []
files = []
hgrepos = []
issue_num = 34246
keywords = ['patch']
message_count = 15.0
messages = ['322481', '322482', '322488', '322493', '322494', '322500', '322501', '322502', '322509', '322544', '323780', '324804', '324809', '324811', '324817']
nosy_count = 7.0
nosy_names = ['barry', 'vstinner', 'r.david.murray', 'serhiy.storchaka', 'scorphus', 'pablogsal', 'miss-islington']
pr_nums = ['8554', '9108', '9111', '9112']
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue34246'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[pablogsal]

The Gentoo Refleaks 3.7 buildbot is reporting that test_smtplib may be leaving with dangling threads:

https://buildbot.python.org/all/#builders/114/builds/179

FAILED (errors=7)
Warning -- threading._dangling was modified by test_smtplib
Before: <_weakrefset.WeakSet object at 0xb6918bcc>
After: <_weakrefset.WeakSet object at 0xb68e761c>
test test_smtplib failed
1 test failed again:
test_smtplib
== Tests result: FAILURE then FAILURE ==

[pablogsal]

New failure on AMD64 Windows8.1 Refleaks 3.6:

https://buildbot.python.org/all/#builders/64/builds/301

[bitdancer]

Ah, sorry. we just modernized test_smtplib's __main__ section, and I think we lost the thread cleanup code that used to be run around the test suite. We need to add the thread cleanup decorators.

[pablogsal]

I think it may be something extra as test_send_message_error_on_non_ascii_addrs_if_no_smtputf8 is failing when you run the test with -R:

./python -W error -m test test_smtplib -R 3:3

Warning -- threading._dangling was modified by test_smtplib
  Before: <_weakrefset.WeakSet object at 0x10ed7dd70>
  After:  <_weakrefset.WeakSet object at 0x10f619830>
test test_smtplib failed -- Traceback (most recent call last):
  File "/Users/pgalindo3/cpython/Lib/test/test_smtplib.py", line 1232, in test_send_message_error_on_non_ascii_addrs_if_no_smtputf8
    smtp.send_message(msg))
  File "/Users/pgalindo3/cpython/Lib/unittest/case.py", line 743, in assertRaises
    return context.handle('assertRaises', args, kwargs)
  File "/Users/pgalindo3/cpython/Lib/unittest/case.py", line 178, in handle
    callable_obj(*args, **kwargs)
TypeError: 'dict' object is not callable

test_smtplib failed

== Tests result: FAILURE ==

1 test failed:
test_smtplib

Total duration: 16 sec 566 ms
Tests result: FAILURE

[pablogsal]

The TypeError is because the test is misusing the assertRaises API:

         self.assertRaises(smtplib.SMTPNotSupportedError,
                          smtp.send_message(msg))

should be

         self.assertRaises(smtplib.SMTPNotSupportedError,
                          smtp.send_message, msg)

[bitdancer]

That's not the code I thought I merged. I'll have to take a look at what actually got merged.

[pablogsal]

That was from commit 48ed88a

[pablogsal]

Oh, sorry, my bad. That was precisely what commit 48ed88a changed. The problem with running with -R still happens:

======================================================================
ERROR: test_name_field_not_included_in_envelop_addresses (test.test_smtplib.SMTPSimTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Users/pgalindo3/cpython/Lib/test/test_smtplib.py", line 1100, in test_name_field_not_included_in_envelop_addresses
    self.assertDictEqual(smtp.send_message(message), {})
  File "/Users/pgalindo3/cpython/Lib/smtplib.py", line 967, in send_message
    rcpt_options)
  File "/Users/pgalindo3/cpython/Lib/smtplib.py", line 861, in sendmail
    (code, resp) = self.mail(from_addr, esmtp_opts)
  File "/Users/pgalindo3/cpython/Lib/smtplib.py", line 532, in mail
    'SMTPUTF8 not supported by server')
smtplib.SMTPNotSupportedError: SMTPUTF8 not supported by server

---

Ran 60 tests in 1.365s

FAILED (errors=7)
Warning -- threading._dangling was modified by test_smtplib
Before: <_weakrefset.WeakSet object at 0x10f445e90>
After: <_weakrefset.WeakSet object at 0x10fe75ad0>
test test_smtplib failed
test_smtplib failed

== Tests result: FAILURE ==

1 test failed:
test_smtplib

Total duration: 2 sec 920 ms
Tests result: FAILURE

[bitdancer]

That's due to another bug, which will get fixed by bpo-32814, unless you'd like to generate a PR with the this fix in it (fixing the mutable default for smtp_options).

[scorphus]

This is just to note that I'm investigating this issue as part of the CPython sprint in EuroPython.

[serhiy-storchaka]

See also bpo-32657.

[pablogsal]

New changeset d5fbe9b by Pablo Galindo (Pablo Aguiar) in branch 'master':
bpo-34246: Use no mutable default args in smtplib (GH-8554)
d5fbe9b

[miss-islington]

New changeset 9835696 by Miss Islington (bot) in branch '3.7':
bpo-34246: Use no mutable default args in smtplib (GH-8554)
9835696

[pablogsal]

New changeset 5b7a2cb by Pablo Galindo in branch 'master':
bpo-34246: Make sure test_smtplib always cleans resources when finished (GH-9108)
5b7a2cb

[pablogsal]

New changeset eb6ab73 by Pablo Galindo in branch '3.6':
[3.6] bpo-34246: Use no mutable default args in smtplib (GH-8554) (bpo-9112)
eb6ab73