[EASY] imaplib noop Debug: bytes vs Unicode bug in debug mode

[StephenEvans]

BPO	26543
Nosy	@warsaw, @vstinner, @bitdancer, @soltysh, @maxking, @ankostis, @serprex, @CuriousLearner, @rotuna
PRs	bpo-26543: fix imaplib.noop() failing when debug > 3 #1503 bpo-26543: Fix IMAP4.noop when debug mode is enabled #15206 bpo-26543: Fix crash on IMAP4.noop when bytes data is passed. #19693
Files	imaplib-noop.py: Python source capable of imaplib.noop() crash. imaplib.patch: ignore; has syntax error imaplib2.patch: Includes refactoring to use a generator

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-06-02.01:18:34.434>
created_at = <Date 2016-03-11.23:23:43.448>
labels = ['easy', 'type-bug', '3.8', 'expert-email', '3.7', '3.9']
title = '[EASY] imaplib noop Debug: bytes vs Unicode bug in debug mode'
updated_at = <Date 2020-06-02.01:18:34.433>
user = 'https://bugs.python.org/StephenEvans'

bugs.python.org fields:

activity = <Date 2020-06-02.01:18:34.433>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-06-02.01:18:34.434>
closer = 'vstinner'
components = ['email']
creation = <Date 2016-03-11.23:23:43.448>
creator = 'Stephen.Evans'
dependencies = []
files = ['42140', '42310', '42312']
hgrepos = []
issue_num = 26543
keywords = ['patch', 'easy']
message_count = 11.0
messages = ['261615', '262533', '262547', '293240', '293243', '349306', '349307', '349311', '349319', '370587', '370588']
nosy_count = 11.0
nosy_names = ['barry', 'vstinner', 'r.david.murray', 'Stephen.Evans', 'maciej.szulik', 'maxking', 'gigaplastik', 'ankostis', 'Demur Rumed', 'CuriousLearner', 'rotuna']
pr_nums = ['1503', '15206', '19693']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue26543'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[StephenEvans]

With the imaplib.Debug=3 (or greater) imaplib.noop() will crash if there are any untagged responses present.

The _dump_ur() function has not been converted to expect bytes in the lambda variable x[1]. An abbreviate example of the 'dict' parameter would be:

{'READ-WRITE': [b''], 'FETCH': [b'1 (FLAGS (\\Recent NonJunk))']}

This _dump_ur() function needs converting to Python 3.x, say, change the map/lambda line to:
l = map(lambda x:'%s: %r' % (x[0], x[1]), l)

A sample crash inducing session produced by the attached file (using another email client to produce untagged responses for NOOP):

  20:01.23 imaplib version 2.58
  20:01.23 new IMAP4 connection, tag=b'LMHB'
  20:01.48 CAPABILITIES: ('IMAP4REV1', 'LITERAL+', 'SASL-IR', 'LOGIN-REFERRALS', 'ID', 'ENABLE', 'IDLE', 'NAMESPACE', 'AUTH=PLAIN', 'AUTH=LOGIN')
Traceback (most recent call last):
  File "E:/temp/imap-idle/imaplib-noop.py", line 18, in <module>
    connection.noop()  # Crashes here.
  File "C:\Python35\lib\imaplib.py", line 656, in noop
    self._dump_ur(self.untagged_responses)
  File "C:\Python35\lib\imaplib.py", line 1210, in _dump_ur
    self._mesg('untagged responses dump:%s%s' % (t, t.join(l)))
  File "C:\Python35\lib\imaplib.py", line 1209, in <lambda>
    l = map(lambda x:'%s: "%s"' % (x[0], x[1][0] and '" "'.join(x[1]) or ''), l)
TypeError: sequence item 0: expected str instance, bytes found

[serprex]

I've attached a patch file of the proposed change which seems correct

[serprex]

Fixes syntax & line length issues in previous patch

[ankostis]

That is a show-stopper, and the patch works fine.
Should I submit a PR with a special test-case with debug >= 3 just for this?

[bitdancer]

That sounds reasonable, if the debug output doesn't pollute the test run output.

[CuriousLearner]

Removing 3.5 as it is in security fix mode. Also, the PR just contains the test case that would trigger the condition and not the actual code.

[CuriousLearner]

Hey ankostis,

Please let us know if you've time to complete the PR.

Thank you :)

[ankostis]

Dear Sanyam,

My apologies, it's been almost a year that i don't need the fix for this
code,
my app has been graciously decommissioned, and i don;t have any time for it.

Best,
Kostis

On Fri, 9 Aug 2019 at 21:35, Sanyam Khurana <report@bugs.python.org> wrote:

Sanyam Khurana <sanyam.khurana01@gmail.com> added the comment:

Hey ankostis,

Please let us know if you've time to complete the PR.

Thank you :)

----------

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue26543\>

---

[ankostis]

BTW, i went back to my code and pasted what seemed to cure the problem,

added in commit 2b99cec in my PR. But i don't have much time to track if
it really works.
I hope it will help you resolve the issue.

Best,
Kostis

[vstinner]

New changeset 8a3d2af by Sanyam Khurana in branch 'master':
bpo-26543: Fix IMAP4.noop when debug mode is enabled (GH-15206)
8a3d2af

[vstinner]

Merged, thanks for the fix.