Strange behavior when I logout() with IMAP4_SSL

[toelke]

BPO	5565
Nosy	@pitrou, @giampaolo, @bitdancer

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-05-03.15:18:03.623>
created_at = <Date 2009-03-26.07:12:24.225>
labels = ['invalid', 'type-bug', 'library']
title = 'Strange behavior when I logout() with IMAP4_SSL'
updated_at = <Date 2010-05-03.15:18:03.621>
user = 'https://bugs.python.org/toelke'

bugs.python.org fields:

activity = <Date 2010-05-03.15:18:03.621>
actor = 'r.david.murray'
assignee = 'none'
closed = True
closed_date = <Date 2010-05-03.15:18:03.623>
closer = 'r.david.murray'
components = ['Library (Lib)']
creation = <Date 2009-03-26.07:12:24.225>
creator = 'toelke'
dependencies = []
files = []
hgrepos = []
issue_num = 5565
keywords = []
message_count = 6.0
messages = ['84175', '85176', '104371', '104820', '104821', '104846']
nosy_count = 4.0
nosy_names = ['pitrou', 'giampaolo.rodola', 'r.david.murray', 'toelke']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue5565'
versions = ['Python 2.6']

[toelke]

While researching some strange logs from out firewall and narrowing it
down to a biff-like imap-client written in python we found the following:

To reproduce:

Start a network-sniffer like wireshark on the loopback-interface
In one shell start some network-listener:
$ nc -l -p 12345

In python, connect to it and send some data:
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> s.connect(("127.0.0.1", 12345))
>>> s.send("Hallo\n");
6

The other shell now looks like:
$ nc -l -p 12345
Hallo

Type in anything and press <Enter> to send. Then quit the nc with C-c.

Then:
>>> s.close()

What I see here is the following:
The two messages are exchanged and nc sends his FIN-Package when
quitting. Python ACKnowledges this package (though intrestingly enough,
with a ack-number one to far (8 instead of 7 in my example)). At the
Moment of the s.close(), it sends another package, containing the same
ACK-Number, the same SEQ-Number(!) and this time the RST-Flag as well.

If I understand correctly, it sends RST, because not everything from the
connection was read by python. Why does it resend the ACK? Why is the
ACK-Number one to high? Why does it reuse the SEQ-Number?

And now to imaplib.IMAP4_SSL. The behavior here seems to me even more
strange:

If I invoke .logout(), the server sends his "BYE" message and after that
a FIN, which python ACKnowledges. At the moment, that the
IMAP4_SSL-object gets out of scope, a RST/ACK-Package is send, that
again re-ACKs the FIN and has the same sequence-number, that the ACK
package had!

Why does .logout() not send a FIN? Why does it not read the complete
Buffer, so that the Socket could close with FIN? And, why does it wait
until getting out of scope with sending this RST? (I assume, that is
when the Object is garbage-collected)

Thank you!

[toelke]

Just FYI, this issue is in python2.6, too.

Only, that in 2.6 the GC does not collect the objects immediately, so
that very soon I have a galore of connections in the CLOSE_WAIT-State.

And I checked, I can not read anymore data out of the socket.

Is this a bug in the ssl-implemantation?

[pitrou]

Do you still witness the issue?
Could you post a small code sample showing the problem with IMAP4_SSL?

(it's not obvious it's a Python problem at all; details of the TCP implementation are obviously handled by the OS, and Python only uses the standard C socket API -- recv(), send() and friends)

[toelke]

The TCP-issues from my post are all resolved. I now know how TCP works; the behaviour of python seems to be correct.

About the imap-behaviour:
me@harga ~$ python
Python 2.5.5 (r255:77872, Apr 21 2010, 08:40:04) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import imaplib
>>> c = imaplib.IMAP4_SSL("post.in.tum.de")
>>> c.login("toelke", "XXX")
('OK', ['LOGIN Ok.'])
>>> c.logout()
('BYE', ['Courier-IMAP server shutting down'])
>>> <C-d>
me@harga ~$

At the time of the logout() the server closes his connection and sends a FIN-packet. If python is closed, it sends not a FIN-Packet but a RST-paket.

The "problem" I have with this is, that the Linux-Packetfilter-Firewall does not expect this RST-Packet either and in out configuration logs this as a connection in the wrong state which is generally a security-problem.

Our workaround for the last year has been

iptables -A OUTPUT -p tcp --tcp-flags ACK,RST ACK,RST -d 131.159.22.43 -j DROP

Thanks for the help!

[toelke]

The TCP-issues from my post are all resolved. I now know how TCP works; the behaviour of python seems to be correct.

About the imap-behaviour:
me@harga ~$ python
Python 2.5.5 (r255:77872, Apr 21 2010, 08:40:04) 
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import imaplib
>>> c = imaplib.IMAP4_SSL("post.in.tum.de")
>>> c.login("toelke", "XXX")
('OK', ['LOGIN Ok.'])
>>> c.logout()
('BYE', ['Courier-IMAP server shutting down'])
>>> <C-d>
me@harga ~$

At the time of the logout() the server closes his connection and sends a FIN-packet. If python is closed, it sends not a FIN-Packet but a RST-paket.

The "problem" I have with this is, that the Linux-Packetfilter-Firewall does not expect this RST-Packet either and in out configuration logs this as a connection in the wrong state which is generally a security-problem.

Our workaround for the last year has been

iptables -A OUTPUT -p tcp --tcp-flags ACK,RST ACK,RST -d 131.159.22.43 -j DROP

Thanks for the help!

[bitdancer]

FWIW on my cisco firewalls the logs contain a lot of 'deny, no connection' messages for RST packets, probably coming from similar scenarios.