IMAP connection unencrypted

In documentation, it is stated that the connection to proxy should be unencrypted, but it doesn't work with IMAP.

2023-03-27 16:49:55,997: New incoming connection to IMAP server at localhost:4993 (unsecured) proxying imap.example.com:993 (SSL/TLS)
2023-03-27 16:49:55,998: Accepting new connection to IMAP server at localhost:4993 (unsecured) proxying imap.example.com:993 (SSL/TLS) via ('::1', 60304, 0, 0)
2023-03-27 16:49:56,007: IMAP (localhost:4993; ::1:60304->imap.example.com:993) --> b'00001 CAPABILITY\r\n'
2023-03-27 16:49:56,008: IMAP (localhost:4993; ::1:60304->imap.example.com:993)     --> b'00001 CAPABILITY\r\n'
2023-03-27 16:49:56,014: IMAP (localhost:4993; ::1:60304->imap.example.com:993) <-- [ Server disconnected ]

To connect, it is needed to use STARTTLS (in The Bat).

[simonrob]

I'm not sure what you mean here, but if your email provider requires STARTTLS please enable this in the proxy's configuration file. Your connection to the proxy should be configured as unencrypted either way. See the examples in the sample configuration file.

It is not the email provider requires STARTTLS, it's the proxy not connecting when unencrypted connection from client.
My provider uses SSL/TLS.

(i.e., do not enable STARTTLS in your client).

It is not connecting until I enable STARTTLS in my client.

[simonrob]

The log file you posted doesn't show anything about STARTTLS. If you enable STARTTLS on the client side then the proxy won't be able to intercept and modify your client's authentication commands.

What provider are you using here?

In The Bat, when using an unencrypted connection, it doesn't connect (to IMAP only).
It only shows "Accepting connection", but no "Client connected".

Log with STARTTLS:

2023-04-02 12:37:58,557: New incoming connection to IMAP server at localhost:4993 (unsecured) proxying imap.yandex.ru:993 (SSL/TLS)
2023-04-02 12:37:58,559: Accepting new connection to IMAP server at localhost:4993 (unsecured) proxying imap.yandex.ru:993 (SSL/TLS) via ('::1', 50524, 0, 0)
2023-04-02 12:37:58,612: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993) --> [ Client connected ]
2023-04-02 12:37:58,637: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993) <-> [ Starting TLS handshake ]
2023-04-02 12:37:58,672: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993) <-> [ TLSv1.3 handshake complete ]
2023-04-02 12:37:58,929: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993)     <-- b'* OK Yandex IMAP4rev1 at mail-imap-production-424.iva.yp-c.yandex.net:993 ready to talk with ::ffff:131.128.179.112:50525, 2023-Apr-02 12:37:58, wbIx1sRpKCg0\r\n'
2023-04-02 12:37:58,931: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993) <-- b'* OK Yandex IMAP4rev1 at mail-imap-production-424.iva.yp-c.yandex.net:993 ready to talk with ::ffff:131.128.179.112:50525, 2023-Apr-02 12:37:58, wbIx1sRpKCg0\r\n'
2023-04-02 12:37:58,981: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993) --> b'TLS0 STARTTLS\r\n'
2023-04-02 12:37:58,983: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993)     --> b'TLS0 STARTTLS\r\n'
2023-04-02 12:37:59,012: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993)     <-- b'TLS0 BAD [CLIENTBUG] STARTTLS SSL/TLS already active. sc=wbIx1sRpKCg0_020937_imap-production-424\r\n'
2023-04-02 12:37:59,014: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993) <-- b'TLS0 BAD [CLIENTBUG] STARTTLS SSL/TLS already active. sc=wbIx1sRpKCg0_020937_imap-production-424\r\n'
2023-04-02 12:37:59,016: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993) --> b'00001 CAPABILITY\r\n'
2023-04-02 12:37:59,017: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993)     --> b'00001 CAPABILITY\r\n'
2023-04-02 12:37:59,047: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993)     <-- b'* CAPABILITY IMAP4rev1 CHILDREN UNSELECT LITERAL+ NAMESPACE XLIST BINARY UIDPLUS ENABLE ID AUTH=PLAIN AUTH=XOAUTH2 IDLE MOVE\r\n'
2023-04-02 12:37:59,050: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993) <-- b'* CAPABILITY IMAP4rev1 CHILDREN UNSELECT LITERAL+ NAMESPACE XLIST BINARY UIDPLUS ENABLE ID AUTH=PLAIN SASL-IR IDLE MOVE\r\n'
2023-04-02 12:37:59,279: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993)     <-- b'00001 OK CAPABILITY Completed.\r\n'
2023-04-02 12:37:59,280: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993) <-- b'00001 OK CAPABILITY Completed.\r\n'
2023-04-02 12:37:59,283: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993) --> b'00002 ID ("name" "The Bat!" "version" "5.8.8" "os" "Windows 7" "os-version" "6.1 build 7601" "vendor" "Ritlabs S.R.L." "support-url" "http://www.ritlabs.com")\r\n'
2023-04-02 12:37:59,284: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993)     --> b'00002 ID ("name" "The Bat!" "version" "5.8.8" "os" "Windows 7" "os-version" "6.1 build 7601" "vendor" "Ritlabs S.R.L." "support-url" "http://www.ritlabs.com")\r\n'
2023-04-02 12:37:59,358: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993)     <-- b'* ID ("name" "Yandex Mail" "vendor" "Yandex" "support-url" "http://feedback.yandex.ru/?from=mail")\r\n'
2023-04-02 12:37:59,359: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993) <-- b'* ID ("name" "Yandex Mail" "vendor" "Yandex" "support-url" "http://feedback.yandex.ru/?from=mail")\r\n'
2023-04-02 12:37:59,587: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993)     <-- b'00002 OK ID Completed.\r\n'
2023-04-02 12:37:59,589: IMAP (localhost:4993; ::1:50524->imap.yandex.ru:993) <-- b'00002 OK ID Completed.\r\n'

STARTTLS is not used, it just doesn't disconnect like plain text connection.

[simonrob]

Neither your provider nor the proxy support STARTTLS for IMAP, so I suspect you are seeing this behaviour because of a misconfiguration or client bug.

Against my better judgement I replicated your setup using "The Bat" and an email account at the same provider you are using. When properly set up with a non-secure local connection and proxy configuration file, everything works as it should – see below.

[IMAP-3993]
server_address = imap.yandex.com
server_port = 993

[your.email.address@yandex.com]
permission_url = https://oauth.yandex.com/authorize
token_url = https://oauth.yandex.com/token
oauth2_scope = mail:imap_full
redirect_uri = http://localhost
client_id = *** your client id here ***
client_secret = *** your client secret here ***

If you did want to use a secure local connection the proxy can support this, but it is usually easier to get things working first before looking at advanced configuration options.

As an aside, "The Bat" has a strange behaviour as demonstrated in your proxy log in that when you enable STARTTLS it continues with the connection even if STARTTLS fails. Given its framing as a security-focused client, this is perhaps worth reporting to its developers.

I'm using version 5.8.
Ver. 10 is not the thing to use.

	time	src	dst	prot	length	info
47	5.273895	::1	::1	TCP	76	54297 > 4993 [SYN] Seq=0 Win=8192 Len=0 MSS=65475 WS=256 SACK_PERM=1	client connecting
48	5.273973	::1	::1	TCP	76	4993 > 54297 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=65475 WS=256 SACK_PERM=1	confirmation
49	5.274011	::1	::1	TCP	64	54297 > 4993 [ACK] Seq=1 Ack=1 Win=8192 Len=0	connection established
50	5.274614	::1	::1	TCP	82	54297 > 4993 [PSH, ACK] Seq=1 Ack=1 Win=8192 Len=18	client sends 00001 CAPABILITY
51	5.274645	::1	::1	TCP	64	4993 > 54297 [ACK] Seq=1 Ack=19 Win=7936 Len=0	confirmation
60	5.397023	192.168.253.216	192.168.253.1	DNS	74	Standard query 0xd98c A imap.yandex.ru
61	5.398302	192.168.253.1	192.168.253.216	DNS	90	Standard query response 0xd98c A imap.yandex.ru A 77.88.21.125
62	5.418241	192.168.253.216	192.168.253.1	DNS	74	Standard query 0x4aa1 A imap.yandex.ru
63	5.419120	192.168.253.1	192.168.253.216	DNS	90	Standard query response 0x4aa1 A imap.yandex.ru A 77.88.21.125
64	5.419763	192.168.253.216	77.88.21.125	TCP	66	54300 > 993 [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=4 SACK_PERM=1	proxy connecting to server
54	5.441637	::1	::1	TCP	64	4993 > 54297 [FIN, ACK] Seq=1 Ack=19 Win=7936 Len=0	proxy initiates connection termination with the client
56	5.441690	::1	::1	TCP	64	54297 > 4993 [ACK] Seq=19 Ack=2 Win=8192 Len=0	confirmation from the client
65	5.459086	77.88.21.125	192.168.253.216	TCP	66	993 > 54300 [SYN, ACK] Seq=0 Ack=1 Win=42300 Len=0 MSS=1410 SACK_PERM=1 WS=256	server waiting for confirmation from the proxy
57	5.472890	::1	::1	TCP	64	54297 > 4993 [FIN, ACK] Seq=19 Ack=2 Win=8192 Len=0	client responds to connection termination with proxy
58	5.472971	::1	::1	TCP	64	4993 > 54297 [ACK] Seq=2 Ack=20 Win=7936 Len=0	connection terminated
76	6.483264	77.88.21.125	192.168.253.216	TCP	66	[TCP Retransmission] 993 > 54300 [SYN, ACK] Seq=0 Ack=1 Win=42300 Len=0 MSS=1410 SACK_PERM=1 WS=256	too late

So, if it will not work as unencrypted and will not continue "if STARTTLS fails", then it will not work at all.

[simonrob]

That was an important detail – a 10-year old client which has plenty of more recent versions may well have bugs that have been resolved in the meantime.

Based on the sequence of events you've described, I suspect this issue is related to #101, where the IMAP client is not following the specification. Clients are meant to wait for the initial server greeting message before sending commands, and it looks like "The Bat" didn't do this until a more recent version. I'd recommend upgrading.