[Mailer] SMTP transport fails for long running processes

[tomme87]

Symfony version(s) affected: 5.0.5+

Description
Happens if sending a email, and after 10 seconds (and before 100 seconds) a new email (on the same connection) is sent, then a TransportException is thrown:

Expected response code "250" but got code "451", with message "451 4.4.2 Timeout waiting for data from client.".

If sending another email an other TransportException is thrown:

Unable to write bytes on the wire.

This happens until I wait for 100 seconds.

How to reproduce
Use symfony/messenger, run message:consume and send a Email, wait for between 10 and 100 seconds then send a new Email.

I have only been able to reproduce this using Amazon SMTP server, but it might be others too.

Possible Solution
setting $pingThreshold in SmtpTransport to 10 seams to solve the issue.

Additional context

14:32:24 INFO      [messenger] Received message Symfony\Component\Mailer\Messenger\SendEmailMessage ["message" => Symfony\Component\Mailer\Messenger\SendEmailMessage^ { …},"class" => "Symfony\Component\Mailer\Messenger\SendEmailMessage"]                                                                                                                           
14:32:24 ERROR     [messenger] Error thrown while handling message Symfony\Component\Mailer\Messenger\SendEmailMessage. Sending for retry #1 using 1000 ms delay. Error: "Expected response code "250" but got code "451", with message "451 4.4.2 Timeout waiting for data from client."." ["message" => Symfony\Component\Mailer\Messenger\SendEmailMessage^ { …},"class" => "Symfony\Component\Mailer\Messenger\SendEmailMessage","retryCount" => 1,"delay" => 1000,"error" => "Expected response code "250" but got code "451", with message "451 4.4.2 Timeout waiting for data from client.".","exception" => Symfony\Component\Messenger\Exception\HandlerFailedException^ { …}]
14:32:24 INFO      [messenger] Received message Symfony\Component\Mailer\Messenger\SendEmailMessage ["message" => Symfony\Component\Mailer\Messenger\SendEmailMessage^ { …},"class" => "Symfony\Component\Mailer\Messenger\SendEmailMessage"]                                                                                                                           
14:32:24 ERROR     [messenger] Error thrown while handling message Symfony\Component\Mailer\Messenger\SendEmailMessage. Sending for retry #1 using 1000 ms delay. Error: "Unable to write bytes on the wire." ["message" => Symfony\Component\Mailer\Messenger\SendEmailMessage^ { …},"class" => "Symfony\Component\Mailer\Messenger\SendEmailMessage","retryCount" => 1,"delay" => 1000,"error" => "Unable to write bytes on the wire.","exception" => Symfony\Component\Messenger\Exception\HandlerFailedException^ { …}]

Looks like it happens after #35633 was introduced.

[dmaicher]

@tomme87 you could use this setter to customize the value?

https://github.com/symfony/symfony/pull/35633/files#diff-1e389e0e3302b8f3db89160f092bd39dR86

[tomme87]

@dmaicher Yes, that requires me to create my own factory and set that value tho. I don't think I should be required to deal with that value.. I just want to send an email.

At the very least the error message should be better and maybe point me to that value. And if this is a value that needs to be changed based on what SMTP server I use it should be configuration option imo.

[oesteve]

I had the same problem, I'm also using amazon SES with smtp transport, I fixed overriding the smtp factory to adjust $pingThreshold, but I would like to find a more elegant solution.

[dmaicher]

Maybe it should be an option on the Dsn?

Like smtp://user:pass@smtp.example.com?ping_threshold=10

[oesteve]

This parameters was introduced in #35633
Maybe we are going to have the same source problem in te future.
And a simple retry after reset connection if there are an especifics 4XX errors or a connection error ?

[oesteve]

Currently the transport throw TransportException until $pingThreshold is exceded, if the $pingThreshold is faster, the number of retries to reconnect to the transport increase, but the problem persists, in long-running processes a lot of messages will be discarded in 10 seconds.

@tomme87 Are you using messenger worker to send messages ?
In my case, messenger tries 3 times before send the message to invalid queue, with #36417 the message will throw transport exception if the socket is closed, but on the next retry, the transport must try to open de connection again. It isn't a clean solution because al least one message was descarted, but the messenger retry mechanism will eventually send it.

[arjanfrans]

I am still seeing this problem on our system. It doesn't seem that the merged PR resolves the problem.

[stof]

@arjanfrans please open an new issue with details