Sending stops after X sends and fails

[iamkeir]

Our sendouts have started failing after what seems to be 10 recipient sends. This seems to be since we upgraded Craft and Campaign (just recently), but may be unrelated. It also feels slightly reminiscent of #104

From the sendout:

Status: Failed
Message: Sending failed. Please check your email settings and the error in the logs.
Recipients: 10 (of 14)
Fails: 1
Expected Recipients: 4

In the Campaign logs, I cannot see any errors, only info messages:

2021-04-16 12:41:36 [][4034][info] Sendout "ANOTHER TEST" initiated by "XXX".
2021-04-16 12:41:37 [][4034][info] An exception has been thrown during the rendering of a template ("session_set_cookie_params(): Cannot change session cookie parameters when headers already sent").
2021-04-16 12:47:12 [][][info] Sending of the sendout "ANOTHER TEST" to XXX@XXX.XXX failed after 3 send attempt(s). Please check that your Campaign email settings are correctly configured and check the error in the Craft log.

From the queue log:

2021-04-16 12:41:37 - error - craft\queue\QueueLogBehavior::afterError
 [426] Sending “ANOTHER TEST” sendout [batch 1] (attempt: 1, pid: 12411) - Error (time: 0.204s): An exception has been thrown during the rendering of a template ("session_set_cookie_params(): Cannot change session cookie parameters when headers already sent").
2021-04-16 12:41:37 - error - Twig\Error\RuntimeError
yii\base\ErrorException: session_set_cookie_params(): Cannot change session cookie parameters when headers already sent in /home/ben/craft/vendor/yiisoft/yii2/web/Session.php:432
2021-04-16 12:47:09 - warning - application
Error sending email: fgets(): SSL operation failed with code 1. OpenSSL Error messages:
error:1408F10B:SSL routines:ssl3_get_record:wrong version numbe
2021-04-16 12:47:10 - warning - application
Error sending email: fwrite(): SSL: Succes
2021-04-16 12:47:11 - warning - application
Error sending email: fwrite(): SSL: Succes
2021-04-16 12:47:13 - warning - application
Error sending email: fwrite(): SSL: Succes

I am reaching out to our hosting as well to see if they can shed some light - but any thoughts greatly appreciated.

Versions

• Plugin version: 1.19.1
• Craft version: 3.6.12

[bencroker]

Thanks for reporting this. We're already aware of the issue and working on a fix, will update here when we've gotten to the bottom of it.

[bencroker]

In the meantime, clearing the compiled template caches (in the Cache Utility) appears to fix this issue.

[iamkeir]

I've cleared all Craft caches but is there a specific one within Campaign? Clearing the Craft caches did not seem to help. We are currently just manually resuming to send to the next 10 recipients until complete. Keep us posted!

[bencroker]

Just the regular compiled template caches should do the trick. You may need to restart FPM if OPcache is enabled, after clearing the caches.

[bencroker]

One thing to note is that we are unable to reliably reproduce the issue, so please be sure update Craft and Campaign and let us know if you have any more info to share about when exactly the issue happens, including what exactly you updated and which version of PHP you are running.

[iamkeir]

@bencroker I'll get everything updated tomorrow and provide as much information as I can.

[iamkeir]

Hi Ben, I've spent the day exploring this in more detail, I'll share what I have.

Craft 3.6.12
Campaign 1.19.1
PHP 7.3
Using SMTP settings

If we use Campaign's test mode, it sends to the full list of intended recipients successfully. If we use live mode, it always gets to 10 recipients then fails on the 11th. I also tried Gmail settings and it sent to all successfully. This therefore indicates that we have some rate limiting in our SMTP.

We see the 2021-04-20 00:11:07 [][][info] Sending of the sendout "TESTTIMEOUT" to xxx@xxx.xxx failed after 3 send attempt(s). Please check that your Campaign email settings are correctly configured and check the error in the Craft log. error in the logs. I am guessing this is the 11th in any sendout.

Now, looking at the historical Campaign logs (and
#104), we’ve actually had this rate limit issue for a long time BUT Campaign never previously used to stop and mark the sendout as failed - it would just skip any failing recipient and carry on to the next, and onwards. As it turns out, a failing email on our hosting actually immediately retries on our mail server and succeeds anyway.

I am wondering therefore if recent updates in Campaign and/or Craft has changed the way these failures are handled and resumed?

In the meantime, setting ‘max batch size’ to 10 seems to be a work-around.

[bencroker]

Version 1.15.4 introduced the maxSendFailsAllowed config setting which defines the failed send attempts that are allowed before failing the entire sendout which defaults to 1. See the discussion in #138 on why this was added.

I would highly recommend using a dedicated email sending service that does not rate limit you to 10 emails in some time period. You can see a list of recommended services and plugins at https://putyourlightson.com/plugins/campaign#email-delivery

Alternatively, if you must stick with your rate limited SMTP server, then setting maxBatchSize to 10 and the batchJobDelay to the number of seconds to wait between batches should work, however it will significantly slow down sendouts.

[iamkeir]

Ah ha! That's the badger! We'll discuss with the team and client - but seems like we have a couple of options in there. Thanks for your patience on this - I'll mark as closed now.

One quick question - what was the maxSendFailsAllowed originally set to?

[bencroker]

One quick question - what was the maxSendFailsAllowed originally set to?

Same as the default now, 1.

[Saboteur777]

I have been hit by this issue recently:

2021-05-03 17:04:01 [172.18.0.3][1][-][error][craft\queue\QueueLogBehavior::afterError]  [9202] Sending “XXXX” sendout [batch 1] (attempt: 1, pid: 31) - Error (time: 1.270s): An exception has been thrown during the rendering of a template ("session_set_cookie_params(): Cannot change session cookie parameters when headers already sent").
2021-05-03 17:04:01 [172.18.0.3][1][-][error][Twig\Error\RuntimeError] yii\base\ErrorException: session_set_cookie_params(): Cannot change session cookie parameters when headers already sent in /var/www/vendor/yiisoft/yii2/web/Session.php:432
Stack trace:
#0 /var/www/vendor/craftcms/cms/src/web/ErrorHandler.php(77): yii\base\ErrorHandler->handleError(2, 'session_set_coo...', '/var/www/vendor...', 432)
#1 [internal function]: craft\web\ErrorHandler->handleError(2, 'session_set_coo...', '/var/www/vendor...', 432, Array)
#2 /var/www/vendor/yiisoft/yii2/web/Session.php(432): session_set_cookie_params(Array)
#3 /var/www/vendor/yiisoft/yii2/web/Session.php(150): yii\web\Session->setCookieParamsInternal()
#4 /var/www/vendor/yiisoft/yii2/web/Session.php(700): yii\web\Session->open()
#5 /var/www/vendor/yiisoft/yii2/web/Session.php(792): yii\web\Session->get('__flash', Array)
#6 /var/www/vendor/craftcms/cms/src/behaviors/SessionBehavior.php(114): yii\web\Session->getFlash('__ab', Array, true)
#7 [internal function]: craft\behaviors\SessionBehavior->getAssetBundleFlashes(true)
#8 /var/www/vendor/yiisoft/yii2/base/Component.php(298): call_user_func_array(Array, Array)
#9 /var/www/vendor/craftcms/cms/src/web/View.php(1798): yii\base\Component->__call('getAssetBundleF...', Array)
#10 /var/www/vendor/craftcms/cms/src/web/View.php(1078): craft\web\View->registerAssetFlashes()
#11 /var/www/storage/runtime/compiled_templates/ad/adc24ff08eb99813261fe42e29e8dd8f549505f3322a0d9dff17e9deecf9f765.php(74): craft\web\View->endBody()
#12 /var/www/vendor/twig/twig/src/Template.php(405): __TwigTemplate_e4eb0221872226289588009241b19943c1d2a20bd4335c4017af8176c81b1aad->doDisplay(Array, Array)
#13 /var/www/vendor/twig/twig/src/Template.php(378): Twig\Template->displayWithErrorHandling(Array, Array)
#14 /var/www/storage/runtime/compiled_templates/6b/6b567135a2a95040904caaaef964c3e176e0c65d6069682f85b3ffaddda15bd3.php(141): Twig\Template->display(Array, Array)
#15 /var/www/vendor/twig/twig/src/Template.php(405): __TwigTemplate_fc6b1a35030d68fc5c63f6cf850075554758b6f1dd203c75e668e37e8f2ce697->doDisplay(Array, Array)
#16 /var/www/vendor/twig/twig/src/Template.php(378): Twig\Template->displayWithErrorHandling(Array, Array)
#17 /var/www/vendor/twig/twig/src/Template.php(390): Twig\Template->display(Array)
#18 /var/www/vendor/twig/twig/src/TemplateWrapper.php(45): Twig\Template->render(Array, Array)
#19 /var/www/vendor/twig/twig/src/Environment.php(318): Twig\TemplateWrapper->render(Array)
#20 /var/www/vendor/craftcms/cms/src/web/View.php(390): Twig\Environment->render('_emails/newslet...', Array)
#21 /var/www/vendor/putyourlightson/craft-campaign/src/elements/CampaignElement.php(655): craft\web\View->renderTemplate('_emails/newslet...', Array)
#22 /var/www/vendor/putyourlightson/craft-campaign/src/elements/CampaignElement.php(442): putyourlightson\campaign\elements\CampaignElement->_getBody('html', Object(putyourlightson\campaign\elements\ContactElement), Object(putyourlightson\campaign\elements\SendoutElement))
#23 /var/www/vendor/putyourlightson/craft-campaign/src/services/SendoutsService.php(618): putyourlightson\campaign\elements\CampaignElement->getHtmlBody(Object(putyourlightson\campaign\elements\ContactElement), Object(putyourlightson\campaign\elements\SendoutElement))
#24 /var/www/vendor/putyourlightson/craft-campaign/src/jobs/SendoutJob.php(157): putyourlightson\campaign\services\SendoutsService->finaliseSending(Object(putyourlightson\campaign\elements\SendoutElement))
#25 /var/www/vendor/yiisoft/yii2-queue/src/Queue.php(246): putyourlightson\campaign\jobs\SendoutJob->execute(Object(craft\queue\Queue))
#26 /var/www/vendor/yiisoft/yii2-queue/src/cli/Queue.php(147): yii\queue\Queue->handleMessage('9202', 'O:40:"putyourli...', '300', 1)
#27 /var/www/vendor/craftcms/cms/src/queue/Queue.php(131): yii\queue\cli\Queue->handleMessage('9202', 'O:40:"putyourli...', '300', 1)
#28 [internal function]: craft\queue\Queue->craft\queue\{closure}(Object(Closure))
#29 /var/www/vendor/yiisoft/yii2-queue/src/cli/Queue.php(117): call_user_func(Object(Closure), Object(Closure))
#30 /var/www/vendor/craftcms/cms/src/queue/Queue.php(140): yii\queue\cli\Queue->runWorker(Object(Closure))
#31 /var/www/vendor/craftcms/cms/src/controllers/QueueController.php(84): craft\queue\Queue->run()
#32 [internal function]: craft\controllers\QueueController->actionRun()
#33 /var/www/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#34 /var/www/vendor/yiisoft/yii2/base/Controller.php(181): yii\base\InlineAction->runWithParams(Array)
#35 /var/www/vendor/craftcms/cms/src/web/Controller.php(190): yii\base\Controller->runAction('run', Array)
#36 /var/www/vendor/yiisoft/yii2/base/Module.php(534): craft\web\Controller->runAction('run', Array)
#37 /var/www/vendor/craftcms/cms/src/web/Application.php(274): yii\base\Module->runAction('queue/run', Array)
#38 /var/www/vendor/craftcms/cms/src/web/Application.php(577): craft\web\Application->runAction('queue/run', Array)
#39 /var/www/vendor/craftcms/cms/src/web/Application.php(253): craft\web\Application->_processActionRequest(Object(craft\web\Request))
#40 /var/www/vendor/yiisoft/yii2/base/Application.php(392): craft\web\Application->handleRequest(Object(craft\web\Request))
#41 /var/www/web/index.php(21): yii\base\Application->run()
#42 {main}
Next Twig\Error\RuntimeError: An exception has been thrown during the rendering of a template ("session_set_cookie_params(): Cannot change session cookie parameters when headers already sent"). in /var/www/templates/_emails/html_base.twig:19
Stack trace:
#0 /var/www/vendor/twig/twig/src/Template.php(378): Twig\Template->displayWithErrorHandling(Array, Array)
#1 /var/www/storage/runtime/compiled_templates/6b/6b567135a2a95040904caaaef964c3e176e0c65d6069682f85b3ffaddda15bd3.php(141): Twig\Template->display(Array, Array)
#2 /var/www/vendor/twig/twig/src/Template.php(405): __TwigTemplate_fc6b1a35030d68fc5c63f6cf850075554758b6f1dd203c75e668e37e8f2ce697->doDisplay(Array, Array)
#3 /var/www/vendor/twig/twig/src/Template.php(378): Twig\Template->displayWithErrorHandling(Array, Array)
#4 /var/www/vendor/twig/twig/src/Template.php(390): Twig\Template->display(Array)
#5 /var/www/vendor/twig/twig/src/TemplateWrapper.php(45): Twig\Template->render(Array, Array)
#6 /var/www/vendor/twig/twig/src/Environment.php(318): Twig\TemplateWrapper->render(Array)
#7 /var/www/vendor/craftcms/cms/src/web/View.php(390): Twig\Environment->render('_emails/newslet...', Array)
#8 /var/www/vendor/putyourlightson/craft-campaign/src/elements/CampaignElement.php(655): craft\web\View->renderTemplate('_emails/newslet...', Array)
#9 /var/www/vendor/putyourlightson/craft-campaign/src/elements/CampaignElement.php(442): putyourlightson\campaign\elements\CampaignElement->_getBody('html', Object(putyourlightson\campaign\elements\ContactElement), Object(putyourlightson\campaign\elements\SendoutElement))
#10 /var/www/vendor/putyourlightson/craft-campaign/src/services/SendoutsService.php(618): putyourlightson\campaign\elements\CampaignElement->getHtmlBody(Object(putyourlightson\campaign\elements\ContactElement), Object(putyourlightson\campaign\elements\SendoutElement))
#11 /var/www/vendor/putyourlightson/craft-campaign/src/jobs/SendoutJob.php(157): putyourlightson\campaign\services\SendoutsService->finaliseSending(Object(putyourlightson\campaign\elements\SendoutElement))
#12 /var/www/vendor/yiisoft/yii2-queue/src/Queue.php(246): putyourlightson\campaign\jobs\SendoutJob->execute(Object(craft\queue\Queue))
#13 /var/www/vendor/yiisoft/yii2-queue/src/cli/Queue.php(147): yii\queue\Queue->handleMessage('9202', 'O:40:"putyourli...', '300', 1)
#14 /var/www/vendor/craftcms/cms/src/queue/Queue.php(131): yii\queue\cli\Queue->handleMessage('9202', 'O:40:"putyourli...', '300', 1)
#15 [internal function]: craft\queue\Queue->craft\queue\{closure}(Object(Closure))
#16 /var/www/vendor/yiisoft/yii2-queue/src/cli/Queue.php(117): call_user_func(Object(Closure), Object(Closure))
#17 /var/www/vendor/craftcms/cms/src/queue/Queue.php(140): yii\queue\cli\Queue->runWorker(Object(Closure))
#18 /var/www/vendor/craftcms/cms/src/controllers/QueueController.php(84): craft\queue\Queue->run()
#19 [internal function]: craft\controllers\QueueController->actionRun()
#20 /var/www/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#21 /var/www/vendor/yiisoft/yii2/base/Controller.php(181): yii\base\InlineAction->runWithParams(Array)
#22 /var/www/vendor/craftcms/cms/src/web/Controller.php(190): yii\base\Controller->runAction('run', Array)
#23 /var/www/vendor/yiisoft/yii2/base/Module.php(534): craft\web\Controller->runAction('run', Array)
#24 /var/www/vendor/craftcms/cms/src/web/Application.php(274): yii\base\Module->runAction('queue/run', Array)
#25 /var/www/vendor/craftcms/cms/src/web/Application.php(577): craft\web\Application->runAction('queue/run', Array)
#26 /var/www/vendor/craftcms/cms/src/web/Application.php(253): craft\web\Application->_processActionRequest(Object(craft\web\Request))
#27 /var/www/vendor/yiisoft/yii2/base/Application.php(392): craft\web\Application->handleRequest(Object(craft\web\Request))
#28 /var/www/web/index.php(21): yii\base\Application->run()
#29 {main}

Clearing the cache solved the issue, the sendout was successful.

Campaign 1.19.2
PHP 7.3.11
Craft Pro 3.6.12.1
Sending with Mailgun

config/campaign.php:

[...]
'maxSendFailsAllowed' => 1000,
[...]

The recipient count for this sendout was 39 addresses, so it was impossible that the max. fail count was hit.

[bencroker]

@Saboteur777 I believe you've encountered issue #225.

[Saboteur777]

@bencroker I think so, yes - thanks!