Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Use quoted_printable_encode() if available for performance reasons #217

Merged
merged 2 commits into from

4 participants

@lstrojny

If we have a PHP version that offers quoted_printable_encode() we should it as it is much faster.

(Pull request courtesy of @maxbeutel and myself)

@fabpot
Owner

It breaks a bunch of tests.

@lstrojny

Would it maybe be better to have NativeQpEncoder and leave the userland implementation untouched?

@fabpot
Owner

This is probably a better idea.

@lstrojny

OK, here is a standalone implementation. We measured 30x (down from 100 mails in 30 seconds to 100 mails in 1 second) more throughput with the native function compared to the userland implementation. And here is how to use it (swift_init.php):

<?php
...
Swift_DependencyContainer::getInstance()
    ->register('mime.qpcontentencoder')
    ->asAliasOf('mime.nativeqpcontentencoder');
@lstrojny

From my POV this pull request is ready to be merged. Comments?

@fabpot
Owner

Can you squash you commits? Thanks.

@lstrojny

Done

@fabpot
Owner

Can you also add a note somewhere in the documentation (probably in including-the-files.rst) that when using PHP >= 5.3.0, this is the recommended class to use (with a snippet of code on how to configure it)?

@lukaswoj

Hi there
I'm so happy to find out this PR. Looks like really fresh topic.
Thank you Lars

I was just struggling with the same problem of high CPU usage when sending many emails in bulk.
In my case the problem came from the fact that emails needs to be dispatched at given moment in time and the amount of them makes whole process taking too long.
Since I knew in advance what will be the content of each email - I even had the idea to asynchronously preencode body of the messages and later pass already encoded string.
But now this does not matter anymore.

Great job!

Thanks again
Cheers

@lstrojny

Documentation added. @lukaswoj glad we could help you.

@fabpot fabpot referenced this pull request from a commit
@fabpot fabpot merged branch InterNations/native-quoted-printable-encode (PR #217)
Commits
-------

ed0312b Adding doc hint
449117d Native quoted printable content encoder

Discussion
----------

Use quoted_printable_encode() if available for performance reasons

If we have a PHP version that offers quoted_printable_encode() we should it as it is much faster.

(Pull request courtesy of @maxbeutel and myself)

---------------------------------------------------------------------------

by fabpot at 2012-06-25T19:37:32Z

It breaks a bunch of tests.

---------------------------------------------------------------------------

by lstrojny at 2012-06-25T19:49:29Z

Would it maybe be better to have NativeQpEncoder and leave the userland implementation untouched?

---------------------------------------------------------------------------

by fabpot at 2012-06-26T05:06:13Z

This is probably a better idea.

---------------------------------------------------------------------------

by lstrojny at 2012-06-26T09:36:20Z

OK, here is a standalone implementation. We measured 30x (down from 100 mails in 30 seconds to 100 mails in 1 second) more throughput with the native function compared to the userland implementation. And here is how to use it (`swift_init.php`):

```php
<?php
...
Swift_DependencyContainer::getInstance()
    ->register('mime.qpcontentencoder')
    ->asAliasOf('mime.nativeqpcontentencoder');

```

---------------------------------------------------------------------------

by lstrojny at 2012-06-30T20:10:59Z

From my POV this pull request is ready to be merged. Comments?

---------------------------------------------------------------------------

by fabpot at 2012-06-30T20:17:48Z

Can you squash you commits? Thanks.

---------------------------------------------------------------------------

by lstrojny at 2012-06-30T20:25:32Z

Done

---------------------------------------------------------------------------

by fabpot at 2012-06-30T20:25:42Z

Can you also add a note somewhere in the documentation (probably in `including-the-files.rst`) that when using PHP >= 5.3.0, this is the recommended class to use (with a snippet of code on how to configure it)?

---------------------------------------------------------------------------

by lukaswoj at 2012-07-01T03:50:08Z

Hi there
I'm so happy to find out this PR. Looks like really fresh topic.
Thank you Lars

I was just struggling with the same problem of high CPU usage when sending many emails in bulk.
In my case the problem came from the fact that emails needs to be dispatched at given moment in time and the amount of them makes whole process taking too long.
Since I knew in advance what will be the content of each email - I even had the idea to asynchronously preencode body of the messages and later pass already encoded string.
But now this does not matter anymore.

Great job!

Thanks again
Cheers

---------------------------------------------------------------------------

by lstrojny at 2012-07-01T13:37:38Z

Documentation added. @lukaswoj glad we could help you.
e85176b
@fabpot fabpot merged commit ed0312b into swiftmailer:master
@fabpot
Owner

merged, thanks!

@c2h5oh

Sweet, email batch that used to take over 4 hours got sent in just under 20 minutes. Why don't we make it default for PHP 5.3.0+ ?

<?php
...

if (version_compare(PHP_VERSION, '5.3.0') >= 0) {
    Swift_DependencyContainer::getInstance()
        ->register('mime.qpcontentencoder')
        ->asAliasOf('mime.nativeqpcontentencoder');
}
@fabpot
Owner

@c2h5oh That's indeed a good idea. We might want to get some more feedback from people like you before switching it to be the default. I've opened a ticket for the change (see #220).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jun 30, 2012
  1. @lstrojny
Commits on Jul 1, 2012
  1. @lstrojny

    Adding doc hint

    lstrojny authored
This page is out of date. Refresh to see the latest.
View
11 doc/including-the-files.rst
@@ -28,3 +28,14 @@ To use Swift Mailer's autoloader:
require_once '/path/to/swift-mailer/lib/swift_required.php';
/* rest of code goes here */
+
+For PHP versions starting with 5.3 it is recommended using the native quoted
+printable encoder. It uses PHP’s native ``quoted_printable_encode()``-function
+to achieve much better performance. To do so, edit ``lib/swift_init.php`` and
+add the following line:
+
+.. code-block:: php
+
+ Swift_DependencyContainer::getInstance()
+ ->register('mime.qpcontentencoder')
+ ->asAliasOf('mime.nativeqpcontentencoder');
View
58 lib/classes/Swift/Mime/ContentEncoder/NativeQpContentEncoder.php
@@ -0,0 +1,58 @@
+<?php
+
+class Swift_Mime_ContentEncoder_NativeQpContentEncoder implements Swift_Mime_ContentEncoder
+{
+ /**
+ * Notify this observer that the entity's charset has changed.
+ * @param string $charset
+ */
+ public function charsetChanged($charset)
+ {
+ if ($charset !== 'utf-8') {
+ throw new RuntimeException(
+ sprintf('Charset "%s" not supported. NativeQpContentEncoder only supports "utf-8"', $charset));
+ }
+ }
+
+ /**
+ * Encode $in to $out.
+ * @param Swift_OutputByteStream $os to read from
+ * @param Swift_InputByteStream $is to write to
+ * @param int $firstLineOffset
+ * @param int $maxLineLength - 0 indicates the default length for this encoding
+ */
+ public function encodeByteStream(
+ Swift_OutputByteStream $os, Swift_InputByteStream $is, $firstLineOffset = 0,
+ $maxLineLength = 0)
+ {
+ $string = '';
+
+ while (false !== $bytes = $os->read(8192)) {
+ $string .= $bytes;
+ }
+
+ $is->write($this->encodeString($string));
+ }
+
+ /**
+ * Get the MIME name of this content encoding scheme.
+ * @return string
+ */
+ public function getName()
+ {
+ return 'quoted-printable';
+ }
+
+ /**
+ * Encode a given string to produce an encoded string.
+ * @param string $string
+ * @param int $firstLineOffset if first line needs to be shorter
+ * @param int $maxLineLength - 0 indicates the default length for this encoding
+ * @return string
+ */
+ public function encodeString($string, $firstLineOffset = 0,
+ $maxLineLength = 0)
+ {
+ return quoted_printable_encode($string);
+ }
+}
View
5 lib/dependency_maps/mime_deps.php
@@ -82,6 +82,9 @@
-> register('mime.qpcontentencoder')
-> asNewInstanceOf('Swift_Mime_ContentEncoder_QpContentEncoder')
-> withDependencies(array('mime.charstream', 'mime.bytecanonicalizer'))
+
+ -> register('mime.nativeqpcontentencoder')
+ -> asNewInstanceOf('Swift_Mime_ContentEncoder_NativeQpContentEncoder')
-> register('mime.7bitcontentencoder')
-> asNewInstanceOf('Swift_Mime_ContentEncoder_PlainContentEncoder')
@@ -99,7 +102,7 @@
-> register('mime.rfc2231encoder')
-> asNewInstanceOf('Swift_Encoder_Rfc2231Encoder')
-> withDependencies(array('mime.charstream'))
-
+
;
unset($swift_mime_types);
View
101 tests/acceptance/Swift/Mime/ContentEncoder/NativeQpContentEncoderAcceptanceTest.php
@@ -0,0 +1,101 @@
+<?php
+
+require_once 'Swift/Tests/SwiftUnitTestCase.php';
+require_once 'Swift/Mime/ContentEncoder/NativeQpContentEncoder.php';
+require_once 'Swift/CharacterStream/ArrayCharacterStream.php';
+require_once 'Swift/CharacterReaderFactory/SimpleCharacterReaderFactory.php';
+require_once 'Swift/ByteStream/ArrayByteStream.php';
+
+class Swift_Mime_ContentEncoder_NativeQpContentEncoderAcceptanceTest
+ extends Swift_Tests_SwiftUnitTestCase
+{
+ /**
+ * @var Swift_Mime_ContentEncoder_NativeQpContentEncoder
+ */
+ protected $_encoder;
+
+ public function setUp()
+ {
+ $this->_samplesDir = realpath(dirname(__FILE__) . '/../../../../_samples/charsets');
+ $this->_encoder = new Swift_Mime_ContentEncoder_NativeQpContentEncoder();
+ }
+
+ public function testEncodingAndDecodingSamples()
+ {
+ $sampleFp = opendir($this->_samplesDir);
+ while (false !== $encodingDir = readdir($sampleFp))
+ {
+ if (substr($encodingDir, 0, 1) == '.')
+ {
+ continue;
+ }
+
+ $sampleDir = $this->_samplesDir . '/' . $encodingDir;
+
+ if (is_dir($sampleDir))
+ {
+
+ $fileFp = opendir($sampleDir);
+ while (false !== $sampleFile = readdir($fileFp))
+ {
+ if (substr($sampleFile, 0, 1) == '.')
+ {
+ continue;
+ }
+
+ $text = file_get_contents($sampleDir . '/' . $sampleFile);
+
+ $os = new Swift_ByteStream_ArrayByteStream();
+ $os->write($text);
+
+ $is = new Swift_ByteStream_ArrayByteStream();
+ $this->_encoder->encodeByteStream($os, $is);
+
+ $encoded = '';
+ while (false !== $bytes = $is->read(8192))
+ {
+ $encoded .= $bytes;
+ }
+
+ $this->assertEqual(
+ quoted_printable_decode($encoded), $text,
+ '%s: Encoded string should decode back to original string for sample ' .
+ $sampleDir . '/' . $sampleFile
+ );
+ }
+ closedir($fileFp);
+ }
+
+ }
+ closedir($sampleFp);
+
+ }
+
+ public function testEncodingAndDecodingSamplesFromDiConfiguredInstance()
+ {
+ $encoder = $this->_createEncoderFromContainer();
+ $this->assertSame('=C3=A4=C3=B6=C3=BC=C3=9F', $encoder->encodeString('äöüß'));
+ }
+
+ public function testCharsetChangeNotImplemented()
+ {
+ $this->_encoder->charsetChanged('utf-8');
+ $this->expectException(new RuntimeException('Charset "charset" not supported. NativeQpContentEncoder only supports "utf-8"'));
+ $this->_encoder->charsetChanged('charset');
+ }
+
+ public function testGetName()
+ {
+ $this->assertSame('quoted-printable', $this->_encoder->getName());
+ }
+
+ // -- Private Methods
+
+ private function _createEncoderFromContainer()
+ {
+ return Swift_DependencyContainer::getInstance()
+ ->lookup('mime.nativeqpcontentencoder')
+ ;
+ }
+
+}
Something went wrong with that request. Please try again.