Skip to content
This repository

Use quoted_printable_encode() if available for performance reasons #217

Merged
merged 2 commits into from almost 2 years ago

4 participants

Lars Strojny Fabien Potencier Łukasz Wojciechowski Maciej Lisiewski
Lars Strojny

If we have a PHP version that offers quoted_printable_encode() we should it as it is much faster.

(Pull request courtesy of @maxbeutel and myself)

Fabien Potencier
Owner
fabpot commented June 25, 2012

It breaks a bunch of tests.

Lars Strojny

Would it maybe be better to have NativeQpEncoder and leave the userland implementation untouched?

Fabien Potencier
Owner
fabpot commented June 25, 2012

This is probably a better idea.

Lars Strojny

OK, here is a standalone implementation. We measured 30x (down from 100 mails in 30 seconds to 100 mails in 1 second) more throughput with the native function compared to the userland implementation. And here is how to use it (swift_init.php):

<?php
...
Swift_DependencyContainer::getInstance()
    ->register('mime.qpcontentencoder')
    ->asAliasOf('mime.nativeqpcontentencoder');

Lars Strojny

From my POV this pull request is ready to be merged. Comments?

Fabien Potencier
Owner
fabpot commented June 30, 2012

Can you squash you commits? Thanks.

Lars Strojny

Done

Fabien Potencier
Owner
fabpot commented June 30, 2012

Can you also add a note somewhere in the documentation (probably in including-the-files.rst) that when using PHP >= 5.3.0, this is the recommended class to use (with a snippet of code on how to configure it)?

Łukasz Wojciechowski

Hi there
I'm so happy to find out this PR. Looks like really fresh topic.
Thank you Lars

I was just struggling with the same problem of high CPU usage when sending many emails in bulk.
In my case the problem came from the fact that emails needs to be dispatched at given moment in time and the amount of them makes whole process taking too long.
Since I knew in advance what will be the content of each email - I even had the idea to asynchronously preencode body of the messages and later pass already encoded string.
But now this does not matter anymore.

Great job!

Thanks again
Cheers

Lars Strojny

Documentation added. @lukaswoj glad we could help you.

Fabien Potencier fabpot referenced this pull request from a commit July 01, 2012
Fabien Potencier merged branch InterNations/native-quoted-printable-encode (PR #217)
Commits
-------

ed0312b Adding doc hint
449117d Native quoted printable content encoder

Discussion
----------

Use quoted_printable_encode() if available for performance reasons

If we have a PHP version that offers quoted_printable_encode() we should it as it is much faster.

(Pull request courtesy of @maxbeutel and myself)

---------------------------------------------------------------------------

by fabpot at 2012-06-25T19:37:32Z

It breaks a bunch of tests.

---------------------------------------------------------------------------

by lstrojny at 2012-06-25T19:49:29Z

Would it maybe be better to have NativeQpEncoder and leave the userland implementation untouched?

---------------------------------------------------------------------------

by fabpot at 2012-06-26T05:06:13Z

This is probably a better idea.

---------------------------------------------------------------------------

by lstrojny at 2012-06-26T09:36:20Z

OK, here is a standalone implementation. We measured 30x (down from 100 mails in 30 seconds to 100 mails in 1 second) more throughput with the native function compared to the userland implementation. And here is how to use it (`swift_init.php`):

```php
<?php
...
Swift_DependencyContainer::getInstance()
    ->register('mime.qpcontentencoder')
    ->asAliasOf('mime.nativeqpcontentencoder');

```

---------------------------------------------------------------------------

by lstrojny at 2012-06-30T20:10:59Z

From my POV this pull request is ready to be merged. Comments?

---------------------------------------------------------------------------

by fabpot at 2012-06-30T20:17:48Z

Can you squash you commits? Thanks.

---------------------------------------------------------------------------

by lstrojny at 2012-06-30T20:25:32Z

Done

---------------------------------------------------------------------------

by fabpot at 2012-06-30T20:25:42Z

Can you also add a note somewhere in the documentation (probably in `including-the-files.rst`) that when using PHP >= 5.3.0, this is the recommended class to use (with a snippet of code on how to configure it)?

---------------------------------------------------------------------------

by lukaswoj at 2012-07-01T03:50:08Z

Hi there
I'm so happy to find out this PR. Looks like really fresh topic.
Thank you Lars

I was just struggling with the same problem of high CPU usage when sending many emails in bulk.
In my case the problem came from the fact that emails needs to be dispatched at given moment in time and the amount of them makes whole process taking too long.
Since I knew in advance what will be the content of each email - I even had the idea to asynchronously preencode body of the messages and later pass already encoded string.
But now this does not matter anymore.

Great job!

Thanks again
Cheers

---------------------------------------------------------------------------

by lstrojny at 2012-07-01T13:37:38Z

Documentation added. @lukaswoj glad we could help you.
e85176b
Fabien Potencier fabpot merged commit ed0312b into from July 01, 2012
Fabien Potencier fabpot closed this July 01, 2012
Fabien Potencier
Owner
fabpot commented July 01, 2012

merged, thanks!

Maciej Lisiewski
c2h5oh commented July 02, 2012

Sweet, email batch that used to take over 4 hours got sent in just under 20 minutes. Why don't we make it default for PHP 5.3.0+ ?

<?php
...

if (version_compare(PHP_VERSION, '5.3.0') >= 0) {
    Swift_DependencyContainer::getInstance()
        ->register('mime.qpcontentencoder')
        ->asAliasOf('mime.nativeqpcontentencoder');
}
Fabien Potencier
Owner
fabpot commented July 02, 2012

@c2h5oh That's indeed a good idea. We might want to get some more feedback from people like you before switching it to be the default. I've opened a ticket for the change (see #220).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 2 unique commits by 1 author.

Jun 30, 2012
Lars Strojny Native quoted printable content encoder 449117d
Jul 01, 2012
Lars Strojny Adding doc hint ed0312b
This page is out of date. Refresh to see the latest.
11  doc/including-the-files.rst
Source Rendered
@@ -28,3 +28,14 @@ To use Swift Mailer's autoloader:
28 28
         require_once '/path/to/swift-mailer/lib/swift_required.php';
29 29
 
30 30
         /* rest of code goes here */
  31
+
  32
+For PHP versions starting with 5.3 it is recommended using the native quoted
  33
+printable encoder. It uses PHP’s native ``quoted_printable_encode()``-function
  34
+to achieve much better performance. To do so, edit ``lib/swift_init.php`` and
  35
+add the following line:
  36
+
  37
+.. code-block:: php
  38
+
  39
+   Swift_DependencyContainer::getInstance()
  40
+       ->register('mime.qpcontentencoder')
  41
+       ->asAliasOf('mime.nativeqpcontentencoder');
58  lib/classes/Swift/Mime/ContentEncoder/NativeQpContentEncoder.php
... ...
@@ -0,0 +1,58 @@
  1
+<?php
  2
+
  3
+class Swift_Mime_ContentEncoder_NativeQpContentEncoder implements Swift_Mime_ContentEncoder
  4
+{
  5
+  /**
  6
+   * Notify this observer that the entity's charset has changed.
  7
+   * @param string $charset
  8
+   */
  9
+  public function charsetChanged($charset)
  10
+  {
  11
+    if ($charset !== 'utf-8') {
  12
+      throw new RuntimeException(
  13
+        sprintf('Charset "%s" not supported. NativeQpContentEncoder only supports "utf-8"', $charset));
  14
+    }
  15
+  }
  16
+
  17
+  /**
  18
+   * Encode $in to $out.
  19
+   * @param Swift_OutputByteStream $os to read from
  20
+   * @param Swift_InputByteStream $is to write to
  21
+   * @param int $firstLineOffset
  22
+   * @param int $maxLineLength - 0 indicates the default length for this encoding
  23
+   */
  24
+  public function encodeByteStream(
  25
+    Swift_OutputByteStream $os, Swift_InputByteStream $is, $firstLineOffset = 0,
  26
+    $maxLineLength = 0)
  27
+  {
  28
+    $string = '';
  29
+
  30
+    while (false !== $bytes = $os->read(8192)) {
  31
+      $string .= $bytes;
  32
+    }
  33
+
  34
+    $is->write($this->encodeString($string));
  35
+  }
  36
+
  37
+  /**
  38
+   * Get the MIME name of this content encoding scheme.
  39
+   * @return string
  40
+   */
  41
+  public function getName()
  42
+  {
  43
+    return 'quoted-printable';
  44
+  }
  45
+
  46
+  /**
  47
+   * Encode a given string to produce an encoded string.
  48
+   * @param string $string
  49
+   * @param int $firstLineOffset if first line needs to be shorter
  50
+   * @param int $maxLineLength - 0 indicates the default length for this encoding
  51
+   * @return string
  52
+   */
  53
+  public function encodeString($string, $firstLineOffset = 0,
  54
+                               $maxLineLength = 0)
  55
+  {
  56
+    return quoted_printable_encode($string);
  57
+  }
  58
+}
5  lib/dependency_maps/mime_deps.php
@@ -82,6 +82,9 @@
82 82
   -> register('mime.qpcontentencoder')
83 83
   -> asNewInstanceOf('Swift_Mime_ContentEncoder_QpContentEncoder')
84 84
   -> withDependencies(array('mime.charstream', 'mime.bytecanonicalizer'))
  85
+
  86
+  -> register('mime.nativeqpcontentencoder')
  87
+  -> asNewInstanceOf('Swift_Mime_ContentEncoder_NativeQpContentEncoder')
85 88
   
86 89
   -> register('mime.7bitcontentencoder')
87 90
   -> asNewInstanceOf('Swift_Mime_ContentEncoder_PlainContentEncoder')
@@ -99,7 +102,7 @@
99 102
   -> register('mime.rfc2231encoder')
100 103
   -> asNewInstanceOf('Swift_Encoder_Rfc2231Encoder')
101 104
   -> withDependencies(array('mime.charstream'))
102  
-  
  105
+
103 106
   ;
104 107
   
105 108
 unset($swift_mime_types);
101  tests/acceptance/Swift/Mime/ContentEncoder/NativeQpContentEncoderAcceptanceTest.php
... ...
@@ -0,0 +1,101 @@
  1
+<?php
  2
+
  3
+require_once 'Swift/Tests/SwiftUnitTestCase.php';
  4
+require_once 'Swift/Mime/ContentEncoder/NativeQpContentEncoder.php';
  5
+require_once 'Swift/CharacterStream/ArrayCharacterStream.php';
  6
+require_once 'Swift/CharacterReaderFactory/SimpleCharacterReaderFactory.php';
  7
+require_once 'Swift/ByteStream/ArrayByteStream.php';
  8
+
  9
+class Swift_Mime_ContentEncoder_NativeQpContentEncoderAcceptanceTest
  10
+  extends Swift_Tests_SwiftUnitTestCase
  11
+{
  12
+  /**
  13
+   * @var Swift_Mime_ContentEncoder_NativeQpContentEncoder
  14
+   */
  15
+  protected $_encoder;
  16
+
  17
+  public function setUp()
  18
+  {
  19
+    $this->_samplesDir = realpath(dirname(__FILE__) . '/../../../../_samples/charsets');
  20
+    $this->_encoder = new Swift_Mime_ContentEncoder_NativeQpContentEncoder();
  21
+  }
  22
+  
  23
+  public function testEncodingAndDecodingSamples()
  24
+  {
  25
+    $sampleFp = opendir($this->_samplesDir);
  26
+    while (false !== $encodingDir = readdir($sampleFp))
  27
+    {
  28
+      if (substr($encodingDir, 0, 1) == '.')
  29
+      {
  30
+        continue;
  31
+      }
  32
+
  33
+      $sampleDir = $this->_samplesDir . '/' . $encodingDir;
  34
+
  35
+      if (is_dir($sampleDir))
  36
+      {
  37
+
  38
+        $fileFp = opendir($sampleDir);
  39
+        while (false !== $sampleFile = readdir($fileFp))
  40
+        {
  41
+          if (substr($sampleFile, 0, 1) == '.')
  42
+          {
  43
+            continue;
  44
+          }
  45
+
  46
+          $text = file_get_contents($sampleDir . '/' . $sampleFile);
  47
+
  48
+          $os = new Swift_ByteStream_ArrayByteStream();
  49
+          $os->write($text);
  50
+
  51
+          $is = new Swift_ByteStream_ArrayByteStream();
  52
+          $this->_encoder->encodeByteStream($os, $is);
  53
+
  54
+          $encoded = '';
  55
+          while (false !== $bytes = $is->read(8192))
  56
+          {
  57
+            $encoded .= $bytes;
  58
+          }
  59
+
  60
+          $this->assertEqual(
  61
+            quoted_printable_decode($encoded), $text,
  62
+            '%s: Encoded string should decode back to original string for sample ' .
  63
+            $sampleDir . '/' . $sampleFile
  64
+            );
  65
+        }
  66
+        closedir($fileFp);
  67
+      }
  68
+
  69
+    }
  70
+    closedir($sampleFp);
  71
+
  72
+  }
  73
+  
  74
+  public function testEncodingAndDecodingSamplesFromDiConfiguredInstance()
  75
+  {
  76
+    $encoder = $this->_createEncoderFromContainer();
  77
+    $this->assertSame('=C3=A4=C3=B6=C3=BC=C3=9F', $encoder->encodeString('äöüß'));
  78
+  }
  79
+
  80
+  public function testCharsetChangeNotImplemented()
  81
+  {
  82
+    $this->_encoder->charsetChanged('utf-8');
  83
+    $this->expectException(new RuntimeException('Charset "charset" not supported. NativeQpContentEncoder only supports "utf-8"'));
  84
+    $this->_encoder->charsetChanged('charset');
  85
+  }
  86
+
  87
+  public function testGetName()
  88
+  {
  89
+    $this->assertSame('quoted-printable', $this->_encoder->getName());
  90
+  }
  91
+
  92
+  // -- Private Methods
  93
+  
  94
+  private function _createEncoderFromContainer()
  95
+  {
  96
+    return Swift_DependencyContainer::getInstance()
  97
+      ->lookup('mime.nativeqpcontentencoder')
  98
+      ;
  99
+  }
  100
+  
  101
+}
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.