Fixing character encoding issues

The current version of the Twilio PHP library will take input passed in and run
it through the PHP `htmlentities` function. This produces "named" character
entities in the output. Named character entities and XML don't get along. Only
the 'quot', 'amp', 'apos', 'lt', and 'gt' entities are defined.

This change replaces the `htmlentities` call with an process that decodes the
input and then runs it through `htmlspecialchars` to produce "numeric" entities
instead.

This should accommodate most western character sets. Non standard (to php's
default htmlspecialchars) multi-byte character sets will fail silently in Twilio
(ie: say nothing) instead of throwing an application error.

README.md

@@ -125,7 +125,7 @@ Library Documentation")
 
 ## Prerequisites
 
-* PHP >= 5.2.1
+* PHP >= 5.2.3
 * The PHP JSON extension
 
 ## Reporting Issues

Services/Twilio/Twiml.php

@@ -92,15 +92,13 @@ public function __call($verb, array $args)
          * want to break their existing code by turning their &'s into 
          * &
          *
-         * So we end up with the following matrix:
+         * We also want to use numeric entities, not named entities so that we
+         * are fully compatible with XML
          *
-         * We want & to turn into & before passing to addChild
-         * We want & to stay as & before passing to addChild
-         *
-         * The following line accomplishes the desired behavior.
+         * The following lines accomplishes the desired behavior.
          */
-        $normalized = htmlentities($noun, null, null, false);
-        //then escape it again
+        $decoded = html_entity_decode($noun, ENT_COMPAT, 'UTF-8');
+        $normalized = htmlspecialchars($decoded, ENT_COMPAT, 'UTF-8', false);
         $child = empty($noun)
             ? $this->element->addChild(ucfirst($verb))
             : $this->element->addChild(ucfirst($verb), $normalized);

tests/TwimlTest.php

@@ -47,6 +47,30 @@ public function testSayConvienceMethod() {
         $this->assertXmlStringEqualsXmlString($expected, $r);
     }
 
+    public function testSayUTF8() {
+        $r = new Services_Twilio_Twiml();
+        $r->say("é tü & må");
+        $expected = '<Response><Say>'
+            . '&#xE9; t&#xFC; &amp; m&#xE5;</Say></Response>';
+        $this->assertXmlStringEqualsXmlString($expected, $r);
+    }
+
+    public function testSayNamedEntities() {
+        $r = new Services_Twilio_Twiml();
+        $r->say("&eacute; t&uuml; &amp; m&aring;");
+        $expected = '<Response><Say>'
+            . '&#xE9; t&#xFC; &amp; m&#xE5;</Say></Response>';
+        $this->assertXmlStringEqualsXmlString($expected, $r);
+    }
+
+    public function testSayNumericEntities() {
+        $r = new Services_Twilio_Twiml();
+        $r->say("&#xE9; t&#xFC; &amp; m&#xE5;");
+        $expected = '<Response><Say>'
+            . '&#xE9; t&#xFC; &amp; m&#xE5;</Say></Response>';
+        $this->assertXmlStringEqualsXmlString($expected, $r);
+    }
+
     public function testPlayBasic() {   
         $r = new Services_Twilio_Twiml();
         $r->play("hello-monkey.mp3");