Fixed a bug in Message.toPostArgs() related to UTF-8 encoded values.

In generating the argument dictionary the .toPostArgs() method (apparently)
assumed that values were all Unicode objects and called
``value.encode('utf-8')`` on them unconditionally. However, the values appear
to be a mixed set of Unicode objects and UTF-8 encoded strings (most being of
the latter group).

Calling .encode('utf-8') on a string will implicitly decode the string into a
Unicode object before encoding it to the selected encoding. This automatic
decoding happens using the ``sys.getdefaultencoding()`` encoding which is by
default 'ascii'. The original call therefore works only as long as the values
are 7-bit ASCII and breaks when they contain higher bit characters.

The patch ensures that the resulting values in the returned dictionary are
UTF-8 encoded strings regardless if the input values were Unicode objects or
UTF-8 strings.

Author: dokai

openid/message.py

@@ -277,7 +277,8 @@ def toPostArgs(self):
 
         for (ns_uri, ns_key), value in self.args.iteritems():
             key = self.getKey(ns_uri, ns_key)
-            args[key] = value.encode('UTF-8')
+            # Ensure the resulting value is an UTF-8 encoded bytestring.
+            args[key] = oidutil.toUnicode(value).encode('UTF-8')
 
         return args
 

openid/test/test_message.py

@@ -462,6 +462,20 @@ def test_toPostArgs(self):
                               'xey': 'value',
                               })
 
+    def test_toPostArgs_bug_with_utf8_encoded_values(self):
+        msg = message.Message.fromPostArgs({'openid.mode':'error',
+                                            'openid.error':'unit test',
+                                            'openid.ns':message.OPENID2_NS
+                                             })
+        msg.setArg(message.BARE_NS, 'ünicöde_key', 'ünicöde_välüe')
+        self.failUnlessEqual(msg.toPostArgs(),
+                             {'openid.mode':'error',
+                              'openid.error':'unit test',
+                              'openid.ns':message.OPENID2_NS,
+                              'ünicöde_key': 'ünicöde_välüe',
+                              })
+
+
     def test_toArgs(self):
         # This method can't tolerate BARE_NS.
         self.msg.delArg(message.BARE_NS, "xey")