Revise section "argument conversion" (#557)

* Remove redundant parenthesis.

modified:   src/ZPublisher/HTTPRequest.py

* Bring field2boolean under test.

... and finally make it easier readable.

modified:   src/ZPublisher/Converters.py
modified:   src/ZPublisher/tests/test_Converters.py

* Bring field2int under test.

modified:   src/ZPublisher/Converters.py
modified:   src/ZPublisher/tests/test_Converters.py

* Bring field2long under test.

modified:   src/ZPublisher/tests/test_Converters.py

* Fix test order.

modified:   src/ZPublisher/tests/test_Converters.py

* Bring field2float under test.

modified:   src/ZPublisher/tests/test_Converters.py

* Format docstring.

modified:   src/ZPublisher/Converters.py

* Bring field2required under test.

modified:   src/ZPublisher/tests/test_Converters.py

* Improve coverage for field2date_international.

modified:   src/ZPublisher/tests/test_Converters.py

* Bring field2text under test.

modified:   src/ZPublisher/tests/test_Converters.py

* Revise subsection "Argument Conversion".

modified:   docs/zdgbook/ObjectPublishing.rst

docs/zdgbook/ObjectPublishing.rst

@@ -662,70 +662,87 @@ Argument Conversion
 The publisher supports argument conversion. For example consider this
 function::
 
-        def onethird(number):
-            "returns the number divided by three"
+        def one_third(number):
+            """returns the number divided by three"""
             return number / 3.0
 
 This function cannot be called from the web because by default the
 publisher marshals arguments into strings, not numbers. This is why
 the publisher provides a number of converters. To signal an argument
 conversion you name your form variables with a colon followed by a
-type conversion code. For example, to call the above function with 66
-as the argument you can use this URL *onethird?number:int=66* The
-publisher supports many converters:
+type conversion code.
 
-- boolean -- Converts a variable to true or false. Variables that are
-  0, None, an empty string, or an empty sequence are false, all others
-  are true.
+For example, to call the above function with 66 as the argument you
+can use this URL ``one_third?number:int=66`` The publisher supports
+many converters:
 
-- int -- Converts a variable to a Python integer.
+- **boolean** -- Converts a variable to ``True`` or ``False``.
+  Variables that are  0, None, an empty string, or an empty sequence
+  are ``False``, all others are ``True``.
 
-- long -- Converts a variable to a Python long integer.
+- **int** -- Converts a variable to a Python integer. Also converts a
+  list/tuple of variables to a list/tuple of integers.
 
-- float -- Converts a variable to a Python floating point number.
+- **long** -- Converts a variable to a Python integer. Strips the
+  trailing "L" symbol at the end of the value. Also converts a
+  list/tuple of variables to a list/tuple of integers.
 
-- string -- Converts a variable to a Python string.
+- **float** -- Converts a variable to a Python floating point number.
+  Also converts a list/tuple of variables to a list/tuple of floats.
 
-- ustring -- Converts a variable to a Python unicode string.
+- **string** -- Converts a variable to a native string. So the result
+  is ``str``, no matter which Python version you are on.
 
-- required -- Raises an exception if the variable is not present or
+- **ustring** -- Converts a variable to a Python unicode string.
+
+- **bytes** -- Converts a variable to a Python bytes object/string.
+
+- **required** -- Raises an exception if the variable is not present or
   is an empty string.
 
-- ignore_empty -- Excludes a variable from the request if the
+- **ignore_empty** -- Excludes a variable from the request if the
   variable is an empty string.
 
-- date -- Converts a string to a *DateTime* object. The formats
-  accepted are fairly flexible, for example '10/16/2000', '12:01:13
-  pm'.
+- **date** -- Converts a string to a **DateTime** object. The formats
+  accepted are fairly flexible, for example ``10/16/2000``, ``12:01:13
+  pm``.
+
+- **date_international** -- Converts a string to a **DateTime** object,
+  but especially treats ambiguous dates as "days before month before
+  year". This useful if you need to parse non-US dates.
 
-- list -- Converts a variable to a Python list of values, even if
+- **list** -- Converts a variable to a Python list of values, even if
   there is only one value.
 
-- tuple -- Converts a variable to a Python tuple of values, even if
+- **tuple** -- Converts a variable to a Python tuple of values, even if
   there is only one value.
 
-- lines -- Converts a string to a Python list of values by splitting
-  the string on line breaks.
+- **lines** -- Converts a variable to a Python list of native strings
+  by splitting the string on line breaks. Also converts list/tuple of
+  variables to list/tuple of native strings.
 
-- tokens -- Converts a string to a Python list of values by splitting
-  the string on spaces.
+- **tokens** -- Converts a variable to a Python list of native strings
+  by splitting the variable on spaces.
 
-- text -- Converts a variable to a string with normalized line
+- **text** -- Converts a variable to a native string with normalized line
   breaks. Different browsers on various platforms encode line
   endings differently, so this converter makes sure the line endings
   are consistent, regardless of how they were encoded by the browser.
 
-- ulines, utokens, utext -- like lines, tokens, text, but using
-  unicode strings instead of plain strings.
+- **ulines**, **utokens**, **utext** -- like **lines**, **tokens**,
+  **text**, but always converts into unicode strings.
 
 If the publisher cannot coerce a request variable into the type
 required by the type converter it will raise an error. This is useful
 for simple applications, but restricts your ability to tailor error
 messages. If you wish to provide your own error messages, you should
 convert arguments manually in your published objects rather than
-relying on the publisher for coercion. Another possibility is to use
-JavaScript to validate input on the client-side before it is submitted
-to the server.
+relying on the publisher for coercion.
+
+.. note::
+  Client-side validation with HTML 5 and/or JavaScript may improve
+  the usability of the application, but it is never a replacement for
+  server side validation.
 
 You can combine type converters to a limited extent. For example you
 could create a list of integers like so::
@@ -734,8 +751,9 @@ could create a list of integers like so::
         <input type="checkbox" name="numbers:list:int" value="2">
         <input type="checkbox" name="numbers:list:int" value="3">
 
-In addition to these type converters, the publisher also supports
-method and record arguments.
+In addition to the mentioned type converters, the publisher also supports
+both method and record arguments and specifying character encodings.
+
 
 Character Encodings for Arguments
 ---------------------------------

src/ZPublisher/Converters.py

@@ -31,8 +31,10 @@
 
 
 def field2string(v):
-    """Converts value to native strings (so always to `str` no matter which
-    python version you are on)"""
+    """Converts value to native strings.
+
+    So always to `str` no matter which Python version you are on.
+    """
     if hasattr(v, 'read'):
         return v.read()
     elif six.PY2 and isinstance(v, text_type):
@@ -91,7 +93,9 @@ def field2int(v):
             return int(v)
         except ValueError:
             raise ValueError(
-                "An integer was expected in the value %r" % escape(v, True)
+                "An integer was expected in the value %r" % escape(
+                    v, quote=True
+                )
             )
     raise ValueError('Empty entry when <strong>integer</strong> expected')
 
@@ -163,8 +167,8 @@ def field2date_international(v):
 
 def field2boolean(v):
     if v == 'False':
-        return not 1
-    return not not v
+        return False
+    return bool(v)
 
 
 class _unicode_converter(object):

src/ZPublisher/HTTPRequest.py

@@ -599,7 +599,7 @@ def processInputs(
                         elif type_name == 'tuple':
                             tuple_items[key] = 1
                             flags = flags | SEQUENCE
-                        elif (type_name == 'method' or type_name == 'action'):
+                        elif type_name == 'method' or type_name == 'action':
                             if delim:
                                 meth = key
                             else:

src/ZPublisher/tests/test_Converters.py

@@ -13,11 +13,135 @@
 
 import unittest
 
+from six import PY2
+from six import PY3
 from six import text_type
 
 
 class ConvertersTests(unittest.TestCase):
 
+    def test_field2boolean_with_empty_string(self):
+        from ZPublisher.Converters import field2boolean
+        to_convert = u''
+        expected = False
+        self.assertEqual(field2boolean(to_convert), expected)
+
+    def test_field2boolean_with_False_as_string(self):
+        from ZPublisher.Converters import field2boolean
+        to_convert = u'False'
+        expected = False
+        self.assertEqual(field2boolean(to_convert), expected)
+
+    def test_field2boolean_with_some_string(self):
+        from ZPublisher.Converters import field2boolean
+        to_convert = u'to_convert'
+        expected = True
+        self.assertEqual(field2boolean(to_convert), expected)
+
+    def test_field2boolean_with_positive_int(self):
+        from ZPublisher.Converters import field2boolean
+        to_convert = 1
+        expected = True
+        self.assertEqual(field2boolean(to_convert), expected)
+
+    def test_field2boolean_with_zero(self):
+        from ZPublisher.Converters import field2boolean
+        to_convert = 0
+        expected = False
+        self.assertEqual(field2boolean(to_convert), expected)
+
+    def test_field2boolean_with_emtpy_list(self):
+        from ZPublisher.Converters import field2boolean
+        to_convert = []
+        expected = False
+        self.assertEqual(field2boolean(to_convert), expected)
+
+    def test_field2float_with_list_of_numbers(self):
+        from ZPublisher.Converters import field2float
+        to_convert = ["1.1", "2.2", "3.3"]
+        expected = [1.1, 2.2, 3.3]
+        rv = field2float(to_convert)
+        self.assertEqual(rv[0], expected[0])
+        self.assertEqual(rv[1], expected[1])
+        self.assertEqual(rv[2], expected[2])
+
+    def test_field2float_with_regular_number(self):
+        from ZPublisher.Converters import field2float
+        to_convert = "1"
+        expected = 1.0
+        self.assertAlmostEqual(field2float(to_convert), expected)
+
+    def test_field2float_with_illegal_value(self):
+        from ZPublisher.Converters import field2float
+        to_convert = "<"
+        self.assertRaises(ValueError, field2float, to_convert)
+
+    def test_field2float_with_empty_value(self):
+        from ZPublisher.Converters import field2float
+        to_convert = ""
+        self.assertRaises(ValueError, field2float, to_convert)
+
+    def test_field2int_with_list_of_numbers(self):
+        from ZPublisher.Converters import field2int
+        to_convert = ["1", "2", "3"]
+        expected = [1, 2, 3]
+        self.assertEqual(field2int(to_convert), expected)
+
+    def test_field2int_with_regular_number(self):
+        from ZPublisher.Converters import field2int
+        to_convert = "1"
+        expected = 1
+        self.assertEqual(field2int(to_convert), expected)
+
+    def test_field2int_with_illegal_value(self):
+        from ZPublisher.Converters import field2int
+        to_convert = "<"
+        self.assertRaises(ValueError, field2int, to_convert)
+
+    def test_field2int_with_empty_value(self):
+        from ZPublisher.Converters import field2int
+        to_convert = ""
+        self.assertRaises(ValueError, field2int, to_convert)
+
+    def test_field2long_with_list_of_numbers(self):
+        from ZPublisher.Converters import field2long
+        to_convert = ["1", "2", "3"]
+        expected = [1, 2, 3]
+        self.assertEqual(field2long(to_convert), expected)
+
+    def test_field2long_with_regular_number(self):
+        from ZPublisher.Converters import field2long
+        to_convert = "1"
+        expected = 1
+        self.assertEqual(field2long(to_convert), expected)
+
+    def test_field2long_with_illegal_value(self):
+        from ZPublisher.Converters import field2long
+        to_convert = "<"
+        self.assertRaises(ValueError, field2long, to_convert)
+
+    def test_field2long_with_empty_value(self):
+        from ZPublisher.Converters import field2long
+        to_convert = ""
+        self.assertRaises(ValueError, field2long, to_convert)
+
+    def test_field2long_strips_trailing_long_symbol(self):
+        from ZPublisher.Converters import field2long
+        to_convert = "2L"
+        expected = 2
+        self.assertEqual(field2long(to_convert), expected)
+
+    def test_field2required_returns_string(self):
+        from ZPublisher.Converters import field2required
+        to_convert = "to_convert"
+        expected = "to_convert"
+        self.assertEqual(field2required(to_convert), expected)
+
+    def test_field2required_raises_ValueError(self):
+        from ZPublisher.Converters import field2required
+        value = ""
+        self.assertRaises(ValueError, field2required, value)
+
     def test_field2string_with_string(self):
         from ZPublisher.Converters import field2string
         to_convert = 'to_convert'
@@ -49,6 +173,13 @@ def test_field2bytes_with_text(self):
         expected = b'to_convert'
         self.assertEqual(field2bytes(to_convert), expected)
 
+    def test_field2date_international_with_proper_date_string(self):
+        from ZPublisher.Converters import field2date_international
+        to_convert = "2.1.2019"
+        from DateTime import DateTime
+        expected = DateTime(2019, 1, 2)
+        self.assertEqual(field2date_international(to_convert), expected)
+
     def test_field2lines_with_list(self):
         from ZPublisher.Converters import field2lines
         to_convert = ['one', b'two']
@@ -78,6 +209,16 @@ def test_field2lines_with_string_with_newlines(self):
         expected = [b'abc', b'def', b'ghi']
         self.assertEqual(field2lines(to_convert), expected)
 
+    def test_field2text_with_string_with_newlines(self):
+        from ZPublisher.Converters import field2text
+        to_convert = 'abc\r\ndef\r\nghi'
+        if PY3:
+            expected = 'abc\ndef\nghi'
+            self.assertEqual(field2text(to_convert), expected)
+        if PY2:
+            expected = b'abc\ndef\nghi'
+            self.assertEqual(field2text(to_convert), expected)
+
     def test_field2ulines_with_list(self):
         from ZPublisher.Converters import field2ulines
         to_convert = [u'one', 'two']