Hack for wide characters overflowing the database field (code and db …

…disagree on what a character is due to encoding differences)

pylib/pyc/utils/check_petition_feeds.py

@@ -40,23 +40,21 @@
             guid   = guid,
             defaults = {
                 'council'     : council,
-                'title'       : rss_entry['title'][:200],
+                'title'       : rss_entry['title'][:190],
                 'url'         : rss_entry['link'],
                 'description' : rss_entry['summary'][:1900],  # see note below
                 'pub_date'    : pub_date,
             },
         )
 
-        # Note on the 1900 cap.
+        # Note on the 190/1900 cap.
         #
-        # The database field can store 2000 chars, but the encoding is SQL so it is
-        # actually 2000 bytes. Hence any entry containing wide characters that gets
-        # truncated may be 2000 chars, but > 2000 bytes causing the insert query to
-        # fail.
+        # The title and description database fields can store 200 or 2000 chars,
+        # but the encoding is SQL so it is actually 2000 bytes. Hence any entry
+        # containing wide characters that gets truncated may be 200(0) chars, 
+        # but > 200(0) bytes causing the insert query to fail.
         #
         #   django.db.utils.DatabaseError: value too long for type character varying(2000)
         #
-        # Rather than mess around with the schema to increse the limit (South not 
-        # used in this project), or trim the string to be < 2000 bytes we just trim it
-        # to 1900 chars and hope that will do the trick.
-
+        # We should switch the encoding to utf8, or do some smarter truncating.
+        # But for now just trimming off more than we need to does the trick.