Unicode characters in pkginfo description may cause package modification notification emails to fail

[mholttech]

I'm getting an error 500 when I copy/paste a pkginfo file. Uploading the file works perfectly

Here is the log:

2015-10-15 20:34:48.592 /admin/package 500 261ms 0kb instance=0 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36 module=default version=a56a3dc98d87
100.43.220.122 - mholt [15/Oct/2015:20:34:48 -0700] "POST /admin/package HTTP/1.1" 500 225 - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36" "rlc-simian.appspot.com" ms=261 cpu_ms=53 cpm_usd=0.000025 instance=0 app_engine_release=1.9.27 
W 2015-10-15 20:34:48.418
No admins defined! Configure admins in Admin Tools -> ACL Groups.
W 2015-10-15 20:34:48.446
No admins defined! Configure admins in Admin Tools -> ACL Groups.
E 2015-10-15 20:34:48.590
'ascii' codec can't encode character u'\xae' in position 726: ordinal not in range(128)
Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/apps/s~rlc-simian/a56a3dc98d87.387476931731189739/simian/mac/admin/__init__.py", line 153, in handle_exception
    super(AdminHandler, self).handle_exception(exception, debug_mode)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~rlc-simian/a56a3dc98d87.387476931731189739/simian/mac/admin/package.py", line 170, in post
    self.UpdatePackageInfoFromPlist(create_new=True)
  File "/base/data/home/apps/s~rlc-simian/a56a3dc98d87.387476931731189739/simian/mac/admin/package.py", line 380, in UpdatePackageInfoFromPlist
    self.NotifyAdminsOfPackageChangeFromPlist(plist_xml)
  File "/base/data/home/apps/s~rlc-simian/a56a3dc98d87.387476931731189739/simian/mac/admin/package.py", line 283, in NotifyAdminsOfPackageChangeFromPlist
    main_body = str(plist.GetXml(indent_num=2))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xae' in position 726: ordinal not in range(128)

Here is the plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>_metadata</key>
    <dict>
        <key>created_by</key>
        <string>admin</string>
        <key>creation_date</key>
        <date>2015-10-16T02:58:46Z</date>
        <key>munki_version</key>
        <string>2.3.1.2535</string>
        <key>os_version</key>
        <string>10.10.3</string>
    </dict>
    <key>autoremove</key>
    <false/>
    <key>catalogs</key>
    <array>
        <string>unstable</string>
    </array>
    <key>description</key>
    <string>Adobe® Flash® Player is a cross-platform browser-based application runtime that delivers uncompromised viewing of expressive applications, content, and videos across screens and browsers.</string>
    <key>display_name</key>
    <string>Adobe Flash Player</string>
    <key>installed_size</key>
    <integer>17140</integer>
    <key>installer_item_hash</key>
    <string>b5eca33f72b1b419a7f9754b53cc8780b03e351520f189a43986a9e1f2aa22fa</string>
    <key>installer_item_location</key>
    <string>AdobeFlashPlayer-19.0.0.207.dmg</string>
    <key>installer_item_size</key>
    <integer>16364</integer>
    <key>minimum_os_version</key>
    <string>10.5.0</string>
    <key>name</key>
    <string>AdobeFlashPlayer</string>
    <key>package_path</key>
    <string>Install Adobe Flash Player.app/Contents/Resources/Adobe Flash Player.pkg</string>
    <key>receipts</key>
    <array>
        <dict>
            <key>installed_size</key>
            <integer>17140</integer>
            <key>packageid</key>
            <string>com.adobe.pkg.FlashPlayer</string>
            <key>version</key>
            <string>19.0.0.207</string>
        </dict>
    </array>
    <key>unattended_install</key>
    <true/>
    <key>uninstall_method</key>
    <string>removepackages</string>
    <key>uninstallable</key>
    <true/>
    <key>version</key>
    <string>19.0.0.207</string>
</dict>
</plist>

[mholttech]

forgot to mention as well that even though I get the error 500, the new package is created in Simian

[juofmgoblue]

Thanks for the report, stacktrace, and offending plist!

The package is still uploaded because all that is failing is the method call to email admins about the new upload: https://github.com/google/simian/blob/master/src/simian/mac/admin/package.py#L380

If I had to quickly guess, I'd say the issue is the unicode registered trademark symbols in: Adobe® Flash®

Can you try removing those to see if that uploads as expected?

[johnrandolph]

yeah, I think we'll need a bug fix there to either handle unicode properly
when sending the email, or do a str.encode('ascii', 'ignore') from the
input package to drop the unicode.

obviously the latter is a bad solution to varying degrees, losing unicode
(R) is one thing, losing an entire non-latin alphabet description or
something isn't really an option.

On Fri, Oct 16, 2015 at 8:53 AM, Justin McWilliams <notifications@github.com

wrote:

Thanks for the report, stacktrace, and offending plist!

The package is still uploaded because all that is failing is the method
call to email admins about the new upload:
https://github.com/google/simian/blob/master/src/simian/mac/admin/package.py#L380

If I had to quickly guess, I'd say the issue is the unicode registered
trademark symbols in: Adobe® Flash®

Can you try removing those to see if that uploads as expected?

—
Reply to this email directly or view it on GitHub
#31 (comment).

[mholttech]

I just tested without the ® and it did resolve the 500 error.

[juofmgoblue]

Thanks for the confirmation, this seems like a relatively straightforward fix then.