Python2 unicode

[mrichar1]

This is an aide-memoire for some of the utf-8 related code for python2...

If we add £3.99 to the clipboard, then (with debugging turned on) we see the in-memory buffer holds:

python3: '£3.99' (we can walk away at this point...)

python2: '\xc2\xa33.99'

In both cases, the JSON history file will contain the 'ascii-fied' "\u00a33.99"

However, with python2 when the history file is read back in, it becomes:

python2 u'\xa33.99'

This causes the smart-update and duplicates code to fail, since these will be treated as unequal when compared.

The solution is to always try to decode('utf-8') the selection, ignoring any failures. This will ensure that all comparisons are of u'' objects.

The other solution of course is to just use python3.