BN's Python plugin breaks in interesting ways if Windows user directory contains non-ASCII characters

[alexrp]

Version and Platform (required):

• Binary Ninja Version: 3.5.4402-dev Personal (c2f291a7)
• OS: Windows
• OS Version: 10.0.23506.1000
• CPU Architecture: x64

Bug Description:
I just reinstalled Windows after some hardware upgrades. For whatever reason, Windows decided to make my user directory C:\Users\Alex Rønne Petersen instead of C:\Users\alex this time around (I suspect because I made the account locally before connecting it to a Microsoft account). That ø, encoded as UTF-16 (0xf8), seems to be wreaking all kinds of havoc on BN's Python plugin, which tries to decode it as UTF-8.

During startup:

Traceback (most recent call last):
  File "C:\Program Files\Vector35\BinaryNinja\plugins\..\python\binaryninja\scriptingprovider.py", line 414, in _create_instance
    assert self.__class__.instance_class is not None
           ^^^^^^^^^^^^^^
AttributeError: 'PythonScriptingProvider' object has no attribute '__class__'

Trying to install a plugin:

Exception ignored on calling ctypes callback function: <bound method PythonScriptingProvider._install_modules of <binaryninja.scriptingprovider.PythonScriptingProvider object at 0x000001704C50DB50>>
Traceback (most recent call last):
  File "C:\Program Files\Vector35\BinaryNinja\plugins\..\python\binaryninja\scriptingprovider.py", line 1413, in _install_modules
    user_dir = binaryninja.user_directory()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Vector35\BinaryNinja\plugins\..\python\binaryninja\__init__.py", line 213, in user_directory
    return core.BNGetUserDirectory()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Vector35\BinaryNinja\plugins\..\python\binaryninja\_binaryninjacore.py", line 32717, in BNGetUserDirectory
    string = str(pyNativeStr(ctypes.cast(result, ctypes.c_char_p).value))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Vector35\BinaryNinja\plugins\..\python\binaryninja\_binaryninjacore.py", line 36, in pyNativeStr
    return arg.decode('utf8')
           ^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf8 in position 15: invalid start byte

Steps To Reproduce:

1. Create a Windows user account containing non-ASCII characters in its name.
2. Try to use BN from that account.

Expected Behavior:
Things should work.

[xusheng6]

Related to: #4460

[alexrp]

I don't suppose there's a way I can relocate my BN user directory until this is fixed? If not, all that's left are not-very-appealing options for working around this... 🤔

[CouleeApps]

You can set the environment variable BN_USER_DIRECTORY to an alternative path that is writable, which may mitigate this.

[alexrp]

Thanks, that mostly works. Update settings seem to break, though:

Error changing update settings: Unable to write to cache (object manifest)
Error changing update settings: Unable to write to cache (object manifest)
Error changing update settings: Unable to write to cache (object manifest)

(This happens when changing to the dev channel.)

[CouleeApps]

That's not surprising, considering the update cache uses a different directory than the normal user directory. I'm not sure there's an easy solution to that right now, but it should only impact the auto updater.

[alexrp]

What does that actually mean in terms of my ability to update to the latest dev builds though? E.g. right now I only see the current build I'm on, so it makes me worry that I just might not be able to update at all:

[psifertex]

That just looks like you have selected the stable channel instead of dev.

Ahh, just reread the previous comments. We'll need to change the update mechanism to resolve the bug or at least have it follow the environment variable.

In the meantime, https://binary.ninja/recover/ is your best bet for updates to the latest dev.

[alexrp]

Hmm, is there a changelog for the dev channel anywhere? I can't see that in BN's welcome page either, so I actually have no indication of when there's a new dev build.

[psifertex]

The changes are available in the welcome page if you're on the development build. Or you can access it via the API directly:

https://github.com/Vector35/binaryninja-api/blob/dev/python/examples/version_switcher.py

[xusheng6]

Related to #4460

[xusheng6]

@alexrp I pushed the fix to dev 3.5.4420 which should be available in a few hours. Please give it a try and see it it works!

[alexrp]

@xusheng6 just updated and moved my BN user directory back under C:\Users\Alex Rønne Petersen, and all seems to be working well!