JEX export corrupts notes on Joplin for Android

[mrjo118]

I've just corrupted my entire note collection (includes a good few hundred notes migrated from Evernote) after making the mistake of relying only on JEX backups and starting a fresh profile on Joplin based on the JEX import. Unfortunately all my backups were made using the Android app and I had only tested the JEX backup of the desktop app for initially deciding to migrate to Joplin and I didn't keep those backups. Doing a file search there's over 4000 hits which I now need to manually replace the characters with by making my best guess what they originally were.

The issue is that JEX backups on the Android app do not correctly store non standard characters and simply replace them with the ? character.

Environment

Joplin version: 2.12.2
Platform: Android
OS specifics: Using Android 10

Steps to reproduce

1. On a blank profile, create a new note which includes pound signs (£) and foreign characters such as é
2. Use the export JEX file facility on the Android app
3. Inspect the contents of the JEX file. Non standard characters and some symbols (and white-space characters) such as pound sounds have been replaced with the ? character

Describe what you expected to happen

Non standard characters should be retained in their original form, as is the case with the desktop app on Windows

Logfile

N/A

[personalizedrefrigerator]

I can confirm this.

This seems to only affect JEX exports made on Android, however — unicode characters in iOS exports work as expected. As such, I suspect it's related to how we're using React Native FS (the tar-stream library is JavaScript, so should behave the same way on both platforms).

[personalizedrefrigerator]

This seems to be an issue with RNFetchBlob.fs.writeFile when given a utf-8 encoding.

For example,

const originalText = 'Test. 🕳️';
await fsDriver.writeFile(filePath, originalText, 'utf-8');

results in

Test. ???

written to filePath.

Here's where we're using RNFetchBlob.fs.writeFile:

joplin/packages/app-mobile/utils/fs-driver-rn.ts

Line 41 in cfb1f11

return RNFetchBlob.fs.writeFile(path, content, encoding);

Interestingly, I'm not seeing this issue reported upstream in RNFetchBlob (search results and query) or in its possibly more-up-to-date fork.

We should be able to work around this by base64-encoding utf-8 data before sending to RNFetchBlob.fs.writeFile.

Work on a fix for this (that needs to be cleaned up) can be found here https://github.com/personalizedrefrigerator/joplin/tree/pr/fix-tar-create

[mrjo118]

Thanks for the quick fix on this. I have since managed to correct all my notes. Just wondering what will determine when this code fix will be released to the Play store?

[mrjo118]

Also, the Android changelog page seems to be broken:
https://joplinapp.org/changelog_android/