[OC 10.0][OS X] libsync is able to download files with 4 byte-unicode names but upload etc

[SamuAlfageme]

4 byte-unicode (a.k.a. emoji) filenames are supported in ownCloud 10.0 (owncloud/core#4513)

Expected behavior

The client should be able to handle this filenames as long as the underlying filesystem supports them as well.

Actual behavior

• Sync a file/folder with an emoji filename from the server into the desktop client (v2.3.1)
• Upload a file/folder to the server with a 4byte-unicode name prompts this error
• Delete/rename the file/folder on the server, the client doesn't keep up with these changes

Solving this will be enough to close some old issues:

• Bad file name renaming #4811
• [macOS] Emojis in file name fatally interrupts complete sync #5231

[guruz]

Note that this might be macOS specific because we have to do file name mangling on macOS https://github.com/owncloud/client/search?utf8=%E2%9C%93&q=normalization&type= .

How does it look on other OS?

@michaelstingl this would mandate an earlier 2.3.2

[SamuAlfageme]

@guruz you're right, I assumed the name-conversion error was platform-independent but just tried with Win10 and no flags were raised there. My bad 😅

[michaelstingl]

@michaelstingl this would mandate an earlier 2.3.2

Works for me

[ogoffart]

I bet that's because iconv can't convert the string on macOS (cf. #2777 )

i.e: c_utf8_from_locale returns a null string in csync_vio_local_readdir. (because the iconv call itself returns null in c_iconv)

This used to block the whole sync before commit f6cd2c7, but now such file are simply ignored.

So I believe this is a bug in iconv. Maybe we simply need to update iconv.
Alternatively, we should not use iconv at all and use a callback from the Qt code to convert from "UTF-8-MAC", whatever that is. (i guess that's NFD vs. NFC normalisation)

[guruz]

Maybe we simply need to update iconv.

We use the system iconv.

@ogoffart Any proposal for your Qt code?
Alternatively 2: Instead of using POSIX/csync _treaddir we could use something from Qt for reading directories.

[guruz]

maybe https://doc.qt.io/qt-5/qfile.html#decodeName-1 ..

[guruz]

This is messy. I'm trying a few things but I think this is too invasive for a patch release.

[guruz]

@michaelstingl @pmaier1 Moving to 2.4, the needed patch is too invasive

[SamuAlfageme]

👍