-z on macOS have bugs

[haakonstorm]

[ap]

What happens if you add -T utf8?

[haakonstorm]

Yes, that works!

• I'm writing a small (fix-file-names.sh), that's how I end up with all these edge cases. I even managed to find this (I think) small bug related to as close to the edge you can get an edge case (rename gave up) :

Of course, it's playing dirty serving rename a diet of filenames that only consists of the most impossible-to-handle characters: '$' and '/' and '*' and such... but it's apparently perfectly valid file names in Finder. I work as a consultant working exclusively with networked macOS and iOS boxes around town, and nothing shocks me anymore - if it can be typed, it will end up in a file or folder name sooner or later. And, somewhere along the way, backblaze, rsync, time machine, xar or ditto or even more probably: one of my shell glue scripts, one of those are going to get thrown way off balance by that ' / ' or ' /" ' ...

Don't know if its worth spending the time to band-aid these incredible corner cases, but now at least you are aware of a few perfect 90.000 degree corner cases.

Perhaps you have to go the route of identifying the inode of the file/folder and calling dd(1) [sp?] to zero-out the corresponding parts in the directory?

Anyways.. thanks for a great utility! It's hard at work over here in Oslo, Norway.

//haakon

[ap]

I even managed to find this (I think) small bug related to as close to the edge you can get an edge case (rename gave up) […] Don't know if its worth spending the time to band-aid these incredible corner cases

It is. It’s also a separate bug that I would prefer filed separately; would you resubmit it please?

[haakonstorm]

[Created #9 for tracking the edge case with file names consisting solely of "impossible" characters.]

[ap]

Thanks.

So getting back the original issue, the problem is that it’s not really possible to know when to decode filenames and when not to. 🙁 In Unix, file names are a series of bytes with any value except 0 and ASCII slash – there is no standard encoding and no way or place to specify one. Various conventions exist, and nowadays UTF-8 is pretty much the norm, but there is no real enforcement of this at any level. So ultimately the user has to specify… and things generally work fine even if rename doesn’t decode file names, anyway. As long as you’re not feeding Unicode file names to the control character stripper, that is.

There’s not much I can do here, unfortunately. 🙁