Skip to content

Conversation

@musm
Copy link
Collaborator

@musm musm commented Nov 5, 2016

No description provided.

@simonster
Copy link
Member

I'm not sure we can assume characters are always UTF-16. In R2015a+ we should probably use mxArrayToUTF8String since it is guaranteed to handle the locale properly.

@musm
Copy link
Collaborator Author

musm commented Nov 5, 2016

http://www.ece.northwestern.edu/local-apps/matlabhelp/techdoc/apiref/mxchar.html

Also mxarray('a') is also broken (not fixed by this)

@simonster
Copy link
Member

Just because MATLAB is storing characters as 16-bit integers does not necessarily mean it's encoding them as UTF-16, which is what transcode assumes.

@simonster
Copy link
Member

Although http://www.mathworks.com/help/matlab/apiref/mxchar.html suggests it is UTF-16 (really UCS-2) at least on new-ish MATLAB, so maybe this is okay.

@simonster simonster merged commit ad5bb52 into JuliaInterop:master Nov 5, 2016
@musm
Copy link
Collaborator Author

musm commented Nov 6, 2016

people on versions older than 2006 are out of luck, I think that's fine since it's 10 years old at this point

@simonster
Copy link
Member

I think there are versions newer than 2006 that don't use UTF-16 internally (mxChar was introduced before 2006a, but looking around online it's clear it wasn't always defined to be Unicode), but I don't think there's a convenient way to convert their internal encodings to UTF-8, and 16-bit ISO-8859-1 is basically the same as UTF-16, so I'm fine with this.

@musm
Copy link
Collaborator Author

musm commented Nov 6, 2016

we might have to revert this. It's not properly handling some unicode glyphs

@twadleigh
Copy link
Contributor

@musm which? It was working fine for me in mexjulia on my tests.

@musm
Copy link
Collaborator Author

musm commented Nov 6, 2016

@twadleigh see the test added at: #78

@twadleigh
Copy link
Contributor

I can't duplicate a test failure in mexjulia.

If I throw the following into Mex.jl:

const funky_string = "λ α γ"
is_funky_string(str) = str == funky_string

and call it from MATAB, I get:

>> str = jl.eval('funky_string')

str =

λ α γ

>> jl.call('is_funky_string', str)

ans =

  logical

   1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants