Error with unicode labels in GR backend

[juliohm]

using Plots; gr()
plot(x->x, xlabel="λ")

BoundsError: attempt to access 1-element Array{UInt8,1} at index [2]

in setindex!(::Array{UInt8,1}, ::Int64, ::Int64) at ./array.jl:415
in latin1(::String) at /home/juliohm/.julia/v0.5/GR/src/GR.jl:456
in text at /home/juliohm/.julia/v0.5/GR/src/GR.jl:488 [inlined]
in gr_text(::Float64, ::Float64, ::String) at /home/juliohm/.julia/v0.5/Plots/src/backends/gr.jl:191
in gr_display(::Plots.Subplot{Plots.GRBackend}, ::Measures.Length{:mm,Float64}, ::Measures.Length{:mm,Float64}, ::Array{Float64,1}) at /home/juliohm/.julia/v0.5/Plots/src/backends/gr.jl:711
in gr_display(::Plots.Plot{Plots.GRBackend}) at /home/juliohm/.julia/v0.5/Plots/src/backends/gr.jl:494
in _show(::Base.AbstractIOBuffer{Array{UInt8,1}}, ::MIME{Symbol("image/svg+xml")}, ::Plots.Plot{Plots.GRBackend}) at /home/juliohm/.julia/v0.5/Plots/src/backends/gr.jl:1089
in show(::Base.AbstractIOBuffer{Array{UInt8,1}}, ::MIME{Symbol("image/svg+xml")}, ::Plots.Plot{Plots.GRBackend}) at /home/juliohm/.julia/v0.5/Plots/src/output.jl:197
in show(::Base.AbstractIOBuffer{Array{UInt8,1}}, ::MIME{Symbol("text/html")}, ::Plots.Plot{Plots.GRBackend}) at /home/juliohm/.julia/v0.5/Plots/src/output.jl:177
in show(::Base.AbstractIOBuffer{Array{UInt8,1}}, ::String, ::Plots.Plot{Plots.GRBackend}) at ./multimedia.jl:33
in #sprint#304(::Void, ::Function, ::Int64, ::Function, ::String, ::Vararg{Any,N}) at ./strings/io.jl:37
in display_dict(::Plots.Plot{Plots.GRBackend}) at /home/juliohm/.julia/v0.5/Plots/src/output.jl:266
in execute_request(::ZMQ.Socket, ::IJulia.Msg) at /home/juliohm/.julia/v0.5/IJulia/src/execute_request.jl:187
in eventloop(::ZMQ.Socket) at /home/juliohm/.julia/v0.5/IJulia/src/eventloop.jl:8
in (::IJulia.##13#19)() at ./task.jl:360

[mkborregaard]

As a workaround for the moment, you can use LaTeXStrings and xlabel = "\lambda"

[kalmarek]

GR doesn't handle unicode characters well, eg:

plot(title="ąęźńśćł")

the result:

any idea for a workaround?

[stevengj]

Matplotlib also has awful Unicode font support by default — it doesn't know how to find a Unicode fallback font, apparently, and so users resort to manually searching for fonts (e.g. https://jdhao.github.io/2017/05/13/guide-on-how-to-use-chinese-with-matplotlib/, https://stackoverflow.com/questions/28905938/how-to-use-unicode-symbols-in-matplotlib, JuliaPy/PyPlot.jl#114, …). However, it just draws boxes — it doesn't actually error on the strings.

[StefanKarpinski]

Also reported here:

https://discourse.julialang.org/t/why-in-plots-plot-cant-correctly-display-chinese/19511

Not so straightforward to work around since there are no LaTeX escapes for Chinese characters.

[StefanKarpinski]

The culprit is the latin1 function in GR:

https://github.com/jheinen/GR.jl/blob/18bd2a104ef06bc945434b/src/GR.jl#L497-L524

It seems to be interpreting Julia String objects as Latin-1? Or maybe trying to convert to Latin-1? Unclear. Perhaps @jheinen could shed some light on what's going on there?

[jheinen]

Yes - right now, some of the underlying graphics output drivers (in GKS) don't support UTF-8. I will allow to bypass the latin1() logic with one of the next releases and raise warning messages in such drivers.

A better solution would be the intrinsic generation of character glyphs based on the FreeType library using a limited set of fonts distributed as part of GR. We have a proof of concept solution for that, but it has not yet been integrated into GR.

The UTF-8 problem is on our radar, but we are currently working on other feature, like pan/zoom, multiple window support, JS integration etc.

[stevengj]

FreeType might be a little too low level. For example, my understanding is that it doesn't implement automated glyph replacement (finding glyphs from a fallback font if the current font doesn't provide it). Matplotlib uses FreeType and that is apparently why people have so many problems rendering Unicode text with it as in the links above — manually choosing the right font to render certain glyphs is a pain!

Ideally there would be a Julia interface to some higher-level library for text rendering — either something like HarfBuzz or abstracting OS-specific APIs. I'm not sure what the best choice is.

[stevengj]

MEP14 for Matplotlib has a summary of text-handling issues for plotting, the shortcomings of FreeType, and the available options. That document also identifies harfbuzz as the most promising cross-platform option.

[jheinen]

I will add UTF support as described above. HarfBuzz - as an abstraction layer - will not improve anything IMO. I tried it on Linux (Ubuntu), Windows and macOS and got three different results: