Lower case Greek should be italic

[AshtonSBradley]

Not sure if this is a deliberate font choice, but there are some strange italicization results for greek versus roman characters in Makie. Is this a MathTeXEngine problem, or Makie?

more examples at this issue MakieOrg/Makie.jl#1245

[AshtonSBradley]

cbar = Colorbar(f[1, 1][1,1],height=15,vertical=false)
Label(f[1, 1][1, 2], L"|\psi|^2\xi^2")

gives something like this

Here it appears that \psi is not italicised, but | is, which seems to invert the normal situation for latex math where greek is italicised:

It seems that instead greek is not leaning, while some math characters are leaning.

[AshtonSBradley]

Note that computer modern is not correctly rendered using the current implementation. It should look like this

https://tug.org/FontCatalogue/computermodern/

whereas I get, e.g.

The integral seems to have survived intact, but \pi and \mu ...

[AshtonSBradley]

I would be happy to have a go at fixing this... any pointers on how to get started?

[Kolaru]

Thanks for the detailed report ! There are several things going on here.

1. Currently we are using New Computer Modern and not erm... "vanilla" Computer Modern, for technical reason since the former is a set of unicode fonts, while the latter has some pre-unicode complications. The result still looks wrong compared to the expectation anyway.
2. This is definitely mostly a MathTeXEngine problem, Makie trust this package about everything related to math rendering.
3. That being said, there is currently a dependency conflict between Makie and MathTeXEngine due to the last breaking update of GeometryBasics.jl (see CompatHelper: bump compat for "GeometryBasics" to "0.4" for package CairoMakie MakieOrg/Makie.jl#1174 and related). This prevents the use of the very last version we have here. The src/protoype.jl renders think with a different pipeline, allowing to circumvent this issue and continue developpement as long at it isn't solve.
4. In the latest (v0.2.0) the wrong italics for delimiters (including e.g. |) should be solved.
5. For italicized greek letter it is more subtle. The main problem is finding where the glyphs for theses characters are located in the font files and making them available to Makie. They are not the main glyphs in the math font NewCMMath-Math nor the one from the italics font NewCM10-Italic... I fear that they may be alternative glyphs in the math font and we lack the ability to load those through TrueTypeAbstraction.jl. I unfortunately lack the latex knowledge to go read what latex engines do for this.
   Alternatively, it may be simpler to implement the vanilla Computer Modern set of fonts, since it doesn't use any of the OpenType tricks AFAIK. This would mostly require introducing the table associating unicode symbol to their (non unicode) code in the correct font. Matplotlib already have a similar table here we could adapt. I'm starting to be more and more convinced using Computer Modern may be the way to go despite this added complexity.

So point 3 and 5 require a fix. For the latter, the starting point is this function

MathTeXEngine.jl/src/engine/texelements.jl

Line 121 in 88b62d1

function TeXChar(char, fontset::FontSet, char_type)

It is given a Char and FontSet and additional information about the context (currently char type and I'm adding font modifier to fix #21, hopefully today) and has the job to return a TeXChar object containing the code of the char inside the font, the font, and whether this symbol should be considered slanted or not. By navigating what it does, you should get an idea of the madness of font handling :D

If you'd like to contribute and have any question about the internal, I'm happy to help.

[AshtonSBradley]

How does LaTeXStrings.jl approach compare to the two approaches outlined in point 5? Is it doing a version of the matplotlib lookup? I ask this because LaTeXStrings seems to produce consistently good output in my limited usage

[Kolaru]

LaTeXStrings.jl doesn't do any layouting or rendering, it is only a convenient type for strings representing LaTeX code. The hard work is always done by the backend.

[AshtonSBradley]

On closer inspection it seems to me that all lower case Greek has not had the conventional slant applied. Could this be fixed in the pipeline logic? Lower case Greek is always slanted/italic by convention. Presumably this wouldn't auto-fix in a different font until the logic is updated... or have I misunderstood the issue?

[AshtonSBradley]

5. For italicized greek letter it is more subtle. The main problem is finding where the glyphs for theses characters are located in the font files and making them available to Makie. They are not the main glyphs in the math font NewCMMath-Math nor the one from the italics font NewCM10-Italic... I fear that they may be alternative glyphs in the math font and we lack the ability to load those through TrueTypeAbstraction.jl.

So I should at least hunt down an example of italicised lower case greek, and figure out whether TrueTypeAbstraction.jl can load it?

[AshtonSBradley]

These lower case greek symbols look perfect. They seem to be part of the main font. Am I missing something here? Looks like they are not part of NewComputerModernMath though. I accessed via view -> language -> greek in fontbook for the italic form of the regular font. Does that help?

[AshtonSBradley]

how would I go about changing the default lower case greek to be italic?

[AshtonSBradley]

@Kolaru do you think there is a way forward using NCM, or does this require deeper surgery?

[AshtonSBradley]

The glyphs are definitely there already.

Here I used the Font Preview vscode extension to read the otf files in MathTeXEngine.jl:

Can I just set the font that MathTeXEngine would use globally to italic? I mean I would much rather specify a few roman cases using \textrm{ } than have nonitalic defaults. In fact that is the normal situation (italic defaults on everything) I think...

Is the font used for math set at the Makie level, or MathTeXEngine?

How do I tell Makie to tell MathTeXEngine to use NewCM10-italic?

Or set the default font in MathTeXEngine?

[AshtonSBradley]

mmm. Fonts are confusing. No italic math file for NewCM.

Yet, the "Math example" here

https://tug.org/FontCatalogue/newcomputermodernroman/

is italic. I don't get this. What is going on here?

[Kolaru]

Sorry for the late answer I was quite busy recently.

These lower case greek symbols look perfect.

Yes they can do the job, they are not the glyphs used in the math example you linked below for some reason that eludes me as well. But we could use them anyway.

Can I just set the font that MathTeXEngine would use globally to italic?

Unfortunately things are a bit more complicated. Currently we separate between symbols and normal characters. Glyphs for symbols are taken from the math font and glyphs for normal character from the italic font. The catch is that greek letters are parsed as :symbol, so having them italicized require a bit more work.

Is the font used for math set at the Makie level, or MathTeXEngine?

How do I tell Makie to tell MathTeXEngine to use NewCM10-italic?

Or set the default font in MathTeXEngine?

The fonts are set in MathTexEngine there:

MathTeXEngine.jl/src/engine/fonts.jl

Lines 28 to 44 in 32ce8c4

	const _default_fonts = Dict(
	:regular => joinpath("NewComputerModern", "NewCM10-Regular.otf"),
	:italic => joinpath("NewComputerModern", "NewCM10-Italic.otf"),
	:bold => joinpath("NewComputerModern", "NewCM10-Bold.otf"),
	:bolditalic => joinpath("NewComputerModern", "NewCM10-BoldItalic.otf"),
	:math => joinpath("NewComputerModern", "NewCMMath-Regular.otf")
	)
	const _default_font_mapping = Dict(
	:text => :regular,
	:delimiter => :regular,
	:digit => :regular,
	:function => :regular,
	:punctuation => :regular,
	:symbol => :math,
	:char => :italic
	)

The first dict define the default fonts with an identifiers (e.g. :italic for the italic font), while the second maps a element type (e.g. :symbol) to a font type (e.g. :math). As mentioned above, currently greek letters are parsed as :symbol so its clashes with symbols like \sum that we want to take from the :math font. The simplest way may be to parse greek letters as a separate type of object so we can easily define a font for them. Ideally I think it should be done by the parser (greek letters can be identified by looking at their unicode code).

do you think there is a way forward using NCM, or does this require deeper surgery?

Patching NCM is possible, but once I have more time I will look into the options in more details nonetheless.

[AshtonSBradley]

awesome! I will give a test drive !

[AshtonSBradley]

I guess MakieOrg/Makie.jl#1952 basically fixes this, apart from (so far) this new issue with backslash that will impact axis labels MakieOrg/Makie.jl#1967