SkiaSharp renderer

[Happypig375]

As far as I can see, this project is limited to Xamarin.iOS only. However, for most people, the reason for choosing Xamarin instead of native Swift/Kotlin is code sharing with C#. Also, as time goes by, the Xamarin.Forms approach is slowly overshadowing Xamarin native. If this library can have Xamarin.Forms support, then we have a viable competitor to the MathJax/KaTeX + WebView approach, which is slow and RAM-consuming.

I can think of two approaches to this now:

1. Use each platform's native APIs (6 platforms to support, so very time-consuming)
2. Use Xamarin.Forms BoxViews with rotation (Might need very hacky ways to do this)

So, although both approaches are not very good, once Xamarin.Forms is supported, then:

1. MathJax/KaTeX + WebView approach can be considered deprecated
2. Popularity boost for this project!

P.S. Before you ask, I don't really know Xamarin.iOS/native iOS APIs well, so currently I can't really help you here.

[verybadcat]

What I am doing is designing it so that the front end can be plugged in. I don't know the relevant Api too well either -- likely SkiaSharp and/or HarfBuzz. But I'm setting things up so that to create a front end, you have to define your own TypesettingContext and GraphicsContext objects, but you do NOT need to go into the Typesetter code. In other words, the process of creating a front end should be limited to telling it things like what font you want to use, how to measure glyphs, and what spacings you want around them. Then of course you need a GraphicsContext object to tell it how to draw. This is still a nontrivial task, but it's a lot less than writing the whole system.

So yeah, if someone starts on that for Xamarin.Forms, I will be thrilled.

[Happypig375]

@verybadcat
I want to help with this, however a hit a very big problem:
I cannot find any framework methods in Android/UWP for direct font manipulation.
This is frustrating.

So, maybe instead of loading a otf file at runtime, encode the glyphs in byte arrays and treat them as bitmaps, and draw the bitmaps directly, which most platforms should support.

Any thoughts?

[verybadcat]

I seem to remember somewhere that .otf files won't work on non-Apple formats. But there has to be a way to load a font. I also wonder if their default fonts have the needed characters.

Incidentally, another way to help the project would be to get more people interested.

[Happypig375]

So far my comment at microsoft/xaml-standard#192 (comment) has not gained any traction...

[Happypig375]

Yay! This repository now has 3 more stars. Hope that this growth can keep going...

[Happypig375]

From a deleted comment (microsoft/xaml-standard#192 (comment)):

Starting with Android API 26, fonts can be loaded as resources in Xamarin Android projects.
https://developer.xamarin.com/guides/android/application_fundamentals/resources_in_android/part_7_-_fonts/

[Happypig375]

Ayyy, another mention: https://forums.xamarin.com/discussion/4483/math-type-mathjax-for-displaying-equations

[dtila]

hello,

I can assist you in porting this to Xamarin Forms. You could to this in many ways:

• using a library like Skia or anything similar. I do not have experience in this, but I have no experience in this
• implement for each platform. Since you have already the support for iOS, you could add something for Android and use native Canvas from Android.

[Happypig375]

@verybadcat Another C# LaTeX library for reference: https://github.com/ForNeVeR/wpf-math

[zwcloud]

@Happypig375 For font manipulation, I recommend Typography or sixlabors/fonts.

[Happypig375]

@zwcloud a bit of this has been discussed at #2. I think they are viable approaches to this.

[verybadcat]

@tilutza just noticed your comment. I would love to get this onto forms.

[charlesroddie]

I could work with @tilutza and @Happypig375 on SkiaSharp support.

[verybadcat]

Terrific. What do y'all need from me to get started? I can certainly give you access to the repo; all I would ask is that you do it in a project on top of the existing ones.

[Happypig375]

@charlesroddie Can you lay out the plans? Probably a table with assignees to avoid duplication of work.

@tilutza Please comment on this.

[charlesroddie]

I will need to read through the CSharpMath code first before making a plan.

The first thing can be scaffolding: on a new branch we need a separate main project for SkiaSharp (with no Xamarin.Forms dependency), and then Xamarin.Forms sample projects (a .net standard library, and platform-specific projects for UWP, iOS, Android). I think we can work with .net standard 2.0 everywhere. Can one of you make a start on this?

This would allow the SkiaSharp project to be used for non-XF projects including WPF, Xamarin without forms, UWP, and anywhere else SkiaSharp is used.

I have done LaTeX parsing and have some familiarity with SkiaSharp. My C# is basic.

[Happypig375]

I have pushed a new branch with a new project and dependencies.

I don't think we can declare SkiaSharp views in a .NET Standard project. We can declare them in platform projects though. Should we make little platform projects that reference the appropriate SkiaSharp platforms and share a shared project among them, like in the current CSharpMath?

[Happypig375]

Ayyy, you are here! 😃

[charlesroddie]

"SkiaSharp provides cross-platform bindings for: ... .NET Standard 1.3..."

[Happypig375]

"SkiaSharp provides cross-platform bindings for: ... .NET Standard 1.3..."

SKSurface etc. are there, but no SKViews.
If we go the .NET Standard route, we may have to require the end user to call a method returning an SKSurface and bind it to the native SKView themselves.

[charlesroddie]

That's fine for the moment. That should just be a couple of lines of code.

[dtila]

I am glad that there are people interested to port this, but my availability is quite limited. However, I would help in terms of decisions, architecture and advises (in the thread or in a chat room).

Regarding using SkiaSharp I think it's a very goo option. I see that there are binding from using .NET Stardard already (referencing Xamarin Forms) https://github.com/mono/SkiaSharp/tree/master/source/SkiaSharp.Views.Forms/SkiaSharp.Views.Forms.NetStandard

There are ways using directly from xamarin firms as presented in the official documentation https://developer.xamarin.com/api/namespace/SkiaSharp.Views.Forms/

However I am no expert in manipulating fonts (just to use them), and my experience in drawing is to do it on a Canvas. There are a couple of things which are not clear for me:

• what does the Font Mapping Map to help to draw on a canvas

[Happypig375]

/cc @verybadcat

[Happypig375]

Btw, have you accepted the collaborator invitation?

[verybadcat]

Based on what @tilutza wrote above, I'm not inviting him yet, but I'm happy to do so if it should become helpful.

SkiaSharp does work on Xamarin.Forms with .netstandard. The only question would be whether it has a font with all the math glyphs.

[charlesroddie]

I can only see the font latinmodern-math.otf in this repo. It may work but presumably we can fall back on a truetype version if it doesn't.

[prepare]

@Happypig375 ,

I'm not sure that this position is correct or not.

Could you provide reference images?

[Happypig375]

@prepare Yup, that is definitely a bug.
That is the wedge symbol, repredenting a logical conjunction (AND operation). It shiuld be typesetted in between the variables, not above.

[Happypig375]

67th commit: Emailed @verybadcat about bugfixing

[Happypig375]

68th commit: Reworking icon; in progess

[Happypig375]

69th commit: Second iteration of the CSharpMath icon, awaiting @verybadcat bugfixes

[Happypig375]

70th commit: TeX typesetted with the new \raisebox command added in this commit
Also added: \kern, \mkern, \hskip, \mskip commands
To add: \TeX

123\raisebox{16mu}{456}789 \\ \text{T\kern-.1667em\raisebox{-.5ex}{E}\kern-.125emX}

[Happypig375]

71th commit: A couple bugfixes, fixed @prepare's comment's bug

[Happypig375]

72th commit: Staticified almost all FrontEnd-implementing classes to reduce memory, GC pressure and increase performance in speed

[Happypig375]

73rd commit: Test buildable again

[Happypig375]

74th commit: Started implementing feature 'Text and Maths'

[Happypig375]

75th commit: I can't believe it! Fixing a wedge display also fixes stuff like this! 😮
Compare (Look at Matrix3):

Before	After

[Happypig375]

76th commit: My fix for 1 \\ { 2 \\ 3 } for @verybadcat's reference
77th commit: Introduction of BiDictionary to replace DictionaryHelpers; fixed accents, fixed overlines, fixed underlines, iosMath examples now should be all bug-free, iosMath examples magnified to be more readable

[Happypig375]

78th commit: In process of fixing x{\scriptstyle y}z

[Happypig375]

79th commit: Fixed limits in text style, \sin, \cos etc are now non-limitable, also added \iint, \iiint etc, removed unused NuGet packages, added new CSharpMath.Utils.Release to prepare for a NuGet spec editor
(All of the projects got modified - what a great view to see)
\textstyle{\int\limits^6_5}\displaystyle{\int\limits^6_5}\textstyle{\int\nolimits^6_5}\displaystyle{\int\nolimits^6_5}

@prepare How's the Typography.TextBreak port going?

[charlesroddie]

@Happypig375 I get the following build errors, similarly to the last time I tried. Fresh clone.

Severity	Code	Description	Project	File	Line	Suppression State
Error	CS0006	Metadata file 'C:\Users\charl\source\repos\csharpmath\CSharpMath.Forms.Example\CSharpMath.Forms.Example\bin\Debug\netstandard2.0\CSharpMath.Forms.Example.dll' could not be found	CSharpMath.Forms.Example.UWP	C:\Users\charl\source\repos\csharpmath\CSharpMath.Forms.Example\CSharpMath.Forms.Example.UWP\CSC	1	Active
Error		Project "C:\Users\charl\source\repos\csharpmath\Typography\Typography.OpenFont\Typography.OpenFont.projitems" was not imported by "C:\Users\charl\source\repos\csharpmath\CSharpMath.Utils\CSharpMath.Utils.csproj" at (19,3), due to the file not existing.	CSharpMath.Utils	C:\Users\charl\source\repos\csharpmath\Typography\Typography.OpenFont\Typography.OpenFont.projitems	19	
Error		Project "C:\Users\charl\source\repos\csharpmath\Typography\Typography.GlyphLayout\Typography.GlyphLayout.projitems" was not imported by "C:\Users\charl\source\repos\csharpmath\CSharpMath.Utils\CSharpMath.Utils.csproj" at (20,3), due to the file not existing.	CSharpMath.Utils	C:\Users\charl\source\repos\csharpmath\Typography\Typography.GlyphLayout\Typography.GlyphLayout.projitems	20	
Error		Project "C:\Users\charl\source\repos\csharpmath\Typography\Typography.OpenFont\Typography.OpenFont.projitems" was not imported by "C:\Users\charl\source\repos\csharpmath\CSharpMath.Rendering\CSharpMath.~Rendering~.csproj" at (51,3), due to the file not existing.	CSharpMath.~Rendering~	C:\Users\charl\source\repos\csharpmath\Typography\Typography.OpenFont\Typography.OpenFont.projitems	51	
Error		Project "C:\Users\charl\source\repos\csharpmath\Typography\Typography.GlyphLayout\Typography.GlyphLayout.projitems" was not imported by "C:\Users\charl\source\repos\csharpmath\CSharpMath.Rendering\CSharpMath.~Rendering~.csproj" at (52,3), due to the file not existing.	CSharpMath.~Rendering~	C:\Users\charl\source\repos\csharpmath\Typography\Typography.GlyphLayout\Typography.GlyphLayout.projitems	52	
Error		Project "C:\Users\charl\source\repos\csharpmath\Typography\PixelFarm.NetStandard\PixelFarm.Triangulation\PixelFarm.Triangulation.projitems" was not imported by "C:\Users\charl\source\repos\csharpmath\CSharpMath.Rendering\CSharpMath.~Rendering~.csproj" at (53,3), due to the file not existing.	CSharpMath.~Rendering~	C:\Users\charl\source\repos\csharpmath\Typography\PixelFarm.NetStandard\PixelFarm.Triangulation\PixelFarm.Triangulation.projitems	53	
Error		Project "C:\Users\charl\source\repos\csharpmath\Typography\Typography.Contours\Typography.Contours.projitems" was not imported by "C:\Users\charl\source\repos\csharpmath\CSharpMath.Rendering\CSharpMath.~Rendering~.csproj" at (54,3), due to the file not existing.	CSharpMath.~Rendering~	C:\Users\charl\source\repos\csharpmath\Typography\Typography.Contours\Typography.Contours.projitems	54	
Error	MSB4019	The imported project "C:\Users\charl\source\repos\csharpmath\Typography\Typography.OpenFont\Typography.OpenFont.projitems" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.	CSharpMath.~Rendering~	C:\Users\charl\source\repos\csharpmath\CSharpMath.Rendering\CSharpMath.~Rendering~.csproj	51	

[charlesroddie]

It would be nice to clean up the API a bit. Current issues and suggestions (although I could easily be on the wrong track as I can't build the project):

• MathPainter seems to be in the SkiaSharp namespace but that is for SkiaSharp itself. Suggest CSharpMath.SkiaSharp.
• CSharpMath.SkiaSharp should contain everything needed to render LaTeX to an SKSurface. There shouldn't be a need to open CSharpMath.Rendering or Typography.OpenFont.

[Happypig375]

@charlesroddie
You didn't clone with submodules, did you? The error means that the Typography project is not found. I guess not a lot of people understand git submodules.

Again, please check the Recurse Submodules box in Visual Studio. If that is not working, please go ahead and download git and use Git Bash to clone with the command given above after navigating to the relevant folder with cd <directory>.

As for the purpose of CSharpMath.Rendering, it is cross-platform and any graphics framework or platform can be plugged in. (For example, Unity). The CSharpMath core project has a similar philosophy.

The CSharpMath.SkiaSharp project is a minimal layer of glue between SkiaSharp and CSharpMath.Rendering. Its primary type is CSharpMath.SkiaSharp.SkiaMathPainter, which inherits CSharpMath. Rendering.MathPainter (as shown in documentation), and its primary method is Draw(SKCanvas).

[Happypig375]

80th commit: NuGet spec editor in progress

[charlesroddie]

Thanks @Happypig375 . I did check recursively clone, but it needed a manual recursive clone after switching branches.

[Happypig375]

81st commit: NuGet spec editor!

[Happypig375]

82nd commit: Release mode testing, 3rd iteration of CSharpMath icon (frozen for 0.1.0), Adding the 3rd iteration icon to Forms Example projects

[Happypig375]

83rd commit: Publishing 0.1.0-pre1

[Happypig375]

0.1.0-pre1 has been published!
Goodbye first issue, you will be remembered 😢
Well, progress report will continue at #12.

P.S. A long issue like this one takes a while to load on my phone, so I guess it is really appropriate to start a new thread. 💭