Full UTF-8 support in problem statement and sample data

[breed]

It would nice if we could use the full set of UTF-8 characters for problem description and sample data. Currently, playing with fontencodings can get some non-ASCII characters rendered properly, but it is tedious to use and is problematic with sample data.

To fix this, we probably need to change the LaTeX engine that we use, but it would greatly simplify the use of different characters. It's the 21st century, lets go 🚀

[gkreitz]

It would be cool to support this.

To fix this, we probably need to change the LaTeX engine that we use

Do you (or anyone else) have any recommendations for another LaTeX engine to use? Note that we need to be able to output both pdf and html formats. It's fine with separate engines for the two formats, we currently use pdflatex and plasTeX (and I assume neither of them has stellar UTF-8 support, but I can't claim to have tested all that much).

[breed]

Do you (or anyone else) have any recommendations for another LaTeX engine to use? Note that we need to be able to output both pdf and html formats. It's fine with separate engines for the two formats, we currently use pdflatex and plasTeX (and I assume neither of them has stellar UTF-8 support, but I can't claim to have tested all that much).

chatgpt (and web search as well) indicates either xetex or luatex for the latex engine. my unicode tests with plasTeX all passed. for example, the following problem statement works with problem2html but not with problem2pdf.

\problemname{Unicode support would be 🔥}

😀programmers like emojis! let's have problems with 🚀👉☣!

[gkreitz]

This turned into a bit of a rabbit hole. lualatex and xetex are advertised as having full utf-8 support, which is true, but getting fonts working turned was way trickier than I expected. I definitely do not have enough expertise in modern latex to be confident in my solution, but I have a PR coming soon which support latin, cyrillic, greek, and emojis in pdf problem statements, hopefully being robust against particular fonts not being installed on a system.

Annoyingly, lualatex renders a bit differently than pdflatex, the pdf:s I've tested end up with a bit more space (and looking a bit uglier, IMHO). I think that's ok - I'm not sure how many users we have that do pdf rendering from latex, and if anyone has the latex expertise to figure out the differences between what pdflatex did and what lualatex does and want to tweak our cls files, PR:s are very welcome.