Fixing UTF8...?

[algj]

I'm sure quite a few of people may have problems with UTF-8.

Solution: you must first convert it from latin1 (ISO-8859-1) to UTF-8. Sounds weird, but works for me.

On Linux:

iconv -f iso-8859-1 -t utf-8 latex.tex > latex2.tex

In JS (thanks ChatGPT):

function convertLatin1UTF8toUTF8(str) {
    const latin1Array = new TextEncoder().encode(str);
    const utf8Array = new Uint8Array(latin1Array.length * 2);
    let utf8Index = 0;
    for (let i = 0; i < latin1Array.length; i++) {
        let code = latin1Array[i];
        if (code < 128) {
            utf8Array[utf8Index++] = code;
        } else {
            utf8Array[utf8Index++] = 0xc0 | (code >> 6);
            utf8Array[utf8Index++] = 0x80 | (code & 0x3f);
        }
    }
    utf8Array.length = utf8Index;
    const decoder = new TextDecoder('utf-8');
    return decoder.decode(utf8Array);
}

Note, UTF-8 in LaTeX isn't without problems, I had to personally use \usepackage[T1]{fontenc}, \usepackage{textcomp} and \usepackage[utf8]{inputenc} in the file. Yet I still could not use cryllic.

@fzimmermann89, you may find this interesting :)