UTF-8 support

[mikrosk]

I know I'm being idealistic here but in practice, what does prevent implementation of UTF-8 support? I'm not very fluent in this area but I'd assume it's both VDI and the kernel. Maybe it's not that hard and the gain would be huge, no more troubles with reading texts from PC.

[mfro0]

Hmmm.
This would basically break everything, I suppose.
Interestingly enough, the VDI is already (kind of) prepared for wide strings (parameter arrays are WORD sized), the rest of the system is not.
UTF-8 has the drawback that you always have to scan through the string to know its memory requirements (which may change everytime the content changes). One reason, I guess, why Windows preferred to use UTF-16 although its wasting memory.

I completely understand the desire for wide strings (especially if your native language requires a lot of special characters), but I'm afraid the effort to integrate this into the system cleanly is just too much.

[ctzio]

Hi,
obviously I am really interested in this as well.
I think most of it would need to be handled on the application level. However seeing that I can use speedo fonts to see most Greek web text (some encodings don't work) could the way speedo is handled be hacked for utf?

[atic-atac]

If you SSH into your Atari, then UTF-8 is fine and works with bash, etc.
It's real Atari console applications that need work.

[mikrosk]

That's what I'd like to see clarified. Maybe there are more or less separate steps and we can find some consensus that, say, GEM dialogs are impossible to adapt to UTF-8 and we're stuck with a hardcoded 8-bit codepage. But if the console, TosWin2, VDI windows would be able to show text in UTF-8... maybe it's enough?

[ctzio]

Highwire supposedly handles utf-8 as long as the fonts suppport it. For Greek I need to use utf-8 and fonts that only contain Latin and Greek glyphs. I suppose that something similar can be used for other open source applications. i think the hardest part would be text input since GEM has some flexibility while gemdos has not. Certainly this solution can't work for multilingual text where multi > 2

[ps68060]

I know virtually nothing about UTF but when I was working on the form input areas in Highwire I did see some widechar support.

Sometimes the text area goes a bit haywire :-) when editing some input and I think it is down to a miscalculation of the cursor position. Without a debugger it was all but impossible to work out what was wrong.

[jflemaire]

I'm no expert in UTF but I did implement conversions to/from UTF-8 in GEMDict a few years ago from the Highwire code and it seems to be working. Which means that nobody complained that it wasn't working, which may mean that nobody uses the program :-)

Anyway, at the GEM level it was straightforward enough to do and I don't see why it wouldn't be possible to do it system wide. Not too sure about VDI output, though. I'd need to check this out. It's been a while. From memory it was fine only with the system font.