accented char - encoding them depends Module?

[espruino-discuss]

Posted at 2018-09-27 by Mrbbp

Hello,

it should be a newb question... sorry.
i'm working with a 8x8 led matrix
I'm working with string with extented latin chars (éèêàâî,,,).
to draw the string on the display with accents we have to convert accents with char hexa value.

"mémé"

become

"m\xe9m\xe9"

i'm a lazy guy and decide to convert the sentence when it's sent to matrix.
i use this code;

phrase = phrase.replace(/[^\x00-\x7F]/g, function(m) {
    return "\\x" + m.charCodeAt(0).toString(16);
});

it's look as it have to look, but the string is sent as is and i saw on the display "m\xe9m\xe9" and not "mémé" as it should.
What i'have missed?

regards

[espruino-discuss]

Posted at 2018-09-27 by Mrbbp

i answer myself

if i don't use my string.replace it look good (with accents)
it seems it's a pb with module.
currently i work with TM1640 Module
last week i was working with Max7219
with the max's Module i use an arrayBuffer

SPI2.setup({mosi:B15, sck:B13});
var disp = require("MAX7219").connect(SPI2, B14);

var g = Graphics.createArrayBuffer(8,8,1); // Create graphics
g.flip = function() { disp.raw(g.buffer); }; // To send to the display

g.drawString("Mémé");
g.flip(); // update what's on the display

with the arrayBuffer, accents must be encoded.

But with the TM1640 Module, we drawString directly in the display

var g = require("TM1640").connect({din: NodeMCU.D7, clk: NodeMCU.D5}, function() {
  g.drawString("Mémé");
  g.setContrast(2); // a value between `0` lowest and `7` highest intensity.
  g.flip();
});

is it the cause of the encoding pb?

regards

[espruino-discuss]

Posted at 2018-09-27 by @allObjects

What made you think that you needed a transformation of the characters outside the ASCII set (128..255)?

For both displays, the graphics object makes the mapping from character into dot matrix in the display buffer in Espruino memory. .flip() then pushes the dots over to the display according to how it is connected and how - in what sequence - the display expects the bits to show up.

The main difference between these modules is only how the involved software components are created and composed to a whole. They all have the Graphics object as the core - which operates on a buffer of pixels in Espruino memory - and a .flip() method that pushes this buffer to the display using the communication component.

For MAX7219 you are the plumber, where for TM1640 the plumbing is all done for you. MAX7219 provides you with more flexibility / extensibility (cascading), but setting up the individual components correctly and 'screw' them together is your responsibility.

Looking at the module code, you see how everything is baked into one an the same code / method .connect(): setting up the communication, creating the Graphics object on a display buffer (8,8,1), and assigning the flip method to the Graphics object, and finally returning the Graphics object.

Cascading is done differently: where as with MAX7291 - which is serial in and serial out - data is always sent to the first block of 8x8 and pushed to the next block with every now block sent to the display, with TM1640 the data needs to be chopped up and sent to each individual TM1640 directly. Latter allows you to address each 8x8 block separately without having to change the other blocks. For simplicity though, flip is always updating.

[espruino-discuss]

Posted at 2018-09-27 by Mrbbp

Hello Markus,
yes that's right (i think i've found a way post#1, i did not tried it with the Max7219 module), BUT
the need to encode accents (ascii > 127) depends the way we use drawString on espruino.
It's a bit annoying...
Is there a way, to homogenize the way it works, with or without a graphicArrayBuffer? (i'm always use the example provided with modules)

therefore it looks like more a wish list, than a real problem.
Ok it's a problem only for non english people that use ascii over 127 ;)

regards

é.

[gfwilliams]

Posted at 2018-09-27 by @gfwilliams

I'm not quite sure I understand the problem?

When Espruino outputs "m\xe9m\xe9" to the REPL it's just playing it safe with the characters it outputs and is quoting anything over 128 - that is still a 4 character string (if you check .length it'll say 4).

Graphics should behave the same either way - your issue is finding a font that has glyphs for characters above 128 as the built-in one doesn't to try and save space. What you need is FontDennis8: http://www.espruino.com/Fonts

Once that font is set you shouldn't have to do anything to the string - it should 'just work'

[espruino-discuss]

Posted at 2018-09-27 by Mrbbp

i am the plumber, but the way it work is different with the same instruction (with the same tool).
drawString needs encoding accents for one and not for the other, why?

[espruino-discuss]

Posted at 2018-09-27 by Mrbbp

i've used the same font on the 2 projects and the same codebase

require("Font4x6").add(Graphics);

ide trace "no error with Font4x6", and my accents are good on the display (TM1640) with no encoding, but not on the Max7219.

i do not understand why!
what i'm doing wrong?

[espruino-discuss]

Posted at 2018-09-27 by Mrbbp

well well well i do not understand the issue. I'm confuse.
i've retried the code on the Max7219 with no encoding accents.. it works fine (few day ago it dont)
if IDE does not find the font (require), it does not find the module too(require), i have no local repo...
i do not understand why sometime i have to encode, and not today!

sorry for this post.
in fact it works fine.
i'm not able to reproduce my prob... who does not seem to be...
sorry

[espruino-discuss]

Posted at 2018-09-27 by @allObjects

Maintenant, je compris, Eric. --- What does not enter my mind though is why should it be different as @gordon points out: if the font supports 8 bit, which - I hope - both of your projects use the same font. What @gordon says is that a string can be passed on in two ways: with 8 bit character set as "mémé", but when only seven bits are considered, it has to be "m\xe9m\xe9". It is almost like a shift-in and shift-out for each single character that is above 127 ASCII value (MS BIT set). I did not know that Espruino (only? or JavaScript in general? or strings in general - latter most likely a standard how to deal w/ 8 bit chars in a 7 bit world) has this feature... but it is nice to know. Somewhere along the line - when the string is transported / chopped up into characters / mapped to the font - this matters and weird things happen... I expect it the cooperation of Graphics and Font object - similar to the following out put created in console. It looks to me that for strings with MSBit set, an encoding happens on transport to and/or from console entry to Espruino and back... (console) echo behaves 7-bit-ish and different from 8-bit-ish console.log().

Connected to /dev/cu.usbmodem14111
>var ps = "mémé";
="m\xE9m\xE9"
>var cs="m\xE9m\xE9"
="m\xE9m\xE9"
>console.log(ps)
mémé
=undefined
>console.log(cs);
mémé
=undefined
>var c = "é"
="\xE9"
>cs.length
=4
>ps.length
=4
>é
Uncaught SyntaxError: Got ?[233] expected EOF
 at line 1 col 1
?[233]
     ^
>var é = 5
Uncaught SyntaxError: Got ?[233] expected EOF
 at line 1 col 5
var ?[233] = 5
         ^
> 

Just for the sake of fun I tried a variable name e-accent-egu - for your name's - éric - sake... ;)