Bug of Espruino IDE (not supporting UTF-8 comments) #6520

espruino-discuss3 · 2025-02-12T16:38:59Z

espruino-discuss3
Feb 12, 2025
Maintainer

Posted at 2017-07-22 by MobiTech

Hi,
I would like to inform you that there seems to be a bug in the Espruino IDE. When I try to enter a comment with ä,ö,ü, the program will not be transfered to the device. I guess that the coding is not supported. It would be nice if we could use UTF-8 signs in the JavaScript comments.

espruino-discuss3 · 2025-02-12T16:39:02Z

espruino-discuss3
Feb 12, 2025
Maintainer Author

Posted at 2017-07-24 by @gfwilliams

Which IDE are you using, and can you give me an example of what's not working? I just tried uploading:

function a() {
  /* ä,ö,ü */
}

And it works great for me.

Posted at 2017-08-07 by MobiTech

I am using Espruino Web IDE on Windows. The code that causes an error:
// Service für ...

If I remove this line the everything works well.

Posted at 2017-08-07 by MobiTech

I am using Espruino Web IDE on Windows. The code that causes an error:

// Service für ...

If I remove this line the everything works well.

Posted at 2017-08-07 by Wilberforce

Is it the ... - is that three . or a character... ?

Posted at 2017-08-07 by MobiTech

The error is caused by the word "für". If I write "fuer", everything works fine.

Posted at 2017-08-08 by @gfwilliams

I just tested this on Windows with the current Web IDE from the Chrome Web Store, and it works great for me.

However, the 'Native' IDE (the one you install with an installer, that is needed for Bluetooth LE on Windows) does seem to have a problem. Could you let us know which one you're actually using?

Posted at 2017-08-08 by @gfwilliams

I just uploaded a new IDE: http://www.espruino.com/files/espruino_ide_win64_0v66.1.exe

If you were using the Native IDE, if you install that IDE then I think your problems should be gone.

Posted at 2017-08-08 by MobiTech

I am using the native IDE

Posted at 2017-08-08 by @gfwilliams

Ok, then install the new IDE linked above and it should be fixed.

Posted at 2017-08-08 by MobiTech

You are right, it works well. Thanks.

Posted at 2017-08-10 by MobiTech

@gfwilliams I tried now an Inline-Comment in a function which is called inside OnInit. If I use a letter like ä,ö or ü, it throws an Syntax error. This seems to be a bug.

Posted at 2017-08-11 by @gfwilliams

Can you give me an example of the code and the exact error you get? I tried this initially, and it worked fine:

function a() {
  /* ä,ö,ü */
  console.log("Hello");
}
a();

Posted at 2017-08-11 by MobiTech

The function a() would be called inside onInit.
For example this line in a service declaration:

0x4000 : {    // Service für

It is a part of a function which is called inside the onInit function.

Posted at 2017-08-11 by @gfwilliams

I just tried:

function onInit() {
  function a() {
     var b =  { 0x4000 : { // Service für 
     }};
  }
  a();
}

and it works fine. Please give me a FULL example with the problem, and the full error message.

Posted at 2021-04-24 by user127815

Hi,
First of all I would be like to say a big thank for your products.
Amazing devices with my favorite programming language :)
I build a smart barrier controller, controlling barrier and managering proxy cards via socket connection controlled by android phone.
My problems is UTF-8 character encoding (hungarian's accented characters) and representating. As you can see on attached images the accented characters are well in the Espruino memory but they appear bad on browser. Interestingly, e.x. the "ő" character also appear well in the browser and in the code, but other chars appear as question mark. How can it be?
HTML code

    html = `
<!doctype html>
  <head>
    <meta charset="UTF-8">
.....
  <body>
    <h1>Sorompó vezérlő</h1>
    <p>Alaplap hőmérséklete: <span id="temp">0</span>&nbsp;&#8451;</p>
    <p>Wiegand olvasó 1 csatorna: <span id="wieg1"></span></p>
    <p>Wiegand olvasó 2 csatorna: <span id="wieg2"></span></p>
    <p>
      LED on:
      <select id="oled">
        <option>off</option>
        <option>on</option>
      </select>
    </p>
  </body>
</html>`;

My other question is, is the operating temperature of the Espruino wifi CPU normal as shown in the attached picture, or does the motherboard require cooling?

Thank you for your helping hand.

Laci

Attachments:

Képernyőkép erről: 2021-04-24 16-49-33.png

Posted at 2021-04-24 by Robin

Sat 2021.04.24

Hi @LACI So that we may better visualize what is going on, would you mind posting the Javascript code snippet that is performing the actual write back to the browser using the indicated var html.

and, . . . please post the view source results that the browser is receiving so that we may compare with the image that is posted above.

Posted at 2021-04-24 by Robin

Sat 2021.04.24

' is the operating temperature of the Espruino wifi CPU normal'

Please post the code snippet that is used to retrieve that temperature reading. Under normal conditions, my Wifi does sometimes appear warm, but that reading seems way off.

Are there any other devices in the circuit? Missing/mis-wired grounds? What voltage is being applied?

Posted at 2021-04-24 by user127815

Hi Robin,
Here is the javascript code with html variable and webserver function.

    html = `
<!doctype html>
  <head>
    <meta charset="UTF-8">
	<style>
		html {font: 20px monospace}
		body { color: #333; text-align: center; padding: 150px; }
		h1 { font-size: 50px; }
		article { display: block; text-align: left; width: 650px; margin: 0 auto; }
		a { color: #dc8100; text-decoration: none; }
		a:hover { color: #333; text-decoration: none; }
        select {font: 20px monospace;padding: 4px;}
	</style>
    <script>
      window.onload = () => {
        var ws = new WebSocket('ws://' + location.host, 'protocolOne');
        var temp = document.getElementById('temp');
        var wieg1 = document.getElementById('wieg1');
        var oled = document.getElementById('oled');
        ws.onmessage = evt => {
          var d = JSON.parse(evt.data);
          switch(d.param) {
              case 'temp':
                temp.innerText = d.value;
                break;
              case 'wieg1':
                wieg1.innerText = d.value;
                break;
          }
        };
        oled.onchange = evt => {
          ws.send(oled.value);
        };
      };
    </script>
  </head>
  <body>
    <h1>Sorompó vezérlő</h1>
    <p>Alaplap hőmérséklete: <span id="temp">0</span>&nbsp;&#8451;</p>
    <p>Wiegand olvasó 1 csatorna: <span id="wieg1"></span></p>
    <p>Wiegand olvasó 2 csatorna: <span id="wieg2"></span></p>
    <p>
      LED on:
      <select id="oled">
        <option>off</option>
        <option>on</option>
      </select>
    </p>
  </body>
</html>`;
function startServer( ) {
  const s = require('ws').createServer(pgHandler);
  s.on('websocket', wsHandler);
  s.listen( 80 );
}
function pgHandler(req, res) {
  res.writeHead(200, { 'Content-Type':'text/html' });
  res.end(html);
}

Chrome received html code

<!doctype html>
  <head>
    <meta charset="UTF-8">
	<style>
		html {font: 20px monospace}
		body { color: #333; text-align: center; padding: 150px; }
		h1 { font-size: 50px; }
		article { display: block; text-align: left; width: 650px; margin: 0 auto; }
		a { color: #dc8100; text-decoration: none; }
		a:hover { color: #333; text-decoration: none; }
        select {font: 20px monospace;padding: 4px;}
	</style>
    <script>
      window.onload = () => {
        var ws = new WebSocket('ws://' + location.host, 'protocolOne');
        var temp = document.getElementById('temp');
        var wieg1 = document.getElementById('wieg1');
        var oled = document.getElementById('oled');
        ws.onmessage = evt => {
          var d = JSON.parse(evt.data);
          switch(d.param) {
              case 'temp':
                temp.innerText = d.value;
                break;
              case 'wieg1':
                wieg1.innerText = d.value;
                break;
          }
        };
        oled.onchange = evt => {
          ws.send(oled.value);
        };
      };
    </script>
  </head>
  <body>
    <h1>Soromp� vez�rlő</h1>
    <p>Alaplap hőm�rs�klete: <span id="temp">0</span>&nbsp;&#8451;</p>
    <p>Wiegand olvas� 1 csatorna: <span id="wieg1"></span></p>
    <p>Wiegand olvas� 2 csatorna: <span id="wieg2"></span></p>
    <p>
      LED on:
      <select id="oled">
        <option>off</option>
        <option>on</option>
      </select>
    </p>
  </body>
</html>

Posted at 2021-04-24 by user127815

I use sd card interface board with 3.3V and I2C SH 1106 oled display (also 3.3V).
I measured the current consumption of the test panel with a usb Keweisi ammeter and the current was between 0.05A and 0.1A.
Seems to be temperature was higest (50, 51 C) when javascript code send socket message and received bit-serial from wiegand reader via optocoupler isolation. Simultanosly ran two pin interrupt, socket server and a setintervall function with temperature sending via socket message.

Posted at 2021-04-24 by Robin

Sat 2021.04.24

From 1st snippet:
L32 ws.send(oled.value);
is returning the value of an Html element defined at:
L19 var oled = document.getElementById('oled');

I don't believe a WebSocket object returns a temperature in it's payload.

http://www.espruino.com/ws#line=38

Would you post the contents of L20 evt please. . . and/or link to datasheet/tutroial that demos that task.

Is there an additional snippet with a function like getTemp()?

p.116

https://www.espruino.com/datasheets/STM32F411xE.pdf
50C seems to be on the low end of their test cal ranges

Forgive me on: 'UTF-8 character encoding (hungarian's accented characters)' I concur with your observation for the '?' char and will have to defer to others for an answer. Your snippets will provide that detail needed for them to comment on.

In the mean time: Does this assist at all?

https://www.espruino.com/Fonts#character-sets
'Espruino does not support unicode natively'

Posted at 2021-04-24 by user127815

Temperature sending code

function broadcast(msg) {
  clients.forEach( cl => cl.send( msg ) );
}

setInterval(function( ) {
  var t = E.getTemperature( ).toFixed( 1 );
  broadcast( '{"param":"temp","value":"'+ t +'"}' );
}, 500);

Posted at 2021-04-24 by Robin

Okay, thanks that clears that up. re: 'E.getTemperature( )'

https://www.espruino.com/datasheets/STM32F411xE.pdf
p.116 50C seems to be on the low end of their test cal ranges

p.11 'The STM32F411xC/xE operate in the –40 to +105 °C temperature range'

p.32 3.30 Temperature sensor - Variation in internal temperature

Also note: > http://www.espruino.com/Reference#l_E_getTemperature 'Note: This is not entirely accurate and varies by a few degrees from chip to chip. It measures the die temperature, so when connected to USB it could be reading 10 over degrees C above ambient temperature.'

Posted at 2021-04-24 by user127815

Thanks for your help. I'm running another lap about the character encoding error.

Posted at 2021-04-25 by @MaBecker

this might help to display the chars correct in browser

https://dev.w3.org/html5/html-author/charref

three ways to code chars, pick your favorite one


ő
&odblac;
&#x00151;
&#337;

Posted at 2021-04-25 by user127815

yes, that will be the solution.
Thanks

Posted at 2021-04-25 by @fanoush

utf-8 can be broken on many layers, I just tried in WEbIDE with czech letters.
print(E.toArrayBuffer("ěščřýá"));
typing this directly in left side console produce '?' and it really stores characters ?

>print(E.toArrayBuffer("?????ýá"))
new Uint8Array([63, 63, 63, 63, 63, 253, 225]).buffer

putting this to right side editor and uploading is better and produces this, also when typing cursor up it can be sen it is sent via \u unicode encoding

new Uint8Array([196, 155, 197, 161, 196, 141, 197, 153, 253, 225]).buffer
>
>print("\u00c4\u009b\u00c5\u00a1\u00c4\u008d\u00c5\u0099\u00fd\u00e1");
Ä�Å¡Ä�Å�ýá
=undefined

so now this looks like utf-8 encoded except last two letters ýá which are more common and not specific to czech, correct encoding is maybe prined here https://mothereff.in/utf-8#%C4%9B%C5%A1%C4%8D%C5%99%C3%BD%C3%A1

so most probably it is broken already when uploading the source code.

however when now tried to upload this print(E.toArrayBuffer("őýá")); the ő looks utf-8 encoded

new Uint8Array([197, 145, 253, 225]).buffer
>
>print(E.toArrayBuffer("\u00c5\u0091\u00fd\u00e1"));
new Uint8Array([197, 145, 253, 225]).buffer

so maybe 'correct' letters in that web page show because they are not utf-8 encoded and are actually also not shown as utf-8

Posted at 2021-04-25 by user127815

you maybe right. it occurred to me that perhaps the character encoding of the operating system could also cause this error. utf-8 character encoding seems good, but it is actually saved incorrectly in the source code. under windows, you can set the BOM in the text editor, which, if I know it, saves the block needed for utf-8/16 encoding in the given text file. this option i cant find for ubuntu gedit.

https://www.w3.org/International/questions/qa-byte-order-mark

Posted at 2021-04-25 by user127815

Do you know what is the maximum length of storable/runnable javascript code in Espruino wifi?

Posted at 2021-04-25 by Robin

Sun 2021.04.25

'what is the maximum length of storable/runnable javascript code in Espruino wifi?'

Use process.memory() to determine the amount of free JsVars

http://www.espruino.com/Reference#l_process_memory
JsVars: https://www.espruino.com/Internals

Although it may be possible to use some fancy techinques to utilize flash in addition to available ROM, typically, Espruino needs as much memory as the code file size being uploaded, in order to unpack and reorganize in memory.

Really rough approximation; take the number of free JsVars and divide by two to account for the need to reorganize during the upload process, then multiply by 16 bytes per JsVar.

I don't have a WiFi handy, but if memory serves, an authentic Espruino Wifi provides around 2200 JsVars.

2200 / 2 * 16 = 17,600 chars rough approximation

Posted at 2021-04-25 by @fanoush

Do you know what is the maximum length of storable/runnable javascript code in Espruino wifi?

require("Storage").getFree() -this is space when saving code to flash. as for html you can even upload parts of your pages as separate files to Storage and then use Storage.read("filename") in place of the strings, maybe the utf-8 encoding could be better with that.

Posted at 2021-04-26 by user127815

i just thought editor minimizing while downloading javascript code, so i can write my code beautifulized with tabulators and newlines. no needed to minimize manually to save flash memory. is it correct?

Posted at 2021-04-26 by user127815

This solution seems to be better than javascript inline html code. i tried to save the html code to the sd card and read it back to a variable while initializing uc but i don't want my html firmware to be public.

Posted at 2021-04-26 by @gfwilliams

Hi! I'm surprised about the CPU temperature - is anything connected that might be causing a short? The Espruino WiFi does run quite hot when connected to WiFi, but normally I'd expect temperatures in the ~40 degrees C range. Above 50 is getting a bit high.

The whole UTF-8 thing in Espruino is a bit of a pain. It stems from some code Amperka added to better handle encoding Russian characters, but I've had to tweak it a few times and honestly it just seems to make matters worse in most cases I have come across. As @MaBecker mentioned, encoding non-ASCII characters in HTML &...; notation is probably the easiest solution (or uploading the files direct to Storage).

Posted at 2021-04-27 by Robin

Mon 2021.04.26

ref #32: 'so i can write my code beautifulized . . . no needed to minimize manually . . . is it correct?'

I had this same question over four years ago and several provided responses to clear things up.

Originally, I had mistaken an observation (noobie incorrect interpretation) for changes that had just occured with the new WebIDE ver ?? around Nov 2016 circa 1V88 Espruino, and was attempting to resolve.

Just released (then) the WebIDE does a bit of housekeeping before upload along with additional cleanup once Espruino takes over. See @gfwilliams 's comments:

'and also strips out whitespace and comments between commands, which would have been ignored by Espruino anyway.'

http://forum.espruino.com/conversations/295641/

'if something happens and data is re-arranged in memory then even that can effect the compression'

http://forum.espruino.com/comments/13488204/

A quick test would be to upload some heavily commented code both inside and outside functions, then upload. Use dump() to see how comments are removed. Then use process.memory() to determine the amount of free JsVars to compare with the source.

Another useful tool: Many helpful ideas on using trace() to view actual memory contents.

http://forum.espruino.com/comments/15227580/

Posted at 2021-04-27 by user127815

I don't know exactly how big my final code will be, but I solved the html processing (including the UTF-8 problem) by reading my html code directly from an sd card. Interestingly, the handling of the SH1106 oled (128x64) is handled very well by Espruino compared to e.g. the Raspberry Pi Pico micropython environment (but maybe I was the only one who was lame).
Question to the software developers, is there a Raspberry Pi Pico version of the interpreter in the pipeline?

Posted at 2021-04-27 by @MaBecker

Question to the software developers, is there a Raspberry Pi Pico version of the interpreter in the pipeline?

http://forum.espruino.com/conversations/359042/#comment15855632

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Espruino JavaScript

Bug of Espruino IDE (not supporting UTF-8 comments) #6520

{{title}}

Replies: 1 comment

{{title}}

Select a reply

Espruino JavaScript

Bug of Espruino IDE (not supporting UTF-8 comments) #6520

espruino-discuss3 Feb 12, 2025 Maintainer

Replies: 1 comment

espruino-discuss3 Feb 12, 2025 Maintainer Author

espruino-discuss3
Feb 12, 2025
Maintainer

espruino-discuss3
Feb 12, 2025
Maintainer Author