Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Byte block doesn't behave in the manner I think it should #62

Closed
cymplecy opened this issue May 26, 2021 · 55 comments
Closed

Byte block doesn't behave in the manner I think it should #62

cymplecy opened this issue May 26, 2021 · 55 comments

Comments

@cymplecy
Copy link
Collaborator

cymplecy commented May 26, 2021

Since we are restarting :)

Back in the mists of time, I raised the point that I didn't think the way byte blocks are interpreted was correct.

We had lots of "discussion" and I never manged to convince you - maybe I did but who can remember after so long :) but this is BIG issue for me so I'm re-raising it again :)

In summary, I believe a byte block should return a single byte (or char) e.g byte(10) should give a single buffer with a value of 10 (e.g. a newline char) not the decimal number 10 as it does at the moment

This should not produce 10
image

it should give

image

@cymplecy cymplecy added invalid This doesn't seem right duplicate This issue or pull request already exists and removed duplicate This issue or pull request already exists invalid This doesn't seem right labels May 26, 2021
@cymplecy cymplecy reopened this May 31, 2021
@cymplecy
Copy link
Collaborator Author

cymplecy commented May 31, 2021

I am VERY confused where we are with this - I'm re-opening but lets wait till all the other stuff is finished and then we can concentrate on this again :)

I've re-read all the stuff we talked about back in 2019. I got myself up to speed back then on JS buffer's but I'd need to wind my brain back into advanced JS/Blockly mode to deal with again

I only want go go round on this subject one last time :)

@cymplecy
Copy link
Collaborator Author

@bartbutenaers
Here is the bit of our messages that refers to this https://discourse.nodered.org/t/getting-blockly-1-2-0-out-of-the-door/14294/6?u=cymplecy

@bartbutenaers
Copy link
Owner

@cymplecy,

I have tried to summarize our discussions from the past, which started to look like the Hundred Years' War ;-)
Please correct me if I have interpreted anything incorrectly!
Then we can start from scratch here, and hopefully @jsccjj can join the discussion for technical insights...

Issue 1 - A byte block should return a single byte (or char)

For example byte(10) should return a single buffer with a value of 10 (e.g. a newline char) not the decimal number 10 as it does at the moment.

image

That makes sense to me, but:

  • That might break existing flows, so we need to inform users about this breaking change.
  • Currently the Byte-block output is type Number. When this output type is changed, we will have to adapt the other blocks to, to accept the two new types.
  • It is not clear yet to me which output type you expect, because type Char doesn't exist here?

Issue 2 - Allow other types of inputs for buffer_set_index block

At this moment the blocks in this branch allow this kind of input:

  • fill_buffer allows String, Buffer and Number"
    image

  • buffer_set_index accepts only Number

You wanted to allow extra input types on the buffer_set_index block (to have both blocks accepting the same input):

image

Although I understand your intentions, it feels a bit like cheating to me...
Because you had to propose to throw away a (major) part of the input data:

  • Strings: You wanted to only keep the first char of the string, and ignore the remaining part of the string (since you only need 1 char). My proposal was to write the entire string into the buffer, starting from the specified index (and only ignore the part of the string that doesn't fit into the buffer length): by generating a buffer.write(someString); statement if length of the string > 1.

  • Numbers: You wanted to limit the input from 0-255. My idea was again to write all the specified numbers into the buffer, starting from the specified index...

Some remarks:

  1. You wanted to implement the same input preprocessing (e.g. only keep the first char of a string) also in the fill_buffer node. Not sure if that is what we want.

  2. You also suggested to remove the Byte-block from the toolbar, as soon as this change was implemented. I'm not sure anymore why that was suggested?

  3. You also wanted to allow variables as input.

    image

    Although that would offer a lot of flexibility, it is not clear to me how we can check which data is in the variable. Otherwise people can store any kind of data in a byte, even an entire jpeg image ... And we only know at runtime which data is stored in a variable, not at code-generation time...

@cymplecy
Copy link
Collaborator Author

cymplecy commented Jun 18, 2021

Before we get into all the options over filling buffers and stuff like that I'd just want to get agreement on the fundemental issue

e.g byte 65 should NOT return the number 65, it should return a single buffer object with a value of 65.

Now, we can either correct this or we can drop the byte block altogether.

I think it will be much easier to drop the block as it's not actually needed to produce buffer objects by the rest of the blocks. We can just use number and text blocks.

I think we should try this approach and see where it leads.

If it proves to be wrong, then roll it back and proceed down the path of making it return a single buffer object, length 1

So in direct response to:

image

  1. Yes - it will be a breaking change - but since NR is going to 2.x.x, now is the perfect time for this
  2. My suggestion is remove it
  3. If we do keep it, it should be buffer object, length 1

@bartbutenaers
Copy link
Owner

Ok, suppose we remove the Byte-block from the toolbar. What happens then with the set-byte-at-index block? It will need to accept other values and get another shadow block. And the we arrive again at the question if we need to truncate the Number/String? Or am I mistaken?

@cymplecy
Copy link
Collaborator Author

"Ok, suppose we remove the Byte-block from the toolbar." OK :)

"What happens then with the set-byte-at-index block? It will need to accept other values and get another shadow block."
Yes - it will need to accept numbers, text and variables

"And the we arrive again at the question if we need to truncate the Number/String"
My opinion (personal) is that it should just set the byte at the wanted position with either the number or the 1st char of the string.

The existing blocks can convert a long text string to a buffer. I would add an explicit block to concatenate two buffers.

I think we also need a length of buffer block

@bartbutenaers
Copy link
Owner

Ok, you win. Had too much discussions already lately :-(
Will see what I can do...

@jsccjj
Copy link
Contributor

jsccjj commented Jun 20, 2021

Hi guys,

I wanted to join the discussion but found that this is a lengthy discussion.... Many historical considerations are there... I think I can only provide my two pennies here ( please excuse me if there is any misunderstanding):

I think we can modify the byte block to return a single buffer object. Then, we modify the set-byte-at-index block to convert the input (now, it is a buffer object, length 1) back to value (number, 0-255). So, the existing users won't be affected while the byte block can return a single buffer object.... I hope this is feasible

@bartbutenaers
Copy link
Owner

I think we also need a length of buffer block

Doesn't that exist already?
See:

image

I would add an explicit block to concatenate two buffers.

I have added a new block:

image

Here is an example flow to concatenate buffer1 (containing "Hello ") and buffer2 (containing " world"):

image

[{"id":"935786d92e05853b","type":"Blockly","z":"c2a7925b.6e143","func":"var buffer1, buffer2;\n\n\nbuffer1 = (Buffer.from('Hello ', \"utf8\"));\nbuffer2 = (Buffer.from('world', \"utf8\"));\nmsg['payload'] = (Buffer.concat([buffer1, buffer2]));\nnode.send([msg]);\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\"k;r1=:Al*Fb:A+@;B^]D\">buffer1</variable>\n    <variable id=\"mtb,+}W$:ly1N9jU*k#v\">buffer2</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"#:g7*k:IR]si7j[1]T*M\" x=\"-312\" y=\"-137\">\n    <field name=\"VAR\" id=\"k;r1=:Al*Fb:A+@;B^]D\">buffer1</field>\n    <value name=\"VALUE\">\n      <block type=\"buffer_from_string\" id=\"KN=V^94,q2MiQMLEyqBL\">\n        <field name=\"ENCODING\">utf8</field>\n        <value name=\"STRING_INPUT\">\n          <shadow type=\"text\" id=\"aqb3oX068$Cxr2[wG_-T\">\n            <field name=\"TEXT\">Hello </field>\n          </shadow>\n        </value>\n      </block>\n    </value>\n    <next>\n      <block type=\"variables_set\" id=\"]_@5:.A^k`?=V6Z{FY90\">\n        <field name=\"VAR\" id=\"mtb,+}W$:ly1N9jU*k#v\">buffer2</field>\n        <value name=\"VALUE\">\n          <block type=\"buffer_from_string\" id=\"}4o_p-|Yu?nx6sV%zgD_\">\n            <field name=\"ENCODING\">utf8</field>\n            <value name=\"STRING_INPUT\">\n              <shadow type=\"text\" id=\"0*5-G:4L~d;xU$=5Dcca\">\n                <field name=\"TEXT\">world</field>\n              </shadow>\n            </value>\n          </block>\n        </value>\n        <next>\n          <block type=\"node_object_set\" id=\")$Yqthfe9^@=f2!AL!av\">\n            <value name=\"object_field\">\n              <shadow type=\"node_msg\" id=\"]gl2-i%AxlP[,^32MPj(\"></shadow>\n            </value>\n            <value name=\"field_name\">\n              <shadow type=\"text\" id=\"f3ySgBn}o+v5p|@]0jOB\">\n                <field name=\"TEXT\">payload</field>\n              </shadow>\n            </value>\n            <value name=\"value_field\">\n              <shadow type=\"text\" id=\"/(;1bC54JyJza87k46:3\">\n                <field name=\"TEXT\"></field>\n              </shadow>\n              <block type=\"buffer_concatenate\" id=\"6h`{e?{}Vq$k|x~ZmEs}\">\n                <value name=\"BUFFER_FIRST\">\n                  <block type=\"variables_get\" id=\"*j0g6t{0f10:fzsm7wi.\">\n                    <field name=\"VAR\" id=\"k;r1=:Al*Fb:A+@;B^]D\">buffer1</field>\n                  </block>\n                </value>\n                <value name=\"BUFFER_SECOND\">\n                  <block type=\"variables_get\" id=\"f}d{cL,3P2h~F`-Q{cTk\">\n                    <field name=\"VAR\" id=\"mtb,+}W$:ly1N9jU*k#v\">buffer2</field>\n                  </block>\n                </value>\n              </block>\n            </value>\n            <next>\n              <block type=\"node_send\" id=\"tweuq)?Rp7^)T]bVi.YU\">\n                <field name=\"OUTPUT_NR\">1</field>\n                <value name=\"MESSAGE_INPUT\">\n                  <shadow type=\"node_msg\" id=\"G7rwZXpq8EQ@JI_i=pq#\"></shadow>\n                </value>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"","x":1140,"y":360,"wires":[["7011a862f6c538be"]]},{"id":"9512866d85dd94d0","type":"inject","z":"c2a7925b.6e143","name":"Start concatenation","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payloadType":"date","x":950,"y":360,"wires":[["935786d92e05853b"]]},{"id":"7011a862f6c538be","type":"debug","z":"c2a7925b.6e143","name":"Concatenated buffer","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1350,"y":360,"wires":[]},{"id":"d7a036fa.2c0298","type":"blockly-config","language":"en","showTrashcan":true,"allowComments":true,"showZoomControl":true,"enableBackPack":"node","backpackContents":[],"toolboxPosition":"left","renderer":"geras","categories":[{"name":"Node-RED","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Objects","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Buffer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Date/time","files":["blockly-contrib/npm/@blockly%2Ffield-date/dist/date_compressed.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Timer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly extension","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly standard","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/basic/toolbox.xml"]},{"name":"Custom category","files":["blockly-contrib/file/c:/temp/myblocks/myBlocksCodeGen.js","blockly-contrib/file/c:/temp/myblocks/myBlocksDefs.js","blockly-contrib/file/c:/temp/myblocks/myToolbox.xml","blockly-contrib/file/c:/temp/myblocks/my_messages/en.js"]}],"customizeToolbox":true,"name":"Left"}]

Which returns this (which is best readable via the "raw" button):

image

@cymplecy
Copy link
Collaborator Author

"Doesn't that exist already?"
Sorry- got confused :)

Concat works for me as well :)

@bartbutenaers
Copy link
Owner

I think we are done developing for 2.0.0-beta1 if we have fixed the byte block, to return a single byte buffer.
To accomplish that, I have asked a new question on the blockly forum:
https://groups.google.com/g/blockly/c/9fEPSGFarNM
As soon as I have an answer, I will continue with it...

@bartbutenaers
Copy link
Owner

Hi guys (@cymplecy, @jsccjj),
I have pushed my (hopefully) last commit for the release...
Content: the byte block returns a buffer of length 1, and the set-byte-at-index block accepts a number/string/buffer.

It is not really 100% working, but the daily job is a bit of a hobby killer for me at the moment ...
Would be nice if you guys could have look whether the generated code is ok for strings and buffers. For numbers the code generation still needs to be implemented (see here), but my head is exploding...

Here is my test flow:

[{"id":"935786d92e05853b","type":"Blockly","z":"c2a7925b.6e143","func":"(Buffer.alloc(0))[0] = 123456;\n(Buffer.alloc(0))[0] = (Buffer.alloc(1, 255))[0];\n(Buffer.alloc(0))[0] = '123456'.charAt(0);\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <block type=\"buffer_set_index\" id=\"+/8nH+xwf5(OS@]2DKKz\" x=\"-463\" y=\"-187\">\n    <value name=\"INDEX\">\n      <shadow type=\"math_number\" id=\"ddwr~I.`,0*vaRov}?/u\">\n        <field name=\"NUM\">1</field>\n      </shadow>\n    </value>\n    <value name=\"BUFFER\">\n      <shadow type=\"buffer_empty\" id=\"/?@Vr{yb4i]X1#ZzNR(D\"></shadow>\n    </value>\n    <value name=\"VALUE\">\n      <block type=\"math_number\" id=\"^q59jH:{pIzwq;@=TH[h\">\n        <field name=\"NUM\">123456</field>\n      </block>\n    </value>\n    <next>\n      <block type=\"buffer_set_index\" id=\"{$~{Vv|F4;v7fS3bN#OZ\">\n        <value name=\"INDEX\">\n          <shadow type=\"math_number\" id=\"}+QIQ-|k(.RauNL/$~BG\">\n            <field name=\"NUM\">1</field>\n          </shadow>\n        </value>\n        <value name=\"BUFFER\">\n          <shadow type=\"buffer_empty\" id=\"n}:p:TdXQ+=Iz+h9w9[X\"></shadow>\n        </value>\n        <value name=\"VALUE\">\n          <block type=\"buffer_byte\" id=\"B2r/`@yJyJ`_yh?{M^Ix\">\n            <field name=\"BYTE_VALUE\">255</field>\n          </block>\n        </value>\n        <next>\n          <block type=\"buffer_set_index\" id=\"7$o^+dCMW=e3Mu/0U5^j\">\n            <value name=\"INDEX\">\n              <shadow type=\"math_number\" id=\"~//2cZ[F[t=`@UN5w0Jj\">\n                <field name=\"NUM\">1</field>\n              </shadow>\n            </value>\n            <value name=\"BUFFER\">\n              <shadow type=\"buffer_empty\" id=\"$Ug$F+(_6yt!Tv7Gp(u;\"></shadow>\n            </value>\n            <value name=\"VALUE\">\n              <block type=\"text\" id=\"hEM2=v%U|E~W7%xh1tZ3\">\n                <field name=\"TEXT\">123456</field>\n              </block>\n            </value>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"","x":1140,"y":360,"wires":[[]]},{"id":"9512866d85dd94d0","type":"inject","z":"c2a7925b.6e143","name":"Set byte in buffer","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payloadType":"date","x":940,"y":360,"wires":[["935786d92e05853b"]]},{"id":"d7a036fa.2c0298","type":"blockly-config","language":"en","showTrashcan":true,"allowComments":true,"showZoomControl":true,"enableBackPack":true,"backpackContents":[],"toolboxPosition":"left","renderer":"geras","categories":[{"name":"Node-RED","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Objects","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Buffer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Date/time","files":["blockly-contrib/npm/@blockly%2Ffield-date/dist/date_compressed.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Timer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly extension","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly standard","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/basic/toolbox.xml"]},{"name":"Custom category","files":["blockly-contrib/file/c:/temp/myblocks/myBlocksCodeGen.js","blockly-contrib/file/c:/temp/myblocks/myBlocksDefs.js","blockly-contrib/file/c:/temp/myblocks/myToolbox.xml","blockly-contrib/file/c:/temp/myblocks/my_messages/en.js"]}],"customizeToolbox":false,"name":"Left"}]

Thanks a lot !!!

@cymplecy
Copy link
Collaborator Author

Not seeing any behaviour change on my system

image
image

@cymplecy
Copy link
Collaborator Author

I went and checked to see if you'd committed it and I see you've removed release-1.1.0 and merged everthing into master :)

I didn't read the error msg when I updated :)

I'll test using master

image

@cymplecy
Copy link
Collaborator Author

Working now :)

image

image

Will do some more testing

@cymplecy
Copy link
Collaborator Author

How to you want to proceed with this. I've got ideas on how to improve things a bit.
Do you want me to:

1 Just post them here one at a time
2. Do all changes/suggestions and then post them here
3. Do all changes/suggestions in a fork that you can then pull?

@bartbutenaers
Copy link
Owner

Hi Simon,
Yes I should have told that I merged everything to master, so we have a nice clean repository...

Thanks for analyzing the buffer related blocks!

I assume you are only changing the block definition and code generator files. Just do your changes in both files, and describe here what and why you have done it. Then we can try it and discuss it with you...

@cymplecy
Copy link
Collaborator Author

1st issue to discuss is that the get byte from buffer doesn't return a byte (buffer length 1)

image

but I'd like to suggest that instead of returning a buffer length 1 - the text should be changed to

get value of index ... of buffer ...

as returning a single byte value isn't going to be used much

or maybe it could be modified with a dropdown to return a value, single char text string or a byte???

@bartbutenaers
Copy link
Owner

Quick feedback, because I had to work today from 7:45 to 21:30 ...

the text should be changed

Damn that is a pity. I had thought we only had to change code. Now I have to ask the translator guys again to translate again... But indeed it makes sense what you suggest.

or maybe it could be modified with a dropdown to return a value, single char text string or a byte???

Personally I don't really like that, because it will simply return the data that is in the buffer at the specified index...

@bartbutenaers
Copy link
Owner

@cymplecy,
Do you like me to implement this?

@cymplecy
Copy link
Collaborator Author

cymplecy commented Jul 26, 2021 via email

@cymplecy
Copy link
Collaborator Author

So to do the change just alter
image
in en.js

@cymplecy
Copy link
Collaborator Author

cymplecy commented Jul 26, 2021

I've been playing with trying to get it to handle the differences between numbers,strings, buffer and variable types and come up with this

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  try {  // to see if we can get the type of the value
    var dataType = block.getInput('VALUE').connection.targetConnection.getCheck()[0];
  } catch { // if not - assume its a variable
    const code = buffer + '[' + index + '] = ' + value + ';\n';
    return code;
  }

  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      // Get the first character of the input string
      value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
      value = value.charCodeAt(0);
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
  }

Basic philosphy is that if it fails to find the datatype, then it's a variable and it returns the code to deal with that

If it's a number - it makes sure its in the range 0 - 255

If its a string - it takes the value of the 1st char of the string

if it's a byte - then it uses the value of the byte

It seems to handle all 4 scenarios in my testing

It will probably fail if the contents of the variable are not numeric but maybe some more JS could make it fail safely

@cymplecy
Copy link
Collaborator Author

cymplecy commented Jul 26, 2021

My testing flow
image

[{"id":"935786d92e05853b","type":"Blockly","z":"0ebed3e2bb0753df","func":"var testBuffer;\n\n\ntestBuffer = (Buffer.alloc(10));\ntestBuffer[0] = 254;\nmsg['payload'] = testBuffer;\nreturn msg;\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"22_3lZTO,U+Yg[$S,6Qf\" x=\"-387\" y=\"-587\">\n    <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n    <value name=\"VALUE\">\n      <block type=\"buffer_alloc\" id=\"gOpuNQ.fpDvk|f~VV8n-\">\n        <value name=\"LENGTH\">\n          <shadow type=\"math_number\" id=\"/h@`xv2)0Cuib]mc}vP`\">\n            <field name=\"NUM\">10</field>\n          </shadow>\n        </value>\n      </block>\n    </value>\n    <next>\n      <block type=\"buffer_set_index\" id=\"7$o^+dCMW=e3Mu/0U5^j\">\n        <value name=\"INDEX\">\n          <shadow type=\"math_number\" id=\"~//2cZ[F[t=`@UN5w0Jj\">\n            <field name=\"NUM\">1</field>\n          </shadow>\n        </value>\n        <value name=\"BUFFER\">\n          <shadow type=\"buffer_empty\" id=\"$Ug$F+(_6yt!Tv7Gp(u;\"></shadow>\n          <block type=\"variables_get\" id=\"KTR5:V1bFT0k`_H6]0#q\">\n            <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n          </block>\n        </value>\n        <value name=\"VALUE\">\n          <block type=\"math_number\" id=\"a(JS7vsw.3WnJXXUpyuo\">\n            <field name=\"NUM\">254</field>\n          </block>\n        </value>\n        <next>\n          <block type=\"node_object_set\" id=\"C3$AJ{J$X4GMcQ~P.Y}C\" inline=\"true\">\n            <value name=\"object_field\">\n              <shadow type=\"node_msg\" id=\"@k~uw$CUXFpD(YZJT^*x\"></shadow>\n            </value>\n            <value name=\"field_name\">\n              <shadow type=\"text\" id=\"~,*uud-2mQ+~{i}a:[+y\">\n                <field name=\"TEXT\">payload</field>\n              </shadow>\n            </value>\n            <value name=\"value_field\">\n              <shadow type=\"text\">\n                <field name=\"TEXT\"></field>\n              </shadow>\n              <block type=\"variables_get\" id=\"N{mmHXgP84oF_Tj0%i]g\">\n                <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n              </block>\n            </value>\n            <next>\n              <block type=\"node_return_message\" id=\"Ztq4^UzX6wdbv9i:{a*F\">\n                <field name=\"OUTPUT_NR\">1</field>\n                <value name=\"MESSAGE_INPUT\">\n                  <shadow type=\"node_msg\" id=\"vY1|C#bvL$?ixB0t4x3k\"></shadow>\n                </value>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"","x":320,"y":60,"wires":[["ce8a88fc.ada2b8"]]},{"id":"9512866d85dd94d0","type":"inject","z":"0ebed3e2bb0753df","name":"Set byte in buffer","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payloadType":"date","x":120,"y":120,"wires":[["935786d92e05853b","d08f055843d1d370","562f46f07824a1f9","61b8cf83e18214e8"]]},{"id":"ce8a88fc.ada2b8","type":"debug","z":"0ebed3e2bb0753df","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":510,"y":60,"wires":[]},{"id":"73d66ccb.24a5b4","type":"debug","z":"0ebed3e2bb0753df","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":510,"y":120,"wires":[]},{"id":"7219ba03.df7314","type":"debug","z":"0ebed3e2bb0753df","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":510,"y":180,"wires":[]},{"id":"d08f055843d1d370","type":"Blockly","z":"0ebed3e2bb0753df","func":"var testBuffer;\n\n\ntestBuffer = (Buffer.alloc(10));\ntestBuffer[0] = 104;\nmsg['payload'] = testBuffer;\nreturn msg;\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"22_3lZTO,U+Yg[$S,6Qf\" x=\"-387\" y=\"-587\">\n    <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n    <value name=\"VALUE\">\n      <block type=\"buffer_alloc\" id=\"gOpuNQ.fpDvk|f~VV8n-\">\n        <value name=\"LENGTH\">\n          <shadow type=\"math_number\" id=\"/h@`xv2)0Cuib]mc}vP`\">\n            <field name=\"NUM\">10</field>\n          </shadow>\n        </value>\n      </block>\n    </value>\n    <next>\n      <block type=\"buffer_set_index\" id=\"+/8nH+xwf5(OS@]2DKKz\">\n        <value name=\"INDEX\">\n          <shadow type=\"math_number\" id=\"ddwr~I.`,0*vaRov}?/u\">\n            <field name=\"NUM\">1</field>\n          </shadow>\n        </value>\n        <value name=\"BUFFER\">\n          <shadow type=\"buffer_empty\" id=\"/?@Vr{yb4i]X1#ZzNR(D\"></shadow>\n          <block type=\"variables_get\" id=\"Nw|k`Ev3nCx6ojPpH}{%\">\n            <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n          </block>\n        </value>\n        <value name=\"VALUE\">\n          <block type=\"text\" id=\"qvN~}Ti8MA!{RR%EFUlP\">\n            <field name=\"TEXT\">hello</field>\n          </block>\n        </value>\n        <next>\n          <block type=\"node_object_set\" id=\"g]f+4#M,V)aj2So-m7M}\" inline=\"true\">\n            <value name=\"object_field\">\n              <shadow type=\"node_msg\" id=\"x!8r;JYSdR{+b!pbO_UM\"></shadow>\n            </value>\n            <value name=\"field_name\">\n              <shadow type=\"text\" id=\"neD;Gi@j#~zpZ}zUZHqJ\">\n                <field name=\"TEXT\">payload</field>\n              </shadow>\n            </value>\n            <value name=\"value_field\">\n              <shadow type=\"text\" id=\"-Bo*3[1Cvj@bf{vOli7F\">\n                <field name=\"TEXT\"></field>\n              </shadow>\n              <block type=\"variables_get\" id=\"SOUV)fY1i=,Os191tH,v\">\n                <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n              </block>\n            </value>\n            <next>\n              <block type=\"node_return_message\" id=\"`02SC[6h{,{u@gs0X4hW\">\n                <field name=\"OUTPUT_NR\">1</field>\n                <value name=\"MESSAGE_INPUT\">\n                  <shadow type=\"node_msg\" id=\"}:0JW{Ow,NS$O$lvS$o`\"></shadow>\n                </value>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"","x":320,"y":120,"wires":[["73d66ccb.24a5b4"]]},{"id":"562f46f07824a1f9","type":"Blockly","z":"0ebed3e2bb0753df","func":"var testBuffer;\n\n\ntestBuffer = (Buffer.alloc(10));\ntestBuffer[0] = (Buffer.alloc(1, 127))[0];\nmsg['payload'] = testBuffer;\nreturn msg;\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"22_3lZTO,U+Yg[$S,6Qf\" x=\"-387\" y=\"-587\">\n    <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n    <value name=\"VALUE\">\n      <block type=\"buffer_alloc\" id=\"gOpuNQ.fpDvk|f~VV8n-\">\n        <value name=\"LENGTH\">\n          <shadow type=\"math_number\" id=\"/h@`xv2)0Cuib]mc}vP`\">\n            <field name=\"NUM\">10</field>\n          </shadow>\n        </value>\n      </block>\n    </value>\n    <next>\n      <block type=\"buffer_set_index\" id=\"{$~{Vv|F4;v7fS3bN#OZ\">\n        <value name=\"INDEX\">\n          <shadow type=\"math_number\" id=\"}+QIQ-|k(.RauNL/$~BG\">\n            <field name=\"NUM\">1</field>\n          </shadow>\n        </value>\n        <value name=\"BUFFER\">\n          <shadow type=\"buffer_empty\" id=\"n}:p:TdXQ+=Iz+h9w9[X\"></shadow>\n          <block type=\"variables_get\" id=\"xOq/5k(-hTm1lL}etL;2\">\n            <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n          </block>\n        </value>\n        <value name=\"VALUE\">\n          <block type=\"buffer_byte\" id=\"B2r/`@yJyJ`_yh?{M^Ix\">\n            <field name=\"BYTE_VALUE\">127</field>\n          </block>\n        </value>\n        <next>\n          <block type=\"node_object_set\" id=\"R@H@}jC/Fp_9K1Gz7S9n\" inline=\"true\">\n            <value name=\"object_field\">\n              <shadow type=\"node_msg\" id=\":=bCe5*%VD@?_]g?3H(5\"></shadow>\n            </value>\n            <value name=\"field_name\">\n              <shadow type=\"text\" id=\"]lB$ZtFJXRW33lj}|p/$\">\n                <field name=\"TEXT\">payload</field>\n              </shadow>\n            </value>\n            <value name=\"value_field\">\n              <shadow type=\"text\">\n                <field name=\"TEXT\"></field>\n              </shadow>\n              <block type=\"variables_get\" id=\"yfj{Js,YA-`;MI0ai@:8\">\n                <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n              </block>\n            </value>\n            <next>\n              <block type=\"node_return_message\" id=\"[aXz(jH!KTMNQw.~F=YL\">\n                <field name=\"OUTPUT_NR\">1</field>\n                <value name=\"MESSAGE_INPUT\">\n                  <shadow type=\"node_msg\" id=\"6PC*c1vQ?0cB8lMdx_X-\"></shadow>\n                </value>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"","x":320,"y":180,"wires":[["7219ba03.df7314"]]},{"id":"dab3681868768c11","type":"debug","z":"0ebed3e2bb0753df","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":510,"y":240,"wires":[]},{"id":"61b8cf83e18214e8","type":"Blockly","z":"0ebed3e2bb0753df","func":"var testVariable, testBuffer;\n\n\ntestVariable = 129;\ntestBuffer = (Buffer.alloc(10));\ntestBuffer[0] = testVariable;\nmsg['payload'] = testBuffer;\nreturn msg;\n\n254;\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\"LFEG^MuL.Q[,CZg~=HO!\">testVariable</variable>\n    <variable id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"bDUeNBJ~rcEy_RX0tw+O\" x=\"-362\" y=\"-612\">\n    <field name=\"VAR\" id=\"LFEG^MuL.Q[,CZg~=HO!\">testVariable</field>\n    <value name=\"VALUE\">\n      <block type=\"math_number\" id=\"Rt8~jyotbH+}q[-tLHHb\">\n        <field name=\"NUM\">129</field>\n      </block>\n    </value>\n    <next>\n      <block type=\"variables_set\" id=\"22_3lZTO,U+Yg[$S,6Qf\">\n        <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n        <value name=\"VALUE\">\n          <block type=\"buffer_alloc\" id=\"gOpuNQ.fpDvk|f~VV8n-\">\n            <value name=\"LENGTH\">\n              <shadow type=\"math_number\" id=\"/h@`xv2)0Cuib]mc}vP`\">\n                <field name=\"NUM\">10</field>\n              </shadow>\n            </value>\n          </block>\n        </value>\n        <next>\n          <block type=\"buffer_set_index\" id=\"7$o^+dCMW=e3Mu/0U5^j\">\n            <value name=\"INDEX\">\n              <shadow type=\"math_number\" id=\"~//2cZ[F[t=`@UN5w0Jj\">\n                <field name=\"NUM\">1</field>\n              </shadow>\n            </value>\n            <value name=\"BUFFER\">\n              <shadow type=\"buffer_empty\" id=\"$Ug$F+(_6yt!Tv7Gp(u;\"></shadow>\n              <block type=\"variables_get\" id=\"KTR5:V1bFT0k`_H6]0#q\">\n                <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n              </block>\n            </value>\n            <value name=\"VALUE\">\n              <block type=\"variables_get\" id=\"I*^,}^9JD$k.czaACOBp\">\n                <field name=\"VAR\" id=\"LFEG^MuL.Q[,CZg~=HO!\">testVariable</field>\n              </block>\n            </value>\n            <next>\n              <block type=\"node_object_set\" id=\"C3$AJ{J$X4GMcQ~P.Y}C\" inline=\"true\">\n                <value name=\"object_field\">\n                  <shadow type=\"node_msg\" id=\"@k~uw$CUXFpD(YZJT^*x\"></shadow>\n                </value>\n                <value name=\"field_name\">\n                  <shadow type=\"text\" id=\"~,*uud-2mQ+~{i}a:[+y\">\n                    <field name=\"TEXT\">payload</field>\n                  </shadow>\n                </value>\n                <value name=\"value_field\">\n                  <shadow type=\"text\">\n                    <field name=\"TEXT\"></field>\n                  </shadow>\n                  <block type=\"variables_get\" id=\"N{mmHXgP84oF_Tj0%i]g\">\n                    <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n                  </block>\n                </value>\n                <next>\n                  <block type=\"node_return_message\" id=\"Ztq4^UzX6wdbv9i:{a*F\">\n                    <field name=\"OUTPUT_NR\">1</field>\n                    <value name=\"MESSAGE_INPUT\">\n                      <shadow type=\"node_msg\" id=\"vY1|C#bvL$?ixB0t4x3k\"></shadow>\n                    </value>\n                  </block>\n                </next>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n  <block type=\"math_number\" id=\"a(JS7vsw.3WnJXXUpyuo\" x=\"-112\" y=\"-413\">\n    <field name=\"NUM\">254</field>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"","x":320,"y":240,"wires":[["dab3681868768c11"]]},{"id":"d7a036fa.2c0298","type":"blockly-config","language":"en","showTrashcan":true,"allowComments":true,"showZoomControl":true,"enableBackPack":true,"backpackContents":[],"toolboxPosition":"left","renderer":"geras","categories":[{"name":"Node-RED","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Objects","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Buffer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Date/time","files":["blockly-contrib/npm/@blockly%2Ffield-date/dist/date_compressed.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Timer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly extension","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly standard","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/basic/toolbox.xml"]},{"name":"Custom category","files":["blockly-contrib/file/c:/temp/myblocks/myBlocksCodeGen.js","blockly-contrib/file/c:/temp/myblocks/myBlocksDefs.js","blockly-contrib/file/c:/temp/myblocks/myToolbox.xml","blockly-contrib/file/c:/temp/myblocks/my_messages/en.js"]}],"customizeToolbox":false,"name":"Left"}]

@cymplecy
Copy link
Collaborator Author

Modified version as it doesn't need the try/catch

  // Get the data type checks offered by the block that is connected to our VALUE field
  var targetCheck = block.getInput('VALUE').connection.targetConnection.getCheck();

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
 // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  var dataType;
  if (targetCheck && Array.isArray(targetCheck) && targetCheck.length === 1) {
     dataType = targetCheck[0];
  }
  else {
     // We assume the input is a variable
     //dataType = "Variable";
     let code = '';
     code += 'if (isNaN(' + value + ')) {\n';
         code += '    ' + value + ' = ' + value + '.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(' + value + ')) {\n';
         code += '        ' + value + ' = ' + value + '[0]\n';
       code += '    } else {\n';
         code += '        ' + value + ' = Math.min(Math.max(' + value + ', 0),255)\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + '] = ' + value + ';\n';
     return code;
  }

  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      // Get the first character of the input string
      value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
      value = value.charCodeAt(0);
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
     //case "Variable":
       // Keep the value as it is
  }


  const code = buffer + '[' + index + '] = ' + value + ';\n';
  return code;
};

test flow

[{"id":"dab3681868768c11","type":"debug","z":"0ebed3e2bb0753df","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":510,"y":240,"wires":[]},{"id":"61b8cf83e18214e8","type":"Blockly","z":"0ebed3e2bb0753df","func":"var testVariable, testBuffer;\n\n\ntestVariable = (msg['payload']);\ntestBuffer = (Buffer.alloc(10));\nif (isNaN(testVariable)) {\n    testVariable = testVariable.charCodeAt(0);\n} else {\n    if (Buffer.isBuffer(testVariable)) {\n        testVariable = testVariable[0]\n    } else {\n        testVariable = Math.min(Math.max(testVariable, 0),255)\n    }\n}\ntestBuffer[0] = testVariable;\nmsg['payload'] = testBuffer;\nreturn msg;\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\"LFEG^MuL.Q[,CZg~=HO!\">testVariable</variable>\n    <variable id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"bDUeNBJ~rcEy_RX0tw+O\" x=\"-362\" y=\"-612\">\n    <field name=\"VAR\" id=\"LFEG^MuL.Q[,CZg~=HO!\">testVariable</field>\n    <value name=\"VALUE\">\n      <block type=\"node_object_get\" id=\"i,Z|iYA$zak88n[WSlEd\">\n        <mutation xmlns=\"http://www.w3.org/1999/xhtml\" action=\"GET\"></mutation>\n        <field name=\"action\">GET</field>\n        <value name=\"object\">\n          <shadow type=\"node_msg\" id=\"=88|h?-N6h;kb;?5`/Uv\"></shadow>\n        </value>\n        <value name=\"field_name\">\n          <shadow type=\"text\" id=\"hu=AkkX*0In*g,::C95h\">\n            <field name=\"TEXT\">payload</field>\n          </shadow>\n        </value>\n      </block>\n    </value>\n    <next>\n      <block type=\"variables_set\" id=\"22_3lZTO,U+Yg[$S,6Qf\">\n        <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n        <value name=\"VALUE\">\n          <block type=\"buffer_alloc\" id=\"gOpuNQ.fpDvk|f~VV8n-\">\n            <value name=\"LENGTH\">\n              <shadow type=\"math_number\" id=\"/h@`xv2)0Cuib]mc}vP`\">\n                <field name=\"NUM\">10</field>\n              </shadow>\n            </value>\n          </block>\n        </value>\n        <next>\n          <block type=\"buffer_set_index\" id=\"7$o^+dCMW=e3Mu/0U5^j\">\n            <value name=\"INDEX\">\n              <shadow type=\"math_number\" id=\"~//2cZ[F[t=`@UN5w0Jj\">\n                <field name=\"NUM\">1</field>\n              </shadow>\n            </value>\n            <value name=\"BUFFER\">\n              <shadow type=\"buffer_empty\" id=\"$Ug$F+(_6yt!Tv7Gp(u;\"></shadow>\n              <block type=\"variables_get\" id=\"KTR5:V1bFT0k`_H6]0#q\">\n                <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n              </block>\n            </value>\n            <value name=\"VALUE\">\n              <block type=\"variables_get\" id=\"I*^,}^9JD$k.czaACOBp\">\n                <field name=\"VAR\" id=\"LFEG^MuL.Q[,CZg~=HO!\">testVariable</field>\n              </block>\n            </value>\n            <next>\n              <block type=\"node_object_set\" id=\"C3$AJ{J$X4GMcQ~P.Y}C\" inline=\"true\">\n                <value name=\"object_field\">\n                  <shadow type=\"node_msg\" id=\"@k~uw$CUXFpD(YZJT^*x\"></shadow>\n                </value>\n                <value name=\"field_name\">\n                  <shadow type=\"text\" id=\"~,*uud-2mQ+~{i}a:[+y\">\n                    <field name=\"TEXT\">payload</field>\n                  </shadow>\n                </value>\n                <value name=\"value_field\">\n                  <shadow type=\"text\">\n                    <field name=\"TEXT\"></field>\n                  </shadow>\n                  <block type=\"variables_get\" id=\"N{mmHXgP84oF_Tj0%i]g\">\n                    <field name=\"VAR\" id=\"j]_c}J2@#6QF,s@)L89z\">testBuffer</field>\n                  </block>\n                </value>\n                <next>\n                  <block type=\"node_return_message\" id=\"Ztq4^UzX6wdbv9i:{a*F\">\n                    <field name=\"OUTPUT_NR\">1</field>\n                    <value name=\"MESSAGE_INPUT\">\n                      <shadow type=\"node_msg\" id=\"vY1|C#bvL$?ixB0t4x3k\"></shadow>\n                    </value>\n                  </block>\n                </next>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"d7a036fa.2c0298","backpackContents":[],"noerr":0,"name":"Insert value at index","x":320,"y":240,"wires":[["dab3681868768c11"]]},{"id":"f53404bf2a6d3c13","type":"inject","z":"0ebed3e2bb0753df","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"Hello","payloadType":"str","x":110,"y":360,"wires":[["61b8cf83e18214e8"]]},{"id":"9c6b5188c9f4420b","type":"inject","z":"0ebed3e2bb0753df","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"123","payloadType":"num","x":110,"y":240,"wires":[["61b8cf83e18214e8"]]},{"id":"38035f664ee39bb3","type":"inject","z":"0ebed3e2bb0753df","name":"Buffer [32]","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[32]","payloadType":"bin","x":120,"y":400,"wires":[["61b8cf83e18214e8"]]},{"id":"56e221cb2c2e5641","type":"inject","z":"0ebed3e2bb0753df","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"-100","payloadType":"num","x":110,"y":280,"wires":[["61b8cf83e18214e8"]]},{"id":"2686afe3689edfc9","type":"inject","z":"0ebed3e2bb0753df","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"512","payloadType":"num","x":110,"y":320,"wires":[["61b8cf83e18214e8"]]},{"id":"d7a036fa.2c0298","type":"blockly-config","language":"en","showTrashcan":true,"allowComments":true,"showZoomControl":true,"enableBackPack":true,"backpackContents":[],"toolboxPosition":"left","renderer":"geras","categories":[{"name":"Node-RED","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Objects","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Buffer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Date/time","files":["blockly-contrib/npm/@blockly%2Ffield-date/dist/date_compressed.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Timer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly extension","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly standard","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/basic/toolbox.xml"]},{"name":"Custom category","files":["blockly-contrib/file/c:/temp/myblocks/myBlocksCodeGen.js","blockly-contrib/file/c:/temp/myblocks/myBlocksDefs.js","blockly-contrib/file/c:/temp/myblocks/myToolbox.xml","blockly-contrib/file/c:/temp/myblocks/my_messages/en.js"]}],"customizeToolbox":false,"name":"Left"}]

@bartbutenaers
Copy link
Owner

@cymplecy,
I have tested it and it works nice: when no variable is used, then the generated code is short and simple. When a variable is used, then the type checking is very nice.
For me this is good enough! Nice work!!
Is there anything else we need here, before we close the beta version gates?

@cymplecy
Copy link
Collaborator Author

I think we are good to go and put it out there for others to test. :)

@cymplecy
Copy link
Collaborator Author

It's just failed for me in a loop test - so wait please :(

@cymplecy
Copy link
Collaborator Author

cymplecy commented Jul 28, 2021

Okay - I found two issues with this test code (Note paylod is string Hello)

image

The simple one is that the index cannot be auto-decremented at beginning of code as it might not be a number but may be a variable so the Block/JS 1/0 index correction needs to take place just before the code is returned. This effects the get value at index block as well

The second one complicates things further.
If the value is an expression (e.g not a plain number,string, buffer or variable) then it needs special treatment.

My solution is to create a tempoary variable called tempVariable and add the code to evaluate it at runtime and then use the result of that evaluation.

It all seems to work but the code is quite a mess now :(

Blockly.JavaScript['buffer_set_index'] = function(block) {
  var   index = Blockly.JavaScript.valueToCode(block, 'INDEX', Blockly.JavaScript.ORDER_ATOMIC) || 1;
  //console.log(index);
  const buffer = Blockly.JavaScript.valueToCode(block, 'BUFFER', Blockly.JavaScript.ORDER_ATOMIC);
  var   value = Blockly.JavaScript.valueToCode(block, 'VALUE', Blockly.JavaScript.ORDER_ATOMIC);
  
  // Blockly is 1-based while Javascript is 0-based, so the index needs to be converted
  // This doesn't work if index is a variable so commented out
  //index--;

  // Get the data type checks offered by the block that is connected to our VALUE field 
  var targetCheck = block.getInput('VALUE').connection.targetConnection.getCheck();

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
 // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  var dataType = null;
  if (targetCheck && Array.isArray(targetCheck) && targetCheck.length === 1) {
     dataType = targetCheck[0];
     //console.log(dataType);
     //console.log(value);
  }
      
  if (dataType === null) {
     // We assume the input is a variable
     let code = '';
     code += 'if (isNaN(' + value + ')) {\n';
         code += '    ' + value + ' = ' + value + '.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(' + value + ')) {\n';
         code += '        ' + value + ' = ' + value + '[0]\n';
       code += '    } else {\n';
         code += '        ' + value + ' = Math.min(Math.max(' + value + ', 0),255)\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + ' - 1] = ' + value + ';\n';
     return code;
  }

  if ((dataType == 'String') && (value.charAt(0) == '(') && (value.charAt(value.length - 1) == ')') ) {
    // We assume the input is a code expression
    let code = '';
    code += 'let __tempValue__ = ' + value + ';\n';
    code += 'if (isNaN(__tempValue__)) {\n';
         code += '    __tempValue__ = __tempValue__.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(__tempValue__)) {\n';
         code += '        __tempValue__ = __tempValue__[0]\n';
       code += '    } else {\n';
         code += '        __tempValue__ = Math.min(Math.max(__tempValue__, 0),255)\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + ' - 1] = __tempValue__;\n';
     return code;
  }
     
  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      //check if string is JS code
      if ((value.charAt(0) != '(') && (value.charAt(value.length - 1) != ')')  ) {
      // Get the first character of the input string
        value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
        value = value.charCodeAt(0);
      }
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
     //case "Variable":
       // Keep the value as it is
  }


  const code = buffer + '[' + index + ' - 1] = ' + value + ';\n';
  return code;
};

Blockly.JavaScript['buffer_get_index'] = function(block) {
  var   index = Blockly.JavaScript.valueToCode(block, 'INDEX', Blockly.JavaScript.ORDER_ATOMIC) || 1;
  const buffer = Blockly.JavaScript.valueToCode(block, 'BUFFER', Blockly.JavaScript.ORDER_ATOMIC);
  
  // Blockly is 1-based while Javascript is 0-based, so the index needs to be converted
  // This doesn't work if index is a variable so commented out
  //index--;

  const code = buffer + '[' + index + ' - 1]';
  return [code, Blockly.JavaScript.ORDER_NONE];
};

@cymplecy
Copy link
Collaborator Author

Slight issue - a modified looping fill buffer script
image

is converting the space between Hello and World to a 0 instead of 32

@bartbutenaers
Copy link
Owner

Hi @cymplecy,

Can you please share your flow, so that I can quickly test it.

And can you please explain a bit more in detail why the __tempValue__ had to be introduced.
I'm pretty sure you do it correctly, but at first sight I don't see how e.g.

code += 'if (isNaN(' + value + ')) {\n';

is not correct for an expression, while:

code += 'let __tempValue__ = ' + value + ';\n';
code += 'if (isNaN(__tempValue__)) {\n';

would work better? Enlighten me please ;-)

@cymplecy
Copy link
Collaborator Author

[{"id":"aad9b2b9721868cb","type":"inject","z":"0ebed3e2bb0753df","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"Hello World","payloadType":"str","x":130,"y":640,"wires":[["1fc93ad399ae75c3"]]},{"id":"6975824ad5fc3cba","type":"debug","z":"0ebed3e2bb0753df","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":550,"y":640,"wires":[]},{"id":"1fc93ad399ae75c3","type":"Blockly","z":"0ebed3e2bb0753df","func":"var testVariable, testBuffer, i2;\n\n\ntestVariable = (msg['payload']);\ntestBuffer = (Buffer.alloc(10));\nvar i2_end = Math.min.apply(null, [testVariable.length, testBuffer.length]);\nvar i2_inc = 1;\nif (1 > i2_end) {\n  i2_inc = -i2_inc;\n}\nfor (i2 = 1; i2_inc >= 0 ? i2 <= i2_end : i2 >= i2_end; i2 += i2_inc) {\n  let __tempValue__ = (testVariable.slice((i2 - 1), i2));\n  if (isNaN(__tempValue__)) {\n      __tempValue__ = __tempValue__.charCodeAt(0);\n  } else {\n      if (Buffer.isBuffer(__tempValue__)) {\n          __tempValue__ = __tempValue__[0]\n      } else {\n          __tempValue__ = Math.min(Math.max(__tempValue__, 0),255)\n      }\n  }\n  testBuffer[i2 - 1] = __tempValue__;\n}\nmsg['payload'] = testBuffer;\nnode.send([msg]);\n","workspaceXml":"<xml xmlns=\"https://developers.google.com/blockly/xml\">\n  <variables>\n    <variable id=\")ZBKTUC^^!J999c!;(dH\">testVariable</variable>\n    <variable id=\"cx:#6yd(cD`NZ32~!cqA\">testBuffer</variable>\n    <variable id=\")jU4tpD],qwF{9((T=aQ\">i</variable>\n  </variables>\n  <block type=\"variables_set\" id=\"6?g29wT;A~@ji_TT/A(Y\" x=\"-63\" y=\"-212\">\n    <field name=\"VAR\" id=\")ZBKTUC^^!J999c!;(dH\">testVariable</field>\n    <value name=\"VALUE\">\n      <block type=\"node_object_get\" id=\"RHT8|1}}=EH.@V?L8_D)\">\n        <mutation xmlns=\"http://www.w3.org/1999/xhtml\" action=\"GET\"></mutation>\n        <field name=\"action\">GET</field>\n        <value name=\"object\">\n          <shadow type=\"node_msg\" id=\"u19[kk_q8Yi6^.Vo~OAC\"></shadow>\n        </value>\n        <value name=\"field_name\">\n          <shadow type=\"text\" id=\"v)Dxd*`^w/X:f_i.I[t(\">\n            <field name=\"TEXT\">payload</field>\n          </shadow>\n        </value>\n      </block>\n    </value>\n    <next>\n      <block type=\"variables_set\" id=\"b$;g^y.O~{njEy%uq/Xi\">\n        <field name=\"VAR\" id=\"cx:#6yd(cD`NZ32~!cqA\">testBuffer</field>\n        <value name=\"VALUE\">\n          <block type=\"buffer_alloc\" id=\"ZWUvTn7gm`*YtjYx8PgZ\">\n            <value name=\"LENGTH\">\n              <shadow type=\"math_number\" id=\"Ay]l-cD9-QMZ2H+5Ki^R\">\n                <field name=\"NUM\">10</field>\n              </shadow>\n            </value>\n          </block>\n        </value>\n        <next>\n          <block type=\"controls_for\" id=\"5|uarU#@t$Q99lhk/5K/\">\n            <field name=\"VAR\" id=\")jU4tpD],qwF{9((T=aQ\">i</field>\n            <value name=\"FROM\">\n              <shadow type=\"math_number\" id=\"qKtGgMI)FI]vgn!@HB9g\">\n                <field name=\"NUM\">1</field>\n              </shadow>\n            </value>\n            <value name=\"TO\">\n              <shadow type=\"math_number\" id=\"84aZ`1[F~A:((UaTWfhp\">\n                <field name=\"NUM\">5</field>\n              </shadow>\n              <block type=\"math_on_list\" id=\"Fui^^pp#8Yde7)S1G%$6\">\n                <mutation op=\"MIN\"></mutation>\n                <field name=\"OP\">MIN</field>\n                <value name=\"LIST\">\n                  <block type=\"lists_create_with\" id=\"GT.:/FS{g%mj=7oA$=HI\">\n                    <mutation items=\"2\"></mutation>\n                    <value name=\"ADD0\">\n                      <block type=\"text_length\" id=\"v_;U{KO9kR0(?=bBn6.$\">\n                        <value name=\"VALUE\">\n                          <shadow type=\"text\" id=\".kCoiLNeD`bLABJ%bh~3\">\n                            <field name=\"TEXT\">abc</field>\n                          </shadow>\n                          <block type=\"variables_get\" id=\"PB(F98[vtB;jDaz//gcT\">\n                            <field name=\"VAR\" id=\")ZBKTUC^^!J999c!;(dH\">testVariable</field>\n                          </block>\n                        </value>\n                      </block>\n                    </value>\n                    <value name=\"ADD1\">\n                      <block type=\"buffer_length\" id=\"vc*{)^RKT!$Po;3c#:*)\">\n                        <value name=\"BUFFER_INPUT\">\n                          <shadow type=\"buffer_empty\" id=\"Q*s^Qo%d%_XP%G,!WBK`\"></shadow>\n                          <block type=\"variables_get\" id=\"ZW`D3E6bm,;NtLhgl7}Z\">\n                            <field name=\"VAR\" id=\"cx:#6yd(cD`NZ32~!cqA\">testBuffer</field>\n                          </block>\n                        </value>\n                      </block>\n                    </value>\n                  </block>\n                </value>\n              </block>\n            </value>\n            <value name=\"BY\">\n              <shadow type=\"math_number\" id=\"$YXlbBa1H}%/U$wz^a4C\">\n                <field name=\"NUM\">1</field>\n              </shadow>\n            </value>\n            <statement name=\"DO\">\n              <block type=\"buffer_set_index\" id=\"jM.Q4g0lQ8-$9uEaws}q\" inline=\"false\">\n                <value name=\"INDEX\">\n                  <shadow type=\"math_number\" id=\"2vMhj{C/D`F3!R2.N1R2\">\n                    <field name=\"NUM\">1</field>\n                  </shadow>\n                  <block type=\"variables_get\" id=\"Q1XiaotCgxMAC?9PQ{`z\">\n                    <field name=\"VAR\" id=\")jU4tpD],qwF{9((T=aQ\">i</field>\n                  </block>\n                </value>\n                <value name=\"BUFFER\">\n                  <shadow type=\"buffer_empty\" id=\"H:Sr)H.$1il1Cih%rp_^\"></shadow>\n                  <block type=\"variables_get\" id=\"c?u34UG8hraXq%Ow_/.5\">\n                    <field name=\"VAR\" id=\"cx:#6yd(cD`NZ32~!cqA\">testBuffer</field>\n                  </block>\n                </value>\n                <value name=\"VALUE\">\n                  <shadow type=\"buffer_byte\" id=\"4hb*}}uAu:Ct]-(D/HWk\">\n                    <field name=\"BYTE_VALUE\">127</field>\n                  </shadow>\n                  <block type=\"text_getSubstring\" id=\"3Xg^Ih=;u@$W8jkf8p2s\">\n                    <mutation at1=\"true\" at2=\"true\"></mutation>\n                    <field name=\"WHERE1\">FROM_START</field>\n                    <field name=\"WHERE2\">FROM_START</field>\n                    <value name=\"STRING\">\n                      <block type=\"variables_get\" id=\"w8iKs:6;*6~cvr~PX$7M\">\n                        <field name=\"VAR\" id=\")ZBKTUC^^!J999c!;(dH\">testVariable</field>\n                      </block>\n                    </value>\n                    <value name=\"AT1\">\n                      <block type=\"variables_get\" id=\"^tMZ8{3o8UfzMx~)8.jy\">\n                        <field name=\"VAR\" id=\")jU4tpD],qwF{9((T=aQ\">i</field>\n                      </block>\n                    </value>\n                    <value name=\"AT2\">\n                      <block type=\"variables_get\" id=\"Wy_``_G}ej#7X8(8dWAz\">\n                        <field name=\"VAR\" id=\")jU4tpD],qwF{9((T=aQ\">i</field>\n                      </block>\n                    </value>\n                  </block>\n                </value>\n              </block>\n            </statement>\n            <next>\n              <block type=\"node_object_set\" id=\"T`]uSK?]/[w:jE:Vx%oq\" inline=\"true\">\n                <value name=\"object_field\">\n                  <shadow type=\"node_msg\" id=\"M]SQx{ba}1W7I@fGv(e8\"></shadow>\n                </value>\n                <value name=\"field_name\">\n                  <shadow type=\"text\" id=\"aFVf2mVPAgU3-c)0c;29\">\n                    <field name=\"TEXT\">payload</field>\n                  </shadow>\n                </value>\n                <value name=\"value_field\">\n                  <shadow type=\"text\" id=\"7*882J=9M!1=u`$y!RM3\">\n                    <field name=\"TEXT\"></field>\n                  </shadow>\n                  <block type=\"variables_get\" id=\"b+e5cTeOeAtOZL6BHJkH\">\n                    <field name=\"VAR\" id=\"cx:#6yd(cD`NZ32~!cqA\">testBuffer</field>\n                  </block>\n                </value>\n                <next>\n                  <block type=\"node_send\" id=\"wJ;Eh_RQE|YJiqa,%7;M\">\n                    <field name=\"OUTPUT_NR\">1</field>\n                    <value name=\"MESSAGE_INPUT\">\n                      <shadow type=\"node_msg\" id=\"h`33W1BFD,t?urGt=g:T\"></shadow>\n                    </value>\n                  </block>\n                </next>\n              </block>\n            </next>\n          </block>\n        </next>\n      </block>\n    </next>\n  </block>\n</xml>","outputs":1,"blocklyConfig":"46e073e1.66e10c","backpackContents":[],"noerr":0,"name":"Fill buffer from payload string","x":340,"y":640,"wires":[["6975824ad5fc3cba"]]},{"id":"46e073e1.66e10c","type":"blockly-config","language":"en","showTrashcan":true,"allowComments":true,"showZoomControl":true,"enableBackPack":true,"backpackContents":["<block xmlns=\"https://developers.google.com/blockly/xml\" type=\"object_create\" inline=\"true\"><mutation xmlns=\"http://www.w3.org/1999/xhtml\" num_fields=\"1\"><field name=\"property name\"></field></mutation><field name=\"field1\">payload</field></block>","<block xmlns=\"https://developers.google.com/blockly/xml\" type=\"node_return_message\"><field name=\"OUTPUT_NR\">1</field><value name=\"MESSAGE_INPUT\"><shadow type=\"node_msg\"></shadow></value></block>","<block xmlns=\"https://developers.google.com/blockly/xml\" type=\"node_object_set\" inline=\"true\"><value name=\"object_field\"><shadow type=\"node_msg\"></shadow></value><value name=\"field_name\"><shadow type=\"text\"><field name=\"TEXT\">payload</field></shadow></value><value name=\"value_field\"><shadow type=\"text\"><field name=\"TEXT\"></field></shadow></value></block>","<block xmlns=\"https://developers.google.com/blockly/xml\" type=\"node_object_get\"><mutation xmlns=\"http://www.w3.org/1999/xhtml\" action=\"GET\"></mutation><field name=\"action\">GET</field><value name=\"object\"><shadow type=\"node_msg\"></shadow></value><value name=\"field_name\"><shadow type=\"text\"><field name=\"TEXT\">payload</field></shadow></value></block>"],"toolboxPosition":"left","renderer":"geras","categories":[{"name":"Node-RED","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/nodeRedBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/nodered/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Objects","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/objectBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/json/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Buffer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/bufferBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/buffer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Date/time","files":["blockly-contrib/npm/@blockly%2Ffield-date/dist/date_compressed.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/dateTimeBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/datetime/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Timer","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/timerBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/timer/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly extension","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksCodeGen.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/extraBlocksDefs.js","blockly-contrib/npm/node-red-contrib-blockly/lib/extra/toolbox.xml","blockly-contrib/npm/node-red-contrib-blockly/messages/en.js"]},{"name":"Blockly standard","files":["blockly-contrib/npm/node-red-contrib-blockly/lib/basic/toolbox.xml"]}],"customizeToolbox":false,"name":""}]

@cymplecy
Copy link
Collaborator Author

cymplecy commented Jul 28, 2021

"And can you please explain a bit more in detail why the tempValue had to be introduced."

In the test flow above, the byte at index is set to

image

value ends up being a string

(testVariable.slice((i2 - 1), i2))

So if we just pass it thru the old code - it sees it as a string (and not a variable) so the old code just ended up delivering 40 (the ASCII code for at (

So I think it needs to be evaluated at runtime (I may have gone too far down the rabbit hole and it might be much simpler to do)

So I came up with the idea of assigning the value of value (pardon using that expression) to a temp variable at runtime - decided on

 __tempValue__

to avoid namespace clashes with real variable names that users might use .

I'm pretty certain that the whole code can be simplfied but that can be done later on down the line - it just needs to work to get the beta out of the door

@cymplecy
Copy link
Collaborator Author

cymplecy commented Jul 28, 2021

I've fixed my " " issue getting treated as a number zero instead of ASCII 32

image

  // Blockly is 1-based while Javascript is 0-based, so the index needs to be converted
  // This doesn't work if index is a variable so commented out
  //index--;

  // Get the data type checks offered by the block that is connected to our VALUE field 
  var targetCheck = block.getInput('VALUE').connection.targetConnection.getCheck();

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
 // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  var dataType = null;
  if (targetCheck && Array.isArray(targetCheck) && targetCheck.length === 1) {
     dataType = targetCheck[0];
     //console.log(dataType);
     //console.log(value);
  }
      
  if (dataType === null) {
     // We assume the input is a variable
     let code = '';
     code += 'if (isNaN(' + value + ')) {\n';
         code += '    ' + value + ' = ' + value + '.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(' + value + ')) {\n';
         code += '        ' + value + ' = ' + value + '[0]\n';
       code += '    } else {\n';
        //JS thinks a single " " is !isNan so we need to treat it as a special case
         code += '        if (' + value + ' == " ") {\n';
           code += '            ' + value + ' = ' + value + '.charCodeAt(0);\n';
         code += '        } else {\n';
           code += '            ' + value + ' = Math.min(Math.max(' + value + ', 0),255)\n';
         code += '        }\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + ' - 1] = ' + value + ';\n';
     return code;
  }

  if ((dataType == 'String') && (value.charAt(0) == '(') && (value.charAt(value.length - 1) == ')') ) {
    // We assume the input is a code expression
    let code = '';
    code += 'let __tempValue__ = ' + value + ';\n';
    //code += 'node.warn(__tempValue__ );\n';
    code += 'if (isNaN(__tempValue__)) {\n';
         //code += 'node.warn("thinks its a string");\n';
         code += '    __tempValue__ = __tempValue__.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(__tempValue__)) {\n';
         //code += 'node.warn("thinks its a buffer");\n';
         code += '        __tempValue__ = __tempValue__[0]\n';
       code += '    } else {\n';
         //code += 'node.warn("thinks its a number");\n';
         //JS thinks " " is !isNan so we need to treat it as a special case
         code += '        if (__tempValue__ == " ") {\n';
         code += '            __tempValue__ = __tempValue__.charCodeAt(0);\n';
         code += '        } else {\n';
         code += '            __tempValue__ = Math.min(Math.max(__tempValue__, 0),255)\n';
         code += '        }\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + ' - 1] = __tempValue__;\n';
     return code;
  }
     
  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      //check if string is JS code
      if ((value.charAt(0) != '(') && (value.charAt(value.length - 1) != ')')  ) {
      // Get the first character of the input string
        value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
        value = value.charCodeAt(0);
      }
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
     //case "Variable":
       // Keep the value as it is
  }


  const code = buffer + '[' + index + ' - 1] = ' + value + ';\n';
  return code;
};

@bartbutenaers
Copy link
Owner

bartbutenaers commented Jul 28, 2021

@cymplecy,

There is nasty thing in Blockly when you generate variable names, like e.g. the ``tempValue` variable. Because as soon as you add more than one instance of such a block, you end up with duplicate variable names...

For example when I duplicate the buffer_set_index block:

image

Then code will be generated that contains an error:

image

P.S. Now you see why I wanted to have syntax error highlighting in this version of the blockly node ;-)

I think there are 3 solutions:

  1. Make the variable name unique by adding a random value in the name. But then the code becomes typical unreadable generated code...
  2. Make the variable name unique by adding a counter in the name.
  3. Generate a function and call the same function from every block.

Personally I would generate a function (which contains your code), but I'm not sure anymore how we have to generate such a function. Because if we have N instances of such a block, we only want to generate the function code only once...

I will search how to do that...

@cymplecy
Copy link
Collaborator Author

I like 1 - I don't think this code is very readable at the moment so having a variable like

buffercodeTempValue_1456723

isn't going to make things much worse :)
2 would be nicer than 1 but then we would have to keep track of a counter and off we go down another rabbit hole :)

I vote for 1 just to get the beta out of the door :)

This whole code can be tidied up later :)

@bartbutenaers
Copy link
Owner

With "later" you mean in 3 years from now ;-)

My time is up for today.
I have asked it on the Blockly forum.
Would be nice if we could generate readable code (i.e. a function with your code inside), because I assume some users try to learn from the generated code ...
I think that such a function could also be called in if dataType === null. Is that correct?

If I don't get a simple to implement example, we will continue with one of the other options...

@cymplecy
Copy link
Collaborator Author

cymplecy commented Jul 29, 2021

Doing a bit of googling, if I switched to using var tempValue instead of let wouldn't that solve the problem?
[2nd edit - seems to work for me - just gives a warning triangle but works ]

image

image

Current code

  // Blockly is 1-based while Javascript is 0-based, so the index needs to be converted
  // This doesn't work if index is a variable so commented out
  //index--;

  // Get the data type checks offered by the block that is connected to our VALUE field 
  var targetCheck = block.getInput('VALUE').connection.targetConnection.getCheck();

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
 // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  var dataType = null;
  if (targetCheck && Array.isArray(targetCheck) && targetCheck.length === 1) {
     dataType = targetCheck[0];
     //console.log(dataType);
     //console.log(value);
  }
      
  if (dataType === null) {
     // We assume the input is a variable
     let code = '';
     code += 'if (isNaN(' + value + ')) {\n';
         code += '    ' + value + ' = ' + value + '.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(' + value + ')) {\n';
         code += '        ' + value + ' = ' + value + '[0]\n';
       code += '    } else {\n';
        //JS thinks a single " " is !isNan so we need to treat it as a special case
         code += '        if (' + value + ' == " ") {\n';
           code += '            ' + value + ' = ' + value + '.charCodeAt(0);\n';
         code += '        } else {\n';
           code += '            ' + value + ' = Math.min(Math.max(' + value + ', 0),255)\n';
         code += '        }\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + ' - 1] = ' + value + ';\n';
     return code;
  }

  if ((dataType == 'String') && (value.charAt(0) == '(') && (value.charAt(value.length - 1) == ')') ) {
    // We assume the input is a code expression
    let code = '';
    code += 'var __tempValue__ = ' + value + ';\n';
    //code += 'node.warn(__tempValue__ );\n';
    code += 'if (isNaN(__tempValue__)) {\n';
         //code += 'node.warn("thinks its a string");\n';
         code += '    __tempValue__ = __tempValue__.charCodeAt(0);\n';
     code += '} else {\n';
       code += '    if (Buffer.isBuffer(__tempValue__)) {\n';
         //code += 'node.warn("thinks its a buffer");\n';
         code += '        __tempValue__ = __tempValue__[0]\n';
       code += '    } else {\n';
         //code += 'node.warn("thinks its a number");\n';
         //JS thinks " " is !isNan so we need to treat it as a special case
         code += '        if (__tempValue__ == " ") {\n';
         code += '            __tempValue__ = __tempValue__.charCodeAt(0);\n';
         code += '        } else {\n';
         code += '            __tempValue__ = Math.min(Math.max(__tempValue__, 0),255)\n';
         code += '        }\n';
       code += '    }\n';
     code += '}\n';
     code += buffer + '[' + index + ' - 1] = __tempValue__;\n';
     return code;
  }
     
  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      //check if string is JS code
      if ((value.charAt(0) != '(') && (value.charAt(value.length - 1) != ')')  ) {
      // Get the first character of the input string
        value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
        value = value.charCodeAt(0);
      }
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
     //case "Variable":
       // Keep the value as it is
  }


  var code = buffer + '[' + index + ' - 1] = ' + value + ';\n';
  return code;
};

Blockly.JavaScript['buffer_get_index'] = function(block) {
  var   index = Blockly.JavaScript.valueToCode(block, 'INDEX', Blockly.JavaScript.ORDER_ATOMIC) || 1;
  const buffer = Blockly.JavaScript.valueToCode(block, 'BUFFER', Blockly.JavaScript.ORDER_ATOMIC);
  
  // Blockly is 1-based while Javascript is 0-based, so the index needs to be converted
  // This doesn't work if index is a variable so commented out
  //index--;

  const code = buffer + '[' + index + ' - 1]';
  return [code, Blockly.JavaScript.ORDER_NONE];
};

@cymplecy
Copy link
Collaborator Author

cymplecy commented Jul 29, 2021

Just consolidated the code and tidied it up

Blockly.JavaScript['buffer_set_index'] = function(block) {
  var   index = Blockly.JavaScript.valueToCode(block, 'INDEX', Blockly.JavaScript.ORDER_ATOMIC) || 1;
  //console.log(index);
  const buffer = Blockly.JavaScript.valueToCode(block, 'BUFFER', Blockly.JavaScript.ORDER_ATOMIC);
  var   value = Blockly.JavaScript.valueToCode(block, 'VALUE', Blockly.JavaScript.ORDER_ATOMIC);

  // Get the data type checks offered by the block that is connected to our VALUE field 
  var targetCheck = block.getInput('VALUE').connection.targetConnection.getCheck();

  // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
 // Find the type of value that has been used as input.
  // See https://groups.google.com/g/blockly/c/9fEPSGFarNM
  var dataType = null;
  if (targetCheck && Array.isArray(targetCheck) && targetCheck.length === 1) {
     dataType = targetCheck[0];
  }
      
  if ((dataType === null) || ((dataType == 'String') && (value.charAt(0) == '(') && (value.charAt(value.length - 1) == ')') )) {
    // We assume the input is a variable
    let code = '';
    if ((dataType == 'String') && (value.charAt(0) == '(') && (value.charAt(value.length - 1) == ')') ) {
    // We assume the input is a code expression so we need to evaluate it before continuing at runtime
      code += 'var __tempValue__ = ' + value + ';\n';
      value = '__tempValue__';
    }
    code += 'if (isNaN(' + value + ')) {\n';
        code += '  ' + value + ' = ' + value + '.charCodeAt(0);\n';
    code += '} else {\n';
      code += '  if (Buffer.isBuffer(' + value + ')) {\n';
        code += '    ' + value + ' = ' + value + '[0];\n';
      code += '  } else {\n';
       //JS thinks a single " " is !isNan so we need to treat it as a special case
        code += '    if (' + value + ' == " ") {\n';
          code += '      ' + value + ' = ' + value + '.charCodeAt(0);\n';
        code += '    } else {\n';
          code += '      ' + value + ' = Math.min(Math.max(' + value + ', 0),255);\n';
        code += '    }\n';
      code += '  }\n';
    code += '}\n';
    code += buffer + '[' + index + ' - 1] = ' + value + ';\n';
    return code;
  }

     
  switch (dataType) {
    case "Number":
      value = Math.min(Math.max(value, 0),255);
      break;
    case "String":
      // Get the first character of the input string
        value = value.replace(/^'(.*)'$/, '$1');  //remove any quotes 
        value = value.charCodeAt(0);
      break;
    case "Buffer":
      // Get the first element from the input buffer
      value = value + '[0]';
      break;
  }

  var code = buffer + '[' + index + ' - 1] = ' + value + ';\n';
  return code;
};

Blockly.JavaScript['buffer_get_index'] = function(block) {
  var   index = Blockly.JavaScript.valueToCode(block, 'INDEX', Blockly.JavaScript.ORDER_ATOMIC) || 1;
  const buffer = Blockly.JavaScript.valueToCode(block, 'BUFFER', Blockly.JavaScript.ORDER_ATOMIC);

  const code = buffer + '[' + index + ' - 1]';
  return [code, Blockly.JavaScript.ORDER_NONE];
};

@bartbutenaers
Copy link
Owner

Ah, I didn't know that about "let". Good catch!!!

The code generator code looks very compact and nice this way!
Good enough for me to announce a beta version.

I will try tonight to:

  1. create a git tag for the beta
  2. create a github release for that tag
  3. write a summary of all changes in the release page
  4. publish a beta version on npm
  5. announce it on discourse (@jsccjj : do you have a username on Discourse, so we can mention you?)

Remark: I still would like to generate a function for the "buffer_set_index" block: because if we have N of such blocks in a workspace, then we generate N times the same duplicate code snippet (resulting heavy readable code ...). Would be nice if we could generate the code snippet once in a function, and call that function N times.

@bartbutenaers
Copy link
Owner

@cymplecy, @jsccjj,
Have taken a few hours holiday...
The release notes are ready: https://github.com/bartbutenaers/node-red-contrib-blockly/releases/tag/v2.0.0-beta.1
Would be nice if you could review them!

@cymplecy
Copy link
Collaborator Author

All good - minor suggestion just to clarify

The "byte" block doesn't return a number anymore, but a buffer of length 1 (containing a number between 0 and 255).

@bartbutenaers
Copy link
Owner

Suggestion is implemented and beta published on npm. Will put it on Discourse later on. Now I'm off ...

@bartbutenaers
Copy link
Owner

Remark: I still would like to generate a function for the "buffer_set_index" block: because if we have N of such blocks in a workspace, then we generate N times the same duplicate code snippet (resulting heavy readable code ...). Would be nice if we could generate the code snippet once in a function, and call that function N times.

@cymplecy,
I now that you have just announced on Discourse that you will never have to write Javascript again ;-)
However I would appreciate if you would do it one more time ...
I have updated the buffer_set_byte block to generate a function, based on Beka's guidelines in the Blockly forum.

When you have now e.g. multiple buffer_set_byte blocks, with a variable as input:

image

Then a single function will be generated (containing your code), which is called by all of the buffer_set_byte blocks:

image

So no code duplication anymore.
Hopefully I haven't corrupted your code. For example I have removed your tempVar because I 'think' that it will now also work because the value is now passed to the function as input parameter. But I might be mistaken. Then we have to put it back in place.

I would appreciate it if you could do your tests one more time.
If this is finished we will publish the 2.0.0.beta-2, since that was the last thing that I wanted in this release ...
Thanks!!

@cymplecy
Copy link
Collaborator Author

cymplecy commented Aug 1, 2021

Initial tests seem to indicate an off by one error somewhere - just investigating further

@bartbutenaers
Copy link
Owner

Simon,
If you can give me a flow and the expected result, then I will also have look.

@cymplecy
Copy link
Collaborator Author

cymplecy commented Aug 1, 2021

I don't want to waste your time if its my testing - give me a few more minutes

@bartbutenaers
Copy link
Owner

No hurry!! Just please don't start swearing here :-D
Because I cannot block you from this repo, since I need you here...

@cymplecy
Copy link
Collaborator Author

cymplecy commented Aug 1, 2021

It's all OK - after restarting and clearing cache and spinning around 10 times your nice tidy code behaves the same as my old rough code :)

We have a go to launch :)

@bartbutenaers
Copy link
Owner

Thanks!!!!!
The function still contains your logic and code...
Then I think you can finally close this Github issue?
Thought this would never happen...
For me beta.2 can be published after the issue is closed.
I am not going to add new features anymore.
We only need the french and russian translations.

@bartbutenaers
Copy link
Owner

@cymplecy : I have created release notes for beta 2...

@cymplecy
Copy link
Collaborator Author

cymplecy commented Aug 1, 2021

Looks good

@cymplecy
Copy link
Collaborator Author

cymplecy commented Aug 1, 2021

I'm officially closing the issue :-)

@cymplecy cymplecy closed this as completed Aug 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants