[Blockly] Add missing openHAB calls and features

[rkoshak]

The problem

The current Blockly implementation is missing a number of calls to openHAB features that make it incomplete for rules development.

Your suggestion

Here is an incomplete list of those calls that are missing. I'm sure there are more and some I'm sure won't be possible without more information from the REST API that isn't supported.

Also, some might be considered "advanced" in which case maybe it shouldn't be supported.

createTimer

I think this is going to be the hardest to support. Unfortunately I also think it's the most important to implement.

It’s a little tricky actually. We would probably want to have a reference to the Timer persist across runs of the rule right? So it can be tested to see if it’s still running and cancelled or rescheduled.

Given that it would be something like:

/** Goes at the top, doesn't need to be imported more than once */
var ScriptExecution = Java.type("org.openhab.core.model.script.actions.ScriptExecution");

/** Function to run when the timer goes off */
var runme = function() {
    // stuff to do when the timer expires
}

/** Create the Timer */
this.myTimer = ScriptExection.createTimer(now.plusSeconds(10), runme);

However, I’ve discovered that if you want to make sure that certain variables are fixed inside the runme function, you need to do something like

/** Goes at the top, doesn't need to be imported more than once */
var ScriptExecution = Java.type("org.openhab.core.model.script.actions.ScriptExecution");

/** Function to run when the timer goes off */
var runmeGenerator = function(itemName) {
    // stuff to do when the timer expires
}

/** Create the Timer */
this.myTimer = ScriptExection.createTimer(now.plusSeconds(10), runme(event.itemName));

The problem is if the rule runs again it runs with the same context and event.itemName will become overwritten by the time the Timer runs. By using a function generator, the itemName gets fixed and won’t be overwritten (I learned this the hard way). But for Blockly I think we might be able to hide that by using a function generator behind the scenes and pass in all the normal stuff like event as arguments by default.

But we also would need to support throw away Timers (i.e. we don’t save it to a variable) and Timers that we want to keep in a variable and check later. So this might require a new variable block and two versions of the Timer block? I’ve never done Blockly myself but my son has been doing Scratch so I’m a little familiar with how the concept is supposed to work.

Visually I would expect to see it where the runme is just defined inside the Timer block instead of forcing the creation of a separate function block. But I don't know how feasible that is to do.

persistent variables

Users will want to set a variable on one run of a rule and be able to retrieve that value the next time the rule runs.

this.myVariable = (this.myVariable === undefined) ? <initial value> : this.myVariable;

That will set the variable to the context that gets reused every time the script runs. If it's undefined we set it to a default value otherwise we keep it with it's old value.

Ephemeris

var logger = Java.type("org.slf4j.LoggerFactory").getLogger("org.openhab.model.script.Rules.Examples");
logger.info("About to test Ephermeris");
var Ephemeris = Java.type("org.openhab.core.model.script.actions.Ephemeris");
logger.info((Ephemeris.isWeekend()) ? "It's the weekend" : "Get back to work!");

There are a lot of calls on Ephemeris that would need to be supported. Given the support for Ephemeris in the conditions this is probably a lower priority

executeCommandLine

// Exec.executeCommandLine
var logger = Java.type("org.slf4j.LoggerFactory").getLogger("org.openhab.model.script.Rules.Examples");
logger.info("About to test executeCommandLine");
var Exec = Java.type("org.openhab.core.model.script.actions.Exec");
var Duration = Java.type("java.time.Duration");
var results = Exec.executeCommandLine(Duration.ofSeconds(1), "echo", "hello");
logger.info("results = " + results);

This one is a little more complex because of the need to use Duration, but it's probably not too bad.

now

Access to now and performing time based comparisons. For example if(now.minusMinutes(1).isBefore(now)).

PersistenceExtensions

var PersistenceExtensions = Java.type(" org.openhab.core.persistence.extensions.PersistenceExtensions");

var lastUpdate = PersistenceExtensions.lastUpdate(ir.getItem("MyItem"), "rrd4j");

As with Ephemeris, there are a lot of functions on PersistenceExtensions.

NOTE: Below this point I don't have examples to post yet. I'll come back when I can with examples of the actual code.

Voice stuff

I don't use TTS or the like but the Say and playStream Actions would need to be supported. It'll be the same pattern: import the class using Java.type and call the methods on the class.

All the remaining core Actions

saveStates/restoreStates, sendHttpXRequest actions, sendBroadcast/sendNotification actions, etc. Everything in the Actions page in the docs should be supported.

Thread.sleep

java.lang.Thread.sleep(1000);

The problems with long running rules is much less pronounced in OH 3.

Binding Actions

This one is probably impossible right now because the names of the methods and arguments can't be hard coded. So there would need to be a REST API call that doesn't yet exist and then code to dynamically create blocks based on those. However, perhaps it's possible to create a "custom code" block and have the users type in their own arbitrary code? That could be a catch-all for what's missing, if Blockly can support it.

Accessing an Item's Group members

It looks like List manipulation is already pretty well supported but I couldn't see a way to access the members of a Group Item.

 context.getNames = function(groupName, filterFunc) {
    var Collectors = Java.type("java.util.stream.Collectors");
    return context.ir.getItem(groupName)
                     .members
                     .stream()
                     .filter(filterFunc)
                     .map(function(i) {
                       return context.getName(i.name);
                     })
                     .collect(Collectors.joining(", "));
}

Note: getName is another function that retrieves the "name" metadata value which is a way I use to apply a human friendly name for my Items. I should probably just use label at this point intead.

More list functions

Map, reduce, filter... See above.

Item metadata

// Accessing Item Metadata
var logger = Java.type("org.slf4j.LoggerFactory").getLogger("org.openhab.model.script.Rules.Examples");
logger.info("Trying to extract a metadata value")
var FrameworkUtil = Java.type("org.osgi.framework.FrameworkUtil");
var _bundle = FrameworkUtil.getBundle(scriptExtension.class);
var bundle_context = _bundle.getBundleContext()
var classname = "org.openhab.core.items.MetadataRegistry"
var MetadataRegistry_Ref = bundle_context.getServiceReference("org.openhab.core.items.MetadataRegistry");
var MetadataRegistry = bundle_context.getService(MetadataRegistry_Ref);
var methods = MetadataRegistry.class.getDeclaredMethods();
logger.info("There are " + methods.length + " methods");
for each (var m in methods) {
  logger.info(m.toString());
}
var Metadata = Java.type("org.openhab.core.items.Metadata");
var MetadataKey = Java.type("org.openhab.core.items.MetadataKey");
var metadata = MetadataRegistry.get(new MetadataKey("name", "vArgus_Status"));
logger.info("vArgus_Status's name is " + metadata.value);

Call another rule

// Run another rule
var FrameworkUtil = Java.type("org.osgi.framework.FrameworkUtil");
var _bundle = FrameworkUtil.getBundle(scriptExtension.class);
var bundle_context = _bundle.getBundleContext()
var classname = "org.openhab.core.automation.RuleManager"
var RuleManager_Ref = bundle_context.getServiceReference(classname);
var RuleManager = bundle_context.getService(RuleManager_Ref);
RuleManager.runNow("tocall");
var map = new java.util.HashMap();
map.put("test_data", "Passed data to called function!")
RuleManager.runNow("tocall", true, map); // second argument is whether to consider the conditions, third is a Map<String, Object> (way to pass data?)

SciptExtensions

This is still on my list of things to figure out. It should be a way we can share variable between Script Actions.

Libraries

Supporting an arbitrary library is probably not feasible (it'd be awesome if it were). But what about library functions created in Blockly? There are some things that I would go through the efforts to reimplement as a Blockly library to import into a rule.

Units of Measurement Support

Let the units be defined and used in comparisons and math operations. Let the user strip the UoM off when necessary. Define constants with UoM units.

I'm sure I'm forgetting some things.

[bigbasec]

@rkoshak Any chance you'd be willing to elaborate on some examples for some of these? I've got about 4-5 of them done so far here and am testing, though it would help to have the js examples like you did for most of these.

New to blockly so this might be a week or so out, but I think this is for the most part manageable. Some are certainly not, as they'd require quite a bit of work to make it happen. Things like running another rule, etc.

For now I am cracking away at the ones with code examples as those are the easiest, more examples the merrier.

Appreciate what you do, thanks,

[bigbasec]

Does anyone mind if I break "ohblocks.js" into multiple files? It's getting somewhat out of hand here.

[ghys]

of course not :)

[bigbasec]

of course not :)

Thanks, appreciate it. PR for a lot of these will be coming this week.

[ghys]

Excellent!

[rkoshak]

The ones I didn't provide examples for are because I don't have examples of them. Some even I don't know how to do yet like ScriptExtensions. As I get some time I'll try to generate some more examples but certain things like say will probably need to come from someone else. I've never used voice stuff.

But in general, all the core actions follow the same pattern (just like executeCommandLine above).

1. Import the right class
2. Call one of the methods on that class

All of the relevant actions classes can be found at http://www.openhab.org/javadoc/latest/org/openhab/core/model/script/actions/package-summary.html

[bigbasec]

Sounds good. I'm working my way through the core.model.script stuff, so should be able to expand things fairly well. Some of the functions I'm still working out how that's going to work in a Blockly world(like Map variables). But for the most part I should be able to cover most things.

I sort of have a good idea of what I want to do here, but I'm sort of green in regards to f7. Going to be making pickers for things like there is currently for the Model. The blockly stuff is mostly done at this point, just cleaning some things up and organizing them better. Then obviously testing a bit here before I push the changes.

I'm actually (on a different branch) going to try to allow Blockly be more dynamically loaded so that there would be more blocks available if you say have binding xyz installed. That's a little bit of a ways off right now, just want to cover the basics and move things along here before I go full out on this.

I don't use Blockly myself, so people to test this stuff a little better than I am(in real world scenarios) would be great. I am trying to create blockly statements for some of my rules here to make sure that I'm covering things the best I can but I'm sure someone has some crazy rule they'll want to work. :)

[rkoshak]

One thing you might want to do is add some definition catches so you don't end up reloading the same class over an over. I don't know if there is a performance issue or not but something like:

this.ScriptExection = (this.ScriptExecution === undefined) ? Java.type("blah.blah.blah.ScriptExection") : this.ScriptExecution;

I haven't looked at what you've done yet but this popped into my mind just now and I wanted to capture it. I don't know if it's smart enough to only add the import if it hasn't already been added when you choose one of the Actions that need an import as a block or if it's a straight "choose block get these lines of code" kind of deal.

[bigbasec]

Yea, as long as you use the Blockly way to do things and not just adding the code for the class it should work(you do need to keep your class function name the same) I think. I know that it works with multiple blocks of the same type and I think I've kept the consistency between calls so I think it'll be ok.

I've honestly written a bunch of things that I'll never use here, but I am trying to do my best to make sure they all work.

[rkoshak]

I suspect that anyone who has the skills to solve this PR is not going to be a user of Blockly in the end.

[tom-ch1]

Please add the implicit variables to the list of features. Not being able to use receivedCommand for a command-triggered rule means you have to duplicate the rule for each command an specify only one command trigger for each. This would be so much simpler if you could just write one rule for all commands and handle it inside the rule.

[rkoshak]

Please add the implicit variables to the list of features.

That's a good point, though the underlying code with Blockly is JavaScript so there is not receivedCommand there either. Instead there is the event object which carries stuff like event.itemState, event.itemName, event.oldState, etc. I think that oldState, command etc. We'll need to iterate over all the variables which I started doing at https://community.openhab.org/t/experimental-next-gen-rules-engine-documentation-4-of-writing-scripts/55963 a long time ago but haven't come back to.

[bigbasec]

I can add whatever can be made available w/ js code. This shouldn't be a problem. Working on an update to this draft, should be done this week.

[svenb1234]

What about "simple" date/time blocks (https://github.com/ioBroker/ioBroker.javascript/blob/master/docs/en/blockly.md#date-and-time-blocks).

Example would be: some rule -> but only if a given script evbaluates to true -> blockly: Current time is equal/smaller/bigger than item sunrise time from astro binding.

There seems to be no way without script to do this in the OH3 UI.

[stefan-hoehn]

Several improvements have been made with this PR: #1179

• add a structured menu with sub groups: audio, voice, timers, variables, notifications
• add code documentation, tooltips and help urls

We should now check what is missing

[svenb1234]

now

Access to now and performing time based comparisons. For example if(now.minusMinutes(1).isBefore(now)).

Did I miss those or are these simple date/time comparisons still missing?, i.e. something like "do something if now() is bigger than HH:mm or in-between HH:mm and HH:mm.

[stefan-hoehn]

Again more improvements have been provided:
#1212

• Add script support
• Add ephemeris support
• Add persistence

[martingruening]

Persistence is great. There is the block last updated date of item 'Item "MyItem"'
This should be last (updated|changed) date of item 'Item "MyItem"' providing a small choice for the desired point in time

[stefan-hoehn]

@rkoshak Can we update this issue to the latest implementation of blockly. I think there is hardly anything missing by now, is it?

@martingruening This has been implemented lately with #1549

@svenb1234 you can find many by now: https://www.openhab.org/docs/configuration/blockly/rules-blockly-date-handling.html

[rkoshak]

Given the reimplantation, i recommend closing this issue and should we need to open a new one if we find something missing later on. I don't see anything here that is obviously missing.

[stefan-hoehn]

Ok, then go ahead and close it. If you have nice ideas regarding new blocks that can only be done with graalsvm, then let me know.

[rkoshak]

I will. I want to play with what's there first to see what comes to mind.