Unable to add new templates (again)

[pb66]

This has been discovered, debugged and fixed previously on the forum (https://community.openenergymonitor.org/t/development-devices-inputs-and-feeds-in-emoncms/4281?u=pb66).

I am currently running emoncms 9.8.31 indexedinputs branch fully up to date at 15/10/18 with the "device-integration beta" switched off. The device module is also fully up to date and I've tried both the master and the indexedinputs branches.

It is currently not possible to introduce any new templates. I have tried adding new templates to both existing and new groups and new folders and every combination of those fails. At each change I have cleared the browser cache.

I have also tried creating a unwanted device to try and pull in the changes (this was previously found to work before this same issue was fixed last time), but the listed device templates doesn't change.

The only way I'm able to get the new template picked up is to delete all keys from Redis via the command line, then the list repopulates with the new templates included.

[pb66]

Found a shortcut here, instead of deleting all Redis for this emoncms instance or even all :device:* or :device:template* entries it is possible to trigger a reload of device templates in redis by just deleting the meta

redis-cli --raw keys "emoncms:device:templates:meta" | xargs redis-cli del

I found this out by using the search pattern device:template:* (subtly different from device:template*) which deleted all template entries in redis but NOT the meta file, resulting in this error

and the same old list without the new addition prevailed.

Just deleting the emoncms:device:templates:meta key seems to work fine.

[adminde]

I'm very sorry, as I completely lost track of this well-known "issue"...

Actually this is a feature, not a bug, as merged in the PR #18.
Money Quote:

Cached template meta data will never be deleted except for the specific API request device/template/reload.json, to enable manual reloading after changes in the database. This request could maybe be restricted to admin users and integrated into the admin panel at some point in the future.

I never continued pushing this to be documented anywhere in the Readme or introduce the Reload button to the admin page though. Shame on me.

[pb66]

I'm not convinced this is the right way to go. But if it is absolutely necessary to not refresh the template meta on a full device page refresh due to resources etc, then a button must be employed and I would prefer to see that button next to the current "new device" button, it can be hidden for non-admins, but who wants to navigate to the admin page or type an api call in another window each time they edit a template during development? We need to make using it easier for the average user. not harder as many users will want(need?) to make their own templates.

Also how does the emoncms update process work with this "feature"? Will users now have to remember to refresh the device list as well as do a database update check after each update?

[adminde]

We need to make using it easier for the average user. not harder as many users will want(need?) to make their own templates.
I completely agree, but having a refresh button like this would create enormous synchronization problems for any multi-user scenario of the module. You cannot allow users to clear all meta data in the redis cache simultaneously while several other users may be in the middle of using it. Same goes for automized processes running in the background, that depend on the cached values.

Maybe another request could be added which does not clear the cache but only checks for changes made? But this would be a lot harder to implement...
I'd strongly argue to go for a (admittedly ugly) temporary solution and just add a Reload button to the admin page. I really think the creation of templates is currently simply a feature for advanced users, as quite a lot of things can go wrong...

To make this accessible to a broader spectrum of users, a GUI at device/template/view or something like this could be necessary to be added. Which naturally is even more work, but could be really worth it, especially for us in combination with emonmuc.