include all dashboard files in a single repo

[paidforby]

Currently, files for the dashboard are split as follows,

peoplesopen-dash - build which is copied into '/www'
sudowrt-packages - a few rpcd configs that are copied into /usr during make install of package.
sudowrt-firmware - infogather that gets put in /www/cgi-bin when the firmware is built.

This makes updating the dashboard awkward and confusing. Most of the sudowrt-packages stuff is a hold over from the old web ui. I'm not sure the best approach to building/deploying the dashboard, it would be nice if everything were unified in a single repo. Any ideas @eenblam @bennlich or @gobengo ?

[bennlich]

Yessssss. Great idea. Good job tracking all these pieces down.

I propose we move everything to peoplesopen-dash and then have a deploy script that just copies all the files over and runs /etc/init.d/rpcd reload to reload any new rpcd scripts. I was actually already working on this script to make it easier to develop the dashboard UI w/o having to do the crazy chrome CORS hack. Will link my PR here.

a few rpcd configs that are copied into /usr during make install of package

Do you know where/when this copying happens exactly? We probably need to change some things in the firmware build for this reorganization to work.

[gobengo]

Good idea. Here's out I'd approach this.

• the web app 'peoplesopen-dash' should be as portable as possible. i.e. it should run easily on my dev laptop, maybe a normal cloud web server, etc. So it can be tested easily and such. i.e. it should not be tightly coupled with sudowrt-firmware, which, if possible, should be considered only one of several valid runtime targets.
• the firmware depends on peoplesopen-dash. i.e. the dash is one of many components composed into the firmware. To use this component the way it wants, it may need to provide its own configs and such. And those firmware-specific configs should live with the firmware, not the dash (but probably be in a standard config format defined by the dash component).
• if I'm understanding right, peoplesopen-dash has a run-time dependency on rpcd, which is provided by sudowrt-firmware (but could be provided elsewhere, e.g. for mocking/testing outside of an embedded env)
• it's unusual to me that peoplesopen-dash/build exists (compiled javascript checked into the source code repository). Usually I'd just keep the source code in the repo, and build all derrived things at build time. If build is successful, give it a good name (e.g. commit hash + build number). If it works really well, consider it for release. firmware would depend on built releases, not the raw source code repo.
  • It's really not much of a hindrance to check it the src code like we're doing. Just taking note..

Sounds like following steps could help:

• move the rpcd configs that the firmware ends up using into the firmware repo
• optionally, include known good working rpcd example config in the peoplesopen-dash. Might make sense to even use that as input in generating the one the firmware uses. Depends on whether any other firmware components use rpcd, or it really is like a wholly owned subcomponent of the dash (dash is only dependents)
• decommission sudowrt-packages if possible. If not possible, inventory things that depend on it. Decide whether we will decommission or not.
• if @bennlich 's deploy script sounds like it would improve developer experience, make it and include in firmware (since that script depends not just on the dash but on details specific to running it in firmware, e.g. how to reload rpcd)

[bennlich]

Thanks for these detailed comments @gobengo ! I think I like the sound of that organization but will reply more in depth when I've had a chance to look more closely. (Two more days of teaching gasp)

Pasting some links to ubus/rpcd docs so I don't forget:
https://wiki.openwrt.org/doc/techref/rpcd
https://wiki.openwrt.org/doc/techref/ubus

[paidforby]

@gobengo good ideas! I feel better about putting all the dashboard configs in the firmware now that we aren't using makenode. The only reason to keep sudowrt-packages would be if we made more packages and actually maintained them through that repo, I don't see that happening anytime soon.

As far as dev environment is concerned, you are correct about the run-time dependencies, @bennlich provided the documentation for both. I think it would be possible to create a node-less dev environment, but you would have to "spoof" the ubus connection somehow. My approach would be to ignore the calls to ubus when in dev env and, instead, use an alias of some sort that references example uci data stored in this repo. Alternatively, we could create a fake rpcd server?

[bennlich]

My approach would be to ignore the calls to ubus when in dev env and, instead, use an alias of some sort that references example uci data stored in this repo.

This is the approach I've started on. Essentially stubbing ubus API calls with captured replies. I'm not super deep in though so if we figure out a better way, can change gears.

[gobengo]

a mock of rpcd is a cool little component, shouldn't be too expensive to make (I think?) at least far enough for dash to use, and may be useful to other systems out there that dep on rpcd.

Way to go @bennlich . Sounds like you're running with it, but being a bit unfamiliar with all this, the first think I'd do is list all the rpcd API calls that the dash makes. That becomes a checklist we can use to track progress on the mock (and split up tasks to parallelize)

[paidforby]

@bennlich hope you are making some progress on the mock ubus idea. I have a possible alternate solution for a "node-less" dev environment. As suggested in the README, we could use a virtual machine. This would have the added benefit of providing a more streamlined dev cycle for the firmware in general. I was able to create a virtual machine image of sudowrt by just changing the Target System and Target Images before running make (as suggested here, https://wiki.openwrt.org/doc/howto/virtualbox). It is still very janky (it's having trouble getting all the packages) but I am able ssh into it through a virtual adapter as though it were a real node.
Here's my first attempt, you should be able to run it by following the instructions at the above link.
openwrt-x86-generic-combined-ext4.img.gz

While creating a virtual machine build of the firmware is a larger project than creating a fake ubus/rpcd, it could be a nice way to generalize our firmware and improve our build process.