autobuild enhanced reflash controller record support

[anwu]

Overview

We needed a way to be able to update PODs OTA with new executives.
The current main way to upgrade executives is through JLink, or using a pingpong bundle produced by firm_pingpong. Up to this point, there was no real way to make a reliable OTA script for production_devices. The only way the OTA TRUB scripts that included an executive update was made, was through remote_bridge. This method is not viable for production. Instead, we want to be able to create these TRUB scripts via the coretools interface. Specifically, we want our iotile build function to be able to pull in the required pingpong bundle associated with the execnrf52832 that we want to load into our device.

These are the following steps required to reach the end goal...

1. Create build products in firm_pingpong that can be iotile pulled from other modules.
2. Modify the execnrf52832 build procedure to pull the firm_pingpong image and to call a script (bundle_executive.py) to create the ping pong bundle that will allow the exec to be updated OTA.
3. Modify the autobuild functionality to handle our new kEnhancedReflashControllerRecord so that we can flash more advanced firmware onto our controller.

This PR addresses the second half of 2 and 3.
Closes #966

Major Changes

iotilecore

• Add enhanced_reflash.py to have a new record type called EnhancedReflashControllerRecord

iotilebuild

• Add autobuild_python_scripts to run a batch of python scripts along with their arguments
• Add autobuild_trub_script_proto which is an improved version of autobuild_trub_script that can now handle specified record types and arguments for individual slot assignments
• Added new underlying functions to integrate the new enhanced reflash record

[x24ling]

Looks good, I tested this with how you described: https://github.com/iotile/pod_controller/pull/223.

I would change the name of the functions build_update_script_proto and autobuild_trub_script_proto to something more like build_update_script_v2 or _enhanced

Also, _build_script_action' looks like it's a enhancement for _build_reflash_script_action, I would probably give build_script_action with a similar suffix to denote that's its for the new process, with something like v2orenhanced`.

Also just to confirm, autobuild_trub_script_proto (or whatever the new name is) can be used to replace autobuild_trub_script right?

[x24ling]

Actually, i need to look a little more on what would be the best way to address this, when i noticed when looking at the SConstruct file of your test branch of production_devices, but you shouldn't have to need to put 'build/deps` in the names of the files in this SConstruct, it should be able to find the files itself.

https://github.com/iotile/pod_controller/pull/223/files#diff-097ed87141bfbb23e296c50dff5fea76R7-R12

[anwu]

I would change the name of the functions build_update_script_proto and autobuild_trub_script_proto to something more like build_update_script_v2 or _enhanced

I'll go with v2.

Also, I'm refraining from renaming _build_script_action to _build_reflash_script_action_v2. The new _build_script_action is more accurate to its name because it will be able to handle script actions that are not reflashing... An example is that it can just send an RPC to reset the device.

Also just to confirm, autobuild_trub_script_proto (or whatever the new name is) can be used to replace autobuild_trub_script right?

Yes. You would just do something like

autobuild_trub_script_v2("os_2059_v1_1_ota.trub",
                      [
                          ("controller", "connrf52832_nrf52832_head.elf", "Reflash Controller", [] ),
                          ("slot 1", "firm_gpiopipe_lpc824_hw2.elf", "Reflash Tile", [])
                      ],
                      (2059, "1.1"))

Actually, i need to look a little more on what would be the best way to address this, when i noticed when looking at the SConstruct file of your test branch of production_devices, but you shouldn't have to need to put 'build/deps` in the names of the files in this SConstruct, it should be able to find the files itself.

Hmm, yeah I wasn't sure about this. I'll take a look and see if there is a cleaner way to grab those build products...

[ekkizogloy]

Looks like it's going in a good direction. I know you chose the name v2... In light of all the version confusion that we have going on between the firmware, hardware, pods, sensorgraphs and tags... is it possible to choose a name that's not indicating a version.

One thing that someone told me a long time ago that helps inform me when trying to select names.
"Name it what it IS, not what it does."
.. the reason being that what something does can change during development.

I leave the naming up to you, just know that this name will live on forever. Once you settle on a name... and merge it to master... it creates a legacy.. your legacy.. one that will outlive you.. outlive us all... in the Github cloud hithermore.. in history throughout history. Know this ... you have a moral responsibility to the children that come after you.

And with this... I will quote a famous movie, (a challenge for all to identify the movie). "You must choose. But choose wisely, ..."

[anwu]

"Name it what it IS, not what it does."

Yeah I believe in this 100%. However, the thing most accurately describes this actually the legacy function names. The original autobuild_trub_script isn't actually accurate to its name. The more accurate name for it would be autobuild_reflash_trub_script, considering it is limited to only reflashing controller/tile. The new autobuild_trub_script_v2 is more accurate to its name considering it has the capability of adding other instructions in the script besides reflashing.

I will give this some more thought...

[anwu]

I decided to rename it to autobuild_trub_records, which is sort of analogous to autobuild_trub_script, since a "trub script" consists of "trub records".

[ekkizogloy]

It's a great name. ... strong.... perfect.