Downloadable modular firmware

[GioCC]

Now and then, there have been talks about making the firmware more modular.

As a quick recap, these are the two main topics:

• A larger number of (existing) devices -> RAM issue
  (discussed originally in Configs with preconditions do not load #154, more recently in Increase max button limit to 50 on Mega Board #174 and Left Padding Function in LCD Configs #207)
• A larger range of supported device types -> Flash memory issue
  (recently mentioned in Different export Logic for 7Segment and LCD Values #206, and surely in other occasions too)

Regarding the second point, since #206 has just been closed and discussion directed elsewhere, I was thinking about a possible idea (maybe something like this has already been proposed? I could find no reference).

The problem with configuring the firmware to exclude certain unused devices and make space for others is that, while that's trivial to achieve during compilation, the end user only receives the finalized version of the firmware.
In order to change it, he should (be able to) get the sources - and the toolchain - and recompile them himself, which is obviously not convenient.

One solution could be following:

• Setup a web page with a simple "mix-your-own" selection
• a simple script could launch a remote compilation with the compilation switches set according to the user's selection (or look if the same combination exists as a cached result)
• Together with the produced firmware, pack an accordingly modified version of the board.json files for the Connector* and a batch file to copy everything in the correct locations (this could more easily be a function in the Connector though).

Of course this would require hosting, but it should not be so demanding; the whole implementation should amount to a simple Docker+Javascript exercise.
Additionally, there should be no further impact on either Connector or firmware.

* Regarding the update to the board.json files for the Connector, see the comment below by @elral on how they could get rid of.

(This should be more of a firmware-related topic, but as @DocMoebiuz required in #206 I'm posting it here.)

[elral]

The modular firmware build was discussed sometimes on Discord.
Regarding the board capabilities we could think about if they could be part of the firmware instead of the .json file (or they override them if available from the firmware). This can also help for custom firmware once the "generel" device is available (just defining within the firmware which devices are available, no need for changed json file).

[GioCC]

I like this idea about determining the board capabilities: if I got it right, there should even be no need for a .json file. Once a board is initially flashed with the firmware (even when not yet configured), there could be an additional query message that reports the capabilities, and the Connector can just take them without even need for a fallback value set from the .json.
Before the firmware is flashed, on the other hand, knowing the board capabilities is useless, because the board couldn't be configured anyway.
Maybe this could already become an issue of its own?

[elral]

if I got it right, there should even be no need for a .json file.

No, I mean to keep the .json file and only to transfer the boad capabilities (which device and how often) to the board.

I think like Sebastian the .json file is still needed for "everything" except the capabilities (which devices are available). Transferring all informations from .json file to the board would mean not less memory requirement in the ROM, and as Sebastian said there are still informations from the .json file required before the board is connected.

[GioCC]

No, I mean to keep the .json file and only to transfer the boad capabilities (which device and how often) to the board.

As usual, you're right. I was thinking only about the capabilities part, overlooking the rest of the information.
Forget that consideration; still, I find very interesting the possibility for the software to supply any desired part of that information overriding the "fallback" info in the .json. Data (key names) would probably better be encoded, in order to avoid unnecessarily storing/transmitting long strings.

[DocMoebiuz]

Hi guys! Thanks for chiming in on this topic.
I personally have a very clear vision for the custom device support.

First, I want to allow other users to add custom output devices to the firmware. For this i think three things are required:

1. a generic interface to send data to the device
2. have one generic way in the UI for the device settings and the respective output config
3. a generic config object to receive some basic information from the device, mainly the pins in use

My assumption is that we can provide a skeleton device as blueprint for easy start, and that people at first will be tech savvy enough to get the firmware compiled etc. - they basically create a custom firmware and they can obviously even turn off stuff and what not.

When we think of the boards settings, there are two options:

1. a user can add a new custom device dynamically, exactly like devices at the moment
2. the device reports itself statically by adding it to the config

i think both options might make sense in different circumstances and i think we could even start with option 2 only.

Custom firmware - ok this requires also a bit of best practices in the beginning, but we have everything in place already: simply report a special board type and tell mobiflight it's not flashable.

Upload of a custom firmware is now actually possible with the new feature that allows you to choose which firmware you'd like to upload.
We still need a board.json for this. I agree that capabilities could also be reported by the board but it would not be required. the board json is required for individual settings like the "don't flash me" or providing a name for the firmware and vid pid.

Later, when there have been interesting custom devices created by the community, we will have to figure out how we want to share them easily and maybe also combine them in custom firmwares.

Until then i think there is a lot to learn and to experiment and i simply would start with the first step.

[GioCC]

Your considerations are clear and I would really like to add my take on them, but it seems to me they might be more related to #213 (which in turn is related to #331).

The discussion here is meant to be basically limited to standard devices (it will of course involve custom devices, but only when they will be eventually introduced).
It would possibly become more relevant in the wake of the future introduction of different MCU boards.

[Gamma36]

As a retired electronics engineer/ software engineer with CAD/CAM experience back to 1981, my view is that most users struggle with software, they just want simple solutions that work.

FLASH/SRAM/EEPROM levels: 3

Mega - all devices in 4k of firmware, Big cockpits need lots of Megas and lots of pins.

Medium . Nano, Micro,

Tiny: Raspberry Pico - just switches/buttons/LEDS, encoders, 4 analog. No displays, shift registers or mux devices The hardware design mandates small firmware,; the Pico is designed for low power, fixed 3.3V I/O, low pin count and surface mounting. The fixed 3.3V I/O eliminates MAX7219 displays which need 4V to 5V

PI-Pico has a maximum of 16mA per pin with the total current from all pins not exceeding 51mA. So maybe just 3 direct LEDs. Throw in support for one DM13A for 16 LEDs. Maybe one LCD, for a cheap encoder/3.3V LCD module..

Cheers
Paul 95 on discord.

[tigert]

The pico actually has way more processing power and memory (~30x) and flash (8x) than a Mega. It does need a level shifter for MAX7219 but the shift registers should work ok.