Firmware Update via webinterface

[TobiTobsen77]

Hello,
I use some of these pretty nice boards with your firmware because I am a fan of wired devices.
Your firmware is working better from update to update.
The only issue with the updates is that I build in the board and the relay into power strips which are not easy to disassemble.
Would there be the possibility to update them over the webinterface like the tasmota sockets?
That would make it much easier than with the stm flasher.

[nielsonm236]

Unfortunately the hardware designer didn't use the SPI interface to attach the ENC28J60 ethernet chip. Instead the designer opted for using a bit-bang interface with GPIO pins. The reason this is important is that the STM8 built-in boot loader requires use of the SPI interface to operate. The boot loader is needed to do the online code updates. Alternative implementations I've looked at won't fit in the available Flash. So ... I'm stuck. I personally would like to have this as well - it sure would have helped with the 1000+ development flash cycles I've been through.
The only workaround I can suggest is to bring the 4 pin header out on a cable to an external connector. Since you aren't the first to request an improvement in this area I think I will look at some some possible solutions to make a more robust cable and connector for easier access in situations like yours. I have one particular installation where I left a 2 foot cable attached so that I could get my laptop close enough to reprogram the device. Not ideal, but beats disassembly.

[nielsonm236]

Mostly a note to myself since I will keep this Issue for further investigation.
STM Application Note AN2659 provides information on creating a boot-loader in firmware. Available Flash is very limited so this won't fit as-is, but there might be a way to create a very limited eNet flash capability.

[TobiTobsen77]

Hello, thank you for the quick reply.
I already thougt about the external connector but my socktes are mounted under tables or behind cupboards.
Not so nice for flashing.
But its very good that you try to find out if its possible over ethernet.
The firmware from april works pretty good for me so its not absolutely necessary to flash a new one.
But I would do it over ethernet of course

[nielsonm236]

OK. I will keep this on my list to attempt implementation. I will probably have to spend a good bit of time finding code reductions to make some space.

[nielsonm236]

Some investigation update with a question
Update: The more I study this the more I am convinced I can't implement it in the available memory space. Looks like I would need at least 4K of additional Flash. So I started looking at some alternatives. If I could add some Flash in the form of an I2C attached memory I could use that space to store a downloaded code image, then go into an offline mode to overwrite the STM8 flash with the downloaded image. The key is to minimize the code that has to stay resident in the STM8 flash, and a simple copy routine would be very small.
Question: I find I can buy a 32Kx8 I2C memory module on eBay for as little as $1.25 USD (including shipping) ... although most sellers are in the $2 to $3 range. For those applications where the physical installation is "hard to reach", would sacrificing 2 IO pins for an I2C interface plus a $2 to $3 cost adder be acceptable for a solution?

[yozik04]

This all is way too much for this feature request. How frequently you need to upgrade? One? Two times in a lifetime? Will you upgrade to latest version just for the sake of “a new version released”. I would upgrade only if I need a fix for a problem.
@nielsonm236 Eventially most common bugs will be fixed and no updates will be needed.

[jmcvieira1]

I agree with @yozik04 this all is way too much for this feature request, but as user i have a question, how many of us did not realize and already have a NetworkModule with the "bigger chip" the STM8S105C6 until today i already by eight modules, purchase at different times at different seller´s all came with the STM8S105C6 the price was the same around 8€, maybe this feature would make more sense on this board´s instead of tinkering around with cables and adapter´s just swap the board on those places were is not possible to update via the four pin cable . but this is just a thought i don´t know anything about programming don´t know if it´s possible to implement this feature even with the STM8S105C6.

[nielsonm236]

The 105 still only has 32K of Flash. It has a larger EEPROM (1KB vs 128B in the 005). So that doesn't help with this feature. Tobi's request is because he places his modules where they are very difficult to access so code updates are a huge pain. I have a personal interest in this feature request mostly because in development and test I have to update code many dozens of times on multiple modules with every new request, although only two of mine are difficult to access.
I do agree that the bug rate will drop off dramatically, and very soon. In fact lately bugs are mostly appearing because of code compression in attempts to squeeze in "one more feature", and we've done about all the squeezing that can be done. If I would stop doing that I bet many months would pass before any new bug is found (if any at all).

[nielsonm236]

To all subscribers for consideration and comment: I spent the last 5 weeks working on the "Firmware update over Ethernet" feature, and I have it working for my personal purposes, but perhaps with some drawbacks for others that may want to use it. The pertinent points:

1. The feature required attachment of I2C EEPROM memory as there needs to be a place to deposit a code update, then that update can be copied to Flash.
2. I found that the "Updater" was large enough that there was no possible way to fit it in Flash with the MQTT code. So, I created a two step process. a) From your runtime code you input a command that loads the Updater from I2C EEPROM, then you use the Updater GUI to identify the .sx file you want to download. The Updater downloads the new .sx file, installs it in Flash, and reboots.
3. The above actually requires that I have two 32KB I2C EEPROM storage areas. One for the new "Operating Code", and one for the "Updater". In this way the Updater remains in the I2C EEPROM and can be called on at any time to allow a new code downlaod.
4. Obviously (3) raises the question of "How did the Updater get into the I2C EEPROM to start with?" Well, you have to install it with the SWIM interface and when booted it copies itself to its reserved area in the I2C EEPROM. Then you can run the Updater to install any .sx file. But if you want to continue using the Updater you have to use .sx files released starting with the first Updater release ( meaning old .sx files don't know anything about the update process, so if they are installed the process no longer works). Of course you can still install any .sx file via SWIM regardless of whether you have the I2C EEPROM attached.
5. I'm using a pair of cheap 32KB (256kb) I2C EEPROMs for the development. Search eBay for "AT24C256 Serial I2C Interface". They cost between $1.50 to $2.00 USD each including shipping and the memory. OR - you can buy a board without the memory for about $0.99 USD then buy a 128KB (1024kb) dip to put on the board. I was able to find some of these for about $1.50 USD each. OR - you can buy a board and two of the 256kb dips and piggy-back them (leaving address pin A0 open on one of the dips).
   That's a lot of info ... and here's more:

• I'm considering expanding use of the I2C EEPROM for future features. So the 128K EEPROM might be the smarter way to go. Future features might include some tricks to free up a bunch of Flash in the MQTT version of the project.
• It takes about 90 seconds to install the Updater, then another 90 seconds to download and install a new .sx file. I don't currently know how to provide progress in the GUI in the very tiny code space, so you just have to be patient when using the process. Can it be faster? Yes, but it requires additional code space which won't fit (for now).
• Anything I do needs to keep some "bare bones" versions of the firmware available for people that don't have any need for "update over Ethernet". So, maybe I have to start releasing several parallel releases that support different sets of features. I already do that given I have an MQTT version and a Browser Only version. The next step would be to add an "MQTT with Enet Upgrade" version and a "Browser Only with Enet Upgrade" version, both of which require attaching the external I2C EEPROM.
• FYI, the MQTT version I have right now WILL compile with the I2C EEPROM code, but it only leaves 106 bytes to spare in the Flash. And this is after a lot of work finding additional code reductions in the MQTT code.
• And another FYI: I decided to work on this feature first for two reasons: 1) I personally wanted it, and 2) It required development of the I2C interface, which I will need for attaching humidity and barometric sensors.

This code is ready for some Beta exposure ... but of course anyone using it would have to buy and attach the I2C EEPROM memory. I would appreciate some discussion from the subscribers to see how much interest there is, as I could use some help testing and experimenting with the code to see if I've overlooked some bugs. Let me know what you think.

I can provide links to hardware on eBay if wanted so that you will know the exact parts I used.

Mike

[nielsonm236]

I'm leaning toward the requirement for a 1024kb I2C EEPROM (24AA1025 or 24LC1025 or 24FC1025). Here's why:

• For users that don't want to add the EEPROM I can still maintain the releases we have now (MQTT and Browser Only). But it is clear the MQTT version does not have enough space for additional features of any significance.
• For users that add the I2C EEPROM it will support the "upgrade over Enet" process, and I expect that I will be able to store many of the infrequently used constants and strings in the I2C EEPROM. This would likely enable creation of a single "Upgradeable" release that combines MQTT and Browser Only features in one code load. And perhaps the upgrade process would be a single step instead of two - although I am less certain of this item.
• If I am successful at moving constants and strings to the I2C EEPROM I believe I will be able to implement the block-mode programming of Flash. This would reduce the re-flash time from the 90 seconds described above to well under 10 seconds.

I'm still interested in thoughts from others. But in a couple of days I will start down the path described above if nobody stops me with some better ideas.

Mike

[yozik04]

Nice progress. I am sure I will not personally use it but looks interesting. =) Do you do checksums of written to the EEPROM?

[nielsonm236]

I checksum the ENet transmissions, but not the writes to the EEPROMs. I need a little more code space to add EEPROM checksums, and think moving some constants and strings to the EEPROMs would give me that space ... perhaps a great deal of space. The EEPROM checksums might not be needed since they should be on very short wires with little chance of corruption. Another issue to resolve is I don't think I have adequate "failure" paths to allow graceful recovery when I do find a problem. Again, need Flash space to harden the code, although the hardening may not be necessary in this application.

[nielsonm236]

Code development and test continues. This was a very involved change. I've got I2C and Off-Board EEPROM working, and if Off-Board EEPROM is detected the main webpages are stored in the Off-Board EEPROM, freeing up about 5K of Flash space for future code. Next step is to try to make Flash programming faster (about 10x faster) than what I have now. If that works this may be ready for Beta. It is currently working well enough that I use the Ethernet programming interface for most of my bench development code updates ... except when I really screw it up and need to unbrick it with the SWIM interface.

[jmcvieira1]

Good job, my EEPROM should arrive this week, as soon as it arrives I'll give you notice.

[nielsonm236]

I just sent some Beta code to Jose Viera. If anyone else wants to join the party let me know.
Mike