Time-to-close and further memory improvements #145

dkerr64 · 2024-04-04T00:31:24Z

fixes for issues #60 (time-to-close delay) and #143 (only reboot for settings change when necessary). Also includes cleanup and removal of unnecessary code / html pages.

Added implementation for #150

jgstroud · 2024-04-04T22:46:34Z

at @dkerr64 can we move that last commit for the OTA to a separate PR?

dkerr64 · 2024-04-04T22:52:17Z

at @dkerr64 can we move that last commit for the OTA to a separate PR?

Yes, done. PR #153

jgstroud · 2024-04-07T15:52:50Z

src/comms.cpp

@@ -859,6 +892,19 @@ void open_door() {
    door_command(DoorAction::Open);
 }

+void TTCdelayLoop() {
+    if (--TTCcountdown > 0) {


There is a "service_timers_loop" in the ratgdo.cpp file. Should this go there instead?

Right now in the transmitSec2 function we write the rolling code to flash every time. I think we need to add something to only write once an hour or something. This should be ok now that the device is stable enough that it stays up for days at a time. The esphome variant does something along these lines.

I think we have to do this now because closing the garage door now will trigger a lot more writes to flash.

There is a "service_timers_loop" in the ratgdo.cpp file. Should this go there instead?

I probably erred in naming this xxxLoop as this function is not part of the main loop. It executes once every 500ms using Ticker(). And it only executes during a time-to-close delay (so for a 10 second delay, it is called 20 times) to cycle the light and on completion of the timeout close the door.

Right now in the transmitSec2 function we write the rolling code to flash every time. I think we need to add something to only write once an hour or something. This should be ok now that the device is stable enough that it stays up for days at a time. The esphome variant does something along these lines.

I think we have to do this now because closing the garage door now will trigger a lot more writes to flash.

If we do this (which makes sense) then we need some sort of error recovery... as we now have EspSaveCrash with a callback then maybe we have to write to the flash whenever it crashes. And is there a way to detect on boot-up that things are out-of-sync and re-initialize it... then I can remove the code file deletes from protocol change.

for reference: https://github.com/ratgdo/esphome-ratgdo/blob/main/components/ratgdo/secplus2.cpp#L25

Ah, so the GDO is tolerant of incrementing >1 in the rolling code, and ESPhome guys took a guess at 60. We could do the same, but note that time-to-close light flash sends two per second, so our math might be a little different... if we write once a minute, then in theory we could have >120 code increments. And we can also handle the crash case with our callback, and manually requested reboots. Which just leaves power outages as uncaught.

btw... what is the expected life of the ESP8266 flash?

src/web.cpp

dkerr64 · 2024-04-07T19:44:41Z

I rebased onto latest main. Sorry for several force-push. Had several conflicts in web.cpp to get right.

I've been running with a browser open for 17 hours now, with server logs going to javascript console. All good.

I was originally thinking of having a separate URL for a separate browser to connect to, and for messages to appear in that browser. But when I looked into it, I decided that camping onto the existing server-sent-event code was easiest and most efficient on ESP memory usage, and just dumping to the browser's javascript console was much simpler than anything else. The server side code is very efficient. Just don't put any RINFO() inside the code that processes it, else infinite loop!!

I added user setting to turn this on, that setting is local to the browser (localStorage) and I hide it if you are on mobile device as makes no sense to send logs to those.

Last piece is to change HomeKit library to capture its logs.

jgstroud · 2024-04-07T19:55:45Z

The rebase and force push is fine. I haven't had time to test this out yet. Sounds like the new memory layout option isn't showing any negative effects yet. How is your free heap looking? Can multiple clients be monitoring the serial log?

dkerr64 · 2024-04-07T20:07:28Z

No negative effects. But we will have to keep an eye on things. Free heap is in good condition.

As for the IRAM heap. We need to keep track...

json string buffer... 1024 bytes.
log lineBuffer... 1024 bytes (could probably reduce to 512 or less, for sprintf() into for each long line.
log msgBuffer... 2048 bytes (holds about last 20 log lines, could increase to 4096?).
savedLogs... 2048 (malloc's and free'd to read saved log file into when display requested).

In other words, we need at least 2x the message buffer space as we must allocate space to read the saved file into. Now in theory I could probably read the file in chunks, but it would get rather complicated because the "head" of the circular buffer could be anywhere in the file. So long as we have the space, I prefer reading it all in at once.

jgstroud · 2024-04-07T20:29:54Z

We can log the free IRAM heap as well, but maybe not add to web ui

https://github.com/esp8266/Arduino/blob/master/libraries/esp8266/examples/irammem/irammem.ino#L465

jgstroud

Do you still want the Keep track of last door state change commit without the NTP client

jgstroud · 2024-04-30T18:37:59Z

I've rebased, reordered some of the commits and squashed some commits. I don't think I can update this PR, so I'm going to replace this PR with a new one

dkerr64 · 2024-04-30T19:08:45Z

Do you still want the Keep track of last door state change commit without the NTP client

Yes please, I'd like to keep that. I removed all the NTP client so it just tracks it from last reboot. I find it useful and its minor stuff.

dkerr64 · 2024-04-30T19:09:39Z

I've rebased, reordered some of the commits and squashed some commits. I don't think I can update this PR, so I'm going to replace this PR with a new one

Yes, definitely some squashing can be done. Thanks. I can make you a collaborator (thought you already were) or you can do a new PR if you prefer. Your choice.

jgstroud · 2024-04-30T19:36:50Z

I've rebased, reordered some of the commits and squashed some commits. I don't think I can update this PR, so I'm going to replace this PR with a new one

Yes, definitely some squashing can be done. Thanks. I can make you a collaborator (thought you already were) or you can do a new PR if you prefer. Your choice.

I was able to update the PR. Thanks. It could probably be cleaned up more, but for the most part, I'm happy with this

jgstroud · 2024-04-30T19:43:55Z

I never moved the crashlog from eeprom to filesystem. It started to seem like a big rewrite of the utility and I'm not sure we gain a whole lot from it. we can do that later though

dkerr64 · 2024-04-30T19:45:31Z

I never moved the crashlog from eeprom to filesystem. It started to seem like a big rewrite of the utility and I'm not sure we gain a whole lot from it. we can do that later though

I'm fine with that. What about overwriting previous crash data, so it only attempts to keep the latest? Save that for future too?

dkerr64 · 2024-04-30T19:47:36Z

I've rebased, reordered some of the commits and squashed some commits. I don't think I can update this PR, so I'm going to replace this PR with a new one

Yes, definitely some squashing can be done. Thanks. I can make you a collaborator (thought you already were) or you can do a new PR if you prefer. Your choice.

I was able to update the PR. Thanks. It could probably be cleaned up more, but for the most part, I'm happy with this

good, thanks. Embarrassingly I still have that missing semicolon commit visible to all !!! But I suppose that demonstrates integrity and that I'm not ashamed to make my trivial errors visible. :-)

jgstroud · 2024-04-30T19:51:38Z

I've rebased, reordered some of the commits and squashed some commits. I don't think I can update this PR, so I'm going to replace this PR with a new one

Yes, definitely some squashing can be done. Thanks. I can make you a collaborator (thought you already were) or you can do a new PR if you prefer. Your choice.

I was able to update the PR. Thanks. It could probably be cleaned up more, but for the most part, I'm happy with this

good, thanks. Embarrassingly I still have that missing semicolon commit visible to all !!! But I suppose that demonstrates integrity and that I'm not ashamed to make my trivial errors visible. :-)

oops, missed that one

…crashes

Nicer upload progress serial logging

send CRC error message to client

This was referenced Apr 4, 2024

Warning beep control on or off #60

Closed

Don't reboot ratgdo unless really necessary (save settings) #143

Closed

jgstroud force-pushed the main branch from 92dd3dc to c630151 Compare April 4, 2024 00:56

dkerr64 force-pushed the time-to-close branch from b0587df to 49239ba Compare April 4, 2024 01:25

jgstroud force-pushed the main branch from ca75001 to c1f621e Compare April 4, 2024 02:06

dkerr64 force-pushed the time-to-close branch 5 times, most recently from a636921 to 4c3b6fa Compare April 4, 2024 19:23

dkerr64 mentioned this pull request Apr 5, 2024

Add setting to adjust WiFi TX Power #150

Closed

dkerr64 force-pushed the time-to-close branch 7 times, most recently from 2f30944 to 8488c94 Compare April 7, 2024 02:33

jgstroud reviewed Apr 7, 2024

View reviewed changes

jgstroud mentioned this pull request Apr 7, 2024

Add support for Battery Status #140

Closed

dkerr64 force-pushed the time-to-close branch 2 times, most recently from 6f2f8f2 to f1faebe Compare April 7, 2024 19:01

Time to close

6679e63

dkerr64 force-pushed the time-to-close branch from f1faebe to 940c65f Compare April 7, 2024 19:11

Try moving web objects to iram heap

2e849f0

jgstroud reviewed Apr 30, 2024

View reviewed changes

jgstroud force-pushed the time-to-close branch from 09cbd58 to 85033ec Compare April 30, 2024 19:35

dkerr64 and others added 15 commits April 30, 2024 14:52

Implement wifi power setting, addresses issue ratgdo#150

3bff860

Convert time-to-close visual to a slider, add CSS for sliders

ea30cd1

Add functions to read & write binary data to file

357967c

Add a memory buffer for log messages that is saved to file if system …

76b798c

…crashes

Reset door opener ID and rolling codes when switching protocols

874be5a

Send server logs to browser javascript console if requested

0d1e2d9

Redirect HomeKit library log messages to use our custom logger function

5b0549a

Increase log message buffer from 2K to 4K and fix set door protocol bug

a392d13

Add arduino_homekit_close() when starting firmware update.

3829b1f

Add MD5 check for firmware update

8f3f9cb

Reduce number of flash writes for rolling code.

de75e43

Update readme and screen captures to match current code.

11f8408

Report upload percent complete to browser

cb93400

Nicer upload progress serial logging

Keep track of last door state change

1adf604

Check flash CRC after firmware upload

068e42a

send CRC error message to client

jgstroud force-pushed the time-to-close branch from 85033ec to 068e42a Compare April 30, 2024 19:52

More crash testing debug code

892c2f9

jgstroud merged commit ff1ad42 into ratgdo:main Apr 30, 2024

jgstroud mentioned this pull request Apr 30, 2024

1.2.0 OTA bricked RATGDO #151

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Time-to-close and further memory improvements #145

Time-to-close and further memory improvements #145

dkerr64 commented Apr 4, 2024 •

edited

jgstroud commented Apr 4, 2024

dkerr64 commented Apr 4, 2024

jgstroud Apr 7, 2024

jgstroud Apr 7, 2024

dkerr64 Apr 7, 2024

dkerr64 Apr 7, 2024

jgstroud Apr 7, 2024

dkerr64 Apr 7, 2024

dkerr64 Apr 7, 2024

dkerr64 commented Apr 7, 2024

jgstroud commented Apr 7, 2024

dkerr64 commented Apr 7, 2024

jgstroud commented Apr 7, 2024

jgstroud left a comment •

edited

jgstroud commented Apr 30, 2024

dkerr64 commented Apr 30, 2024

dkerr64 commented Apr 30, 2024

jgstroud commented Apr 30, 2024

jgstroud commented Apr 30, 2024

dkerr64 commented Apr 30, 2024

dkerr64 commented Apr 30, 2024 •

edited

jgstroud commented Apr 30, 2024

Time-to-close and further memory improvements #145

Time-to-close and further memory improvements #145

Conversation

dkerr64 commented Apr 4, 2024 • edited

jgstroud commented Apr 4, 2024

dkerr64 commented Apr 4, 2024

jgstroud Apr 7, 2024

Choose a reason for hiding this comment

jgstroud Apr 7, 2024

Choose a reason for hiding this comment

dkerr64 Apr 7, 2024

Choose a reason for hiding this comment

dkerr64 Apr 7, 2024

Choose a reason for hiding this comment

jgstroud Apr 7, 2024

Choose a reason for hiding this comment

dkerr64 Apr 7, 2024

Choose a reason for hiding this comment

dkerr64 Apr 7, 2024

Choose a reason for hiding this comment

dkerr64 commented Apr 7, 2024

jgstroud commented Apr 7, 2024

dkerr64 commented Apr 7, 2024

jgstroud commented Apr 7, 2024

jgstroud left a comment • edited

Choose a reason for hiding this comment

jgstroud commented Apr 30, 2024

dkerr64 commented Apr 30, 2024

dkerr64 commented Apr 30, 2024

jgstroud commented Apr 30, 2024

jgstroud commented Apr 30, 2024

dkerr64 commented Apr 30, 2024

dkerr64 commented Apr 30, 2024 • edited

jgstroud commented Apr 30, 2024

dkerr64 commented Apr 4, 2024 •

edited

jgstroud left a comment •

edited

dkerr64 commented Apr 30, 2024 •

edited