Replies: 1 comment
|
@matssun, In master, we've fixed dozens of places where the PD previously would go offline. This is done by treating link failure different from an application layer failure and based on my testing things are a lot more stable nowadays. IOW, a PD going offline should already mean a PD is truly offline/faulty. Besides the OSDP specified 8 second grace period (as you observed) is plenty for any reasonable PD which respects the SLAs. To re-introduce a bounded back-off, you need a case where it actually helps more than a patch. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
What happens
When a PD stops answering, the CP probes it for about 8 s and then marks it offline for a fixed
OSDP_ONLINE_RETRY_WAIT_MAX_MS(300 s). If the PD is answering again a moment after the probe gave up (a reader that reboots, a cable that is reseated, a power blip), nothing contacts it for the remaining five minutes.I measured this on current master (
59eb4c8, built from source) with one CP and one PD over an in-memory channel. The PD is switched off until the CP has declared it offline (8.7 s), then switched back on at once:The log line is the familiar one:
Going offline for 300 seconds; Was in 'Online' state.How I understand the history
I tried to find out whether this is intended, and it looks like a deliberate trade-off, so I would like to check that I have read it right:
005e8b0(2021, closing Add bounded exponential back-off PD/CP offline retries #48) added a bounded exponential back-off: retry after 1 s, 2 s, 4 s ... up to 300 s, because a flat 1 s retry was "too wasteful" for a PD that is powered off.182f049(2024-07-31) added the probe state (re-send the last command for 8 s, the spec's inactivity window) to stop spurious secure-channel drops when a PD misses the 200 ms window, especially during file transfers. Its message says this "also means that we can get rid of the exponential back-off that was employed in this path before", and the wait became the constant 300 s.What I could not tell from the commit messages is whether dropping the back-off was meant to cover a PD that comes back shortly after the 8 s probe, or only a PD that is really unplugged.
What I would like to ask
OSDP_ONLINE_RETRY_WAIT_INIT_MS(default 1 s), doubling on every further failure up toOSDP_ONLINE_RETRY_WAIT_MAX_MS, starting over once the PD is online?disable_pd()/enable_pd()as the way to retry sooner)?Why not just lower the maximum, or let the application retry
OSDP_ONLINE_RETRY_WAIT_MAX_MSis one knob for two jobs: it also makes a PD that is gone for good get polled that often, forever.disable_pd()followed byenable_pd()(with a short pause between them) returnedTrue/Trueand the CP saw the PD online 0.2 s later. So the application can implement the policy itself. The cost is that every integrator has to discover that and write the loop, and the sensible policy looks the same for everyone.The cost of what I am proposing
Each retry still includes the probe, about 8 s of the last command re-sent every 200 ms. With the patch and a PD that stays dead, the consecutive "Going offline" waits were 1, 2, 4, 8, 16, 32 s, with 9, 11, 13, 17 and 24 s between the verdicts. Over those first 74 s a dead PD is probed for more than half of the time, and only later does the load fall to about 8 s in every 300 s. That is the reason the initial wait is a tunable. If that early traffic is a concern, a larger default for the initial wait (or a different shape) would be fine with me.
Patch
I have a small patch with a unit test and have not opened a PR, as I would rather settle the direction first. It changes
osdp_cp.c,osdp_common.handosdp_tunables.h, adds the new tunable to the Zephyr and ESP-IDF Kconfig/CMake files in the same way as the maximum (with a compile-time error if it is zero or larger than the maximum), and keeps the wait in its own field becausepd->wait_msis also the command retry delay.The new
cp_offlinesuite rewindspd->tstampinstead of sleeping, runs in the normal and the zero-copy binaries, and checks the initial wait, the doubling up to the maximum, and the reset once the PD is online. It fails on current master and passes with the patch. The existing suites are unchanged: 23 of 23 suites pass with the leanmake check, and 49 of 49 tests pass withcmake --target check, with and withoutOPT_BUILD_OSDP_TRS. The Python pytest suite, run against a build of the patch and a build of unmodified master, gives the same result for both: 328 passed, 4 skipped. I could not runscripts/clang-format-check.sh(clang-format is not installed here), so formatting is by hand.Thanks for the library, and for reading this far. I am happy to adjust the approach, or to drop it if the current behaviour is what you want.
All reactions