RFC: Introducing Watchdog, a cross-layer anomaly detection module #18987
Conversation
CWatchdog is a cross-layer anomalies detection module aims to cover block issuance, peer management, local clock or net level. If any anomalie is detected an event is triggered to inform an application layer or any internal consumer of a future watchdog interface. This code is only integrated in next commit.
CWatchdogInterface is consumed by another module or application willingly to take corrective actions based on anomalies detected. CWatchdogInterface only callback is BlockHeaderAnomalies, meaning block header aren't received at a normal rate modulo block variance.
CWatchdog is integrated in NodeContext to make it accessible to any futute rpc calls. Some application may fine-tune watchdog according to their requirements. This code is not scheduled yet.
Run ScanAnomalies to process any heuristics and trigger watchdog interface callback. As of present commit, there is no heuristic implemented, callback is blankly called to exerce watchdog interface.
Insert watchdog LogHeader after a successful ProcessNewBlockHeaders, therefore updating last time watchdog have seen a step forward on headers tree construction. Watchdog loggers should be designed on being invasive-minimal at they may encumber hot paths.
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
|
🐙 This pull request conflicts with the target branch and needs rebase. |
|
I haven't looked at this PR closer, but I think anomaly detection could be done via USDT in an external application hooking into Bitcoin Core internals too.
This could be done with the
I've added a few tracepoints (not in a PR yet) to addrman to externally observe the
Let if you think this might be an alternative to a module in Bitcoin Core. Happy to chat. |
|
Honestly, I've not been that far in the design of a specific hook interface servicing watchdog/statistical tools. This PoC was relying on a One solution I'm working on is pursuing on the work done by multiprocess with the newly introduced abstract interfaces ( Of course, w.r.t to event collection, the USDT approach you're proposing sounds the right tool. One could imagine a watchdog module where the low-level event collection is gathered through tracing and the high-level corrective logic relying on a
I need to think more, but I think tracepoints could cover the whole range of proposed heuristics: local clock, packets RTT, peer rotation frequency, mempool feerate groups, ... ?
First impression, I think this tracepoint approach is far better than yet-another-non-critical-module in core for sure. Yeah happy to chat, on this PR, a new issue or irc :) |
There hasn't been much activity lately and the patch still needs rebase. What is the status here?
|
|
Closing for now. Let us know when this should be reopened. |
This PR proposes to introduce a new subsystem for anomaly detection and notifications of those for corrective behaviors. It showcases a proof-of-concept laying out the design of such a new system with a new heuristic.
Prevention of eclipse-attacks and network partitions is critical for application security and user funds. Beyond fork detection and obvious protocol misbehavior, Bitcoin Core doesn't implement any active anomalies detection. These vulnerabilities may be even more destructive for time-sensitive protocol with harder security assumptions, such as those requiring real-time block processing with regards to honest majority network. Active detection at the base-layer, where it's easier to monitor, may be used in two ways:
a) With an application, like a payment system or a LN node, cutting its deposit flow or closing its channels in reaction or stopping HTLC processing, and therefore limiting exposure.
b) With an internal Bitcoin Core module, triggering rescue header-fetching -- see the AltNet subsystem proposal as an example of consumer usage.
Even if notifications must be interpreted according to application requirements, such automatic reactions, well-implemented, would be a positive increase for their security.
Detection may rely on a wide range of cross-layer heuristics, including the local clock, packets RTT, ASN distribution among addrman, abnormal peer rotation, stalling or delayed block issuance, mempool congestion and bandwidth consumption surges. You may combine heuristics to increase confidence, but due to the p2p nature of the network or Poisson block interval false positives must be taken into account. Ideally consumer would be able to fine-tune their false-positives exposure with regards to application security and the cost of their reaction.
This PR introduces a new module (
CWatchdog), with low-reliance on other modules beyond synchronous events harvesting (LogHeader). Anomalie detections (ScanAnomalie) is scheduled eachSCAN_ANOMALIES_INTERVALon its own thread. Integration is done withNodeContext, making the module accessible to future RPCs. A subscription interface (CWatchdogInterface) is added, the model ofCValidationInterface. This can host the actual fork detection in the future after refactoringCheckForkWarningConditions.This new module, less intertwined with current code can be disabled by default, and only activated with
--enable-watchdogby node operators willingly to use it.