Skip to content

fault_manager: a fault confirmed during another fault's post-roll never gets a recording #559

Description

@mfaferek93

Summary

When a fault confirms while another fault's post-roll recording is still running,
it gets no rosbag at all - not then, and not later. The log line is:

Already recording post-fault data, skipping confirmation for '<CODE>'

Faults arrive in bursts from a single root cause, so this drops the black box for
precisely the correlated faults, which are the ones worth replaying. With the
shipped default snapshots.rosbag.duration_after_sec: 1.0, anything confirming
within a second of the first fault is affected.

Cause

RosbagCapture::on_fault_confirmed returns early while recording_post_fault_
is set. The guard exists because one writer is open at a time, but the
confirmation is dropped rather than deferred or attached, so nothing ever writes
a bag for that fault.

Observed

Two faults from one source within the same second: the first has three bags
across its flaps, the second has a freeze frame and zero bags.

Fix direction

A burst is one root cause in one window - one recording can serve several faults.
Attaching the in-flight recording to the additional fault (one metadata row per
fault code, same file_path) fits the existing schema. Deferring the flush
instead does not work: the ring buffer is not filled during a post-roll, so a
queued confirmation would flush an empty buffer.

Bag lifetime then needs refcounting, so deleting one fault of a burst does not
unlink a file another fault still references, and the storage quota counts a
shared bag once.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions