Skip to content

[BUG] Fault injection not working #20

@bburda

Description

@bburda

Description

The fault injection scripts (inject-collision.sh, inject-controller-failure.sh, inject-nav-failure.sh, inject-localization-failure.sh) in the TurtleBot3 integration demo were not working as expected. After running all inject scripts, only one fault was visible in the FaultManager, and some scripts produced no faults at all.

The root cause was a flawed approach: scripts attempted to trigger faults by deactivating Nav2 lifecycle nodes, but lifecycle_manager automatically reactivates them, preventing persistent fault conditions. Additionally, Nav2 doesn't publish ERROR-level diagnostics for normal navigation failures (path planning failures, goal aborts), as these are considered normal operational events, not system faults.

Steps to Reproduce

  1. Start TurtleBot3 demo:

    cd demos/turtlebot3_integration
    ./run-demo.sh
  2. Run fault injection scripts:

    ./inject-nav-failure.sh
    ./inject-collision.sh
    ./inject-controller-failure.sh
    ./inject-localization-failure.sh
  3. Check faults:

    curl http://localhost:8080/api/v1/faults | jq

Expected vs Actual Behavior

Expected:

  • Each inject script should create specific, distinct faults
  • Faults should be visible via REST API at /api/v1/faults
  • Multiple faults should be reported simultaneously
  • Fault codes should be descriptive: NAVIGATION_FAILURE, COLLISION_WARNING, CONTROLLER_FAILURE, LOCALIZATION_FAILURE

Actual:

  • After running all 4 inject scripts, only 1 fault was visible
  • Some scripts (collision, controller-failure) produced no faults
  • Scripts using lifecycle node deactivation failed because lifecycle_manager auto-reactivated nodes
  • Nav2 doesn't report navigation failures (goal aborts, path planning failures) as ERROR diagnostics - they're treated as normal operational events

Environment

  • OS: Ubuntu 24.04 (Docker container)
  • ROS 2 Distro: Jazzy
  • ros2_medkit: Latest (commit prior to fix)
  • Demo: TurtleBot3 + Nav2 + Gazebo

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions