Skip to content

11. Label markers

Jump to bottom
Marco Marini edited this page Jun 26, 2026 · 4 revisions

Formal Robotics Specification – Camera Event and Lidar Sensor Event Correlation

1. System Overview

The system models a robotic perception stack that integrates lidar-based spatial mapping with QR code recognition events. The objective is to maintain a consistent representation of the environment and to correlate heterogeneous sensor events in time.

2. Label Marker Representation

The LabelMarker is a 2D occupancy structure.

Each marker stores the following state variables:

  • label: the QR-code label
  • location: the 2D location of label marker
  • weight: the weight of signal (confidence)
  • t_marker: timestamp of the last camera event related to this marker
  • t_clean: timestamp of the clean marker

Each marker represents a local estimate of environmental occupancy derived from lidar observations and camera recognition event. All the label markers are grouped as a map of unique label id markers.

3. LiDAR Event Processing Model

3.1 Input Definition

A LiDAR event is defined as:

$$ L = (t, r, S) $$

where:

  • t is the timestamp
  • r is the sensor direction
  • S is the set of measured returns

3.2 Update Function

Let:

  • $M_i$: be an existing marker matching the recognized label
  • $\hat{p}$: be the fused position estimate from LiDAR and camera
  • $\lambda > 0$: temporal decay factor

Define:

$$ \alpha = e^{-\lambda (t - t_{marker})} $$

3.3 Marker Update Logic

Case 1 — Valid observation (matched marker)

If a marker is detected within the correlated camera–LiDAR angular sector:

$$ M_i.location \leftarrow (1-\alpha) M_i.location + \alpha \hat{p} $$

$$ M_i.weight \leftarrow 1 $$

$$ M_i.t_{marker} \leftarrow t $$

Case 2 — New marker

If label is not present in the map:

  • create $M_i$
  • initialize:
    • $location = \hat{p}$
    • $weight = 1$
    • $t_{marker} = t$

Case 3 — No observation

If no marker is detected:

  • increment no_detection_counter

Define LiDAR change condition:

$$ |S_t - S_{t-1}| > \delta $$

If LiDAR state changes, store current scan context.

If:

$$ noDetectionCounter \geq \theta_{clean} $$

then:

  • reduce remaining marker weights in cleaning region by $-\frac{t - t_{marker}}{\tau_{clean}} w$ clamping to 0, 1
  • remove markers where $w = 0$

3.4 Receptive (Cleaning) Region Definition

The receptive region $R(t) \subset \mathbb{R}^2$ is the set of points in the planar workspace that are jointly observable by both camera and LiDAR at time $t$, expressed in the robot-centered reference frame.

3.4.1 Camera Field of View Constraint

Let:

  • $p=(x,y)$ be a point in the robot-centered frame
  • $θ(p)$ be the bearing angle of $p$
  • $∥p∥$ be the Euclidean distance
  • $r_C(t)$ is the current camera pointing direction
  • $r_L(t)$ is the current LiDAR pointing direction (or beam angle)

3.4.2 Camera Field-of-View Constraint

A point is visible to the camera if:

$$ |θ(p)−r_C(t)| \le \frac{θ_{cam}}{2} $$

where:

  • $θ_{cam}$ is the horizontal field of view of the camera

3.4.3 LiDAR Angular Consistency Constraint

A point is consistent with LiDAR sensing if:

$$ |θ(p)−r_L(t)| \le \frac{\Delta θ_{lidar}}{2} $$

where:

  • $θ_{lidar}$ is the horizontal field of view of the LiDAR

3.4.4 Range Constraint

A point must lie within the effective sensing range:

$$ ∥p∥≤\min(d(t), d_{max}) $$

where:

  • $d(t)$ is the instantaneous LiDAR measured distance along direction $θ(p)$
  • $d_{max}$ is the maximum reliable sensing distance

3.4.5 Final Definition of the Receptive Region

The receptive region is defined as:

$$ R(t)={p∈R^2 \left| ∣θ(p)−r_C(t)∣≤\frac{θ_{cam}}{2} ∧ ∣θ(p)−r_L(t)∣≤\frac{Δθ_{lidar}}{2} ∧ ∥p∥≤\min(d(t), d_{max}) \right| } $$

4. Camera Event Model

4.1 Definition

A camera event is defined as:

C = (t, m, d)

where:

  • t is the timestamp
  • m is the recognized marker
  • d is the marker direction relative to the camera frame

4.2 Temporal Correlation Constraint

Each camera event must be associated with a LiDAR event such that:

Let:

  • t_p be the proximity event timestamp
  • t_c be the camera event timestamp
  • Δt_corr be the correlation window

A valid association satisfies:

t_p < t_c ≤ t_p + Δt_corr

5. Robot State Classification

The system defines a discrete state space for camera-proximity correlation:

5.1 Missing Camera Event

Occurs if:

  • no camera event exists, or
  • the camera event occurs before the proximity event

5.2 Valid Camera Event

Valid ⇔ (t_c > t_p) ∧ (t_c ≤ t_p + Δt_corr)

5.3 Invalid Camera Event

Invalid ⇔ t_c > t_p + Δt_corr

In this case, spatial correspondence cannot be reliably established.

6. System Assumptions

  • Sensor timestamps are globally ordered or synchronized
  • Lidar and camera events are processed in discrete time steps
  • Geometric projections assume a calibrated and fixed sensor frame relationship

7. Version

Release: 0.26.0

Clone this wiki locally