30. NN Input Specification

Design Document

---

This is a Draft document not yet implemented in software.

---

Neural Network Input Specification (World-State Encoding)

---

1. Objective

Define a clean production-ready input interface that converts robot world-state data into structured tensors suitable for neural network consumption.

This specification defines inputs only (no fusion, no downstream modeling).

---

2. Input Modalities (Raw World State)

At each timestep t, the system provides three raw modalities:

---

2.1 Occupancy Grid

A dense spatial representation of the environment:

• occupancy probability
• free-space confidence
• contact signal (optional channel)

O(t) ∈ ℝ^{H × W × C_o}

---

2.2 Landmark Graph (QR-based)

A sparse semantic graph of detected QR markers:

G(t) = (V, E)

Where:

• V = QR marker nodes
• E = spatial / visibility edges (optional)

Each node:

v_i = (id, position_2d, orientation, confidence)

---

2.3 Robot Pose (Ego-State)

Robot state in 2D space:

x(t) = (x, y, θ)

---

3. Encoding Functions (Input Transformation Layer)

Each modality is transformed into a neural-ready latent representation.

Encoders are considered part of the perception-to-tensor interface and not part of downstream neural architecture.

Encoders are deterministic preprocessing modules that do not perform temporal reasoning or policy computation.

---

3.1 Occupancy Encoder

O'(t) = f_occ(O(t))

O'(t) ∈ ℝ^{H × W × d_o}

---

3.2 Graph Encoder

G'(t) = f_graph(G(t))

G'(t) ∈ ℝ^{N × d_g}

Graphs are padded or masked to a maximum size N_max for batching.

---

3.3 Pose Encoder

P'(t) = f_pose(x(t))

Recommended encoding:

f_pose(x, y, θ) = MLP([x, y, sinθ, cosθ])

P'(t) ∈ ℝ^{d_p}

---

4. Neural Network Input Format

4.1 Single timestep input

The final input at time t is a keyed tensor map:

X(t) = {
    "occupancy": O'(t),
    "graph":     G'(t),
    "pose":      P'(t)
}

---

4.2 Batch format (training/inference)

X ∈ {
    "occupancy": ℝ^{B × T × H × W × d_o},
    "graph":     ℝ^{B × T × N × d_g},
    "pose":      ℝ^{B × T × d_p}
}

Where:

• B = batch size
• T = temporal window length

---

5. Design Guarantees

This input specification ensures:

• Modularity: each modality is independently encodable
• Consistency: shared temporal indexing across modalities
• Scalability: supports variable graph size (N nodes)
• Compatibility: suitable for CNNs, GNNs, Transformers

---

6. Constraints

• No fusion or joint embedding is defined here
• No temporal modeling is assumed inside encoders
• All modalities must be independently encodable
• Output tensors must remain numerically stable and fixed-dimension per modality (except graph nodes)

---

7. Summary

This document defines a production-ready neural input interface for robotics systems based on:

• dense occupancy grids
• sparse landmark graphs
• ego-pose state vectors

It serves as a strict contract between robot perception systems and neural network architectures.

---

8. Version

Release: Draft

---