episode 7 the emotions

Episode 7: The Emotions / Las Emociones

"Eight keys, eight faces — now ORION can show emotions." "Ocho teclas, ocho caras — ahora ORION puede mostrar sus sentimientos"

Commit and release version: Release: Episode 7 - The Emotions

📑 Contents

• 🎥 Watch the Short
• 📖 Episode Description
• 🚀 Usage
• 🔄 Changes from Episode 6
• 💡 Lessons & Learnings
• 🔌 Vambrace Connections Up to This Episode
• 🎯 Challenge Posed
• ⏭️ Next Episode

---

🎥 Watch the Short

English Version

Versión en Español

▶️ Watch in English | ▶️ Ver en Español

---

📖 Episode Description

With movement and arm control already working, the team turns ORION's face (well, screen) into a communication tool. Felipe wires and program the 4×4 membrane keypad to the Vambrace, while also building a new prototype. David is ready for testing it and aim for future design improvements.

---

🚀 Usage

Note: Wiring details for all components are in the Connections section below.

1. Configure WiFi credentials

Copy the example template and fill in your values:

cp lib/vambrace_secret/vambrace_secret_ex.hpp lib/vambrace_secret/vambrace_secret.hpp

Then edit vambrace_secret.hpp:

#define WIFI_SSID    "YOUR_NETWORK_NAME"
#define WIFI_PASS    "YOUR_PASSWORD"
#define AGENT_IP_STR "192.168.X.X"   // IP of the machine running the micro-ROS agent
#define AGENT_PORT   8888

2. Connect the 4×4 membrane keypad to the ESP32

Wire the keypad following the Wiring Diagram in the Connections section below.

3. Flash the firmware

pio run --target upload
pio device monitor --baud 115200

On a successful boot you should see:

Vambrace ESP32 started!
Connected. IP: 192.168.X.X
NTP synced!
Joystick calibrated: center_x=XXXX, center_y=XXXX
Hardware initialized.

4. Start the µ-ROS agent on ORION

ros2 run micro_ros_agent micro_ros_agent udp4 --port 8888

5. Verify emotion publishing

Press keys 1–8 on the keypad and verify the emotion index changes:

ros2 topic echo /emotion/int

Expected output when pressing key 3 (fear):

data: 2
---

The topic only publishes when a key is pressed — it does not publish continuously.

---

💡 Lessons & Learnings

Key Concept: Matrix scanning

A 4×4 membrane keypad has 16 keys but only 8 pins — 4 rows and 4 columns. The key at the intersection of a row and a column is detected by driving that row LOW and reading which column also goes LOW.

     C0   C1   C2   C3
R0 [ 1  ][ 2  ][ 3  ][ A  ]
R1 [ 4  ][ 5  ][ 6  ][ B  ]
R2 [ 7  ][ 8  ][ 9  ][ C  ]
R3 [ *  ][ 0  ][ #  ][ D  ]

The scan drives one row LOW at a time while all others stay HIGH. If a column reads LOW, it means the key at that intersection is being pressed. This is repeated for each row at 20 Hz.

Why GPIO 2 and GPIO 4 were discarded

The ESP32 has several strapping pins — pins that the chip reads at boot to decide its operating mode. Two of them caused problems in this episode:

GPIO 2 has an external pull-down resistor on most ESP32 DevKit V1.0 boards. This resistor is there by design to ensure the chip boots correctly. The problem: when used as INPUT_PULLUP, the external pull-down fights the internal pull-up and wins — the pin always reads LOW regardless of whether a key is pressed. The result was the keypad appearing to constantly detect column 3, printing A non-stop.

GPIO 4 is also a strapping pin with similar behavior on certain board revisions — unreliable as a digital input with pull-up.

The fix: move all column pins (INPUT_PULLUP) to clean non-strapping GPIOs (13, 14, 25, 26). GPIOs 2 and 4 were discarded for the keypad matrix connection. The final assignment of the remaining pins (LEDs, switches and other elements) is left pending and will be reviewed in a future episode.

Events vs. continuous state

The emotion topic /emotion/int is fundamentally different from /cmd_vel or the arm topics. Velocity and arm position are continuous state — the robot needs a constant stream to know where to be. Emotion is an event — it should only change when the operator makes a deliberate choice.

Publishing emotion on every loop cycle (33 Hz) caused ORION to immediately revert to neutral after every keypress, because the default value 4 (neutral) was overwriting the new emotion 50ms later.

The fix: publish /emotion/int only when a new keypress is detected.

// Wrong — publishes neutral (4) on every loop, overwriting the keypress
msg_emotion.data = cmd.emotion();  // always 4
rcl_publish(&pub_emotion, ...);

// Correct — only publishes when a key was actually pressed
if (cmd.keypress() >= '1' && cmd.keypress() <= '8')
{
    msg_emotion.data = cmd.keypress() - '1';
    rcl_publish(&pub_emotion, ...);
    cmd.clearKeypress();
}

Key concept: producer vs. consumer

vambrace_hardware detects which key was pressed and stores it in TeleoperationCmd. It does NOT clear it. vambrace_ros reads the keypress, publishes it, and then clears it. This separation ensures the data survives long enough to be used — if hardware cleared it immediately after storing it, the publish layer would always see an empty value.

What We Learned:

• Matrix scanning: 16 keys with only 8 pins — rows are outputs, columns are inputs, scan one row at a time.
• Strapping pin hazard: GPIO 2 and 4 cannot be used as INPUT_PULLUP on DevKit V1.0 — their boot-time resistors override the software pull-up.
• Events vs. continuous state: some topics should only publish on change, not on every loop iteration.
• Producer/consumer pattern: the module that writes data should not be the one that clears it — only the consumer knows when it's done.
• Debounce: mechanical contacts bounce for ~10–50ms when pressed. Without debounce, a single keypress registers as dozens of events. A timestamp comparison (now - last_keypress_ms >= KEYPAD_DEBOUNCE_MS) filters the noise.
• Anti-ghosting via confirmation scan: debounce prevents a key from repeating within 50ms, but does NOT prevent phantom keys caused by simultaneous presses. A ghost occurs when two keys share a row or column — the scan incorrectly reports a third key at their intersection. The fix: after detecting a key, wait 5ms and scan again. Only accept the keypress if both scans return the same key.
• Transition-based detection: polling for a non-null key on every scan causes re-fires if the user holds a key across multiple scan cycles. Tracking last_scanned_key and only acting when the detected key changes (press or release) ensures exactly one event per physical keypress, regardless of how long the key is held.

Technical Highlights:

• Keypad scans at 20 Hz (KEYPAD_INTERVAL_MS = 50) — aligned with joystick and potentiometer polling to eliminate variable latency.
• Debounce window of 50ms (KEYPAD_DEBOUNCE_MS) — ignores repeated triggers within that window.
• TeleoperationCmd extended with setKeypress(), keypress(), clearKeypress() — hardware and ROS layers remain independent.
• vambrace_hardware stores the detected key char as-is (any key on the pad — '1'–'8', but also A–D, *, #, 0). vambrace_ros is the one that maps it to the emotion index (0–7) and only acts on '1'–'8'; keys outside that range are ignored (reserved for macros in Ep. 8). Neither module knows about the other.

---

🔄 Changes from Episode 6

New in vambrace_config.hpp

// 4x4 Membrane keypad pins
#define KEYPAD_ROW0_PIN    21     // module pin 1 → row 0 (1 2 3 A)
#define KEYPAD_ROW1_PIN    19     // module pin 2 → row 1 (4 5 6 B)
#define KEYPAD_ROW2_PIN    18     // module pin 3 → row 2 (7 8 9 C)
#define KEYPAD_ROW3_PIN     5     // module pin 4 → row 3 (* 0 # D)
#define KEYPAD_COL0_PIN    13     // module pin 5 → col 0 (1 4 7 *)
#define KEYPAD_COL1_PIN    14     // module pin 6 → col 1 (2 5 8 0)
#define KEYPAD_COL2_PIN    25     // module pin 7 → col 2 (3 6 9 #)
#define KEYPAD_COL3_PIN    26     // module pin 8 → col 3 (A B C D)
#define KEYPAD_DEBOUNCE_MS  50
#define KEYPAD_INTERVAL_MS  50

Addition	Purpose
constexpr char KEYPAD_MAP[4][4]	Character map moved here from vambrace_hardware.cpp — single source of truth for keypad layout
KEYPAD_INTERVAL_MS changed 100 → 50	Aligns keypad scan to 20 Hz, same as joystick and potentiometer

New in vambrace_app.hpp / vambrace_app.cpp

Addition	Purpose
char keypress_ field	Stores the last detected key
setKeypress(char key)	Called by hardware on valid keypress
keypress() const	Read-only access for the ROS layer
clearKeypress()	Called by ROS after publishing

New in vambrace_hardware.cpp

Addition	Purpose
KEYPAD_MAP[4][4] (moved to config)	—
KEYPAD_ROWS[4] / KEYPAD_COLS[4]	Pin arrays for scan loop
scan_keypad()	Matrix scan — returns pressed key or '\0'
last_scanned_key	Tracks the previously detected key — fires only on transitions, preventing re-fire on held keys
Confirmation scan (delayMicroseconds(5000) + second scan_keypad())	Anti-ghosting: confirms the detected key is stable before accepting it
Keypad init in vambrace_hardware_init()	Sets row pins as OUTPUT HIGH, col pins as INPUT_PULLUP
Keypad scan in vambrace_hardware_update()	20 Hz scan with transition detection, debounce, and anti-ghosting

Changed in vambrace_ros.cpp

Emotion publish moved from unconditional to keypress-gated — only fires when cmd.keypress() is in range '1'–'8'.

Change	Purpose
keypress_to_emotion(char key) helper added	Makes the ASCII-to-index mapping ('1' → 0, …, '8' → 7) explicit and easy to extend

No changes to

• main.cpp
• vambrace_ros.cpp publisher initialization

---

🔌 Vambrace Connections Up to This Episode

Joystick (Ep. 5)

ESP32 GPIO	Signal	Description
34	VRx	X axis — angular velocity (ADC1, input-only)
35	VRy	Y axis — linear velocity (ADC1, input-only)
32	SW	Sprint button (INPUT_PULLUP)

Potentiometers (Ep. 6)

ESP32 GPIO	Signal	Description
33	WIPER	Left arm potentiometer (ADC1)
36	WIPER	Right arm potentiometer (ADC1, input-only)

4×4 Membrane Keypad (Ep. 7)

ESP32 GPIO	Module pin	Signal	Description
21	1	ROW0	Row 0 — keys 1 2 3 A (OUTPUT)
19	2	ROW1	Row 1 — keys 4 5 6 B (OUTPUT)
18	3	ROW2	Row 2 — keys 7 8 9 C (OUTPUT)
5	4	ROW3	Row 3 — keys * 0 # D (OUTPUT)
13	5	COL0	Col 0 — keys 1 4 7 * (INPUT_PULLUP)
14	6	COL1	Col 1 — keys 2 5 8 0 (INPUT_PULLUP)
25	7	COL2	Col 2 — keys 3 6 9 # (INPUT_PULLUP)
26	8	COL3	Col 3 — keys A B C D (INPUT_PULLUP)

Wiring Diagram

Joystick Module          ESP32 DevKit V1.0
┌──────────────┐         ┌──────────────────┐
│ VCC ─────────┼────────→│ 3.3V             │
│ GND ─────────┼────────→│ GND              │
│ VRx ─────────┼────────→│ GPIO 34 (ADC1)   │
│ VRy ─────────┼────────→│ GPIO 35 (ADC1)   │
│ SW  ─────────┼────────→│ GPIO 32          │
└──────────────┘         └──────────────────┘

Left Arm Potentiometer   ESP32 DevKit V1.0
┌──────────────┐         ┌──────────────────┐
│ VCC ─────────┼────────→│ 3.3V             │
│ GND ─────────┼────────→│ GND              │
│ WIPER ───────┼────────→│ GPIO 33 (ADC1)   │
└──────────────┘         └──────────────────┘

Right Arm Potentiometer  ESP32 DevKit V1.0
┌──────────────┐         ┌──────────────────┐
│ VCC ─────────┼────────→│ 3.3V             │
│ GND ─────────┼────────→│ GND              │
│ WIPER ───────┼────────→│ GPIO 36 (ADC1)   │
└──────────────┘         └──────────────────┘

4x4 Membrane Keypad      ESP32 DevKit V1.0
┌──────────────┐         ┌──────────────────┐
│ Pin 1 (ROW0)─┼────────→│ GPIO 21          │
│ Pin 2 (ROW1)─┼────────→│ GPIO 19          │
│ Pin 3 (ROW2)─┼────────→│ GPIO 18          │
│ Pin 4 (ROW3)─┼────────→│ GPIO 5           │
│ Pin 5 (COL0)─┼────────→│ GPIO 13          │
│ Pin 6 (COL1)─┼────────→│ GPIO 14          │
│ Pin 7 (COL2)─┼────────→│ GPIO 25          │
│ Pin 8 (COL3)─┼────────→│ GPIO 26          │
└──────────────┘         └──────────────────┘

Emotion Index Reference

Key	Emotion index	Emotion
1	0	angry
2	1	disgust
3	2	fear
4	3	happy
5	4	neutral
6	5	sad
7	6	surprise
8	7	wink

---

🎯 Challenge Posed

"There next step is to add actions to the pending keys - Felipe"

The Vambrace now controls movement, arms, and expression. The next challenge is implementing default actions with the other keys to add more options to interact with the robot.

---

⏭️ Next Episode

➡️ Episode 8: The Actions

---

Characters: Felipe, David, Danna, ORION | Date: 2026-06-20

⬅️ Previous: The Potentiometers | 🏠 Home | Next ➡️: The Actions