Real-time biometric pipeline for adaptive haptic feedback in VR
User study: 3 haptic patterns × 10 participants, HR time-series analysis
📊 Calmpilot is a user study investigating how haptic feedback patterns affect physiological arousal in VR. The system uses real-time heart rate data from wearables to automatically trigger calming haptics when arousal exceeds thresholds. We compared three patterns—slow vibration, breathing guidance, and butterfly hug—in a controlled VR public speaking experiment (N=10). Breathing-guided haptics achieved the most sustained heart rate reduction (avg. -4 BPM).
📊 Calmpilot은 VR 프로그램을 활용한 노출 치료에서 햅틱 피드백 패턴이 불안 감소에 미치는 영향을 연구한 프로젝트입니다. 웨어러블 디바이스의 실시간 심박수 데이터를 활용해 임계값을 초과하면 자동으로 이완 햅틱 패턴을 트리거합니다. 임상적으로 검증된 세 가지 햅틱 패턴—느린 진동, 심호흡 유도, 나비포옹법—을 설계하고 VR 발표 환경에서 통제된 실험을 통해 효과를 검증했습니다. 분석 결과 심호흡 유도 햅틱이 가장 유의미하고 지속적인 심박수 감소(평균 4 BPM 감소)를 보였습니다.
- 🎯 Overview
- 📚 Research Background
- ⚙️ System Architecture
- 🔬 Experiment Design
- 📊 Data Analysis
- 🏆 Publications & Awards
Project: Calmpilot
Type: Academic Research
Duration: 2023.09 ~ 2024.02
Advisor: Seokhee Jeon (KHU Haptics and Virtual Reality Lab)
| Position | Role | Name | Affiliation |
|---|---|---|---|
| 💻 Research | First Author Research Design & Client Development |
Jaehyun Byun | Kyung Hee Univ. Computer Science |
| 💻 Research | Co-Author Data Analysis |
Jihye Ryu | Kyung Hee Univ. Software Convergence |
| 💻 Research | Co-Author Backend Development |
Hyeon Roh | Kyung Hee Univ. Industrial & Management Engineering |
| 🎓 Advisor | Academic Advisor | Seokhee Jeon | Kyung Hee Univ. Haptics and Virtual Reality Lab |
Social anxiety disorder causes excessive fear in social interactions—4 out of 10 university students experience these symptoms. While exposure therapy with cognitive feedback shows superior outcomes, two barriers exist: (1) CBT specialists require 1+ years of training, and (2) real-time third-party intervention is nearly impossible during social interactions.
사회 공포증은 사회적 상호작용에서 과도한 불안을 유발하며, 대학생 10명 중 4명이 이를 경험합니다. 노출 요법에 인지적 피드백을 병합하면 효과가 우수하지만 두 가지 장벽이 존재합니다: (1) 인지행동치료 전문가 양성에 최소 1년 이상 소요, (2) 사회적 상호작용 중에는 제3자의 실시간 개입이 어려움.
This research designed a biometric-driven automatic haptic feedback pipeline (Galaxy Watch → WebSocket → Unity → bhaptics) for VR exposure therapy, and compared which haptic pattern most effectively reduces tension through a controlled user study.
본 연구는 VR 노출 치료 중 생체 신호 기반 자동화된 햅틱 피드백 파이프라인(Galaxy Watch → WebSocket → Unity → bhaptics) 을 설계하고, 통제된 사용자 연구를 통해 어떤 햅틱 패턴이 긴장 완화에 가장 효과적인지 비교했습니다.
| Study | Key Finding | Our Pattern |
|---|---|---|
| Azevedo et al. (2017) | Vibration 20% slower than resting HR reduced anxiety | Slow vibration |
| Haynes et al. (2022) | Breathing-guided haptic most effective for tension relief | Breathing guidance |
| Butterfly Hug (EMDR) | Alternating shoulder taps used in trauma therapy | Butterfly hug |
이종 플랫폼 데이터 파이프라인
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ Wearable Layer │ │ Streaming Layer │ │ Client Layer │ │ Output Layer │
│ Galaxy Watch 6 │ │ HypeRate │ │ Unity Engine │ │ bhaptics Suit │
├──────────────────┤ ├──────────────────┤ ├──────────────────┤ ├──────────────────┤
│ • HR Sensor │ │ • WebSocket API │ │ • Data Reception │ │ • 40 Vibration │
│ • 1Hz Sampling │───→│ • Real-time │───→│ • Threshold │───→│ Motors │
│ • BPM Output │ │ Relay │ │ Detection │ │ • Pattern Play │
│ │ │ • Multi-device │ │ • CSV Logging │ │ • Haptic Design │
└──────────────────┘ │ Support │ │ • Trigger Logic │ └──────────────────┘
└──────────────────┘ └──────────────────┘
실시간 심박수 스트리밍
| Component | Implementation |
|---|---|
| Sensor | Galaxy Watch 6 optical HR sensor, 1-second interval sampling 갤럭시 워치 6 광학 심박 센서, 1초 간격 샘플링 |
| Middleware | HypeRate WebSocket relay—unified API for diverse wearables HypeRate WebSocket 중계—다양한 웨어러블 통합 API |
| Reception | Unity HypeRate SDK, real-time BPM variable access Unity HypeRate SDK, 실시간 BPM 변수 접근 |
| Threshold | HR ≥ 115 BPM triggers haptic feedback (pilot test max: 130, normal: 60-100) 심박수 115 이상 시 햅틱 피드백 트리거 (파일럿 테스트 최대: 130, 정상: 60-100) |
타임스탬프 동기화 로깅 시스템
// CSV Logging with StreamWriter
public class BiometricLogger : MonoBehaviour
{
private StreamWriter writer;
private string filepath;
void Start()
{
filepath = $"Data/HR_{participantID}_{DateTime.Now:yyyyMMdd_HHmmss}.csv";
writer = new StreamWriter(filepath);
writer.WriteLine("Timestamp,HeartRate,FeedbackType,ScenePhase");
}
void OnHeartRateReceived(int bpm)
{
string timestamp = DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss.fff");
string feedbackType = currentFeedback.ToString();
string phase = currentScene.ToString();
writer.WriteLine($"{timestamp},{bpm},{feedbackType},{phase}");
writer.Flush();
}
}
임상적으로 검증된 세 가지 패턴
Patterns were designed using bhaptics Designer—a web-based haptic authoring tool for TactSuit's 40 vibration motors.
패턴은 bhaptics Designer를 사용해 설계했습니다—TactSuit의 40개 진동 모터를 위한 웹 기반 햅틱 저작 도구입니다.
| Pattern | Clinical Basis | Implementation |
|---|---|---|
| 🫀 Slow Vibration | Azevedo (2017): Sub-heartbeat rhythm induces physiological entrainment 심박 이하 리듬이 생리적 동조 유도 |
10-second continuous vibration at 20% below resting HR frequency 안정 시 심박수보다 20% 느린 주파수로 10초간 지속 진동 |
| 🌬️ Breathing Guide | Haynes (2022): Most effective for tension relief 긴장 완화에 가장 효과적 |
Abdominal-centered expanding/contracting circular vibration pattern 복부 중앙 기준 확장/수축하는 원형 진동 패턴 |
| 🦋 Butterfly Hug | EMDR-based trauma therapy technique EMDR 기반 외상 치료 기법 |
Alternating single taps on left/right shoulders 양쪽 어깨를 번갈아 단발적으로 두드림 |
The VR public speaking simulation was built in Unity Engine with psychological pressure elements:
VR 발표 시뮬레이션은 Unity Engine으로 심리적 압박 요소를 포함하여 구현했습니다:
| Element | Implementation | Purpose |
|---|---|---|
| Audience NPCs | Animator Controller with Idle/Clapping/Questioning states Idle/Clapping/Questioning 상태를 가진 Animator Controller |
Social pressure simulation 사회적 압박 시뮬레이션 |
| Event Sequences | Timeline + Playable Director for host, warnings, disruptions 사회자, 경고, 돌발 상황을 위한 Timeline + Playable Director |
Controlled stressor delivery 통제된 스트레스 요인 전달 |
| Interactions | Ray Interactor for PPT control, script checking PPT 제어, 대본 확인을 위한 Ray Interactor |
Realistic task engagement 현실적인 과제 몰입 |
| UI Elements | World Space Canvas for recording indicator, timer 녹화 표시, 타이머를 위한 World Space Canvas |
Performance pressure 수행 압박 |
| Scene Transitions | Async Scene Loading with fade effects 페이드 효과가 적용된 비동기 씬 로딩 |
Immersion maintenance 몰입 유지 |
| Variable | Definition |
|---|---|
| Control | Presentation content & interactions (constant) |
| Independent | Haptic type: None / Slow / Breathing / Butterfly |
| Dependent | Heart rate change (BPM Δ) |
Participants: N=10 (5M/5F), ages 20-25, university students
Procedure: Pre-survey → Equipment fitting (VR + Watch + bhaptics) → Waiting room → Presentation → Post-SUS
- 4 trials per participant (within-subjects, counterbalanced)
Raw CSV data (HR + timestamp + feedback type) was cleaned by handling missing values with median imputation, encoding categorical feedback types to integers, and validating 1-second interval uniformity for synchronized analysis.
원시 CSV 데이터(심박수 + 타임스탬프 + 피드백 유형)를 전처리: 결측치는 중앙값 대체, 범주형 피드백 유형은 정수 인코딩, 1초 간격 균일성 검증 후 동기화 분석에 활용.
| Condition | Mean HR (BPM) | Δ from Baseline |
|---|---|---|
| No Feedback | 111 | — |
| With Feedback | 107 | -4 BPM |
| Pattern | Immediacy | Duration | Effectiveness |
|---|---|---|---|
| 🌬️ Breathing Guide | Immediate | Longest | Most Effective |
| 🫀 Slow Vibration | Immediate | Moderate | Effective |
| 🦋 Butterfly Hug | Delayed (2-3 reps) | Short | Least Effective |
- Feedback-triggered group: HR spiked to 120-130 BPM upon waiting room entry
- Breathing guide & slow vibration: immediate HR reduction post-trigger
- Butterfly hug: required 2-3 repetitions before measurable effect
| Metric | Score |
|---|---|
| SUS Score | 78.95 / 100 (Good) |
| Perceived Effectiveness | 4.0 / 5.0 |
Designing Haptic Feedback for Social Phobia Improvement and VRET Environment for Data Analysis
Korea Computer Congress (KCC) 2024
Jaehyun Byun, Jihye Ryu, Seokhee Jeon
| Award | Event | Year |
|---|---|---|
| 🏆 학부생 우수논문상 | Korea Computer Congress (KCC) 2024 | 2024 |
| 🌏 Honored Partner Startup Exhibition | Vietnam Mobile Summit 2024 | 2024 |
- 권석만. 현대 이상심리학. 서울 학지사. 2012
- Azevedo, R.T., et al. The calming effect of a new wearable device during the anticipation of public speech. Sci Rep 7, 2285. 2017
- Haynes AC, et al. A calming hug: Design and validation of a tactile aid to ease anxiety. PLoS One. 2022
- Deusdado & Antunes. VR rehabilitation with bHaptics TactSuit. 2023
- Heimberg, R.G. Cognitive-behavioral therapy for social anxiety disorder. Biological Psychiatry, 51, 101-108. 2002
For questions about this research, please contact:
Jaehyun Byun — bjh1750@khu.ac.kr