📌 SmartHeadphones

This project aims to assist autistic people in office environments, where coworkers and other sources of audible sources of distraction are present by automatically controlling ANC of headphones via wake words and creating adaptive masking noise.

---

📖 Table of Contents

• Overview
• Publications
• Features
• Installation
• Usage
• Project Structure
• Team & Contact
• License

---

🔍 Overview

Detailed description of the project:

• Context: This project was developed in the context of research into assistive technologies for neurodivergent people in shared work environments. Many people with ADHD or autism use Active Noise Cancellation (ANC) and masking sounds to regulate sensory input and maintain focus. While these strategies can significantly improve concentration, they can also unintentionally create barriers to communication in collaborative environments such as offices or labs.
• Goal: The goal of this prototype is to demonstrate how audio masking and environmental awareness can coexist. The application combines masking sound playback with wake-word detection, allowing users to stay focused while still being able to respond when someone intentionally tries to get their attention.
• Target Audience: This prototype is designed for:
  • Neurodivergent users (especially people with ADHD and autism) who use auditory masking for focus
  • Researchers exploring assistive technologies for sensory regulation
  • Developers interested in human-computer interaction, accessibility, and ambient intelligence

Core Concept

The central idea of this project is to balance sensory regulation with social accessibility.

Masking sounds and ANC are effective tools for reducing sensory overload and improving concentration. However, they also make it difficult for coworkers to initiate communication. This prototype addresses that problem by introducing a wake-word–based interaction channel.

Instead of removing masking entirely, the system listens for a predefined wake word (e.g., the user’s name). When the wake word is detected:

1. The masking sound is temporarily reduced or paused.

2. The user becomes aware that someone wants to speak to them.

3. Normal conversation can take place without removing headphones or disabling ANC.

This approach preserves the benefits of sensory isolation while enabling intentional and seamless social interaction in shared environments.

---

📚 Publications

This prototype is based on the following scientific publications:

Main Publication

Customized Wake-Word Integration in ANC-Enabled Headphones: Enhancing Assistive Technology for Noise-Sensitive Employees in Office
Jan Schmalfuß-Schwarz, David Gollasch, Christin Engel, Meinhardt Branig, and Gerhard Weber. (2024) In: Proceedings of Mensch und Computer 2024 (MuC '24). Association for Computing Machinery, New York, NY, USA, 533–537.
DOI

Open Sesame! Use of Headphones at Work Considering Social Acceptance
Schmalfuß-Schwarz, J., Gollasch, D., Engel, C., Branig, M., Weber, G. (2024)
In: Miesenberger, K., Peňáz, P., Kobayashi, M. (eds) Computers Helping People with Special Needs. ICCHP 2024. Lecture Notes in Computer Science, vol 14751. Springer, Cham.
DOI

---

✨ Features

• ✅ ANC Control Integration with headphone noise cancellation features to help users control their sensory environment.
• ✅ Masking sounds Play customizable background masking sounds (e.g., noise, ambience, loops, playlists) to support focus and sensory regulation.
• ✅ Wakeword detection Continuously listens for a configurable wake word that signals someone is trying to get the user’s attention.
• 🔄 Automatic masking volume When the wake word is detected, masking audio is automatically reduced or paused to allow communication.

---

💻 Installation

Prerequisites

• Python 3.13+

Quick Start

# Clone the repository
git clone https://github.com/AutARK-Research-Project/SmartHeadphones-v2
cd SmartHeadphones-v2

# Create virtual environment (recommended)
python -m venv .venv
source .venv/bin/activate  # Linux/Mac
# or: .venv\Scripts\activate  # Windows

# Install dependencies
pip install -r requirements.txt --ignore-requires-python
# --ignore-requires-python is needed because one dependency is arbitrarily stuck below python 3.12

# Run the application
python -m app.main

---

🚀 Usage

Basic Usage

python -m app.main

Integration with HeadphoneANCDriver

To enable controlling of ANC (Automatic Noise Cancelling), for Huawei Headphones and a few Jabra Headphones, use the AutARK HeadphoneANCDriver. Download from GitHub Releases, and (under Windows) place the file anc_controller.exe in /AppData/Local/AutARK/Smart Headphones/backend/.

---

📁 Project Structure

SmartHeadphones/
├── 📁app
|   ├── 📁controller
|   ├── 📁model
|   |   ├── 📁database
|   |   |   ├── 📁entity
|   |   |   ├── 📁repository
|   |   |   └── db_manager.py
|   |   ├── 📁headphone
|   |   ├── 📁microphone
|   |   ├── 📁volume
|   |   └── 📁wakeword
|   ├── 📁resources
|   |   ├── 📁icons
|   |   └── 📁masking_sounds
|   ├── 📁utils
|   ├── 📁view
|   |   ├── 📁add_headphones
|   |   ├── 📁gui_elements
|   |   ├── 📁start_view
|   |   |   ├── 📁start_view_dialogs
|   |   |   ├── 📁start_view_panels
|   |   |   └── start_view.py
|   |   └── main_window.py
|   └── main.py
├── .gitignore
├── LICENSE
├── README.md
└── requirements.txt

---

🛠️ Development

Setting Up the Development Environment

# Install dependencies
pip install -r requirements.txt --ignore-requires-python

Code Style

We use Black for Python formatting:

black app/

Contributing

Contributions are welcome! Please note:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/new-feature)
3. Commit your changes (git commit -m 'Add: New feature')
4. Push the branch (git push origin feature/new-feature)
5. Open a Pull Request

---

🔑 Key Components

Component A

Path: app/model/headphone/headphone_manager.py, app/model/microphone/microphone_manager.py

The HeadphoneManager handles adding headphones, saving them for faster selection in the database, and then connecting the one that is selected. This requires audio device parsing and fingerprinting, to reliably detect whether two headphones are the same. The MicrophoneManager works analogously.

Component B

Path: app/model/wakeword/

This project uses realtime speech to text transcription to detect wakewords, and Silero VAD to detect end of speech. Only one input stream is opened and the output is routed to the one that is currently active. When a wakeword or the end of speech is detected, the masking sound and anc controllers are notified via push-observer-pattern.

---

⚠️ Known Limitations

• Tested on: Windows 11

---

👥 Team & Contact

Developed by:

• Jan Schmalfuß-Schwarz – Main Author
• Richard Fuchs – Main Author

Research Group:
AutARK
Chair of Human-Computer Interaction Faculty of Computer Science Dresden University of Technology

Contact:
📧 autark@tu-dresden.de

---

📄 License

This project is licensed under the MIT License.

---

🙏 Acknowledgments

This project was funded by Bundesministerium für Arbeit und Soziales (BMAS) (FT 1 – 58330).

---

Created as part of the AutARK research project