🧠 SenseLink

Hands-Free Cursor Control for Adobe Express — Powered by Face Expressions & AI

Features • How It Works • Gestures • Installation • Technologies • Accessibility

---

🌟 What is SenseLink?

SenseLink is a revolutionary hands-free accessibility add-on for Adobe Express that empowers users to control their mouse cursor using only facial expressions. Designed for individuals with motor impairments, RSI, or anyone seeking an alternative input method, SenseLink transforms your face into a fully-functional mouse controller.

🎯 No hardware required — just your webcam and your face!

---

✨ Features

🖱️ Hands-Free Mouse Control

Feature	Description
Head Movement	Move your head left/right/up/down to control cursor position
Mouth Open	Open your mouth briefly to perform a left click
Both Eyes Blink	Blink both eyes simultaneously for a right click
Hold Mouth Open	Keep your mouth open to drag and select (press & hold)
Calibration	One-click calibration to center your neutral position

🎙️ Voice-Powered Features

• Speech-to-Text — Dictate text directly onto your canvas
• Voice Commands — Control the add-on using voice ("Add text", "Search images")

🎨 AI-Powered Design Intelligence

Feature	Powered By
Design Advisor	Hugging Face Qwen2.5-VL-72B — Analyzes your canvas and provides professional UX feedback
Smart Image Suggestions	AI analyzes your design theme and suggests matching stock photos from Unsplash
Auto-Improve Design	One-click AI-generated enhancements applied to your canvas

🖼️ Integrated Image Search

• Search Unsplash directly from the add-on
• Click any image to instantly add it to your canvas
• AI-powered suggestions based on your design's mood and colors

---

🎮 Gesture Controls

Gesture	Action	Description
🔄	Move Head	Move your head to control cursor position
👄	Open Mouth	Quick mouth open performs a Left Click
😮	Keep Mouth Open	Hold mouth open to Drag & Select
😑	Blink Both Eyes	Blink both eyes together for Right Click

How It Works

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   FACE MESH     │───▶│  GESTURE        │───▶│  MOUSE          │
│   DETECTION     │    │  RECOGNITION    │    │  ACTION         │
└─────────────────┘    └─────────────────┘    └─────────────────┘
        │                      │                      │
   MediaPipe             Mouth/Eye              RobotJS
   Face Mesh               Analysis            Cross-Platform

Gesture Details

Action	How To Perform	Cooldown
Move Cursor	Move your head left/right/up/down	Real-time
Left Click	Open your mouth briefly	500ms
Drag & Select	Keep your mouth open (hold)	-
Right Click	Blink both eyes simultaneously	600ms

---

🛠️ Installation

Prerequisites

• Node.js 18+
• npm or yarn
• Modern browser with webcam support (Chrome/Edge recommended)
• Adobe Express account with Developer Mode enabled

Quick Start

# 1. Clone the repository
git clone https://github.com/yourusername/senselink.git
cd senselink

# 2. Install frontend dependencies
npm install

# 3. Install backend dependencies
cd facecontrol-backend
npm install
cd ..

# 4. Configure environment variables
# Create facecontrol-backend/.env file:
HF_API_KEY=your_huggingface_api_key
UNSPLASH_ACCESS_KEY=your_unsplash_access_key

Running the Add-on

Terminal 1 — Backend Server:

cd facecontrol-backend
npm run dev

Terminal 2 — Frontend Dev Server:

npm run start

Launch in Adobe Express:

1. Open Adobe Express
2. Enable Developer Mode in Settings
3. Load the add-on from https://localhost:5241
4. Grant camera and microphone permissions
5. Click Start and Calibrate to begin!

---

🏗️ Project Structure

senselink/
├── src/                          # 🎨 Frontend (Adobe Express Add-on)
│   ├── index.html               # Main UI with premium glassmorphism design
│   ├── index.js                 # Face tracking, gesture detection, Adobe SDK
│   ├── styles.css               # 2200+ lines of beautiful CSS
│   ├── code.js                  # Document Sandbox for canvas manipulation
│   └── manifest.json            # Add-on configuration
│
├── facecontrol-backend/          # ⚙️ Backend (Node.js WebSocket Server)
│   ├── server.js                # WebSocket server, mouse control, AI APIs
│   ├── .env                     # API keys (HF, Unsplash)
│   └── package.json
│
└── dist/                         # 📦 Built add-on files

---

🔧 Technologies

Core Stack

Technology	Purpose	Logo
Adobe Express SDK	Add-on Framework & Canvas API
MediaPipe Face Mesh	468 Facial Landmark Detection
Hugging Face AI	Qwen2.5-VL-72B Vision Model
Unsplash API	HD Stock Photo Search
RobotJS	Cross-Platform Mouse Control

Tech Stack Breakdown

Layer	Technology	Purpose
Frontend	HTML5, CSS3, JavaScript	Premium UI with glassmorphism design
Face Detection	MediaPipe Face Mesh	Real-time 468-point facial landmark tracking
Smoothing	Kalman Filter + Bezier	Ultra-smooth cursor movement at 125fps
Communication	WebSocket	Ultra-low latency (~8ms) client-server connection
Mouse Control	@jitsi/robotjs	Native mouse/keyboard control (Win/Mac/Linux)
AI Vision	Hugging Face Qwen2.5-VL	Design analysis and improvement suggestions
Image Search	Unsplash API	Professional stock photo integration
Voice Input	Web Speech API	Browser-native speech recognition

---

⚙️ Configuration

Sensitivity Settings

Setting	Range	Default	Description
Sensitivity	0.5 - 3.0	2.0	Controls cursor movement range
Smoothness	0.1 - 0.9	0.4	Controls movement fluidity
Show Tracking	On/Off	On	Display face mesh overlay

Environment Variables

Create facecontrol-backend/.env:

# Hugging Face API Key (for AI features)
HF_API_KEY=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

# Unsplash API Key (for image search)
UNSPLASH_ACCESS_KEY=your_unsplash_access_key_here

---

♿ Accessibility Impact

SenseLink was designed with accessibility as the core mission:

Impact	Benefit
🧑‍🦽 Motor Impairment Support	Users with limited hand mobility can fully navigate Adobe Express
🖐️ RSI Prevention	Reduce repetitive strain by alternating input methods
🌍 No Special Hardware	Works with any standard webcam — no eye trackers required
🔒 Privacy-First	All facial processing runs locally — no video data leaves your machine
📋 WCAG 2.1 AA Compliant	Meets accessibility guidelines for keyboard-less navigation

---

📊 Performance

Metric	Value
Latency	~8ms (WebSocket)
Frame Rate	125 FPS mouse updates
Face Detection	30 FPS via MediaPipe
Memory Usage	< 150MB RAM
CPU Usage	~15% average

---

🚀 Development

Build Commands

# Development server
npm run start

# Production build
npm run build

# Package for submission
npm run package

# Clean build artifacts
npm run clean

Backend Commands

cd facecontrol-backend

# Development with hot-reload
npm run dev

# Production
npm start

---

🛣️ Roadmap

• 🎯 Eye Gaze Tracking — Direct eye-to-cursor mapping
• 📱 Mobile Support — Touch-based calibration for tablets
• 🌐 Multi-language — UI localization
• 🔊 Audio Feedback — Sound cues for actions
• ⌨️ Virtual Keyboard — On-screen keyboard via gestures
• 🎨 Custom Gestures — User-defined gesture mappings

---

🤝 Contributing

Contributions are welcome! Here's how you can help:

1. 🍴 Fork the repository
2. 🔧 Create a feature branch (git checkout -b feature/amazing-feature)
3. 💾 Commit your changes (git commit -m 'Add amazing feature')
4. 📤 Push to the branch (git push origin feature/amazing-feature)
5. 🔃 Open a Pull Request

---

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

Copyright (c) 2026 Piyush Joshi

---

🙏 Acknowledgments

• Adobe — For the Express Add-on SDK
• Google MediaPipe — For the incredible Face Mesh model
• Hugging Face — For accessible AI vision models
• Unsplash — For the beautiful stock photo API
• RobotJS Team — For cross-platform automation

---

Built with ❤️ for inclusive design

---

_{🧠 SenseLink — Making creativity accessible to everyone}