EDIVA – Emotion Driven Intelligent Voice Assistant

EDIVA (Emotion Driven Intelligent Voice Assistant) is a Python-based desktop voice assistant that runs locally on Windows. It listens to your voice, understands your intent, detects your emotional tone, performs safe desktop actions (like opening apps or writing notes), and replies to you with an empathetic voice response.[file:1]

Built as a first‑year B.Tech CSE project to explore AI, human–computer interaction, and safe automation on personal computers.[file:1]

---

✨ Key Features

• 🎙️ Hands‑free voice control of your Windows desktop (open apps, take notes, basic automation).[file:1]
• 🧠 Intent understanding using a lightweight NLU layer with rule‑based parsing and entities.[file:1]
• 💗 Emotion detection (very negative, negative, neutral, positive, very positive) from the user’s speech using VADER sentiment analysis.[file:1]
• 🗣️ Emotion‑aware TTS – reply style, speaking rate, and volume adapt to the detected emotion (calm & slow when sad, bright & fast when happy).[file:1]
• 🛟 Safety & crisis handling – checks for risky or self‑harm phrases and uses safer, supportive responses, with support for local crisis contacts.[file:1]
• 🖥️ Whitelisted device control – only allows predefined safe actions like opening specific applications and writing/saving notes.[file:1]
• 🧩 Modular architecture – each function (audio, STT, NLU, emotion, persona, safety, device control, TTS, storage) lives in its own Python module for easy extension.[file:1]

---

🧱 Project Architecture

High‑level flow:[file:1]

1. User speaks into the microphone.
2. audio_io captures and preprocesses audio.
3. stt converts speech to text.
4. nlu detects intent + entities and emotion.
5. persona builds a conversational reply.
6. safety inspects the request and reply for unsafe content.
7. device_control executes a whitelisted action on the PC (open app, write note, etc.).
8. tts speaks the final emotionally‑aware response back to the user.[file:1]

---

📁 Folder Structure

ediva-assistant/
├── assistant_core.py       # Main orchestrator – runs the EDIVA loop
├── utils.py                # Common helper utilities
├── modules/
│   ├── __init__.py
│   ├── audio_io.py         # Audio capture & preprocessing
│   ├── stt.py              # Speech-to-text interface
│   ├── nlu.py              # Intent, entities & emotion detection
│   ├── persona.py          # Dialogue manager & persona engine
│   ├── safety.py           # Safety & crisis-handling logic
│   ├── device_control.py   # Whitelisted desktop actions
│   ├── tts.py              # Text-to-speech wrapper
│   └── storage.py          # Logging & simple memory
├── resources/
│   └── crisis_contacts.json  # Local crisis contacts & helplines
├── tests/
│   └── ...                  # (optional) unit tests for modules
├── requirements.txt
└── README.md

This matches the modular voice‑assistant architecture described in the project report.[file:1]

---

🛠️ Tech Stack

• Language: Python 3.x
• Core libraries:
  • SpeechRecognition – speech‑to‑text interface
  • pyttsx3 – offline‑friendly text‑to‑speech
  • vaderSentiment – sentiment analysis for emotion detection
  • PyAutoGUI – GUI automation (opening apps, typing notes, saving files)
  • numpy, scipy, pydub, librosa – audio utilities (optional)[file:1]
• Platform: Windows desktop (tested on Windows 11)
• Tools: VS Code, CMD / PowerShell, Python virtual environment, Git, GitHub.[file:1]

---

🚀 Getting Started

1. Clone the repository

git clone https://github.com/<your-username>/ediva-assistant.git
cd ediva-assistant

2. Create and activate a virtual environment

python -m venv .venv
.\.venv\Scripts\activate    # on Windows

3. Install dependencies

pip install --upgrade pip
pip install -r requirements.txt

Note: Some STT or audio backends may require additional system dependencies (e.g., microphone drivers, optional offline models). Adjust according to your environment.[file:1]

4. Configure application paths & crisis contacts

• In modules/device_control.py, update the whitelisted app paths, e.g. your WhatsApp, Notepad, VS Code, etc. paths so “open WhatsApp” actually finds the correct .exe on your system.[file:1]
• In resources/crisis_contacts.json, add your local emergency numbers or helplines if you want to enable crisis‑support messaging.[file:1]

5. Run EDIVA

python assistant_core.py

Speak commands like:

• “Open WhatsApp”
• “Take a note I have lab at 2 pm, save it as DBMS_lab”
• “Stop now”

EDIVA will listen, detect emotion, execute the command, and reply with voice.

---

🧪 Example Use Cases

• Productivity: open frequently used apps hands‑free, create quick notes, and save them without touching the keyboard.
• Companion: express your mood; EDIVA detects emotional tone and replies in a more supportive way.
• Research / learning: a base project to explore speech interfaces, basic NLU, sentiment analysis, and safe agent design.[file:1]

---

⚠️ Safety & Ethics

EDIVA is a student research project, not a medical or crisis‑response tool.[file:1]

• The safety module only performs simple keyword‑based checks and uses static crisis messages / contacts configured locally.[file:1]
• It must not be relied on for real mental‑health emergencies or life‑critical decisions.

Always advise users to contact professional help lines or emergency services for real crises.

---

📈 Future Improvements

Planned or possible extensions:[file:1]

• Replace STT with a stronger offline engine (e.g., VOSK) and add multilingual support.
• Upgrade to neural, human‑like TTS voices.
• Integrate a local or API‑based LLM for richer, open‑domain conversation.
• Add a simple GUI dashboard for configuring EDIVA (app whitelist, crisis contacts, logs, persona).
• Implement more robust logging, analytics, and test coverage.

---

📚 Project Report

This repository is accompanied by a detailed PDF report describing the architecture, design choices, and evaluation of EDIVA as an Emotion Driven Intelligent Voice Assistant.[file:1]

---

🙋‍♂️ Author

Sachin Tiwari
SY B.Tech CSE-AIML

Feel free to open issues or submit pull requests if you want to extend or improve EDIVA.