Apex Assistant

A powerful, privacy-focused voice assistant designed to run locally on your machine. Leverage the power of offline speech recognition to control your system, launch applications, and manage tasks without sending your voice data to the cloud.

🚀 Key Features

• 🔒 Privacy-First: Built with Vosk offline speech recognition models. Your voice data stays on your device.
• 💻 System Control: Execute system commands directy via voice.
• 📂 App Launcher: Pre-configured support for launching over 30+ popular Windows applications including:
  • Development: VS Code, IntelliJ IDEA, Eclipse, Cursor, Git Bash, Postman, Docker.
  • Productivity: Microsoft Office (Word, Excel, PowerPoint), Notion, Evernote.
  • Browsers: Google Chrome, Edge.
  • Social: WhatsApp, Instagram, Discord, Teams.
  • Tools: Calculator, Camera, Settings, Task Manager.
• 🌐 Online Query Fallback: Seamlessly switches to online mode for general knowledge queries (via web scraping) when offline commands aren't recognized.
• 🛠 Extensible Skill System: Easily add custom commands and triggers via a simple skills.json configuration file.
• 🎨 Dashboard Widgets: A suite of interactive widgets for real-time monitoring and productivity:
  • Weather: Live temperature and conditions (auto-detected location or default).
  • System Status: Real-time CPU and RAM usage visualization.
  • Storage Monitor: Visual progress bars for all connected disk drives.
  • Live News: A scrolling ticker of the latest technology news headlines.
  • Quick Launch: One-click access to tools like VS Code, Terminal, and Settings.
  • Custom Timer: A built-in countdown timer with pause/reset functionality.

🛠 Tech Stack

• Language: Java 17
• Speech Recognition: Vosk (Offline)
• UI Framework: JavaFX 21
• System Integration: JNA (Java Native Access)
• Data Parsing: Jackson (JSON), Jsoup (HTML/Web Scraping)
• Build Tool: Maven

🏗 Architecture

graph TD
    User([User Voice]) --> Audio[Audio Input]
    Audio --> Wake[Wake Word Detector]
    
    subgraph Core_System [Core System]
        Wake -- "Wake Word Detected" --> Bus{Voice Event Bus}
        Bus -- "Activate" --> Rec[Offline Speech Recognizer]
        Rec -- "Final Result" --> Bus
        Bus --> Parser[Intent Parser]
        Parser --> SkillMgr{Skill Manager}
    end
    
    subgraph Actions [Execution & Feedback]
        SkillMgr -- "Match Found" --> Cmd[Command Executor]
        SkillMgr -- "No Match" --> Online[Online Query Handler]
        Cmd --> Sys[System / Apps]
        Online --> Web[Web Scraping]
        Parser --> TTS[Text-to-Speech]
    end

    TTS --> Speaker([Speaker Output])
    Bus -.-> UI[Dashboard UI]

Loading

📋 Prerequisites

Before you begin, ensure you have the following installed:

• Java Development Kit (JDK) 17 or higher.
• Maven 3.8+.
• Vosk Model: You will need to download a Vosk language model (e.g., vosk-model-small-en-us-0.15) and place it in the models directory.

⚙️ Installation

1. Clone the repository

   git clone https://github.com/subhapreet21/apex-voice-assistant.git
   cd privacy-first-offline-voice-assistant

2. Download Speech Model

   • Download a model from the Vosk Models page.
   • Extract the zip file.
   • Rename the extracted folder to model (or update the path in your config) and place it inside the models/ directory in the project root.

3. Build the Project

   mvn clean install

▶️ Usage

Running the Application

You can run the application directly using Maven:

mvn javafx:run

Or build a JAR file and run it:

mvn clean package
java -jar target/privacy-first-offline-voice-assistant-1.0-SNAPSHOT.jar

Voice Commands

Once the application is running, try saying commands like:

• "Open Notepad"
• "Open Visual Studio Code"
• "What is the capital of France?" (Online mode)
• "Open Calculator"

Adding Custom Skills

To add a new skill or command, edit the skills.json file in the root directory:

{
  "name": "My Custom App",
  "triggers": ["open my app", "start my app"],
  "commands": [
    {
      "platform": "windows",
      "command": "path/to/your/application.exe"
    }
  ]
}

📁 Project Structure

src/
├── main/
│   ├── java/com/assistant/
│   │   ├── MainApp.java           # Application Entry Point
│   │   ├── online/                # Online query handling & scraping
│   │   ├── offline/               # Vosk offline recognition logic
│   │   ├── skills/                # Skill interaction & JSON parsing
│   │   ├── system/                # Command execution (JNA/ProcessBuilder)
│   │   └── ui/                    # JavaFX Controllers & Views
│   └── resources/
│       ├── layout.fxml            # UI Layout
│       ├── styles.css             # UI Styling
│       └── logback.xml            # Logging Configuration
├── skills.json                    # Voice Command Mappings
└── pom.xml                        # Maven Dependencies

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the project
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.