This is the sample code repository for the blog series "ESP32 + MCP over MQTT: Building Emotional Companion Agents from 0 to 1". This series introduces how to use MCP over MQTT, ESP32 hardware (or similar hardware), and various peripherals, LLMs, VLMs, ASR (Automatic Speech Recognition), and TTS (Text-to-Speech) technologies to build an intelligent agent end-to-end. It enables devices to perceive the world, hear, see, and speak. It can understand human natural language and even inject emotions and personality, with broad applications in emotional companion toys, embodied intelligence, and various other scenarios.
- Part 1: ESP32 + MCP over MQTT: Building Emotional Companion Agents from 0 to 1 | Code
- Part 2: ESP32 + MCP over MQTT: Hardware Device Capability Encapsulation | Code
- Part 3: ESP32 + MCP over MQTT: Controlling Smart Hardware Devices through Large Models | Code
- Part 4: ESP32 + MCP over MQTT: Implementing Voice Interaction for Smart Devices | Code
- Part 5: ESP32 + MCP over MQTT: Building Personalized Emotional Agents Based on Large Models | Code
- Part 6: ESP32 + MCP over MQTT: Giving Agents "Eyes": Image Capture + Multimodal Understanding | Code
- Hardware
- ESP32: Runs MCP Server Tools developed based on ESP32 MCP over MQTT, encapsulating hardware capabilities
- Peripherals: Microphone, speaker, camera, etc.
- Connection and Transport Layer
- MQTT Broker (EMQX) + MCP service discovery, management, and access control
- Multimedia Stream Server for processing audio stream data
- App + AI Service Layer
- ASR (Automatic Speech Recognition), TTS (Text-to-Speech), large model inference, multimodal large model photo recognition
- Emotion, personality support, and memory system
- ESP32 MCP over MQTT SDK
- MQTT Protocol Related Materials: Developer Guide | EMQX Documentation
- EMQX Serverless Free Registration - Secure, Scalable Serverless MQTT Messaging Service
- MCP over MQTT Python SDK Code and Examples
- Alibaba Model Service Lingji DashScope
- Large Model Service Platform Bailian - Enterprise Large Model Development Platform - Bailian AI Application Building - Alibaba Cloud
- MQTT Client Tools
- ESP32 Official Website