Skip to content

0.4.1

Choose a tag to compare

View all tags
@aleph-ra aleph-ra released this 18 Jun 15:26
· 434 commits to main since this release
0.4.1
a3d2cb5

🚀 EmbodiedAgents 0.4.1 Release: Faster, More Flexible, and Ready for Anything!

We're thrilled to announce the release of EmbodiedAgents 0.4.1! This version marks a significant leap forward, bringing substantial performance improvements, enhanced flexibility for integrating various models (any server that is OpenAI API compatible), and a refined user experience.

✨ What's New?

This release is packed with exciting new features designed to make development smoother and your agents smarter:

  • Project Renamed! We've officially rebranded from "ROS Agents" to EmbodiedAgents to better reflect the package's purpose and capabilities.
  • Blazing Fast Streaming: Experience significantly faster handling of model output! We've added streaming support to LLM, MLLM, STT (Speech-to-Text), and TTS (Text-to-Speech) components, making your agents more responsive than ever.
  • Onboard Object Detection: The Vision component now includes a local classifier model for efficient, onboard object detection.
  • Universal Model Integration: Integrate with any model serving platform that offers an OpenAI-style API using our new generic HTTP client.
  • Planning with Vision-Language Models: We've added support for planning MLLM models, starting with RoboBrain2.0 by BAAI, which allows for task based outputs from MLLM component (e.g. grounding, pointing etc.), opening the door to multimodal planning and perception-driven action.
  • Enhanced Ollama Support: The OllamaClient now boasts embeddings model support, expanding your options for local model inference.
  • Direct ChromaDB Integration: We've added a ChromaDB client for direct interaction with a ChromaDB server, supporting embeddings models served via Ollama or sentence-transformers. No extra dependencies needed!
  • Simplified & Streamlined: We've reduced external Python package dependencies and simplified models and clients for a cleaner, more efficient codebase.
  • Python 3.8 Compatibility Fixes: We've addressed backward compatibility issues to ensure smoother installations for Python 3.8 users.
  • Dynamic System Prompts: The LLM/MLLM components now support set_system_prompt, allowing for dynamic system prompt configuration without modifying model settings.

🛠️ Under the Hood Improvements

Beyond the big features, we've also implemented a host of improvements and fixes:

  • Improved connection error messages and updated installation instructions for a smoother onboarding experience.
  • Added a Debian packaging workflow for easier deployment.
  • Introduced Ollama-specific inference options for better control over your Ollama models.
  • Integrated the MeloTTS model and enhanced the Text-to-Speech component with a "say text" method for event-driven invocation.
  • Implemented streaming playback for input in the Speech-to-Text component.
  • Refactored components to remove sounddevice as a dependency for Text-to-Speech, simplifying setup.
  • Added warnings for local models when GPU is set but runtime isn't available.
  • Enhanced the Speech-to-Text component with a hypothesis buffer for confirmed transcripts and asynchronous receiving for streaming websockets.
  • Improved asynchronous publishing of responses in the LLM component when streaming.
  • Ensured smoother termination by marking child threads as daemons.
  • Added a break_character to the LLM component config for chunking streaming output.
  • Introduced support for RGBD messages (in Realsense style).

For a comprehensive list of all changes, please refer to the [full changelog](0.3.3...0.4.0).

📖 Dive Deeper

Ready to explore these new features? Check out our updated documentation for detailed information, examples, and installation guide.

Assets 8
Loading