An SDK for commercial device makers to integrate Alexa directly into connected products.
Clone or download
Permalink
Failed to load latest commit information.
ACL Version 1.9.0 alexa-client-sdk Aug 28, 2018
ADSL Version 1.8 alexa-client-sdk Jun 27, 2018
AFML Version 1.8 alexa-client-sdk Jun 27, 2018
AVSCommon Version 1.9.0 alexa-client-sdk Aug 28, 2018
ApplicationUtilities Version 1.9.0 alexa-client-sdk Aug 28, 2018
BluetoothImplementations Version 1.9.0 alexa-client-sdk Aug 28, 2018
CapabilitiesDelegate Version 1.9.0 alexa-client-sdk Aug 28, 2018
CapabilityAgents Version 1.9.0 alexa-client-sdk Aug 28, 2018
CertifiedSender Version 1.7.0 of the avs-device-sdk Apr 18, 2018
ContextManager Version 1.7.1 alexa-client-sdk May 4, 2018
ESP Version 1.9.0 alexa-client-sdk Aug 28, 2018
Integration Version 1.9.0 alexa-client-sdk Aug 28, 2018
KWD Version 1.9.0 alexa-client-sdk Aug 28, 2018
MediaPlayer Version 1.9.0 alexa-client-sdk Aug 28, 2018
PlaylistParser Version 1.9.0 alexa-client-sdk Aug 28, 2018
RegistrationManager Version 1.7.0 of the avs-device-sdk Apr 18, 2018
SampleApp Version 1.9.0 alexa-client-sdk Aug 28, 2018
Storage Version 1.8.1 alexa-client-sdk Jul 9, 2018
ThirdParty Version 1.8 alexa-client-sdk Jun 27, 2018
build Version 1.9.0 alexa-client-sdk Aug 28, 2018
doc v1.3.0 released 12/08/2017: Dec 9, 2017
tools Version 1.9.0 alexa-client-sdk Aug 28, 2018
.clang-format Version 1.2.0 alexa-client-sdk Oct 31, 2017
.gitignore Version 1.2.0 alexa-client-sdk Oct 31, 2017
CHANGELOG.md Adding a new limitation entry. Sep 8, 2018
CMakeLists.txt Version 1.9.0 alexa-client-sdk Aug 28, 2018
CONTRIBUTING.md CONTRIBUTING.md added to the repository with information on reporting… Oct 19, 2017
LICENSE.txt Version 0.1 - Alexa Communications Library - This component serves as… Feb 10, 2017
NOTICE.txt Version 1.9.0 alexa-client-sdk Aug 28, 2018
README.md Adding a new limitation entry. Sep 8, 2018
issue_template.md Version 1.4.0 alexa-client-sdk Jan 12, 2018
pkg-config.pc.cmake Version 1.0.1 alexa-client-sdk Aug 17, 2017

README.md

What is the Alexa Voice Service (AVS)?

The Alexa Voice Service (AVS) enables developers to integrate Alexa directly into their products, bringing the convenience of voice control to any connected device. AVS provides developers with access to a suite of resources to quickly and easily build Alexa-enabled products, including APIs, hardware development kits, software development kits, and documentation.

Learn more »

Overview of the AVS Device SDK

The AVS Device SDK provides C++-based (11 or later) libraries that leverage the AVS API to create device software for Alexa-enabled products. It is modular and abstracted, providing components for handling discrete functions such as speech capture, audio processing, and communications, with each component exposing the APIs that you can use and customize for your integration. It also includes a sample app, which demonstrates the interactions with AVS.

Get Started

You can set up the SDK on the following platforms:

You can also prototype with a third party development kit:

Or if you prefer, you can start with our SDK API Documentation.

Learn More About The AVS Device SDK

Watch this tutorial to learn about the how this SDK works and the set up process.

Tutorial

SDK Architecture

This diagram illustrates the data flows between components that comprise the AVS Device SDK for C++.

SDK Architecture Diagram

Audio Signal Processor (ASP) - Third-party software that applies signal processing algorithms to both input and output audio channels. The applied algorithms are designed to produce clean audio data and include, but are not limited to acoustic echo cancellation (AEC), beam forming (fixed or adaptive), voice activity detection (VAD), and dynamic range compression (DRC). If a multi-microphone array is present, the ASP constructs and outputs a single audio stream for the array.

Shared Data Stream (SDS) - A single producer, multi-consumer buffer that allows for the transport of any type of data between a single writer and one or more readers. SDS performs two key tasks:

  1. It passes audio data between the audio front end (or Audio Signal Processor), the wake word engine, and the Alexa Communications Library (ACL) before sending to AVS
  2. It passes data attachments sent by AVS to specific capability agents via the ACL

SDS is implemented atop a ring buffer on a product-specific memory segment (or user-specified), which allows it to be used for in-process or interprocess communication. Keep in mind, the writer and reader(s) may be in different threads or processes.

Wake Word Engine (WWE) - Software that spots wake words in an input stream. It is comprised of two binary interfaces. The first handles wake word spotting (or detection), and the second handles specific wake word models (in this case "Alexa"). Depending on your implementation, the WWE may run on the system on a chip (SOC) or dedicated chip, like a digital signal processor (DSP).

Audio Input Processor (AIP) - Handles audio input that is sent to AVS via the ACL. These include on-device microphones, remote microphones, an other audio input sources.

The AIP also includes the logic to switch between different audio input sources. Only one audio input source can be sent to AVS at a given time.

Alexa Communications Library (ACL) - Serves as the main communications channel between a client and AVS. The ACL performs two key functions:

  1. Establishes and maintains long-lived persistent connections with AVS. ACL adheres to the messaging specification detailed in Managing an HTTP/2 Connection with AVS.
  2. Provides message sending and receiving capabilities, which includes support JSON-formatted text, and binary audio content. For additional information, see Structuring an HTTP/2 Request to AVS.

Alexa Directive Sequencer Library (ADSL): Manages the order and sequence of directives from AVS, as detailed in the AVS Interaction Model. This component manages the lifecycle of each directive, and informs the Directive Handler (which may or may not be a Capability Agent) to handle the message.

Activity Focus Manager Library (AFML): Provides centralized management of audiovisual focus for the device. Focus is based on channels, as detailed in the AVS Interaction Model, which are used to govern the prioritization of audiovisual inputs and outputs.

Channels can either be in the foreground or background. At any given time, only one channel can be in the foreground and have focus. If multiple channels are active, you need to respect the following priority order: Dialog > Alerts > Content. When a channel that is in the foreground becomes inactive, the next active channel in the priority order moves into the foreground.

Focus management is not specific to Capability Agents or Directive Handlers, and can be used by non-Alexa related agents as well. This allows all agents using the AFML to have a consistent focus across a device.

Capability Agents: Handle Alexa-driven interactions; specifically directives and events. Each capability agent corresponds to a specific interface exposed by the AVS API. These interfaces include:

  • SpeechRecognizer - The interface for speech capture.
  • SpeechSynthesizer - The interface for Alexa speech output.
  • Alerts - The interface for setting, stopping, and deleting timers and alarms.
  • AudioPlayer - The interface for managing and controlling audio playback.
  • Notifications - The interface for displaying notifications indicators.
  • PlaybackController - The interface for navigating a playback queue via GUI or buttons.
  • Speaker - The interface for volume control, including mute and unmute.
  • System - The interface for communicating product status/state to AVS.
  • TemplateRuntime - The interface for rendering visual metadata.
  • Bluetooth - The interface for managing Bluetooth connections between peer devices and Alexa-enabled products.

Security Best Practices

In addition to adopting the Security Best Practices for Alexa, when building the SDK:

  • Protect configuration parameters, such as those found in the AlexaClientSDKCOnfig.json file, from tampering and inspection.
  • Protect executable files and processes from tampering and inspection.
  • Protect storage of the SDK's persistent states from tampering and inspection.
  • Your C++ implementation of AVS Device SDK interfaces must not retain locks, crash, hang, or throw exceptions.
  • Use exploit mitigation flags and memory randomization techniques when you compile your source code, in order to prevent vulnerabilities from exploiting buffer overflows and memory corruptions.

Important Considerations

  • Review the AVS Terms & Agreements.
  • The earcons associated with the sample project are for prototyping purposes only. For implementation and design guidance for commercial products, please see Designing for AVS and AVS UX Guidelines.
  • Please use the contact information below to-
  • IMPORTANT: The Sensory wake word engine referenced in this document is time-limited: code linked against it will stop working when the library expires. The library included in this repository will, at all times, have an expiration date that is at least 120 days in the future. See Sensory's GitHub page for more information.

Release Notes and Known Issues

Note: Feature enhancements, updates, and resolved issues from previous releases are available to view in CHANGELOG.md.

v1.9.0 released 08/28/2018:

Enhancements

  • Added Android SDK support, which includes new implementations of the MediaPlayer, audio recorder, and logger.
  • Added the InteractionModel interface, which enables Alexa Routines.
  • Optional configuration changes have been introduced. Now a network interface can be specified to connect to the SDK via curl.
  • Build options can be configured to support Android.
  • Added GUI 1.1 support. The PlaybackController has been extended to support new control functionality, and the System interface has been updated to support SoftwareInfo.

Bug Fixes

  • Installation script execution time has been reduced. Now a single branch clone is used, such as the master branch.
  • Issue 846 - Fixed a bug where audio stuttered on slow network connections.
  • Removed the SpeakerManager constructor check for non-zero speakers.
  • Issue 891 - Resolved incorrect offset in the PlaybackFinished event.
  • Issue 727 - Fixed an issue where the sample app behaved erratically upon network disconnection/reconnection.
  • Issue 910 - Fixed a GCC 8+ compilation issue. Note: issues related to -Wclass-memaccess will still trigger warnings, but won't fail compilation.
  • Issue 871 Issue 880 - Fixed compiler warnings.
  • Fixed an issue where PlaybackStutterStarted and PlaybackStutterFinished events were not being sent due to a missing Gstreamer queue element.
  • Fixed a bug where the CapabilitiesDelegate database was not being cleared upon logout.
  • Fixed in issue that caused the following compiler warning “class has virtual functions but non-virtual destructor”.
  • Fixed a bug where BlueZDeviceManager was not properly destroyed.
  • Fixed a bug that occurred when the initializer list was converted to std::unordered_set.
  • Fixed a build error that occurred when building with BUILD_TESTING=Off.

Known Issues

  • If a device is not connected to the Internet at the time that an alarm or reminder is scheduled to sound, the SDK will not play any sound for the alarm or reminder at that scheduled time.
  • The ACL may encounter issues if audio attachments are received but not consumed.
  • SpeechSynthesizerState currently uses GAINING_FOCUS and LOSING_FOCUS as a workaround for handling intermediate state. These states may be removed in a future release.
  • The Alexa app doesn't always indicate when a device is successfully connected via Bluetooth.
  • Connecting a product to streaming media via Bluetooth will sometimes stop media playback within the source application. Resuming playback through the source application or toggling next/previous will correct playback.
  • When a source device is streaming silence via Bluetooth, the Alexa companion app indicates that audio content is streaming.
  • The Bluetooth agent assumes that the Bluetooth adapter is always connected to a power source. Disconnecting from a power source during operation is not yet supported.
  • On some products, interrupted Bluetooth playback may not resume if other content is locally streamed.
  • On Raspberry Pi, when streaming audio via Bluetooth, sometimes the audio stream stutters.
  • These CapabilitiesDelegateTest tests have been temporarily disabled to prevent build errors for the Android build: CapabilitiesDelegateTest.withCapabilitiesHappyCase, CapabilitiesDelegateTest.republish, CapabilitiesDelegateTest.testClearData.
  • make integration is currently not available for Android. In order to run integration tests on Android, you'll need to manually upload the test binary file along with any input file. At that point, the adb can be used to run the integration tests.
  • On Raspberry Pi running Android Things with HDMI output audio, beginning of speech is truncated when Alexa responds to user TTS.
  • When the sample app is restarted and network connection is lost, Alerts don't play.
  • When network connection is lost, lost connection status is not returned via local TTS.