Skip to content
This repository has been archived by the owner on Mar 7, 2024. It is now read-only.

Commit

Permalink
add support for aws lex streaming interface
Browse files Browse the repository at this point in the history
  • Loading branch information
davehorton committed Jan 11, 2021
1 parent 9dd97af commit d0f10ff
Show file tree
Hide file tree
Showing 9 changed files with 1,561 additions and 0 deletions.
10 changes: 10 additions & 0 deletions modules/mod_aws_lex/Makefile.am
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
include $(top_srcdir)/build/modmake.rulesam
MODNAME=mod_aws_lex

mod_LTLIBRARIES = mod_aws_lex.la
mod_aws_lex_la_SOURCES = mod_aws_lex.c aws_lex_glue.cpp parser.cpp
mod_aws_lex_la_CFLAGS = $(AM_CFLAGS)
mod_aws_lex_la_CXXFLAGS = $(AM_CXXFLAGS) -std=c++11 -I${switch_srcdir}/libs/aws-sdk-cpp/aws-cpp-sdk-core/include -I${switch_srcdir}/libs/aws-sdk-cpp/aws-cpp-sdk-lex/include -I${switch_srcdir}/libs/aws-sdk-cpp/build/.deps/install/include

mod_aws_lex_la_LIBADD = $(switch_builddir)/libfreeswitch.la
mod_aws_lex_la_LDFLAGS = -avoid-version -module -no-undefined -L${switch_srcdir}/libs/aws-sdk-cpp/build/.deps/install/lib -L${switch_srcdir}/libs/aws-sdk-cpp/build/aws-cpp-sdk-core -L${switch_srcdir}/libs/aws-sdk-cpp/build/aws-cpp-sdk-lex -laws-cpp-sdk-lex -laws-cpp-sdk-core -laws-c-event-stream -laws-checksums -laws-c-common -lpthread -lcurl -lcrypto -lssl -lz
58 changes: 58 additions & 0 deletions modules/mod_aws_lex/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# mod_aws_lex

A Freeswitch module that connects to [AWS Lex](https://docs.aws.amazon.com/lex/) using the streaming API.

Once a Freeswitch channel is connected to a Lex bot, media is streamed to Lex, which returns information describing the "intent" that was detected, along with transcriptions and audio prompts and text to play to the caller. The handling of returned audio by the module is two-fold:
1. If an audio clip was returned, it is *not* immediately played to the caller, but instead is written to a temporary file on the Freeswitch server.
2. Next, a Freeswitch custom event is sent to the application containing the details of the dialogflow response as well as the path to the audio file.

This allows the application whether to decide to play the returned audio clip (via the mod_dptools 'play' command), or to use a text-to-speech service to generate audio using the returned prompt text.

## API

### Commands
The freeswitch module exposes the following API commands:

```
aws_lex_start <uuid> bot alias region
```
Attaches media bug to channel and performs streaming recognize request.
- `uuid` - freeswitch channel uuid
- `bot` - name of Lex bot
- `alias` - alias of Lex bot
- `region` - AWS region name (e.g 'us-east-1')

```
aws_lex_dtmf <uuid> dtmf-entry
```
Notify Lex of a dtmf entry

```
aws_lex_play_done <uuid>
```
Notify Lex that an audio prompt has completed playing. The application needs to call this if barge-in is enabled.
```
aws_lex_stop <uuid>
```
Stop dialogflow on the channel.

### Channel variables
* `ACCESS_KEY_ID` - AWS access key id to use to authenticate; if not provided an environment variable of the same name is used if provided
* `SECRET_ACCESS_KEY` - AWS secret access key to use to authenticate; if not provided an environment variable of the same name is used if provided
* `x-amz-lex:barge-in-enabled` - enables speech barge-in, if set to true or 1
* `x-amz-lex:start-silence-threshold-ms` - no-input timeout in milliseconds (Lex defaults to 4000 if not provided)

### Events
* `lex::intent` - an intent has been detected.
* `lex::transcription` - a transcription has been returned
* `lex::text_response` - a text response has been returned; the telephony application can play this using text-to-speech if desired.
* `lex::audio_provided` - an audio response (.mp3 format) has been returned; the telephony application can play this file if TTS is not being used
* `lex::playback_interruption` - the caller has spoken during prompt playback; the telephony application should kill the current audio prompt
* `lex::error` - dialogflow has returned an error
## Usage
When using [drachtio-fsrmf](https://www.npmjs.com/package/drachtio-fsmrf), you can access this API command via the api method on the 'endpoint' object.
```js
ep.api('aws_lex_start', `${ep.uuid} BookTrip Gamma us-east-1`);
```
## Examples
TODO..
Loading

0 comments on commit d0f10ff

Please sign in to comment.