ChatQLv2: An AWS AppSync Chat Starter App written in React
- re:Invent 2018 Session/Slides
- Getting Started
- Building, Deploying and Publishing with the Amplify CLI
- Back End Setup, Back End and Front End Building, Deploying and Publishing with the Amplify Console
- Clean Up
This is a Starter React Progressive Web Application (PWA) that uses AWS AppSync to implement offline and real-time capabilities in a chat application with AI/ML features such as image recognition, text-to-speech, language translation, sentiment analysis as well as conversational chatbots developed as part of the re:Invent session Bridging the Gap Between Real Time/Offline and AI/ML Capabilities in Modern Serverless Apps. In the chat app, users can search for users and messages, have conversations with other users, upload images and exchange messages. The application demonstrates GraphQL Mutations, Queries and Subscriptions with AWS AppSync integrating with other AWS Services:
- Amazon Cognito for user management as well as AuthN/Z
- Amazon DynamoDB with 4x NoSQL Data Sources (Users, Messages, Conversations, ConvoLink)
- Amazon Elasticsearch Data Source for full text search on messages and users
- AWS Lambda as a Serverless Data Source connecting to AI Services
- Amazon Comprehend for sentiment and entity analysis as well as language detection
- Amazon Rekognition for object, scene and celebrity detection on images
- Amazon Lex for conversational chatbots
- Amazon Polly for text-to-speech on messages
- Amazon Translate for language translation
- Amazon S3 for Media Storage
You can use this for learning purposes or adapt either the application or the GraphQL Schema to meet your needs.
- AWS Account with appropriate permissions to create the related resources
- NodeJS with NPM
- AWS CLI with output configured as JSON
(pip install awscli --upgrade --user)
- AWS Amplify CLI configured for a region where AWS AppSync and all other services in use are available
(npm install -g @aws-amplify/cli)
- AWS SAM CLI
(pip install --user aws-sam-cli)
- Create React App
(npm install -g create-react-app)
- Install JQ
- If using Windows, you'll need the Windows Subsystem for Linux (WSL)
Back End Setup
Note: This solution uses Amazon Lex. The service is only supported in us-east-1, us-west-2 and eu-west-1. We recommending launching this entire solution in one of these regions.
First, clone this repository and navigate to the created folder:
git clone https://github.com/aws-samples/aws-appsync-chat-starter-react.git cd aws-appsync-chat-starter-react
Install the required modules:
Set the region we are deploying resources to:
export AWS_REGION=$(jq -r '.providers.awscloudformation.Region' amplify/#current-cloud-backend/amplify-meta.json) echo $AWS_REGION
Make sure ALL services are supported in this region or else you'll get errors in the next steps.
Add an Amazon Cognito User Pool auth resource. Use the default configuration.
amplify add auth
Add an AppSync GraphQL API with Amazon Cognito User Pool for the API Authentication. Follow the default options. When prompted with "Do you have an annotated GraphQL schema?", select "YES" and provide the schema file path
amplify add api
Add S3 Private Storage for Content to the project with the default options. Select private read/write access for Auth users only:
amplify add storage
Now it's time to provision your cloud resources based on the local setup and configured features. When asked to generate code, answer "NO" as it would overwrite the current custom files in the
Wait for the provisioning to complete. Once done, a
src/aws-exports.jsfile with the resources information is created.
At this point you have an usable serverless chat application with no AI features. The next steps are only needed to deploy and configure the integration with services that provide image recognition, text-to-speech, language translation, sentiment analysis as well as conversational chatbots. From here you can skip to step 13 if there's no interest to setup the AI integration.
Look up the S3 bucket name created for user storage:
export USER_FILES_BUCKET=$(sed -n 's/.*"aws_user_files_s3_bucket": "\(.*\)".*/\1/p' src/aws-exports.js) echo $USER_FILES_BUCKET
Retrieve the API ID of your AppSync GraphQL endpoint
export GRAPHQL_API_ID=$(jq -r '.api[(.api | keys)].output.GraphQLAPIIdOutput' ./amplify/#current-cloud-backend/amplify-meta.json) echo $GRAPHQL_API_ID
Retrieve the project's deployment bucket and stackname . It will be used for packaging and deployment with SAM
export DEPLOYMENT_BUCKET_NAME=$(jq -r '.providers.awscloudformation.DeploymentBucketName' ./amplify/#current-cloud-backend/amplify-meta.json) export STACK_NAME=$(jq -r '.providers.awscloudformation.StackName' ./amplify/#current-cloud-backend/amplify-meta.json) echo $DEPLOYMENT_BUCKET_NAME echo $STACK_NAME
Now we need to deploy 3 Lambda functions (one for AppSync and two for Lex) and configure the AppSync Resolvers to use Lambda accordingly. First, we install the npm dependencies for each lambda function. We then package and deploy the changes with SAM.
cd ./backend/chuckbot-lambda; npm install; cd ../.. cd ./backend/moviebot-lambda; npm install; cd ../.. sam package --template-file ./backend/deploy.yaml --s3-bucket $DEPLOYMENT_BUCKET_NAME --output-template-file packaged.yaml export STACK_NAME_AIML="$STACK_NAME-extra-aiml" sam deploy --template-file ./packaged.yaml --stack-name $STACK_NAME_AIML --capabilities CAPABILITY_IAM --parameter-overrides appSyncAPI=$GRAPHQL_API_ID s3Bucket=$USER_FILES_BUCKET --region $AWS_REGION
Wait for the stack to finish deploying then retrieve the functions' ARN
export CHUCKBOT_FUNCTION_ARN=$(aws cloudformation describe-stacks --stack-name $STACK_NAME_AIML --query "Stacks.Outputs" --region $AWS_REGION --output json | jq -r '. | select(.OutputKey == "ChuckBotFunction") | .OutputValue') export MOVIEBOT_FUNCTION_ARN=$(aws cloudformation describe-stacks --stack-name $STACK_NAME_AIML --query "Stacks.Outputs" --region $AWS_REGION --output json | jq -r '. | select(.OutputKey == "MovieBotFunction") | .OutputValue') echo $CHUCKBOT_FUNCTION_ARN echo $MOVIEBOT_FUNCTION_ARN
Let's set up Lex. We will create 2 chatbots: ChuckBot and MovieBot. Execute the following commands to add permissions so Lex can invoke the chatbot related functions:
aws lambda add-permission --statement-id Lex --function-name $CHUCKBOT_FUNCTION_ARN --action lambda:\* --principal lex.amazonaws.com --region $AWS_REGION aws lambda add-permission --statement-id Lex --function-name $MOVIEBOT_FUNCTION_ARN --action lambda:\* --principal lex.amazonaws.com --region $AWS_REGION
Update the bots intents with the Lambda ARN:
jq '.fulfillmentActivity.codeHook.uri = $arn' --arg arn $CHUCKBOT_FUNCTION_ARN backend/ChuckBot/intent.json -M > tmp.txt ; cp tmp.txt backend/ChuckBot/intent.json; rm tmp.txt jq '.fulfillmentActivity.codeHook.uri = $arn' --arg arn $MOVIEBOT_FUNCTION_ARN backend/MovieBot/intent.json -M > tmp.txt ; cp tmp.txt backend/MovieBot/intent.json; rm tmp.txt
And, deploy the slot types, intents and bots:
aws lex-models put-slot-type --cli-input-json file://backend/ChuckBot/slot-type.json --region $AWS_REGION aws lex-models put-intent --cli-input-json file://backend/ChuckBot/intent.json --region $AWS_REGION aws lex-models put-bot --cli-input-json file://backend/ChuckBot/bot.json --region $AWS_REGION aws lex-models put-slot-type --cli-input-json file://backend/MovieBot/slot-type.json --region $AWS_REGION aws lex-models put-intent --cli-input-json file://backend/MovieBot/intent.json --region $AWS_REGION aws lex-models put-bot --cli-input-json file://backend/MovieBot/bot.json --region $AWS_REGION
Finally, execute the following command to install your project package dependencies and run the application locally:
Access your ChatQLv2 app at http://localhost:3000. Sign up at least 2 different users, authenticate with each user to get them registered in the backend Users table, then search for new users to start a conversation and test real-time/offline messaging as well as other features using different devices or browsers.
Interacting with Chatbots
In order to initiate or respond to a chatbot conversation, you need to start the message with either
@moviebotto trigger or respond to the specific bot, for example:
- @chuckbot Give me a Chuck Norris fact
- @moviebot Tell me about a movie
Each subsequent response needs to start with the bot handle (@chuckbot or @moviebot) so the app can detect the message is directed to Lex and not to the other user in the same conversation. Both users will be able to view Lex chatbot responses in real-time powered by GraphQL subscriptions.
Alternatively you can start a chatbot conversation from the message drop-down menu:
- Just selecting
ChuckBotwill display options for further interaction
- Send a message with a nothing but a movie name and selecting
MovieBotsubsequently will retrieve the details about the movie
- Just selecting
Interacting with other AWS AI Services
- Click or select uploaded images to trigger Amazon Rekognition object, scene and celebrity detection.
- From the drop-down menu, select LISTEN -> TEXT TO SPEECH to trigger Amazon Polly and listen to messages in different voices based on the message automatically detected source language (supported languages: English, Mandarin, Portuguese, French and Spanish).
- To perform entity and sentiment analysis on messages via Amazon Comprehend, select ANALYZE -> SENTIMENT from the drop-down menu.
- To translate the message select the desired language under TRANSLATE in the drop-down menu (supported languages: English, Mandarin, Portuguese, French and Spanish). In the translation pane, click on the microphone icon to listen to the translated message.
Building, Deploying and Publishing with the Amplify CLI
amplify add hostingfrom the project's root folder and follow the prompts to create an S3 bucket (DEV) and/or a CloudFront distribution (PROD).
Build, deploy, upload and publish the application with a single command:
If you are deploying a CloudFront distribution, be mindful it needs to be replicated across all points of presence globally and it might take up to 15 minutes to do so.
Access your public ChatQL application using the S3 Website Endpoint URL or the CloudFront URL returned by the
amplify publishcommand. Share the link with friends, sign up some users, and start creating conversations, uploading images, translating, executing text-to-speech in different languages, performing sentiment analysis and exchanging messages. Be mindful PWAs require SSL, in order to test PWA functionality access the CloudFront URL (HTTPS) from a mobile device and add the site to the mobile home screen.
Back End Setup, Back End and Front End Building, Deploying and Publishing with the Amplify Console
(More info here)
- Fork this repository into your own GitHub account and clone it
- Repeat Steps 3 to 6 from the Back End Setup in the previous section. Do not perform step 7 (
- Commit the changes to your forked repository. A new folder
amplifywill be commited with the project details.
- Connect your repository to the Amplify Console as per the instructions here, making sure the name of the branch in your repository matches the name of the environment configured on
amplify init(i.e. master). When prompted with "We detected a backend created with the Amplify Framework. Would you like Amplify Console to deploy these resources with your frontend?", select "YES" and provide or create an IAM role with appropriate permissions to build the backend resources
- Wait for the build, deployment and verification steps
At this point you have an usable serverless chat application with no AI features. The next steps are only needed to deploy and configure the integration with services that provide image recognition, text-to-speech, language translation, sentiment analysis as well as conversational chatbots. From here you can skip to step 8 if there's no interest to setup the AI integration.
- Now perform steps 7 to 12 from the Back End Setup
- Access your app from the hosted site generated by the Amplify Console(https://master.xxxxxxxx.amplifyapp.com)
To clean up the project, you can simply delete the bots, delete the stack created by the SAM CLI:
aws lex-models delete-bot --name `jq -r .name backend/ChuckBot/bot.json` --region $AWS_REGION aws lex-models delete-bot --name `jq -r .name backend/MovieBot/bot.json` --region $AWS_REGION aws lex-models delete-intent --name `jq -r .name backend/ChuckBot/intent.json` --region $AWS_REGION aws lex-models delete-intent --name `jq -r .name backend/MovieBot/intent.json` --region $AWS_REGION aws lex-models delete-slot-type --name `jq -r .name backend/ChuckBot/slot-type.json` --region $AWS_REGION aws lex-models delete-slot-type --name `jq -r .name backend/MovieBot/slot-type.json` --region $AWS_REGION aws cloudformation delete-stack --stack-name $STACK_NAME_AIML --region $AWS_REGION
to delete the resources created by the Amplify CLI.