Update to version v5.4.0

Update to version v5.4.0

CHANGELOG.md

@@ -5,6 +5,23 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [5.4.0] - 2023-07-27
+
+__*Note: we recommend that you first deploy these changes in a non-production environment. This is true for all releases, but especially important for minor and major releases.*__
+
+### Added
+
+- New (optional) query disambiguation and text generation features through the use of Large Language Models (LLMs) to enable enhanced conversational chat and response synthesis. Details at [README](docs/LLM_Retrieval_and_generative_question_answering/README.md)
+  - In order to provide this functionality, the solution will provision an inference endpoint hosted on Amazon SageMaker
+  - If enabled, this has cost implications. Please [refer to the IG](https://docs.aws.amazon.com/solutions/latest/qnabot-on-aws/plan-your-deployment.html#cost) to see cost estimates
+- [App Registry integration](https://docs.aws.amazon.com/servicecatalog/latest/arguide/intro-app-registry.html), QnABot will now register an application in System Manager to enable various application management tools
+
+### Updated
+
+- Lambda runtimes updated to NodeJS 18
+- Python runtimes updated to Python 3.10
+- Security patches for npm and pip packages
+
 ## [5.3.5] - 2023-07-12
 
 ### Updated

CONTRIBUTING.md

@@ -1,25 +1,59 @@
-# Contributing to the QnABot
+# Contributing Guidelines
 
-Contributions to the QnABot should be made via GitHub [pull
-requests](https://github.com/aws-solutions/qnabot-on-aws/pulls) and discussed using
-GitHub [issues](https://github.com/aws-solutions/qnabot-on-aws/issues).
+Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
+documentation, we greatly value feedback and contributions from our community.
 
-## Before you start
+Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
+information to effectively respond to your bug report or contribution.
 
-If you would like to make a significant change, it's a good idea to first open
-an issue to discuss it.
+## Reporting Bugs/Feature Requests
 
-## Making the request
+We welcome you to use the GitHub issue tracker to report bugs or suggest features.
 
-Development takes place against the `develop` branch of this repository and pull
-requests should be opened against that branch.
+When filing an issue, please check [existing open](https://github.com/aws-solutions/qnabot-on-aws/issues), or [recently closed](https://github.com/aws-solutions/qnabot-on-aws/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
+reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
 
-## Testing
+- A reproducible test case or series of steps
+- The version of our code being used
+- Any modifications you've made relevant to the bug
+- Anything unusual about your environment or deployment
 
-Any contributions should pass all tests, including those not run by our
-current CI system.
+## Contributing via Pull Requests
 
-You may run all test by running the `make test`.
+Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
 
-For significant changes, we may ask you to sign a [Contributor License
-Agreement](http://en.wikipedia.org/wiki/Contributor_License_Agreement).
+1. You are working against the latest source on the _develop_ branch.
+2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
+3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
+
+To send us a pull request, please:
+
+1. Fork the repository.
+2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
+3. Ensure local tests pass.
+4. Commit to your fork using clear commit messages.
+5. Send us a pull request, answering any default questions in the pull request interface.
+6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
+
+GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
+[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
+
+## Finding contributions to work on
+
+Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws-solutions/qnabot-on-aws/labels/help%20wanted) issues is a great place to start.
+
+## Code of Conduct
+
+This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
+For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
+opensource-codeofconduct@amazon.com with any additional questions or comments.
+
+## Security issue notifications
+
+If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
+
+## Licensing
+
+See the [LICENSE](https://github.com/aws-solutions/qnabot-on-aws/blob/main/LICENSE.txt) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
+
+We may ask you to sign a [Contributor License Agreement (CLA)](https://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

NOTICE.txt

@@ -22,6 +22,7 @@ This software includes third party software subject to the following copyrights:
 @babel/preset-stage-2 under the Massachusetts Institute of Technology (MIT) license
 ajv under the Massachusetts Institute of Technology (MIT) license
 alexa-sdk under the Apache License Version 2.0
+arrow under the Apache License Version 2.0
 async-mutex under the Massachusetts Institute of Technology (MIT) license
 autosize under the Massachusetts Institute of Technology (MIT) license
 aws-lex-web-ui under the Amazon Software License
@@ -37,29 +38,39 @@ babel-preset-env under the Massachusetts Institute of Technology (MIT) license
 babel-preset-es2015 under the Massachusetts Institute of Technology (MIT) license
 babel-preset-es2015-ie the Massachusetts Institute of Technology (MIT) license
 beautifulsoup under Massachusetts Institute of Technology (MIT) License
+beautifulsoup4 under the Massachusetts Institute of Technology (MIT) license
 bluebird under the Massachusetts Institute of Technology (MIT) license
 body-parser under the Massachusetts Institute of Technology (MIT) license
+boto3 under the Apache License Version 2.0
+botocore under the Apache License Version 2.0
 bowser under the Massachusetts Institute of Technology (MIT) license
 canvasapi under Massachusetts Institute of Technology (MIT) License
+cfn-lambda under the Massachusetts Institute of Technology (MIT) license
 cfn-response under the Amazon Software License
 chalk under the Massachusetts Institute of Technology (MIT) license
 chrome-aws-lambda under the Massachusetts Institute of Technology (MIT) license
 clean-deep under the Massachusetts Institute of Technology (MIT) license
 clipboard under the Massachusetts Institute of Technology (MIT) license
 commander under the Massachusetts Institute of Technology (MIT) license
 copy-webpack-plugin under the Massachusetts Institute of Technology (MIT) license
-crypto-js under the Massachusetts Institute of Technology (MIT) license
+crhelper under the Apache License Version 2.0
 css-loader under the Massachusetts Institute of Technology (MIT) license
+defusedxml under the Python Software Foundation License Version 2
 dir-loader under the Massachusetts Institute of Technology (MIT) license
+elasticsearch under the Apache License Version 2.0
 exports-loader under the Massachusetts Institute of Technology (MIT) license
 express under the Massachusetts Institute of Technology (MIT) license
 faker under the Massachusetts Institute of Technology (MIT) license
 file-saver under the Massachusetts Institute of Technology (MIT) license
+filelock under the Unlicense license
 handlebars under the Massachusetts Institute of Technology (MIT) license
 handlebars-loader under the Massachusetts Institute of Technology (MIT) license
 highlight.js under BSD-3-Clause license
 html-webpack-plugin under the Massachusetts Institute of Technology (MIT) license
+http-aws-es under the Massachusetts Institute of Technology (MIT) license
 idle-vue under the Massachusetts Institute of Technology (MIT) license
+intercept-stdout under the Massachusetts Institute of Technology (MIT) license
+jmespath under the Massachusetts Institute of Technology (MIT) license
 js-cache under the Massachusetts Institute of Technology (MIT) license
 jsdom under the Massachusetts Institute of Technology (MIT) license
 jsdom-global under the Massachusetts Institute of Technology (MIT) license
@@ -71,6 +82,8 @@ JSONPath under the Massachusetts Institute of Technology (MIT) license
 jsonschema under the Massachusetts Institute of Technology (MIT) license
 jsonwebtoken under the Massachusetts Institute of Technology (MIT) license
 jszip under the Massachusetts Institute of Technology (MIT) license
+jwks-rsa under the Massachusetts Institute of Technology (MIT) license
+langchain under the Massachusetts Institute of Technology (MIT) license
 lodash under the Massachusetts Institute of Technology (MIT) license
 lodash-webpack-plugin under the Massachusetts Institute of Technology (MIT) license
 marked under the Massachusetts Institute of Technology (MIT) license
@@ -85,20 +98,27 @@ pug under the Massachusetts Institute of Technology (MIT) license
 pug-loader under the Massachusetts Institute of Technology (MIT) license
 pug-plain-loader under the Massachusetts Institute of Technology (MIT) license
 pug-runtime under the Massachusetts Institute of Technology (MIT) license
+py-serializable under the Apache License Version 2.0
+python-dateutil under the Apache License Version 2.0 and BSD License
+pytz under the Massachusetts Institute of Technology (MIT) license
 query-string under the Massachusetts Institute of Technology (MIT) license
 querystring under the Massachusetts Institute of Technology (MIT) license
 querystring-browser under the Massachusetts Institute of Technology (MIT) license
 quick-lru under the Massachusetts Institute of Technology (MIT) license
 range under the Massachusetts Institute of Technology (MIT) license
 raw-loader under the Massachusetts Institute of Technology (MIT) license
 raw-text under the Massachusetts Institute of Technology (MIT) license
+read-excel-file under the Massachusetts Institute of Technology (MIT) license
 recursive-readdir under the Massachusetts Institute of Technology (MIT) license
 require-dir under the Massachusetts Institute of Technology (MIT) license
 roboto-fontface under the Apache License Version 2.0
+s3transfer under the Apache License Version 2.0
 sass under the Massachusetts Institute of Technology (MIT) license
 sass-loader under the Massachusetts Institute of Technology (MIT) license
 simple-encryptor under the Massachusetts Institute of Technology (MIT) license
 slackify-markdown under the Massachusetts Institute of Technology (MIT) license
+soupsieve under the Massachusetts Institute of Technology (MIT) license
+static-eval under the Massachusetts Institute of Technology (MIT) license
 strip-ansi under the Massachusetts Institute of Technology (MIT) license
 style-loader under the Massachusetts Institute of Technology (MIT) license
 stylus under the Massachusetts Institute of Technology (MIT) license
@@ -123,5 +143,4 @@ webpack-bundle-analyzer under the Massachusetts Institute of Technology (MIT) li
 webpack-cli under the Massachusetts Institute of Technology (MIT) license
 webpack-dev-server under the Massachusetts Institute of Technology (MIT) license
 webpack-merge under the Massachusetts Institute of Technology (MIT) license
-webpack-s3-plugin under the Massachusetts Institute of Technology (MIT) license
-xlsx under the Apache License Version 2.0
+webpack-s3-plugin under the Massachusetts Institute of Technology (MIT) license

README.md

@@ -22,19 +22,21 @@ The high-level process flow for the solution components deployed with the AWS Cl
 
 1.	The admin deploys the solution into their AWS account, opens the Content Designer UI or [Amazon Lex](https://aws.amazon.com/lex/) web client, and uses [Amazon Cognito](https://aws.amazon.com/cognito/) to authenticate.
 
-2. After authentication, [Amazon CloudFront](http://aws.amazon.com/cloudfront/) and [Amazon S3](http://aws.amazon.com/s3/) deliver the contents of the Content Designer UI.
+2. After authentication, [Amazon API Gateway](http://aws.amazon.com/api-gateway/) and [Amazon S3](http://aws.amazon.com/s3/) deliver the contents of the Content Designer UI.
 
-3. The admin configures questions and answers in the Content Designer and the UI sends requests to [Amazon API Gateway](http://aws.amazon.com/api-gateway/) to save the questions and answers.
+3. The admin configures questions and answers in the Content Designer and the UI sends requests to Amazon API Gateway to save the questions and answers.
 
 4. The `Content Designer` [AWS Lambda](http://aws.amazon.com/lambda/) function saves the input in [Amazon OpenSearch Service](http://aws.amazon.com/opensearch-service/) in a questions bank index. If using [text embeddings](docs/semantic_matching_using_LLM_embeddings/README.md), these requests will first pass through a ML model hosted on [Amazon SageMaker](https://aws.amazon.com/sagemaker/) to generate embeddings before being saved into the question bank on OpenSearch.
 
 5. Users of the chatbot interact with Amazon Lex via the web client UI or [Amazon Connect](https://aws.amazon.com/connect/).
 
 6. Amazon Lex forwards requests to the `Bot Fulfillment` AWS Lambda function. Users can also send requests to this Lambda function via [Amazon Alexa](https://developer.amazon.com/en-US/alexa) devices.
 
-7. The `Bot Fulfillment` AWS Lambda function takes the users input and uses [Amazon Comprehend](https://aws.amazon.com/comprehend/) and [Amazon Translate](https://aws.amazon.com/translate/) (if necessary) to translate non-English requests to English and then looks up the answer in in Amazon OpenSearch Service. If using [text embeddings](docs/semantic_matching_using_LLM_embeddings/README.md), these requests will first pass through a ML model hosted on Amazon SageMaker to generate an embedding to compare with those saved in the question bank on OpenSearch. If an [Amazon Kendra](https://aws.amazon.com/kendra/) index is configured, the `Bot Fulfillment` function also sends a request to that index.
+7. The `Bot Fulfillment` AWS Lambda function takes the users input and uses [Amazon Comprehend](https://aws.amazon.com/comprehend/) and [Amazon Translate](https://aws.amazon.com/translate/) (if necessary) to translate non-English requests to English and then looks up the answer in in Amazon OpenSearch Service. If using LLM features such as [text generation](docs/LLM_Retrieval_and_generative_question_answering/README.md) and  [text embeddings](docs/semantic_matching_using_LLM_embeddings/README.md), these requests will first pass through various ML models hosted on Amazon SageMaker to generate the search query and embeddings to compare with those saved in the question bank on OpenSearch.
 
-8. User interactions with the `Bot Fulfillment` function generate logs and metrics data, which is sent to [Amazon Kinesis Data Firehose](http://aws.amazon.com/kinesis/data-firehose/) then to Amazon S3 for later data analysis.
+8.	If an [Amazon Kendra](https://aws.amazon.com/kendra/) index is [configured for fallback](docs/Kendra_Fallback_README.md), the `Bot Fulfillment` AWS Lambda function forwards the request to Kendra if no matches were returned from the OpenSearch question bank. The text generation LLM can optionally be used to create the search query and to synthesize a response given the returned document excerpts.
+
+9. User interactions with the `Bot Fulfillment` function generate logs and metrics data, which is sent to [Amazon Kinesis Data Firehose](http://aws.amazon.com/kinesis/data-firehose/) then to Amazon S3 for later data analysis.
 
 Refer to the [implementation guide](https://docs.aws.amazon.com/solutions/latest/qnabot-on-aws) for detailed instructions on deploying QnABot in your AWS account.