Skip to content

didier-durand/qstensils

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

97 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Q logo

qstensils: tools and utility scripts for Amazon Q

AWS Python License: MIT Codacy Badge build workflow

Provided Tools

Disclaimer: the scripts provided in this repository reflect the state of service Amazon Q for Business as it was launched in December 2023 by AWS in Preview mode. Due to this Preview mode, the features of Amazon Q and their implementation can change at any time during Preview and for General Availability. Such changes may require updates to this repository.

This project gathers diverse tools and utility scripts to explore and operate Amazon Q for Business. We will add new scripts based on your demand: feel free to cut a ticket here if you have a need or idea!

We currently provide the following utilities:

  1. q_list_applications to inventory the applications existing in a given region of an AWS account. The returned json structure details the various components (indices, data source, retrievers, etc.) of those Amazon Q applications.
  2. q_list_data_source_sync_jobs to list the history of index synchronization jobs executed on a given Q data source. This script adds additional metric like total job duration, document scan rate and average scan duration per document.
  3. q_list_documents to list all the documents of an Amazon Q index and get all their associated metadata, in particular their status. The returned list can be filtered (via inclusion or exclusion) to return only a fraction of those documents for example based on their indexing status.
  4. q_list_conversations to obtain all list of all past conversations between a given application and a user as remembered by Amazon Q.
  5. q_chat to be able to script conversations (based on single or multiple messages) with the assistant of Amazon Q.

All those scripts return json structures that can be further processed in shell pipelines with various utilities like jq, sed, awk, etc.

Those scripts rely on the Python AWS SDK. All APIs related to Q for business are described in details in the SDK boto3 public documentation.

About Amazon Q

IMAGE ALT TEXT

Amazon Q is a fully managed, generative-AI powered assistant that can be configured to answer questions, provide summaries, generate content, and complete tasks based on data in your enterprise. Amazon Q provides immediate and relevant information to its users, and helps streamline tasks and accelerate problem-solving.

An Amazon Q application relies on a corpus of documents to build its specific Q index. This corpus of documents is stored in one or more document repositories (S3, Jira, Quip, etc.) called Q data sources. The answers to user questions by the assistant will be prepared through the leverage of RAG technology.

Retrieval-Augmented Generation (RAG) is a natural language processing (NLP) technique. It is composed of a language model-based system, usually a Large Language Model (LLM), that accesses external knowledge sources to complete tasks. This enables more contextuality, factual consistency, improves reliability of the generated responses, and helps to mitigate the problem of "hallucinations".

Security

The scripts of this project assume that the AWS user reflected by environment variables named AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY has proper IAM credentials in terms of authorizations to access the APIs of Amazon Q in the AWS account. See IAM policy examples in the Security section of Q Documentation for all details.

Usage

from the /src directory of this project, the following commands can be used to get all command options

for list_applications.py

%python3 q_list_applications.py -h
usage: q_list_applications.py [-h] [-v]

list applications, indexes, retrievers, web experiences, plugins, etc. running in Amazon Q for business

options:
  -h, --help     show this help message and exit
  -v, --verbose  verbose mode

q_list_data_source_sync_jobs

 % python3 q_list_data_source_sync_jobs.py -h
usage: q_list_data_source_sync_jobs.py [-h] [-a APP_ID] [-i IDX_ID] [-d DS_ID] [-v]

list synchronization jobs executed for a given data source of an Amazon Q application

options:
  -h, --help            show this help message and exit
  -a APP_ID, --app_id APP_ID
                        Q application id
  -i IDX_ID, --idx_id IDX_ID
                        Q data source id
  -d DS_ID, --ds_id DS_ID
                        Q data source id
  -v, --verbose         verbose mode

for q_list_documents.py

% python3 q_list_documents.py  -h                                                                                                   
usage: q_list_documents.py [-h] [-a APP_ID] [-i IDX_ID] [-incl INCLUDE] [-excl EXCLUDE] [-inv] [-v]

list documents indexed by Amazon Q

options:
  -h, --help            show this help message and exit
  -a APP_ID, --app_id APP_ID
                        Q application id
  -i IDX_ID, --idx_id IDX_ID
                        Q index id
  -incl INCLUDE, --include INCLUDE
                        comma-separated list of status to include
  -excl EXCLUDE, --exclude EXCLUDE
                        comma-separated list of status to exclude
  -inv, --inventory     with document inventory
  -v, --verbose         verbose mode

for q_list_conversations.py

% python3 q_list_conversations.py -h
usage: q_list_conversations.py [-h] [-a APP_ID] [-u USR_ID] [-v]

list documents indexed by Amazon Q

options:
  -h, --help            show this help message and exit
  -a APP_ID, --app_id APP_ID
                        Q application id
  -u USR_ID, --usr_id USR_ID
                        Q user id
  -v, --verbose         verbose mode

for q_chat.py

% python3 q_chat.py  -h
usage: q_chat.py [-h] [-a APP_ID] [-u USR_ID] [-p PROMPT] [-f FILE] [-c CNV_ID] [-m MSG_ID] [-d] [-v]

ask a question to a Q application and get answer

options:
  -h, --help            show this help message and exit
  -a APP_ID, --app_id APP_ID
                        Q application id
  -u USR_ID, --usr_id USR_ID
                        Q index id
  -p PROMPT, --prompt PROMPT
                        question prompt or path to file with list of prompts
  -f FILE, --file FILE  path to attachment file
  -c CNV_ID, --cnv_id CNV_ID
                        Q conversation id (only to continue an existing conversation)
  -m MSG_ID, --msg_id MSG_ID
                        Q parent message id (only to continue an existing conversation)
  -d, --details         full conversation details
  -v, --verbose         verbose mode