Merge branch 'vNext-Dev' of https://github.com/microsoft/PubSec-Info-…

…Assistant into geearl/6323-large-tables

.github/ISSUE_TEMPLATE/bug_report.md

@@ -7,30 +7,49 @@ assignees: ''
 
 ---
 
+*Before you open an issue, please check if a similar issue already exists or has been closed before.*
+
+ You can also find details on [Troubleshooting Common Issues](https://github.com/microsoft/PubSec-Info-Assistant/blob/main/docs/deployment/troubleshooting.md). You can use these tools to help gather additional logs and details to include in  your issue.
+
+ :warning: Please DO NOT include confidential information in your issue on GitHub. :warning:
+
+### Bug Details
 **Describe the bug**
 A clear and concise description of what the bug is.
 
-**To Reproduce**
-Steps to reproduce the behavior:
-1. Go to '...'
-2. Click on '....'
-3. Scroll down to '....'
-4. See error
+**Steps To Reproduce**
+
+1. ...
+2. ...
+3. ...
 
-**Expected behavior**
-A clear and concise description of what you expected to happen.
+**What is the expected behavior?**
 
 **Screenshots**
 If applicable, add screenshots to help explain your problem.
 
-**Desktop (please complete the following information):**
- - OS: [e.g. iOS]
- - Browser [e.g. chrome, safari]
- - Version [e.g. 22]
+### Information Assistant details
 
-**Alpha version details**
- - GitHub branch: [e.g. main]
- - Latest commit: [obtained by running `git log -n 1 <branchname>`
+Please provide the following details. You can simply include a screenshot of your Info panel as well.
+
+>GitHub branch: [e.g. main]
+>
+>Version or Latest commit: [obtained by running `git log -n 1 <branchname>`
+>
+>What region is your Azure Open AI Service in?
+>
+>What ChatGPT model are you using?
+>
+>model name: (i.e. gpt-3.5-turbo, gpt-4)
+>
+>model version: (i.e. 0613)
+>
+>What embeddings model are you using?
 
 **Additional context**
 Add any other context about the problem here.
+
+**If the bug is confirmed, would you be willing to submit a PR?**
+
+- [ ] Yes
+- [ ] No

.vscode/settings.json

@@ -13,5 +13,9 @@
         "latex",
         "plaintext",
         "shellscript"
+    ],
+    "cSpell.words": [
+        "Codespaces",
+        "referenceable"
     ]
 }

Makefile

@@ -58,4 +58,3 @@ destroy-inf: check-subscription
 
 functional-tests: extract-env ## Run functional tests to check the processing pipeline is working
 	@./scripts/functional-tests.sh	
-

README.md

@@ -8,12 +8,13 @@
 - [Features](#features)
 - [Azure account requirements](#azure-account-requirements)
 - [Azure Deployment](./docs/deployment/deployment.md)
-  - [Codespaces Setup](./docs/deployment/deployment.md#development-environment-configuration)
+  - [GitHub Codespaces Setup](./docs/deployment/deployment.md#development-environment-configuration)
   - [Cost Estimation](./docs/deployment/deployment.md#sizing-estimator)
   - [Configuring ENV parameters](./docs/deployment/deployment.md#configure-env-files)
   - [Authenticating to Azure](./docs/deployment/deployment.md#log-into-azure-using-the-azure-cli)
   - [Deploying to Azure](./docs/deployment/deployment.md#deploy-and-configure-azure-resources)
-  - [Troubleshooting Deployment & Startup Issues](./docs/deployment/worbook_usage.md)
+  - [Troubleshooting Common Issues](./docs/deployment/troubleshooting.md)
+  - [Considerations for Production Adoption](./docs/deployment/considerations_production.md)
 - [Enabling optional features](./docs/features/optional_features.md)
 - [Using the app](/docs/deployment/using_ia_first_time.md)
 - [Responsible AI](#responsible-ai)
@@ -36,6 +37,8 @@ This industry accelerator showcases integration between Azure and OpenAI's large
 
 The accelerator adapts prompts based on the model type for enhanced performance. Users can customize settings like temperature and persona for personalized AI interactions. It offers features like explainable thought processes, referenceable citations, and direct content for verification.
 
+Please [see this video](https://aka.ms/InfoAssist/video) for use cases that may be achievable with this accelerator.
+
 ## Features
 
 The IA Accelerator contains several features, many of which have their own documentation.
@@ -46,7 +49,7 @@ The IA Accelerator contains several features, many of which have their own docum
 
 For a detailed review see our [Features](./docs/features/features.md) page.
 
-![Process Flow](/docs/process_flow.drawio.png)
+![Process Flow](/docs/process_flow.png)
 
 ## Azure account requirements
 
@@ -74,14 +77,16 @@ For a detailed review see our [Features](./docs/features/features.md) page.
   * Your Azure account also needs `Microsoft.Resources/deployments/write` permissions on the subscription level.
   * Your Azure account also needs `microsoft.directory/applications/create` and `microsoft.directory/servicePrincipals/create`, such as [Application Administrator](https://learn.microsoft.com/en-us/entra/identity/role-based-access-control/permissions-reference#application-administrator) Entra built-in role.
 * **To have accepted the Azure AI Services Responsible AI Notice** for your subscription. If you have not manually accepted this notice please follow our guide at [Accepting Azure AI Service Responsible AI Notice](./docs/deployment/accepting_responsible_ai_notice.md).
-* (Optional) Have [Visual Studio Code](https://code.visualstudio.com/) installed on your development machine. If your Azure tenant and subscription have conditional access policies or device policies required, you may need to open your Codespace in VS Code to satisfy the required polices.
+* (Optional) Have [Visual Studio Code](https://code.visualstudio.com/) installed on your development machine. If your Azure tenant and subscription have conditional access policies or device policies required, you may need to open your GitHub Codespaces in VS Code to satisfy the required polices.
 
 ## Deployment
 
 Please follow the instructions in [the deployment guide](/docs/deployment/deployment.md) to install the IA Accelerator in your Azure subscription.
 
 Once completed, follow the [instructions for using IA Accelerator for the first time](/docs/deployment/using_ia_first_time.md).
 
+You may choose to **[view the deployment and usage click-through guides](https://aka.ms/InfoAssist/deploy)** to see the steps in action. These videos may be useful to help clarify specific steps or actions in the instructions.
+
 ## Responsible AI
 
 The Information Assistant (IA) Accelerator and Microsoft are committed to the advancement of AI driven by ethical principles that put people first.
@@ -114,7 +119,7 @@ This project has the following structure:
 
 File/Folder | Description
 ---|---
-.devcontainer/ | Dockerfile, devcontainer configuration, and supporting script to enable both Codespaces and local DevContainers.
+.devcontainer/ | Dockerfile, devcontainer configuration, and supporting script to enable both GitHub Codespaces and local DevContainers.
 app/backend/ | The middleware part of the IA website that contains the prompt engineering and provides an API layer for the client code to pass through when communicating with the various Azure services. This code is python based and hosted as a Flask app.
 app/enrichment/ | The text-based file enrichment process that handles language translation, embedding the text chunks, and inserting text chunks into the Azure AI Search hybrid index. This code is python based and is hosted as a Flask app that subscribes to an Azure Storage Queue.
 app/frontend/ | The User Experience layer of the IA website. This code is Typescript based and hosted as a Vite app and compiled using npm.

app/backend/approaches/chatreadretrieveread.py

@@ -40,18 +40,18 @@ class ChatReadRetrieveReadApproach(Approach):
     ASSISTANT = "assistant"
 
     system_message_chat_conversation = """You are an Azure OpenAI Completion system. Your persona is {systemPersona} who helps answer questions about an agency's data. {response_length_prompt}
-    User persona is {userPersona} Answer ONLY with the facts listed in the list of sources above in {query_term_language}
-    Your goal is to provide accurate and relevant answers based on the facts listed above in the provided source documents. Make sure to reference the above source documents appropriately and avoid making assumptions or adding personal opinions.
+    User persona is {userPersona} Answer ONLY with the facts listed in the list of sources below in {query_term_language} with citations.If there isn't enough information below, say you don't know and do not give citations. For tabular information return it as an html table. Do not return markdown format.
+    Your goal is to provide answers based on the facts listed below in the provided source documents. Avoid making assumptions,generating speculative or generalized information or adding personal opinions.
+       
     
-    Emphasize the use of facts listed in the above provided source documents.Instruct the model to use source name for each fact used in the response.  Avoid generating speculative or generalized information. Each source has a file name followed by a pipe character and 
-    the actual information.Use square brackets to reference the source, e.g. [info1.txt]. Do not combine sources, list each source separately, e.g. [info1.txt][info2.pdf].
+    Each source has a file name followed by a pipe character and the actual information.Use square brackets to reference the source, e.g. [info1.txt]. Do not combine sources, list each source separately, e.g. [info1.txt][info2.pdf].
     Never cite the source content using the examples provided in this paragraph that start with info.
-    
+      
     Here is how you should answer every question:
     
-    -Look for relevant information in the above source documents to answer the question in {query_term_language}.
-    -If the source document does not include the exact answer, please respond with relevant information from the data in the response along with citation.You must include a citation to each document referenced.      
-    -If you cannot find any relevant information in the above sources, respond with I am not sure.Do not provide personal opinions or assumptions.
+    -Look for information in the source documents to answer the question in {query_term_language}.
+    -If the source document has an answer, please respond with citation.You must include a citation to each document referenced only once when you find answer in source documents.      
+    -If you cannot find answer in below sources, respond with I am not sure.Do not provide personal opinions or assumptions and do not include citations.
     
     {follow_up_questions_prompt}
     {injected_prompt}
@@ -172,7 +172,8 @@ async def run(self, history: Sequence[dict[str, str]], overrides: dict[str, Any]
             model=self.model_name,
             messages=messages,
             temperature=0.0,
-            max_tokens=32,
+            # max_tokens=32, # setting it too low may cause malformed JSON
+            max_tokens=100,
             n=1)
 
         generated_query = chat_completion.choices[0].message.content
@@ -244,6 +245,18 @@ async def run(self, history: Sequence[dict[str, str]], overrides: dict[str, Any]
         citation_lookup = {}  # dict of "FileX" moniker to the actual file name
         results = []  # list of results to be used in the prompt
         data_points = []  # list of data points to be used in the response
+
+        #  #print search results with score
+        # for idx, doc in enumerate(r):  # for each document in the search results
+        #     print(f"File{idx}: ", doc['@search.score'])
+
+        # cutoff_score=0.01
+
+        # # Only include results where search.score is greater than cutoff_score
+        # filtered_results = [doc for doc in r if doc['@search.score'] > cutoff_score]
+        # # print("Filtered Results: ", len(filtered_results))
+
+
 
         for idx, doc in enumerate(r):  # for each document in the search results
             # include the "FileX" moniker in the prompt, and the actual file name in the response
@@ -321,7 +334,7 @@ async def run(self, history: Sequence[dict[str, str]], overrides: dict[str, Any]
                 system_message,
                 self.model_name,
                 history,
-                history[-1]["user"] + "Sources:\n" + content + "\n\n",
+                history[-1]["user"] + "Sources:\n" + content + "\n\n", # 3.5 has recency Bias that is why this is here
                 self.response_prompt_few_shots,
                 max_tokens=self.chatgpt_token_limit - 500
             )
@@ -346,11 +359,12 @@ async def run(self, history: Sequence[dict[str, str]], overrides: dict[str, Any]
 
         elif self.model_name.startswith("gpt-4"):
             messages = self.get_messages_from_history(
-                "Sources:\n" + content + "\n\n" + system_message,
-                # system_message + "\n\nSources:\n" + content,
+                system_message,
+                # "Sources:\n" + content + "\n\n" + system_message,
                 self.model_name,
                 history,
-                history[-1]["user"],
+                # history[-1]["user"],
+                history[-1]["user"] + "Sources:\n" + content + "\n\n", # GPT 4 starts to degrade with long system messages. so moving sources here 
                 self.response_prompt_few_shots,
                 max_tokens=self.chatgpt_token_limit
             )

app/frontend/package.json

@@ -26,10 +26,10 @@
     "@types/dompurify": "^2.4.0",
     "@types/react": "^18.0.27",
     "@types/react-dom": "^18.0.10",
-    "@vitejs/plugin-react": "^4.0.1",
+    "@vitejs/plugin-react": "^4.2.1",
     "prettier": "^2.8.3",
     "typescript": "^4.9.3",
-    "vite": "^4.1.0",
+    "vite": "^5.0.10",
     "postcss-nesting": "^11.2.2"
   }
 }

app/frontend/src/components/Example/ExampleList.tsx

@@ -11,9 +11,9 @@ export type ExampleModel = {
 };
 
 const EXAMPLES: ExampleModel[] = [
-    { text: "What impact does China have on climate change?", value: "What impact does China have on climate change?" },
+    { text: "How is strengthening student data privacy accomplished?", value: "How is strengthening student data privacy accomplished?" },
     { text: "What are Microsoft's primary sources of revenue?", value: "What are Microsoft's primary sources of revenue?" },
-    { text: "Provide a report comparing the Arleigh Burke destroyers with the Chinese Type 055.", value: "Provide a report comparing the Arleigh Burke destroyers with the Chinese Type 055." }
+    { text: "What are some flavors of Breyers?", value: "What are some flavors of Breyers?" }
 ];
 
 interface Props {

docs/container_webapp_debug.md

@@ -1,8 +1,8 @@
-# Debugging the Container Based Web App Locally in VSCode
+# Debugging the Enrichment Web App Locally in VSCode
 
-Embeddings processing is performed in a docker container which is deployed from the Azure Container Registry (ACR) to the Webapp. The system then uses this fucntionality to create embeddings when messages arrive in the embeddings queue. At some point you may wish to step through this code line by line in VS Code. Prior to debugging, ensure you stop the webapp in the Azure portal, or it will pick up and test messages you deliver to the embeddings queue before your code can read the message.
+Embeddings processing is performed in an Azure App Service. The system uses this functionality to perform enrichments, create embeddings, and index documents when messages arrive in the **embeddings-queue** queue. At some point you may wish to step through this code line by line in VS Code. Prior to debugging, ensure you stop the webapp in the Azure portal, or it will pick up and test messages you deliver to the embeddings queue before your code can read the message.
 
-To start debugging firstly add breakpoints to the code and then simply select the Run & Debug menu option in the left bar, or Ctrl+Shift+D. Next select Python: FastAPI - enrichment app and hit the play button. This will then initiate the code and stop on your first breakpoint.
+To start debugging firstly add breakpoints to the code and then simply select the Run & Debug menu option in the left bar, or Ctrl+Shift+D. Next select `Python: Enrichment Webapp`  and hit the play button. This will then initiate the code and stop on your first breakpoint.
 
 One tip is to save a copy of a message in the embeddings queue which triggers your logic. Then you will be able to just resubmit this message again and again to initiate and trace your code.
 

docs/costestimator.md

@@ -1,16 +1,16 @@
-# IA Accelerator, version 0.4 delta - Estimation
+# IA Accelerator, version 1.0 - Estimation
 
-The Azure pricing calculator helps estimate costs by considering the amount of data to be processed and stored, as well as the expected performance level. It allows users to customize and combine different Azure services for IA Accelerator, version 0.4 delta, and provides cost estimates based on the chosen configurations.
+The Azure pricing calculator helps estimate costs by considering the amount of data to be processed and stored, as well as the expected performance level. It allows users to customize and combine different Azure services for IA Accelerator, version 1.0, and provides cost estimates based on the chosen configurations.
 
 
 | Solution            | Environment  |    Azure Pricing Calculator Link                                          |
 | :------------------:|:-----------------------------:|:------------------------------------------------:|
-| IA Accelerator, version 0.4 delta | Sandbox  |  [Sample Azure Estimation](https://azure.com/e/06376dd6bb454d088b7af5fdd6e00d48) | 
+| IA Accelerator, version 1.0 | Sandbox  |  [Sample Azure Estimation](https://azure.com/e/9849721efce04059be9ed8d5735a7a58) | 
 
 ---
 ### Azure Services
 
-The following list of Azure Services will be deployed for IA Accelerator, version 0.4 gamma:
+The following list of Azure Services will be deployed for IA Accelerator, version 0.4 delta:
 
 - App Service [:link:](https://azure.microsoft.com/en-ca/pricing/details/app-service/linux/)
 - Azure Function(App Service plan) [:link:](https://azure.microsoft.com/en-ca/pricing/details/functions/#pricing)
@@ -23,14 +23,14 @@ The following list of Azure Services will be deployed for IA Accelerator, versio
 - Azure Active Directory [:link:](https://www.microsoft.com/en-sg/security/business/microsoft-entra-pricing?rtc=1)
 - Azure AI Video Indexer [:link:](https://azure.microsoft.com/en-us/pricing/details/video-indexer/)
 - Log Analytics workspace [:link:](https://azure.microsoft.com/en-ca/pricing/details/monitor/)
-- Storage account  [:link:]()
+- Storage account  [:link:](https://learn.microsoft.com/en-us/azure/storage/common/storage-account-overview)
 
 ---
 **NOTE:**
 
 - The proposed the cost estimation prepared based on Sandbox environment, estimation may vary customer to customer.
 - For detailed Azure OpenAI Service [pricing](https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/#pricing)
-- To calculate total count of tokens [OpenAI Tokenizer](https://platform.openai.com/tokenizer)
+- Customers with latency-sensitive scenarios can opt for provisioned throughput which allows customers to reserve model processing capacity. [Read More](/docs/deployment/considerations_production.md#gpt-model---throttling)
 
 ---