diff --git a/ai-optimizer/desktop/evaluating/evaluating.md b/ai-optimizer/desktop/evaluating/evaluating.md new file mode 100644 index 00000000..22e2102f --- /dev/null +++ b/ai-optimizer/desktop/evaluating/evaluating.md @@ -0,0 +1,151 @@ +# Evaluating Performance + +## Introduction + +We are confident that adjusting certain parameters can improve the quality and accuracy of the chatbot’s responses. However, can we be sure that a specific configuration remains reliable when scaled to hundreds or even thousands of different questions? + +In this lab, you will explore the *Testbed* feature. The Testbed allows you to evaluate your chatbot at scale by generating a Q&A test dataset and automatically running it against your current configuration. + +**Note**: The example shown in this lab relies on gpt-4o-mini. Feel free to use your local LLMs (e.g. llama3.1) if you choose to or can't use OpenAI LLMs. + +Estimated Time: 15 minutes + +### Objectives + +In this lab, you will: + +* Explore the *Testbed* tab +* Generate a Q&A Test dataset +* Perform an evaluation on the Q&A Testset + +### Prerequisites + +* All previous labs successfully completed + +## Task 1: Navigate to the Testbed tab + +Access the *Testbed* from the left-hand menu: + +![testbed](./images/testbed.png) + +As a first step, you can either upload an existing Q&A test set—either from a local file or from a saved collection in the database—or generate a new one from a local PDF file. + +## Task 2: Generate a Q&A Test dataset + +The AI Optimizer allows you to generate as many questions and answers as you need, based on a single document from your knowledge base. To enable test dataset generation, simply select the corresponding radio button: + +![generate](./images/generatenew.png) + +1. Upload a document + + Upload the same document that was used to create the vector store. You can easily download it from [this link](https://docs.oracle.com/en/database/oracle/oracle-database/23/vecse/ai-vector-search-users-guide.pdf). + +2. Increase the number of questions to be generated to 10 or more + + Keep in mind that the process can take a significant amount of time, especially if you are using a local LLM without sufficient hardware resources. If you choose to use a remote OpenAI model instead, the generation time will be less affected by the number of Q&A pairs to create. + +3. Leave the default option for: + * Q&A Language Model: **gpt-4o-mini** + * Q&A Embedding Model: **text-embedding-3-small** + +4. Click on **Generate Q&A** button and wait until the process is over: + + ![patience](./images/patience.png) + +5. Browse the questions and answers generated: + + ![qa-browse](./images/qa-browse.png) + + Note that the **Question** and **Answer** fields are editable, allowing you to modify the proposed Q&A pairs based on the **Context** (which is randomly extracted and not editable) and the **Metadata** generated by the Testbed engine. + + In the *Metadata* field you'll find a **topic** tag that classifies each Q&A pair. The topic list is generated automatically by analyzing the document content and is assigned to each Q&A pair. It will be used in the final report to break down the **Overall Correctness Score** and highlight areas where the chatbot lacks precision. + + You can also export the generated Q&A dataset using the **Download** button. This allows you to edit and review it—e.g., in Visual Studio Code. + + ![qa-json](./images/qa-json.png) + +6. Update the **Test Set Name** + + Replace the automatically generated default name to make it easier to identify the test dataset later, especially when running repeated tests with different chatbot configurations. For example, change it from: + + ![default-test-set](./images/default-test-set.png) + + to something more descriptive, like: + + ![test-rename](./images/test-rename.png) + +## Task 3: Evaluate the Q&A Testset + +Now you are ready to perform an evaluation on the Q&As you generated in the previous step. + +1. In the left-hand menu: + + * Under **Language Model Parameters**, select **gpt-4o-mini** from the **Chat model** dropdown list. + + * Ensure **Enable RAG?** is selected (if it wasn't already) + + * In the **Select Alias** dropdown list, choose the **TEST2** value. + + * Leave all other parameters unchanged + +2. With **gpt-4o-mini** selected as the evaluation model, click the **Start Evaluation** button and wait a few seconds. All questions from your dataset will be submitted to the chatbot using the configuration defined in the left pane: + + ![start-eval](./images/start-eval.png) + +3. Let's examine the result report, starting with the first section: + + ![result](./images/result-topic.png) + + This section displays: + + * The chatbot's **Evaluation Settings**, as configured in the left-hand pane before launching the massive test. + + * The **RAG Settings** including the database and vector store used, the name of the embedding **model** used, and all associated parameters (e.g., **chunk size**, **top-k**). + + * The **Overall Correctness Score**, representing the percentage of questions for which the LLM judged the chatbot's response as correct compared to the reference answer + + * The **Correctness By Topic**, which breaks down the results based on the automatically generated topics assigned to each Q&A pair in the dataset. + + The second section of the report contains details on each question submitted, with a focus on the **Failures** collection and the **Full Report** list. To view all fields, scroll horizontally. In the image below, the second frame has been scrolled to the right: + + ![result](./images/result-question.png) + + The main fields displayed are: + + * **question**: the submitted question + * **reference_answer**: the expected answer used as a benchmark + * **reference_context**: the source document section used to generate the Q&A pair + * **agent_answer**: the response provided by the chatbot based on the current configuration and vector store + * **correctness_reason**: an explanation (if any) of why the response was considered incorrect. If correct, this field will display **None**. + + You can download the results in different formats: + + * Click the **Download Report** button to generate an HTML summary of the *Overall Correctness Score* and *Correctness by Topic* + + * To export the **Full Report** and the **Failures** list, download them as .csv files using the download icons shown in the interface: + + ![csv](./images/download-csv.png) + +## Task 4 (optional): Try a different Q&A Testset + +Now let's perform a test using an external saved test dataset, which you can download [here](https://raw.githubusercontent.com/markxnelson/developer/refs/heads/main/ai-optimizer/getting_started-30_testset.json). This file contains 30 pre-generated questions. + +If you wish to remove any Q&A pairs that you consider irrelevant or unhelpful, you can edit the file, save it, and then reload it as a local file—following the steps shown in the screenshot below: + + ![load-tests](./images/load-tests.png) + +Next, let’s update the Chat Model parameters by setting the **Temperature** to **0** in the left-hand pane under the **Language Model Parameters** section. +Why? Q&A datasets are typically generated with a low level of creativity to minimize randomness and focus on expressing core concepts clearly—avoiding unnecessary "frills" in the answers. +Now, repeat the test to see whether there are improvements in the **Overall Correctness Score**. + +* To compare with previous results, open the dropdown under **Previous Evaluations for...** and click on the **View** button to display the associated report. + + ![previous](./images/previous.png) + +* You can repeat the tests as many times as needed, changing the **Vector Store**, **Search Type**, and **Top K** parameters to apply the same tuning strategies you've used previously with individual questions—now extended to a full test using curated and reproducible data. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Lorenzo De Marchis, May 2025 diff --git a/ai-optimizer/desktop/evaluating/images/default-test-set.png b/ai-optimizer/desktop/evaluating/images/default-test-set.png new file mode 100644 index 00000000..11a047ea Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/default-test-set.png differ diff --git a/ai-optimizer/desktop/evaluating/images/download-csv.png b/ai-optimizer/desktop/evaluating/images/download-csv.png new file mode 100644 index 00000000..22f87dbb Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/download-csv.png differ diff --git a/ai-optimizer/desktop/evaluating/images/generatenew-ollama.png b/ai-optimizer/desktop/evaluating/images/generatenew-ollama.png new file mode 100644 index 00000000..8889862e Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/generatenew-ollama.png differ diff --git a/ai-optimizer/desktop/evaluating/images/generatenew.png b/ai-optimizer/desktop/evaluating/images/generatenew.png new file mode 100644 index 00000000..9c1fdc47 Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/generatenew.png differ diff --git a/ai-optimizer/desktop/evaluating/images/load-tests.png b/ai-optimizer/desktop/evaluating/images/load-tests.png new file mode 100644 index 00000000..89a38e14 Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/load-tests.png differ diff --git a/ai-optimizer/desktop/evaluating/images/patience.png b/ai-optimizer/desktop/evaluating/images/patience.png new file mode 100644 index 00000000..0d9cc381 Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/patience.png differ diff --git a/ai-optimizer/desktop/evaluating/images/previous.png b/ai-optimizer/desktop/evaluating/images/previous.png new file mode 100644 index 00000000..7afb2529 Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/previous.png differ diff --git a/ai-optimizer/desktop/evaluating/images/qa-browse.png b/ai-optimizer/desktop/evaluating/images/qa-browse.png new file mode 100644 index 00000000..a6df294c Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/qa-browse.png differ diff --git a/ai-optimizer/desktop/evaluating/images/qa-json.png b/ai-optimizer/desktop/evaluating/images/qa-json.png new file mode 100644 index 00000000..20a3b6da Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/qa-json.png differ diff --git a/ai-optimizer/desktop/evaluating/images/result-question.png b/ai-optimizer/desktop/evaluating/images/result-question.png new file mode 100644 index 00000000..ae9c1bcf Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/result-question.png differ diff --git a/ai-optimizer/desktop/evaluating/images/result-topic.png b/ai-optimizer/desktop/evaluating/images/result-topic.png new file mode 100644 index 00000000..2704b259 Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/result-topic.png differ diff --git a/ai-optimizer/desktop/evaluating/images/start-eval.png b/ai-optimizer/desktop/evaluating/images/start-eval.png new file mode 100644 index 00000000..13e7a05a Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/start-eval.png differ diff --git a/ai-optimizer/desktop/evaluating/images/test-rename.png b/ai-optimizer/desktop/evaluating/images/test-rename.png new file mode 100644 index 00000000..0a516a63 Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/test-rename.png differ diff --git a/ai-optimizer/desktop/evaluating/images/testbed.png b/ai-optimizer/desktop/evaluating/images/testbed.png new file mode 100644 index 00000000..59ad8ddd Binary files /dev/null and b/ai-optimizer/desktop/evaluating/images/testbed.png differ diff --git a/ai-optimizer/desktop/experimenting/experimenting.md b/ai-optimizer/desktop/experimenting/experimenting.md new file mode 100644 index 00000000..ea33b045 --- /dev/null +++ b/ai-optimizer/desktop/experimenting/experimenting.md @@ -0,0 +1,290 @@ +# Experimenting with Models, Parameters and Techniques + +## Introduction + +In this lab, we will experiment with different configurations, adjusting parameters and using various LLMs to observe how the answers vary. + +> **NOTE**: The answers received by any LLM are based on probabilistic models. Therefore, the answer received in your own testing might differ from the ones shown in this documentation. + +Estimated Lab Time: 15 minutes + +### Objectives + +In this lab, you will: + +* Test different parameter configurations and compare the resulting answers +* Apply various distance metrics to the same set of questions and evaluate how the answers differ +* Experiment with different vector stores and compare the answers + +### Prerequisites + +This lab assumes you have: + +* All previous labs successfully completed + +## Task 1: Play with the Parameters + +Using the **TEST1** vector store, you will experiment with the main parameters configured as shown below to observe how answer quality changes. We will begin by adjusting the **Temperature** parameter. For the first test, set the Temperature to 0 to minimize creativity in the responses. Then, repeat the same question with a different Temperature value to observe how the answers change. + +**IMPORTANT**: After each question—regardless of whether you're repeating the same one with different settings—be sure to clear the history by clicking the **Clear** button. We will start by testing the first question. + +* LLM: llama3.1 +* TopK: 4 +* Search Type: Similarity +* Temperature: 0 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I determine the accuracy of my vector indexes? + +A: The user wants to determine the accuracy of their vector indexes in Oracle Database 23c. According to the provided documentation, there are several ways to achieve this: +1. Use the DBMS_VECTOR.INDEX_ACCURACY_QUERY procedure: This procedure provides an accuracy report for a top-K index search for a specific query vector and a specific target accuracy. +2. Use the DBMS_VECTOR.INDEX_ACCURACY_REPORT function: This function captures accuracy values achieved by approximate searches using a particular vector index for a certain period of time. +3. Use the INDEX_VECTOR_MEMORY_ADVISOR procedure to determine the vector memory size. +Here is an excerpt from the documentation: +To verify the accuracy of a vector index, you can use the DBMS_VECTOR.INDEX_ACCURACY_QUERY function. This function takes four parameters: OWNER_NAME, INDEX_NAME, QV (query vector), and TOP_K (top K results). +The user can also refer to the Oracle Database AI Vector Search User's Guide for more information on determining the accuracy of their vector indexes. +Document references: +* DBMS_VECTOR.INDEX_ACCURACY_QUERY procedure in the provided text +* DBMS_VECTOR.INDEX_ACCURACY_REPORT function in the provided text +* INDEX_VECTOR_MEMORY_ADVISOR procedure in the provided text +``` + +* LLM: llama3.1 +* TopK: 4 +* Search Type: Similarity +* Temperature: 1 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I determine the accuracy of my vector indexes? + +A: To determine the accuracy of your vector indexes in Oracle Database 23c AI, you can use the DBMS_VECTOR.INDEX_ACCURACY_QUERY procedure. +According to the provided documentation, this procedure provides an accuracy report for a top-K index search for a specific query vector and a specific target accuracy +Here's how you can do it: +Declare the necessary variables: + +declare + q_v VECTOR; + report varchar2(128); + +Initialize the query vector q_v with the desired vector (in this case, [0,1,1,0,0]): + +q_v := to_vector('[0,1,1,0,0]'); + +Execute the DBMS_VECTOR.INDEX_ACCURACY_QUERY procedure: + +exec DBMS_VECTOR.INDEX_ACCURACY_QUERY('OWNER_NAME', 'INDEX_NAME', q_v, TOP_K, target_accuracy); + +Replace 'OWNER_NAME', 'INDEX_NAME', TOP_K, and target_accuracy with your actual values. +This will provide you with an accuracy report for the specified vector index. +Additionally, you can use the DBMS_VECTOR.INDEX_ACCURACY_REPORT function to capture accuracy values achieved by approximate searches using a particular vector index for a certain period of time. +``` + +**COMMENT**: As you can see, setting the Temperature to 1 increases the variability of the output, often resulting in an answer that differs significantly from the previous one. + +We will now experiment with the **TopK** parameter, while keeping Temperature fixed at 0 to minimize output variability. + +**IMPORTANT**: After each question—regardless of whether you're repeating the same one with different settings—be sure to clear the history by clicking the **Clear** button. + +Let's move onto a different question and see the results: + +* LLM: llama3.1 +* TopK: 4 +* Search Type: Similarity +* Temperature: 0 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I convert string text to embeddings using PL/SQL? + +A: To generate a vector embedding from an input text string in Oracle Database 23c, you can use the UTL_TO_EMBEDDING PL/SQL function. This function is designed to take a text string as input and return its equivalent vector embedding. +To convert a text string to an embedding using this function, follow these steps: +The database needs to have access to an external vector embedding model stored either internally in the database or accessed through a third-party BINARY vector embedding. +Construct your query such that it makes a call on UTL_TO_EMBEDDING. This function likely requires at least one parameter which would be the text string to convert into embeddings. +For example, if you have a table with a column named 'text' where each row contains the string for conversion: + +SELECT UTL_TO_EMBEDDING(text) FROM YOUR_TABLE; + +Make sure that your database setup includes the access details required for whatever vector embedding model is being referenced. Without knowing more about how UTL_TO_EMBEDDING function expects setup on such models within Oracle databases, it's not possible to offer more specifics than this. +``` + +* LLM: llama3.1 +* TopK: 10 +* Search Type: Similarity +* Temperature: 0 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I convert string text to embeddings using PL/SQL? + +A: To convert a string text to an embedding using PL/SQL in Oracle Database 23c (not 23ai, as there is no such version), you can use the DBMS_VECTOR package. +Here's a simple example: + +DECLARE + input CLOB; + v VECTOR; + +BEGIN + input := 'Your string here'; + SELECT dbms_vector.utl_to_embedding(input, json('{"provider":"database", "model":"doc_model"}')) INTO v FROM dual; + + DBMS_OUTPUT.PUT_LINE(DBMS_VECTOR.util_serialize(v)); +END; +/ +In this code: +input is the string you want to convert. +"{'provider\":\"database\", \"model\":\"doc_model\"}" is a JSON object that specifies the vectorization model (in this case, an in-database model named 'doc_model'). +The DBMS_OUTPUT.PUT_LINE(DBMS_VECTOR.util_serialize(v)) statement then outputs a JSON representation of the generated embedding. +Note: Make sure you have loaded your ONNX model into Oracle Database first by using dbms_math.load_onnx_model package (which is not shown in this code snippet). +Note that there are similar methods for converting strings to binary embeddings outside the database through APIs, and it requires more parameters depending on which kind of embedding or library you use. +``` + +**COMMENT**: As you can see, increasing the TopK parameter prompts the LLM to retrieve a larger number of vectors from the vector database. This provides the model with more contextual information to rely on when using RAG, resulting in significantly different answers compared to the example with TopK set to 4. + +## Task 2: Use a different Distance Metric + +We will now observe how using a different distance metric can lead to a different answer. For this test, we will set Temperature to 0 and TopK to 4. + +**IMPORTANT**: After each question—regardless of whether you're repeating the same one with different settings—be sure to clear the history by clicking the **Clear** button. + +* LLM: llama3.1 +* TopK: 4 +* Search Type: Similarity +* Temperature: 0 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I determine the accuracy of my vector indexes? + +A: To determine the accuracy of your vector indexes in Oracle Database 23c (not 23ai), you can use the DBMS_VECTOR package. +According to the documentation provided, you have two options: +DBMS_VECTOR.INDEX_ACCURACY_QUERY: This function allows you to verify the accuracy of a vector index for a given query vector, top-K, and target accuracy. You can call this function by passing in the OWNER_NAME, INDEX_NAME, QV (query vector), TOP_K (top K results), and TARGET_ACCURACY as parameters. +DBMS_VECTOR.INDEX_ACCURACY_REPORT: This function captures the accuracy values achieved by approximate searches using a particular vector index for a certain period of time. +Here's an example code snippet that demonstrates how to use DBMS_VECTOR.INDEX_ACCURACY_QUERY: + +DECLARE + q_v VECTOR; + report CLOB; + +BEGIN + q_v := TO_VECTOR('[0,1,1,0,0]'); + + REPORT := DBMS_VECTOR.INDEX_ACCURACY_QUERY( + OWNER_NAME => 'VECTOR_USER', + INDEX_NAME => 'VIDX_HNSW', + QV => q_v, + TOP_K => 10, + TARGET_ACCURACY => 0.8); + + DBMS_OUTPUT.PUT_LINE(report); +END; + +Make sure to replace the placeholder values with your actual OWNER_NAME, INDEX_NAME, and vector values. +I hope this helps! +``` + +* LLM: llama3.1 +* Top K: 4 +* Fetch K: 20 +* Degree of Diversity: 0.5 +* Search Type: Maximal Marginal Relevance +* Temperature: 0 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I determine the accuracy of my vector indexes? + +A: To determine the accuracy of your vector indexes in Oracle Database 23c (note: not 23ai), you can use the DBMS_VECTOR.INDEX_ACCURACY_QUERY procedure. +According to the provided documentation and example usage, you need to: +Declare a query vector (q_v) using the TO_VECTOR function. +Call the INDEX_ACCURACY_QUERY procedure with q_v as a parameter, along with other optional parameters as needed (e.g., target accuracy). +Here's an adapted code snippet from the example usage: + +DECLARE + q_v VECTOR; + report VARCHAR2(128); + +BEGIN + q_v := to_vector('[0,1,1,0,0]'); + + -- Set other parameters as needed, e.g., target accuracy + REPORT := DBMS_VECTOR.INDEX_ACCURACY_QUERY(q_v, ); +END; +/ +This will provide an accuracy report for the specified query vector and target accuracy. +Alternatively, you can use the TOVECTOR function to insert vector data into your application. +``` + +**COMMENT**: In this case, even though the MMR retrieves 20 chunks and removes results that differ by less than 50% in diversity, ultimately providing 4 top-K results that should offer more varied information, the results are better than with a plain similarity search using top-K = 4. This approach can, for example, increase efficiency in production, as it directly provides the 4 top-K chunks, rather than adding an extra step in the execution process that involves computing and fetching the 4 most diverse chunks from the 20 top-K results retrieved. + +## (Optional) Task 3: Compare two different vector stores + +> **NOTE**: You can perform this task only if you have access to [OpenAI APIs](https://platform.openai.com/settings/organization/api-keys). + +We can also explore how different combinations of **LLMs** and vector stores can produce markedly different answers. To test this, we’ll experiment with various pairings—and introduce a completely new question, purely for exploration purposes. + +**IMPORTANT**: After each question—regardless of whether you're repeating the same one with different settings—be sure to clear the history by clicking the **Clear** button. + +* LLM: llama3.1 +* TopK: 4 +* Search Type: Similarity +* Temperature: 0 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I create a vector column? + +A: To create a vector column in Oracle Database 23c1, you can use the VECTOR data type. The syntax to declare a vector column is as follows: + +CREATE TABLE table_name ( + id NUMBER, + embedding VECTOR +); + +You don't need to specify the number of dimensions or their format initially. This allows you to enter vectors of different dimensions with various formats. You can use the TO_VECTOR function with specific dimensions and format if needed, as shown in a note in your passage. +Here is an example SQL statement that creates a vector column without specifying dimensions and format (simplified): + +CREATE TABLE my_vectors ( + id NUMBER, + embedding VECTOR +); + +You can then insert data into the embedding column using either a string representation of the vector or the explicit use of the TO_VECTOR function with specific number of dimensions, format, or storage format. +``` + +* LLM: gpt-4o-mini +* TopK: 4 +* Search Type: Similarity +* Temperature: 0 +* Vector Store: TEST2 + +```text +Q: In Oracle Database 23ai, how do I create a vector column? + +A: To create a vector column in Oracle Database 23ai, you can declare a table with a column of the VECTOR data type. Here is a simple example: + +CREATE TABLE my_vectors (id NUMBER, embedding VECTOR); + +In this example, the embedding column is defined as a VECTOR type. You do not need to specify the number of dimensions or their format if you want flexibility. If you want to impose constraints, you can specify them like this: + +CREATE TABLE my_vectors (id NUMBER, embedding VECTOR(768, INT8)); + +In this case, each vector must have 768 dimensions formatted as INT8. For more detailed information, see the section on creating tables using the VECTOR data type in the documentation. +``` + +**COMMENT**: Although the overall content remains essentially the same, we can observe notable differences in both the syntax and structure of the answers. The OpenAI-based configuration produces a more concise and precise response, whereas the Ollama-based output is more verbose—and even contains a typo. These differences are largely due to the distinct chunking sizes used by the two vector stores. + +These few examples offer just a glimpse of the cross-testing possibilities available to users. The range of parameters you can experiment with is much broader. The **AI Optimizer and Toolkit** is specifically designed to help you identify the configuration that best suits your needs. + +## Learn More + +* Cross-test vector stores and models. For example, you can test **llama3.1** while using the **text-embedding-3-small** vector store. +* Try setting the Temperature parameter to its maximum value—you’ll see how creative an LLM can become! +* Experiment with other available parameters across different questions to explore the full range of behavior. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Lorenzo De Marchis, May 2025 diff --git a/ai-optimizer/desktop/experimenting/images/sample1.png b/ai-optimizer/desktop/experimenting/images/sample1.png new file mode 100644 index 00000000..2d9ad473 Binary files /dev/null and b/ai-optimizer/desktop/experimenting/images/sample1.png differ diff --git a/ai-optimizer/desktop/experimenting/images/sample2.png b/ai-optimizer/desktop/experimenting/images/sample2.png new file mode 100644 index 00000000..145c9d39 Binary files /dev/null and b/ai-optimizer/desktop/experimenting/images/sample2.png differ diff --git a/ai-optimizer/desktop/explore/explore.md b/ai-optimizer/desktop/explore/explore.md new file mode 100644 index 00000000..b77a341d --- /dev/null +++ b/ai-optimizer/desktop/explore/explore.md @@ -0,0 +1,143 @@ +# Explore The Environment + +## Introduction + +In this lab, you will explore the environment that was created in the *Get Started* lab. You will set up the database connection, configure the OCI credentials, add new LLMs and Embedding models and test out the *Chat* feature for the first time. + +Estimated Time: 15 minutes + +### Objectives + +In this lab, you will: + +* Set the connection to your Oracle Database 23ai +* Set your Oracle Cloud Infrastructure credentials for OCI connectivity +* Configure the LLMs and embedding models you will use in the following labs +* Test the *Chat* feature for the first time + +### Prerequisites + +This lab assumes you have: + +* An Oracle Database 23ai up and running +* Completed the *Get Started* Lab + +## Task 1: Set the DB connection + +Let's check if the DB is correctly connected. + +1. Navigate to the *Databases* tab on the left side pane: + + ![Navigate to the Databases tab](images/database-navigation.jpg) + +2. To configure the Oracle Database 23ai Free, you will need to enter the credentials: + + * Enter the Database Username: `WALKTHROUGH` + * Enter the Database Password for the database user: `OrA_41_OpTIMIZER` + * Enter the Database Connection String: `//localhost:1521/FREEPDB1` + * Save + + ![Database configuration](images/database-config.png) + +3. If you are using a compatible database and have any SelectAI profile active, you'll find the list of Select AI profiles to activate. If you haven't created one yet, you will need to create a [Select AI](https://www.oracle.com/it/autonomous-database/select-ai/) profile and you'll find the list of active profiles within the tenancy eventually + + ![Select AI Profiles](images/selectai-profiles.png) + + The Select AI feature is currently available and fully supported in: + * Oracle Autonomous Database 23ai + * Oracle Database 23.7+ + +## Task 2: Configure LLMs + +Let's check the models available for use. You'll need to navigate to the *Models* tab: + +![models menu](images/models.jpg) + +* The default LLMs for chat completions are: + + ![llms](images/llms.png) + +* The default LLMs for embeddings are: + + ![embeddings](images/emb.png) + + Let's add another LLM to the Ollama models and enable it. Open a terminal a window and follow these steps: + +* Pull the llama3.2 LLM: + + ```bash + + podman exec -it ollama ollama pull llama3.2 + + ``` + * Now, you will need to enable it in the model list. Go back to the *Models* tab, under the *Language Models* section and press the *Add* button. Now fill in the form that just appeared with the values shown in the following snapshot: + + ![new_llama32](images/addllama32.png) + + * (optional) If you are willing to use models by OpenAI, you will need to configure your **OPENAI API KEY**. To configure one, click the *Edit* button beside the model you would like to use (e.g., **gpt-4o-mini**) and add your own API key in the corresponding box: + + ![openai-key](images/openai-api.png) + + Now you are all set for using the *Chat* feature with the LLMs you just configured! + +## Task 3: Test the *Chat* feature + +The two LLMs availble could be tested right away, in order to understand their behavior with generic questions. First of all, navigate to the chat GUI + +![chat](images/chat.jpg) + +scroll down the left-side menu to find the **Toolkit** menu: + +![toolkit menu](images/toolkit-menu.png) + +select the **LLM Only** option and then choose the **llama3.1** model: + +![chat models](images/chatmodel.png) + +and ask generic questions like: + +```text + +In Oracle Database 23ai, how do I determine the accuracy of my vector indexes? + +``` + +```text + +In Oracle Database 23ai, how do I convert string text to embedding using PL/SQL? + +``` + +> **NOTE**: *if you see a **Database has no Vector Stores. Disabling Vector Search.** message above, don't panic! That's because you haven't created a vector store yet and thus you can't use the RAG functionality* or *if you see a **Database not SelectAI Compatible. Disabling SelectAI.** message above, don't panic! That's because you are using a Database that's not compatible with SelectAI or doesn't have a valid Select AI profile yet*. + +As you will probably be able to notice, even if the questions refer to Oracle Database 23ai, the LLM will mention Oracle Database 23c. This is known as **Knowledge Cutoff**, meaning that probably the LLM was trained before Oracle Database 23c was even renamed as 23ai. + +Moreover, the LLM tends to answer in a generic way, with no specific mention to the actual script needed to perform the operations requested. In Lab 3, we will compare these results with the one obtained using Retrieval-Augmented Generation (RAG), where more context is given to the LLMs. + +## (Optional) Task 4: Set OCI Credentials + +The Optimizer lets you configure the connection to your OCI tenant for retrieving objects from the *Object Storage* and accessing LLMs from the OCI GenAI service. In the *OCI* configuration tab, you can add your **Oracle Cloud Infrastructure** (OCI) credentials to authenticate to your OCI tenancy. This will enable access to objects and documents stored in your cloud compartments. + +1. Navigate to the *OCI* tab on the left-hand pane: + + ![Navigate to the Databases tab](images/oci-navigation.jpg) + +2. Insert your OCI credentials. Detailed information on how to get the required credentials is available in the [Oracle Cloud Infrastructure Documentation](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/apisigningkey.htm#Required_Keys_and_OCIDs). + + If you have previously created a `.oci/config` file, the framework will automatically read this file at startup and load the credentials from the Default profile for authentication. To create one, follow the instructions available in the [Quickstart Documentation](https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/cliinstall.htm#Quickstart). + + After entering your credentials, click the `Save` button. If the credentials are correct, a green confirmation pop-up will appear, indicating successful authentication to your tenancy. + + ![OCI credentials success](images/oci-credentials-success.png) + +## Learn More + +* (optional) Now click the **Clear** button under the **History and Context** section, and choose the other available LLM, **gpt-4o-mini**. Then, ask the same questions and compare the results. Note that the History is enabled by default. The **Clear** button resets the *context window* and starts a fresh interaction with a model. + +* (optional) Play with the **Temperature** parameter (and also the other parameters if you wish to!) and compare the quality of the answers, for each LLM that is available. Clear the history by pressing the **Clear** button after each cycle. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Lorenzo De Marchis, June 2025 diff --git a/ai-optimizer/desktop/explore/files/starter-file.sql b/ai-optimizer/desktop/explore/files/starter-file.sql new file mode 100644 index 00000000..9d7e762e --- /dev/null +++ b/ai-optimizer/desktop/explore/files/starter-file.sql @@ -0,0 +1,101 @@ +/* NOTE: Files cannot contain empty lines (line breaks) */ +/* Specify the base URL that you copied from your files in OCI Object Storage in the define base_URL line below*/ +/* change idthydc0kinr to your real namespace. The name is case-sensitive. */ +/* change ADWCLab to your real bucket name. The name is case-sensitive. */ +/* change us-phoenix-1 to your real region name. The name is case-sensitive. */ +/* you can find these values on the OCI Console .. Storage .. Object Storage screen */ +set define on +define base_URL='https://objectstorage.us-phoenix-1.oraclecloud.com/n/idthydc0kinr/b/ADWCLab/o' +/* copy Channels table */ +begin + dbms_cloud.copy_data( + table_name =>'CHANNELS', + credential_name =>'OBJ_STORE_CRED', + file_uri_list =>'&base_URL/chan_v3.dat', + format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value 'true') + ); +end; +/ +/* copy Countries table */ +begin + dbms_cloud.copy_data( + table_name =>'COUNTRIES', + credential_name =>'OBJ_STORE_CRED', + file_uri_list =>'&base_URL/coun_v3.dat', + format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value 'true') + ); +end; +/ +/* Copy customers */ +begin + dbms_cloud.copy_data( + table_name =>'CUSTOMERS', + credential_name =>'OBJ_STORE_CRED', + file_uri_list =>'&base_URL/cust1v3.dat', + format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value 'true', 'dateformat' value 'YYYY-MM-DD-HH24-MI-SS') + ); +end; +/ +begin + dbms_cloud.copy_data( + table_name =>'SUPPLEMENTARY_DEMOGRAPHICS', + credential_name =>'OBJ_STORE_CRED', + file_uri_list =>'&base_URL/dem1v3.dat', + format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value 'true') + ); +end; +/ +begin + dbms_cloud.copy_data( + table_name =>'SALES', + credential_name =>'OBJ_STORE_CRED', + file_uri_list =>'&base_URL/dmsal_v3.dat', + format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value 'true', 'dateformat' value 'YYYY-MM-DD') + ); +end; +/ +begin + dbms_cloud.copy_data( + table_name =>'PRODUCTS', + credential_name =>'OBJ_STORE_CRED', + file_uri_list =>'&base_URL/prod1v3.dat', + format => json_object('delimiter' value '|', 'quote' value '^', 'ignoremissingcolumns' value 'true', 'dateformat' value 'YYYY-MM-DD-HH24-MI-SS', 'blankasnull' value 'true') + ); +end; +/ +begin + dbms_cloud.copy_data( + table_name =>'PROMOTIONS', + credential_name =>'OBJ_STORE_CRED', + file_uri_list =>'&base_URL/prom1v3.dat', + format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value 'true', 'dateformat' value 'YYYY-MM-DD-HH24-MI-SS', 'blankasnull' value 'true') + ); +end; +/ +begin + dbms_cloud.copy_data( + table_name =>'SALES', + credential_name =>'OBJ_STORE_CRED', + file_uri_list =>'&base_URL/sale1v3.dat', + format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value 'true', 'dateformat' value 'YYYY-MM-DD', 'blankasnull' value 'true') + ); +end; +/ +begin + dbms_cloud.copy_data( + table_name =>'TIMES', + credential_name =>'OBJ_STORE_CRED', + file_uri_list =>'&base_URL/time_v3.dat', + format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value 'true', 'dateformat' value 'YYYY-MM-DD-HH24-MI-SS', 'blankasnull' value 'true') + ); +end; +/ +begin + dbms_cloud.copy_data( + table_name =>'COSTS', + credential_name =>'OBJ_STORE_CRED', + file_uri_list =>'&base_URL/costs.dat', + format => json_object('ignoremissingcolumns' value 'true', 'dateformat' value 'YYYY-MM-DD', 'blankasnull' value 'true') + ); +end; +/ diff --git a/ai-optimizer/desktop/explore/images/addllama32.png b/ai-optimizer/desktop/explore/images/addllama32.png new file mode 100644 index 00000000..89844627 Binary files /dev/null and b/ai-optimizer/desktop/explore/images/addllama32.png differ diff --git a/ai-optimizer/desktop/explore/images/chat.jpg b/ai-optimizer/desktop/explore/images/chat.jpg new file mode 100644 index 00000000..d8f21e75 Binary files /dev/null and b/ai-optimizer/desktop/explore/images/chat.jpg differ diff --git a/ai-optimizer/desktop/explore/images/chatmodel.png b/ai-optimizer/desktop/explore/images/chatmodel.png new file mode 100644 index 00000000..73c6b9fc Binary files /dev/null and b/ai-optimizer/desktop/explore/images/chatmodel.png differ diff --git a/ai-optimizer/desktop/explore/images/database-config.png b/ai-optimizer/desktop/explore/images/database-config.png new file mode 100644 index 00000000..fa1ec707 Binary files /dev/null and b/ai-optimizer/desktop/explore/images/database-config.png differ diff --git a/ai-optimizer/desktop/explore/images/database-navigation.jpg b/ai-optimizer/desktop/explore/images/database-navigation.jpg new file mode 100644 index 00000000..93f55303 Binary files /dev/null and b/ai-optimizer/desktop/explore/images/database-navigation.jpg differ diff --git a/ai-optimizer/desktop/explore/images/emb.png b/ai-optimizer/desktop/explore/images/emb.png new file mode 100644 index 00000000..99f00981 Binary files /dev/null and b/ai-optimizer/desktop/explore/images/emb.png differ diff --git a/ai-optimizer/desktop/explore/images/llms.png b/ai-optimizer/desktop/explore/images/llms.png new file mode 100644 index 00000000..b93b7b1d Binary files /dev/null and b/ai-optimizer/desktop/explore/images/llms.png differ diff --git a/ai-optimizer/desktop/explore/images/models.jpg b/ai-optimizer/desktop/explore/images/models.jpg new file mode 100644 index 00000000..a6ad39a3 Binary files /dev/null and b/ai-optimizer/desktop/explore/images/models.jpg differ diff --git a/ai-optimizer/desktop/explore/images/oci-credentials-success.png b/ai-optimizer/desktop/explore/images/oci-credentials-success.png new file mode 100644 index 00000000..b21e8118 Binary files /dev/null and b/ai-optimizer/desktop/explore/images/oci-credentials-success.png differ diff --git a/ai-optimizer/desktop/explore/images/oci-navigation.jpg b/ai-optimizer/desktop/explore/images/oci-navigation.jpg new file mode 100644 index 00000000..09c5cb8c Binary files /dev/null and b/ai-optimizer/desktop/explore/images/oci-navigation.jpg differ diff --git a/ai-optimizer/desktop/explore/images/openai-api.png b/ai-optimizer/desktop/explore/images/openai-api.png new file mode 100644 index 00000000..e72162ba Binary files /dev/null and b/ai-optimizer/desktop/explore/images/openai-api.png differ diff --git a/ai-optimizer/desktop/explore/images/selectai-profiles.png b/ai-optimizer/desktop/explore/images/selectai-profiles.png new file mode 100644 index 00000000..a8ea36bb Binary files /dev/null and b/ai-optimizer/desktop/explore/images/selectai-profiles.png differ diff --git a/ai-optimizer/desktop/explore/images/toolkit-menu.png b/ai-optimizer/desktop/explore/images/toolkit-menu.png new file mode 100644 index 00000000..bd9a6792 Binary files /dev/null and b/ai-optimizer/desktop/explore/images/toolkit-menu.png differ diff --git a/ai-optimizer/desktop/get-started/files/db-initialize.sql b/ai-optimizer/desktop/get-started/files/db-initialize.sql new file mode 100644 index 00000000..d1b2a267 --- /dev/null +++ b/ai-optimizer/desktop/get-started/files/db-initialize.sql @@ -0,0 +1,11 @@ +alter system set vector_memory_size=512M scope=spfile; + +alter session set container=FREEPDB1; + +CREATE USER "WALKTHROUGH" IDENTIFIED BY OrA_41_OpTIMIZER + DEFAULT TABLESPACE "USERS" + TEMPORARY TABLESPACE "TEMP"; +GRANT "DB_DEVELOPER_ROLE" TO "WALKTHROUGH"; +ALTER USER "WALKTHROUGH" DEFAULT ROLE ALL; +ALTER USER "WALKTHROUGH" QUOTA UNLIMITED ON USERS; +EXIT; \ No newline at end of file diff --git a/ai-optimizer/desktop/get-started/get-started.md b/ai-optimizer/desktop/get-started/get-started.md new file mode 100644 index 00000000..987cd620 --- /dev/null +++ b/ai-optimizer/desktop/get-started/get-started.md @@ -0,0 +1,222 @@ +# Get started - Installation and Setup + +## Introduction + +The **AI Optimizer** is available to install in your own environment, which may be a developer’s desktop, on-premises data center environment, or a cloud provider. It can be run either on bare-metal, within a container, or in a Kubernetes Cluster. + +This walkthrough will guide you through a basic installation of the **Oracle AI Optimizer and Toolkit** (the **AI Optimizer**). It will allow you to experiment with GenAI, using Retrieval-Augmented Generation (RAG) with Oracle Database 23ai at the core. + +You will run four container images to establish the “Infrastructure”: + +* On-Premises LLM - **llama3.1** +* On-Premises Embedding Model - **mxbai-embed-large** +* Vector Storage - **Oracle Database 23ai Free** +* The AI Optimizer + +> **NOTE**: The walkthrough will reference podman commands. If applicable to your environment, podman can be substituted with docker. If you are using docker, make the walkthrough easier by aliasing the podman command: + +````bash + +alias podman=docker + +```` + +Estimated Time: 60 minutes + +### Objectives + +* Perform a full on-premise installation of the **Oracle AI Optimizer and Toolkit** + +### Prerequisites + +This lab assumes you have: + +* An Integrated Development Editor (like **Visual Studio Code**) +* Python 3.11 +* Container Runtime e.g. docker/podman (for running in a Container) +* Internet Access (docker.io and container-registry.oracle.com) +* **100G** of free disk space. +* **12G** of usable memory. +* Sufficient GPU/CPU resources to run the LLM, embedding model, and database + +## Task 1: LLM - llama3.1 + +To enable the _ChatBot_ functionality, access to a **LLM** is required. This workshop will use [Ollama](https://ollama.com/) to run the _llama3.1_ **LLM**. + +1. Start the _Ollama_ container: + + The Container Runtime is native: + + ```bash + + podman run -d --gpus=all -v ollama:$HOME/.ollama -p 11434:11434 --name ollama docker.io/ollama/ollama + + ``` + + If you don't have access to a GPU, you will have to omit the '--gpus=all' parameter: + + ```bash + + podman run -d -v ollama:$HOME/.ollama -p 11434:11434 --name ollama docker.io/ollama/ollama + + ``` + + If you created an unusable podman container by mistake, you can remove any unused volumes with this command: + + ```bash + + podman volume prune + + ``` + + > **Note:** + AI Runners like Ollama, LM Studio, etc. will not utilize Apple Silicon's "Metal" GPU when running in a container. This may change as the landscape evolves. You can install and run Ollama natively outside a container and it will take advantage of the "Metal" GPU. Later in the Workshop, when configuring the models, the API URL for the Ollama model will be your hosts IP address. + +2. Pull the **LLM** into the container: + + ```bash + + podman exec -it ollama ollama pull llama3.1 + + ``` + +3. Test the **LLM**: + + ```bash + + curl http://127.0.0.1:11434/api/generate -d '{ + "model": "llama3.1", + "prompt": "Why is the sky blue?", + "stream": false + }' + + ``` + + Unfortunately, if the above `curl` does not respond within 5-10 minutes, the rest of the workshop will be unbearable. + If this is the case, please consider using different hardware. + +## Task 2: Embedding - mxbai-embed-large + +To enable the **RAG** functionality, access to an embedding model is required. In this workshop you will use [Ollama](https://ollama.com/) to run the _mxbai-embed-large_ embedding model. + +1. Pull the embedding model into the container: + + ```bash + + podman exec -it ollama ollama pull mxbai-embed-large + + ``` + +## Task 3: The AI Optimizer + +The **AI Optimizer** provides an easy to use front-end for experimenting with **LLM** parameters and **RAG**. + +1. Download and Unzip the latest version of the **AI Optimizer**: + + ```bash + + curl -L -o ai-optimizer.tar.gz https://github.com/oracle-samples/ai-optimizer/archive/refs/heads/main.tar.gz + mkdir ai-optimizer + tar zxf ai-optimizer.tar.gz --strip-components=1 -C ai-optimizer + + ``` + +2. Create and activate a Python Virtual Environment: + + ```bash + + cd ai-optimizer/src + python3.11 -m venv .venv --copies + source .venv/bin/activate + pip3.11 install --upgrade pip wheel setuptools + + ``` + +3. Install the Python modules: + + ```bash + + pip3.11 install -e ".[all]" + source .venv/bin/activate + + ``` + +4. Start the AI Optimizer: + + ```bash + + streamlit run launch_client.py --server.port 8501 + + ``` + + If you are running on a remote host, you may need to allow access to the `8501` port. + + For example, in Oracle Linux 8/9 with `firewalld`: + + ```bash + + firewall-cmd --zone=public --add-port=8501/tcp + + ``` + +## Task 4: Vector Storage - Oracle Database 23ai Free + +AI Vector Search in Oracle Database 23ai provides the ability to store and query by similarity text and multimedia data. The AI Optimizer uses these capabilities to provide more accurate and relevant **LLM** responses via Retrieval-Augmented Generation (**RAG**). [Oracle Database 23ai Free](https://www.oracle.com/uk/database/free/get-started/) provides an ideal, no-cost vector store for this workshop. + +To start Oracle Database 23ai Free: + +1. Start the container: + + ```bash + + podman run -d --name ai-optimizer-db -p 1521:1521 container-registry.oracle.com/database/free:latest + + ``` + +2. Alter the `vector_memory_size` parameter and create a [new database user](../client/configuration/db_config#database-user): + + ```bash + + podman exec -it ai-optimizer-db sqlplus '/ as sysdba' + + ``` + + ```sql + alter system set vector_memory_size=512M scope=spfile; + + alter session set container=FREEPDB1; + + CREATE USER "WALKTHROUGH" IDENTIFIED BY OrA_41_OpTIMIZER + DEFAULT TABLESPACE "USERS" + TEMPORARY TABLESPACE "TEMP"; + GRANT "DB_DEVELOPER_ROLE" TO "WALKTHROUGH"; + ALTER USER "WALKTHROUGH" DEFAULT ROLE ALL; + ALTER USER "WALKTHROUGH" QUOTA UNLIMITED ON USERS; + EXIT; + ``` + +3. Bounce the database for the `vector_memory_size` to take effect: + + ```bash + + podman container restart ai-optimizer-db + + ``` + +Now you are all set! With the "Infrastructure" in-place, you are ready to configure the AI Optimizer. + +In a web browser, navigate to `http://localhost:8501` . + +![Chatbot](images/chatbot-no-models.png) + +Notice that there are no language models configured to use. We will deal with the configuration in the next Lab. + +> **NOTE**: In the next steps of this lab, you will need to check the items inside your database 23ai. This lab uses the VS Code **SQL Developer** plugin, but feel free to use the tool you prefer: + +![sql-developer-plugin](images/sql-developer-plugin.png) + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Lorenzo De Marchis, June 2025 diff --git a/ai-optimizer/desktop/get-started/images/chatbot-no-models.png b/ai-optimizer/desktop/get-started/images/chatbot-no-models.png new file mode 100644 index 00000000..3ebcd554 Binary files /dev/null and b/ai-optimizer/desktop/get-started/images/chatbot-no-models.png differ diff --git a/ai-optimizer/desktop/get-started/images/sql-developer-plugin.png b/ai-optimizer/desktop/get-started/images/sql-developer-plugin.png new file mode 100644 index 00000000..795caa33 Binary files /dev/null and b/ai-optimizer/desktop/get-started/images/sql-developer-plugin.png differ diff --git a/ai-optimizer/desktop/introduction/introduction.md b/ai-optimizer/desktop/introduction/introduction.md new file mode 100644 index 00000000..54e91a49 --- /dev/null +++ b/ai-optimizer/desktop/introduction/introduction.md @@ -0,0 +1,42 @@ +# Introduction + +## About this Workshop + +The **Oracle AI Optimizer and Toolkit** (the **AI Optimizer**) provides a streamlined environment where developers and data scientists can explore the potential of Generative Artificial Intelligence (**GenAI**) combined with Retrieval-Augmented Generation (**RAG**) capabilities. By integrating Oracle Database AI Vector Search, the AI Optimizer enables users to enhance existing Large Language Models (**LLMs**) through RAG. This method significantly improves the performance and accuracy of AI models, helping to avoid common issues such as knowledge cutoff and hallucinations. + +* **GenAI**: Powers the generation of text, images, or other data based on prompts using pre-trained LLMs. +* **RAG**: Enhances LLMs by retrieving relevant, real-time information allowing models to provide up-to-date and accurate responses. +* **Vector Database**: A database, including Oracle Database 23ai, that can natively store and manage vector embeddings and handle the unstructured data they describe, such as documents, images, video, or audio. + +Estimated Time: 120 minutes + +### Objectives + +In this workshop, you will learn how to: + +* Install the **Oracle AI Optimizer and Toolkit** (the **AI Optimizer**) +* Explore the AI Optimizer environment +* Embed documents in a vector format within **Oracle Database 23ai** +* Use Retrieval Augmented Generation (**RAG**) techniques +* Experiment with different models, parameters and techniques +* Evaluate model performances with the *Testbed* functionality +* Export a RAG-based *Springboot* Microservice +* Use the AI Optimizer as an API Server + +### Prerequisites + +This lab assumes you have: + +* An **Oracle Database 23ai** up and running (see ["Get Started"](https://markxnelson.github.io/developer/ai-optimizer/workshops/desktop/index.html?lab=get-started) to get one) +* An Integrated Development Editor (like Visual Studio Code) +* Python 3.11 (for running Bare-Metal) +* Container Runtime e.g. docker/podman (for running in a Container) +* Access to an Embedding and Chat Model: + * On-Premises Models + * (optional) API Keys for Third-Party Models (e.g., OpenAI API Keys that you can get [here](https://platform.openai.com/settings/organization/api-keys)) + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Lorenzo De Marchis, May 2025 diff --git a/ai-optimizer/desktop/microservice/images/diff-llm-springai.png b/ai-optimizer/desktop/microservice/images/diff-llm-springai.png new file mode 100644 index 00000000..7a2aef82 Binary files /dev/null and b/ai-optimizer/desktop/microservice/images/diff-llm-springai.png differ diff --git a/ai-optimizer/desktop/microservice/images/download-springai.png b/ai-optimizer/desktop/microservice/images/download-springai.png new file mode 100644 index 00000000..9b6ddac7 Binary files /dev/null and b/ai-optimizer/desktop/microservice/images/download-springai.png differ diff --git a/ai-optimizer/desktop/microservice/images/microservice-started.png b/ai-optimizer/desktop/microservice/images/microservice-started.png new file mode 100644 index 00000000..d52cd9f8 Binary files /dev/null and b/ai-optimizer/desktop/microservice/images/microservice-started.png differ diff --git a/ai-optimizer/desktop/microservice/images/select-test1.png b/ai-optimizer/desktop/microservice/images/select-test1.png new file mode 100644 index 00000000..126822f9 Binary files /dev/null and b/ai-optimizer/desktop/microservice/images/select-test1.png differ diff --git a/ai-optimizer/desktop/microservice/microservice.md b/ai-optimizer/desktop/microservice/microservice.md new file mode 100644 index 00000000..72b3fcb1 --- /dev/null +++ b/ai-optimizer/desktop/microservice/microservice.md @@ -0,0 +1,130 @@ +# (Optional) Generating Microservice Code + +## Introduction + +The AI Optimizer allows you to export the configured chatbot as a ready-to-run microservice built using Java, Spring Boot and the Spring AI framework. This microservice runs independently from the AI Optimizer, relying solely on the created vector store table and the selected LLM servers. + +In the current release, only fully self-contained Ollama configurations (embedding + chat model) or OpenAI-based configurations are supported. + +Estimated Time: 10 minutes + +### Objectives + +In this lab, you will: + +* Export a RAG configuration as a *SpringAI* microservice +* Start the Springboot microservice +* Test the RAG-based microservice + +### Prerequisites + +To run the microservice exported you need: + +* JDK 21.x +* Apache Maven 3.8.x +* curl command + +## Task 1: Export the SpringAI Microservice (Ollama version) + +* First, navigate to the *Chat* tab, select **llama3.1** as the **Chat model** and choose **TEST1** as the vector store alias: + + ![select-test1](./images/select-test1.png) + + This configuration will use the Ollama LLM server provider for both the embedding model and the chat model. + +* Now navigate to the *Settings* tab from the left side menu. Here, you should find the *Download SpringAI* button available. + + If you see a message like this: + + ![notollama](./images/diff-llm-springai.png) + + don't worry — simply select **llama3.1** as the Chat model, and the button will appear. + +* Click the `Download SpringAI` button to export the Ollama-based microservice: + + ![download-springai](./images/download-springai.png) + +## Task 2: Start the SpringAI microservice + +If you have completed *Task 1* from this lab, you should have downloaded a *spring_ai.zip* file. Follow these steps to start it: + +* Unzip the file in a subdirectory + +* Open a terminal window, navigate to the subdirectory, and set executable permissions on `start.sh` using the command: `chmod 755 ./start.sh`. + +* Now start the microservice with: + + ```bash + + ./start.sh + + ``` + + If the Spring Boot application starts successfully, you should see something like this in your terminal logs + + ![microservice-started](./images/microservice-started.png) + +Now you are all set for testing the microservice! + +## Task 3: Test the Microservice + +The microservice you just started exposes a web service that accepts HTTP GET requests at: + +* `http://localhost:9090/v1/chat/completions` — to use RAG via the OpenAI-compatible REST API. +* `http://localhost:9090/v1/service/llm` — to chat directly with the underlying LLM. +* `http://localhost:9090/v1/service/search/`— to search for documents similar to the message provided. + +To test it, run a curl command like this in a new terminal: + + ```bash + + curl -N http://localhost:9090/v1/chat/completions \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer your_api_key" \ + -d '{ + "model": "server", + "messages": [{"role": "user", "content": "In Oracle Database 23ai, how do I determine the accuracy of my vector indexes?"}], + "stream": false + }' + + ``` + +The response using RAG with the TEST1 vector store will look like this: + + ``` json + {"id":"chatcmpl-iHfRbDEYySEuGLESa8faQKZbcpVQ","object":"chat.completion","created":"1746805121","model":"llama3.1", + + "choices":[{"message":{"role":"assistant","content":"According to the documentation, you can determine the accuracy of your vector indexes using the following methods:\n\n1. Use the `DBMS_VECTOR.INDEX_ACCURACY_QUERY` function to verify the accuracy of a vector index for a given query vector, top-K, and target accuracy.\n2. Use the `DBMS_VECTOR.INDEX_ACCURACY_REPORT` function to capture from your past workloads, accuracy values achieved by approximate searches using a particular vector index for a certain period of time.\n\nAdditionally, you can use the `GET_INDEX_STATUS` procedure to get information about the current status of a vector index, such as its stage and percentage completion."}, + + "index":0,"finish_reason":null}]} + ``` + +You can see how the microservice retrieves relevant context from the vector store, producing an answer similar to the one previously obtained using the RAG functionality within the AI Optimizer. + +You can also send a curl request without leveraging RAG: + + ```bash + + curl --get --data-urlencode 'message=In Oracle Database 23ai, how do I determine the accuracy of my vector indexes?' localhost:9090/v1/service/llm + + ``` + + and it will produce an ungrounded answer like this: + + ```json + {"completion":"There is no such thing as \"Oracle Database 23ai\". The current version of Oracle Database is Oracle Database 21c and Oracle Database 22c.\n\nHowever, if you're using a recent version of Oracle Database (18c or later), you can use the `DBMS_VECTOR_INDEX_STATS` package to gather statistics about your vector indexes. This package provides functions to retrieve information about the accuracy of the vector index, such as:\n\n* `GET_BUCKETS_ACCURACY`: Returns the number of buckets that have an accuracy greater than or equal to a specified threshold.\n* `GET_INVERTED_LIST_ACCURACY`: Returns the accuracy of the inverted list for each bucket.\n\nTo use these functions, you'll need to create a vector index on your table and gather statistics about it using the following steps:\n\n1. Create a vector index on your table:\n```sql\nCREATE INDEX my_vector_index ON my_table (my_column) VECTORIZED;\n```\n2. Gather statistics about the vector index:\n```sql\nBEGIN\n DBMS_VECTOR_INDEX_STATS.GATHER_TABLE_STATS('MY_SCHEMA', 'MY_TABLE');\nEND;\n```\n3. Use the `DBMS_VECTOR_INDEX_STATS` package to retrieve information about the accuracy of the vector index:\n```sql\nSELECT GET_BUCKETS_ACCURACY(my_vector_index, 0.5) FROM DUAL; \n-- Returns the number of buckets with an accuracy greater than or equal to 0.5\n```\nPlease note that this is a simplified example and you may need to adjust the syntax depending on your specific use case.\n\nAlso, keep in mind that vector indexes are a feature introduced in Oracle Database 18c, so if you're using an earlier version of Oracle, you won't have access to these features."} + ``` + +You can see that the answer is very generic compared to the RAG-enhanced one. + +## Task 4 (Optional): Execute the OpenAI variant + +> **NOTE**: You can perform this task only if you have access to [OpenAI APIs](https://platform.openai.com/settings/organization/api-keys). + +Proceed as in the previous step, selecting **TEST2** as the vector store alias and **gpt-4o-mini** as the **Chat model**. In the terminal where you run the Spring Boot microservice, make sure that your **OPENAI API KEY** is correctly set as environment variable. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Lorenzo De Marchis, May 2025 diff --git a/ai-optimizer/desktop/other-livelabs/other-livelabs.md b/ai-optimizer/desktop/other-livelabs/other-livelabs.md new file mode 100644 index 00000000..6835d049 --- /dev/null +++ b/ai-optimizer/desktop/other-livelabs/other-livelabs.md @@ -0,0 +1,7 @@ +# Other LiveLabs you might like + +- [Autonomous Database Dedicated](https://livelabs.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=677) + +- [Manage and Monitor Autonomous Database](https://livelabs.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=553) + +- [Scaling and Performance in the Autonomous Database](https://livelabs.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=608) diff --git a/ai-optimizer/desktop/prepare/images/database-vector-store-list.png b/ai-optimizer/desktop/prepare/images/database-vector-store-list.png new file mode 100644 index 00000000..7d2fcbd8 Binary files /dev/null and b/ai-optimizer/desktop/prepare/images/database-vector-store-list.png differ diff --git a/ai-optimizer/desktop/prepare/images/populate-vector-store.png b/ai-optimizer/desktop/prepare/images/populate-vector-store.png new file mode 100644 index 00000000..975ed16b Binary files /dev/null and b/ai-optimizer/desktop/prepare/images/populate-vector-store.png differ diff --git a/ai-optimizer/desktop/prepare/images/query-vector-db.png b/ai-optimizer/desktop/prepare/images/query-vector-db.png new file mode 100644 index 00000000..1f303b6e Binary files /dev/null and b/ai-optimizer/desktop/prepare/images/query-vector-db.png differ diff --git a/ai-optimizer/desktop/prepare/images/split-embed.jpg b/ai-optimizer/desktop/prepare/images/split-embed.jpg new file mode 100644 index 00000000..af3d0bcc Binary files /dev/null and b/ai-optimizer/desktop/prepare/images/split-embed.jpg differ diff --git a/ai-optimizer/desktop/prepare/images/text-embedding-3-small.png b/ai-optimizer/desktop/prepare/images/text-embedding-3-small.png new file mode 100644 index 00000000..f90e82e6 Binary files /dev/null and b/ai-optimizer/desktop/prepare/images/text-embedding-3-small.png differ diff --git a/ai-optimizer/desktop/prepare/images/vector-store-populated.png b/ai-optimizer/desktop/prepare/images/vector-store-populated.png new file mode 100644 index 00000000..9184b700 Binary files /dev/null and b/ai-optimizer/desktop/prepare/images/vector-store-populated.png differ diff --git a/ai-optimizer/desktop/prepare/prepare.md b/ai-optimizer/desktop/prepare/prepare.md new file mode 100644 index 00000000..fa9f619e --- /dev/null +++ b/ai-optimizer/desktop/prepare/prepare.md @@ -0,0 +1,99 @@ +# Prepare the Vector Store + +## Introduction + +In this lab, we will perform a first vector store ingestion to test that it works. We will split&embed one document and store its vector embeddings inside the Oracle 23ai Database set up earlier. We will then be able to inspect the content of the tables that will be created. + +Estimated Lab Time: 15 minutes + +### Objectives + +In this lab, you will: + +* Split/Embed Documents and create a vector store +* Inspect the Vector Database content + +### Prerequisites + +This lab assumes you have: + +* All previous labs successfully completed + +## Task 1: Split/Embed Documents + +In the *Split/Embed* tab, the framework allows you to upload various types of documents and transform their content into vector embeddings in a format that is interpretable by LLMs. + +1. Navigate to the *Split/Embed* tab + + ![split-embed-interface](./images/split-embed.jpg) + +2. Choose the embedding model and its parameters + + Here, you can choose from the embedding models you selected during the configuration phase we had in Lab 1 using a drop-down menu. + For this example, choose **mxbai-embed-large**. + + The **chunk size** defines the length of each segment into which the document will be split, while the **chunk overlap** represents the percentage of overlap between consecutive chunks relative to the chunk size. + + Additionally, you can select different distance metrics and index types to experiment with various vector representations of the same document, allowing you to identify the configuration that best meets your needs. + +3. Select your document source + + Once the embedding model has been selected, scroll down to the *Load and Split Documents* section to upload the document you wish to store in your **Oracle Database 23ai**. + + ![populate-vector-store](images/populate-vector-store.png) + + You can choose from three different file sources: + + * **OCI**: Navigate through your tenancy to select documents from the Object Storage. Ensure that your OCI credentials are properly configured in advance. + * **Local**: Upload a document directly from your local environment. + * **Web**: Import a document from a publicly accessible web URL. + + In this example, we will embed a document from the web, available at [this link](https://docs.oracle.com/en/database/oracle/oracle-database/23/vecse/ai-vector-search-users-guide.pdf). We will give the alias ***TEST1*** to this vector store. + +4. Populate the Vector Store. + + After selecting your document source, you can click on the **Populate Vector Store** button and start the embedding process. + + Once the process is complete, a green confirmation prompt will appear, indicating the number of chunks that have been generated and successfully stored in the database. + + ![vector-store-populated](images/vector-store-populated.png) + + This means that 3016 vectors representations of the information from the input document have been created and stored. + +## Task 2: Inspect the Vector DB + +Now that you've embedded your document, you can query the content of the Vector Store. In this example we will be using the SQL Developer plugin for VS Code, in order to connect to Oracle Database 23ai. + +In your VS Code IDE, click on the SQL Developer plugin icon to use it. Then, open a new SQL script and execute this sql command: + +```sql +select * from WALKTHROUGH.TEST1_MXBAI_EMBED_LARGE_512_103_COSINE_HNSW; +``` +You will then retrieve the rows from the newly created table. + + ![query-vector-db](images/query-vector-db.png) + +What you see in the image above are chunks of text from the input document, which have been transformed into vector format and stored in the Oracle database. Essentially, you’ve replicated the knowledge contained in the document within your database! + +## (Optional) Task 3: Create a different Vector Store + +> **NOTE**: You can perform this task only if you have access to [OpenAI APIs](https://platform.openai.com/settings/organization/api-keys). + +By following the sames steps from Task 1, you can create another vector store using the same document but with a different embedding model, **text-embedding-3-small** from the OpenAI models. Give the alias ***TEST2*** to this vector store. + +In this case, you will get a smaller number of chunks, since the model supports a chunk size of 8191 instead of the 512 given by *mxbai-embed-large*: + +![text-embedding-3-small](images/text-embedding-3-small.png) + +## Learn More + +* (optional) See the list of vector stores: + + You can navigate to the *Database* tab in the navigation menu to see the list of all the vector stores that have been created. If needed, you can easily delete them with a single click. + + ![database-vector-store-list](images/database-vector-store-list.png) + +## Acknowledgements +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes +* **Last Updated By** - Lorenzo De Marchis, May 2025 \ No newline at end of file diff --git a/ai-optimizer/desktop/rag/images/.gitkeep b/ai-optimizer/desktop/rag/images/.gitkeep new file mode 100644 index 00000000..e69de29b diff --git a/ai-optimizer/desktop/rag/images/enable-vector-search.png b/ai-optimizer/desktop/rag/images/enable-vector-search.png new file mode 100644 index 00000000..4eda62c4 Binary files /dev/null and b/ai-optimizer/desktop/rag/images/enable-vector-search.png differ diff --git a/ai-optimizer/desktop/rag/images/rag-query-2.png b/ai-optimizer/desktop/rag/images/rag-query-2.png new file mode 100644 index 00000000..d944b5d6 Binary files /dev/null and b/ai-optimizer/desktop/rag/images/rag-query-2.png differ diff --git a/ai-optimizer/desktop/rag/images/rag-query.png b/ai-optimizer/desktop/rag/images/rag-query.png new file mode 100644 index 00000000..34412fd9 Binary files /dev/null and b/ai-optimizer/desktop/rag/images/rag-query.png differ diff --git a/ai-optimizer/desktop/rag/images/reference.png b/ai-optimizer/desktop/rag/images/reference.png new file mode 100644 index 00000000..93d82a90 Binary files /dev/null and b/ai-optimizer/desktop/rag/images/reference.png differ diff --git a/ai-optimizer/desktop/rag/images/select-vector-store.png b/ai-optimizer/desktop/rag/images/select-vector-store.png new file mode 100644 index 00000000..6e124128 Binary files /dev/null and b/ai-optimizer/desktop/rag/images/select-vector-store.png differ diff --git a/ai-optimizer/desktop/rag/rag.md b/ai-optimizer/desktop/rag/rag.md new file mode 100644 index 00000000..81d00d51 --- /dev/null +++ b/ai-optimizer/desktop/rag/rag.md @@ -0,0 +1,93 @@ +# Using Retrieval Augmented Generation + +## Introduction + +Now that you’ve created the vector stores, you can begin testing the knowledge base built in *Lab 2*. In this Lab, you will test the knowledge stored in the `TEST1` vector store, which serves as an example of an augmented LLM based on publicly available sources. + +> **NOTE**: The answers received by any LLM are based on probabilistic models. Therefore, the answer received in your own testing might differ from the ones shown in this documentation. + +Estimated Time: 5 minutes + +### Objectives + +In this lab, you will: + +* Enable the RAG functionality in the *Chat* window +* Ask questions and receive answers enhanced by relevant context stored in the vector database +* Compare results across different configurations + +### Prerequisites + +This lab assumes you have: + +* All previous labs successfully completed + +## Task 1: Enable RAG + +To enable the *RAG* functionality, follow these steps: + +1. Navigate to the *ChatBot* tab from the left-side menu + +2. Select the **Vector Search** option from the **Toolkit** section and then select **llama3.1** as the chat model. + + ![enable-vector-search](./images/enable-vector-search.png) + +3. In the **Select Alias** dropdown, choose the `TEST1` vector store table. The remaining fields in the **Vector Store** will be automatically populated, as each one represents a search parameter used to configure the selected vector store. + + ![select-vector-store](./images/select-vector-store.png) + + In this case, selecting the alias is sufficient to retrieve the correct store. However, you also get visibility into the parameters that were used to create the chunks and their corresponding embedding vectors. + +## Task 2: Ask questions with RAG enabled + +Now that RAG is enabled, we can test it by asking the same questions submitted in *Lab 1*. Previously, the LLM provided generic answers, as the content was either not part of its training data or not sufficiently addressed in the questions. + +For this guided experiment, perform the following: + +1. Ask the ChatBot: + +```text + +In Oracle Database 23ai, how do I determine the accuracy of my vector indexes? + +``` + +![rag-query](images/rag-query.png) + +```text + +In Oracle Database 23ai, how do I convert string text to embeddings using PL/SQL? + +``` + +![rag-query-2](images/rag-query-2.png) + +You can see how the answers differ from the ones we received earlier. The responses should be different and include references to DBMS_VECTOR and links to the embedded documentation where this information can be found. + +You can also click on one of the *Reference* buttons, to see where the information was retrieved from in the file: + +![reference](images/reference.png) + +## (Optional) Task 3: Repeat Task 2 using OpenAI models + +> **NOTE**: You can perform this task only if you have access to [OpenAI APIs](https://platform.openai.com/settings/organization/api-keys). + +As an additional learning exercise, you can repeat the tests performed in Task 2 of this lab using OpenAI LLMs and vector stores: + +* Select `TEST2` as the vector store to run the questions against +* Choose **gpt-4o-mini** from the **Chat model** dropdown menu to have the same LLM provider. + +afterwards, compare the results to what you got while using *llama3.1*! + +## Learn More + +You can learn more about RAG by playing with more parameters and prompt techniques: + +* Play with the *Temperature* parameter to discover how much the answers provided get enriched. +* Follow up a question with another one asking for “more” to see that the chat history is taken into account when generating a new response. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Lorenzo De Marchis, June 2025 diff --git a/ai-optimizer/desktop/server/images/api-server-activity.png b/ai-optimizer/desktop/server/images/api-server-activity.png new file mode 100644 index 00000000..28a97b02 Binary files /dev/null and b/ai-optimizer/desktop/server/images/api-server-activity.png differ diff --git a/ai-optimizer/desktop/server/images/api-server-config.png b/ai-optimizer/desktop/server/images/api-server-config.png new file mode 100644 index 00000000..f0db2217 Binary files /dev/null and b/ai-optimizer/desktop/server/images/api-server-config.png differ diff --git a/ai-optimizer/desktop/server/images/api-server-settings.png b/ai-optimizer/desktop/server/images/api-server-settings.png new file mode 100644 index 00000000..47534007 Binary files /dev/null and b/ai-optimizer/desktop/server/images/api-server-settings.png differ diff --git a/ai-optimizer/desktop/server/images/curl-response.png b/ai-optimizer/desktop/server/images/curl-response.png new file mode 100644 index 00000000..a83e46f7 Binary files /dev/null and b/ai-optimizer/desktop/server/images/curl-response.png differ diff --git a/ai-optimizer/desktop/server/server.md b/ai-optimizer/desktop/server/server.md new file mode 100644 index 00000000..9dbdb1c9 --- /dev/null +++ b/ai-optimizer/desktop/server/server.md @@ -0,0 +1,96 @@ +# (Optional) Using the API Server + +## Introduction + +In this lab, you will test the functionality of the API Server. The AI Optimizer uses an API Server to make its features accessible to external clients. You can run the API Server either as part of the AI Optimizer GUI client or as a standalone process. + +Estimated Time: 10 minutes + +### Objectives + +In this lab, you will: + +* Start the AI Optimizer as an API Server +* Review the configuration parameters +* Make an API call to the server + +### Prerequisites + +This lab assumes you have: + +* All previous labs successfully completed + +## Task 1: Start the API Server + +Navigate to the *API Server* tab from the left-hand pane: + +![Server Configuration](images/api-server-config.png) + +You’ll see a `Restart Server` button (in red). This is because the API Server starts automatically with the AI Optimizer. Follow the steps below to edit the server configuration and restart it: + +* Change the API Server Port to 8001 +* Update the API Server Key (e.g., abcd, or any value you prefer) +* Click the Restart Server button to apply the new parameters + +## Task 2: Inspect Server Configuration + +When the API Server starts, it creates a default `server` client with minimal settings. This client is used when calling the API Server externally, outside of the AI Optimizer GUI. + +To copy your current GUI client settings to the server client for external use, click the Copy AI Optimizer Settings button. + +![Server Settings](images/api-server-settings.png) + +You can inspect the configuration of the `server` client by expanding the `{...}` brackets. + +## Task 3: Perform an API call to the server + +Now that the API Server is running, you can perform API calls against it. + +* Open a terminal window in your preferred IDE + +* Copy and run the following curl command in the terminal: + + ```bash + + #Bearer Key to be set accordingly to task 1. + curl -X POST "http://localhost:8001/v1/chat/completions" \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer abcd" \ + -H "client: server" \ + -d '{ + "model": "llama3.1", + "messages": [ + { + "role": "user", + "content": "In Oracle Database 23ai, how do I determine the accuracy of my vector indexes?" + } + ] + }' | jq . + + ``` + + Here, you're essentially repeating the same type of request as in *Lab 3*. You’ve passed the API Server Key you configured in Task 1 along with the question content as parameters. You should receive a response similar to the screenshot below: + + ![curl-response](images/curl-response.png) + + As you can see, the API Server responded with a generic answer—this is expected because RAG was not enabled. You can enable RAG just like you did in the previous labs, and the API Server will provide more context-aware responses. + +## Learn More + +* Enable RAG from the Chatbot tab, then repeat Task 3. Observe how the API Server's response changes with RAG enabled. + +* To explore the Oracle AI Optimizer and Toolkit API documentation, visit: + + ```bash + + http://localhost:8001/v1/docs# + + ``` + + > **Note**: The port number in the URL (in this case 8001) must match the API Server Port you configured in Task 1 of this lab. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Lorenzo De Marchis, May 2025 diff --git a/ai-optimizer/getting_started-30_testset.json b/ai-optimizer/getting_started-30_testset.json new file mode 100644 index 00000000..18b8a04f --- /dev/null +++ b/ai-optimizer/getting_started-30_testset.json @@ -0,0 +1,422 @@ +[ + { + "id":"7b6cf8d4-44bc-4c81-8221-3ca3bd6e0a33", + "question":"What are the two types of users in the HR Web Application?", + "reference_answer":"The two users in the HR Web Application are hradmin and hrstaff.", + "reference_context":"Document 52: 9\nCreate Login and Logout Functionality\nThe HR Web Application has two users, namely hradmin and hrstaff.\nOnce you login to the HR Application, you see the landing page, with details of the web\napplication. The hradmin and hrstaff have different privileges and access to different\nfeatures.\nThis chapter shows the required classes and additional code required to build the Login and\nLogout functionality in the application.\n\u2022 Create a XML file tomcat-users.xml for login functionality.\n\u2022 Create a HTML page login.html to login the user.\n\u2022 Create a HTML page login-failed.html to display the error message.\n\u2022 Create a web.xml to authenticate the users during login.\n\u2022 Create a HTML page about.html to show more details about the application.\n\u2022 Create a landing page index.html and define the html pages for redirection.\n\u2022 Add code to the servlet WebController.java to process logout.\n9-1", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"52", + "topic":"Login and Logout Functionality" + } + }, + { + "id":"3cecbe31-3eee-469c-955c-70d32295cd9c", + "question":"What should you do after updating the connection variable with your database connection details?", + "reference_answer":"In the File menu, select Save All to save the changes.", + "reference_context":"Document 21: 4. Update the connection variable with the details of your database connection.\n5. In the File menu, select Save All to save the changes.\n6. In the Maven window on the right, double-click on clean to clean the source code.\nThe build progress is shown in the Run window at the bottom.\n7. Similarly, double-click on compile and package in the same order.\nChapter 4\nCompile the Application in IntelliJ\n4-6", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"21", + "topic":"Others" + } + }, + { + "id":"b96db39f-42cb-4e33-a104-bdb8a75e9ba3", + "question":"What should you do after adding the dependency under the tag in the pom.xml file?", + "reference_answer":"In the Project window, expand the src folder, expand main and then Java folder. Under Java, expand com.oracle.jdbc.samples. In the bean folder, double-click JdbcBeanImpl.", + "reference_context":"Document 20: 2. Add the following dependency under the tag in pom.xml file. See JDBC\nDrivers for more information.\n3. In the Project window, expand the src folder, expand main and then Java folder. Under\nJava, expand com.oracle.jdbc.samples. In the bean folder, double-click\nJdbcBeanImpl.\nChapter 4\nCompile the Application in IntelliJ\n4-5", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"20", + "topic":"Java Web Application Development" + } + }, + { + "id":"91c78fc2-a0d9-45fb-9c89-25053d316fd9", + "question":"What happens when the edit button is pressed for an employee record?", + "reference_answer":"If the edit button is pressed already, the function will make the column readonly and allow the user to edit the field names once the employee record is displayed.", + "reference_context":"Document 45: +' No records found for the given Fist_Name<\/div>'\n }\n else {\n var i;\n var out = \"\";\n \/\/ keys is global so that it can be used later as well\n keys = Object.keys(arr[0]);\n \/\/ Print headers\n out += \"
Trash<\/th>Edit<\/th>\"\n for(i = 0; i < keys.length; ++i) {\n out += \"\"+keys[i]+\"<\/th>\"\n }\n out += \"<\/tr>\";\n \/\/ Print values\n for(j = 0; j < arr.length; j++) {\n pk = arr[j][keys[0]];\n out += '
'\n +'<\/span>'\n +'<\/a><\/td>'\n +''\n +'<\/span>'\n +'<\/a><\/td>';\n \/\/ 0 is the primary key\n for(i = 0; i < keys.length; ++i) {\n \/\/ creating an id to each column\n out += \" \"+arr[j][keys[i]]+\"<\/\ntd>\";\n }\n out += \"<\/tr>\"\n }\n out += \"<\/table>\";\n }\n $('#id-emp').html(out);\n}\n5. Add the allowEditSalary(pk) function to make the field names editable once the employee\nrecord is displayed.\nfunction allowEditSalary(pk) {\n \/\/ If the edit button is pressed already\n if(typeof currentPK != 'undefined' && currentPK == pk) {\n console.log('Make column readonly');\n for(i = 1; i < keys.length; ++i) {\n var x = '#' +pk +\"_\" +keys[i];\n var value = $(x).text().trim();\n console.log(value);\n $(x).val(value);\n }\n $('#UpdateButton').hide();\nChapter 7\nCreate a New HTML for Search by Employee Id\n7-7", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"45", + "topic":"Java Employee Management" + } + }, + { + "id":"ed733096-71d5-400d-ab25-28748aa31a40", + "question":"What is the first step to update an employee record in the database?", + "reference_answer":"You must search for an employee in the records.", + "reference_context":"Document 39: 7\nUpdate an Employee Record\nThe Update functionality modifies an employee record in the database according to the user\nedits on the web page.\nFirst, you must search for an employee in the records. Once you retrieve the information\nrelated to the employee, you will find the Edit button to modify details related to the\nemployee.\nIn this chapter, you learn to add code required to build the Update functionality. You will learn\nhow to:\n1. Declare a new method getEmployeeByFn(String) in JavaBean.java.\n2. Declare a new method updateEmployee(int) in JavaBean.java.\n3. Implement a new method getEmployeeByFn(String) in JavaBeanImpl.java.\n4. Implement a new method updateEmployee(int) in JavaBeanImpl.java.\n5. Add new code to WebController.java to process the request and response.\n6. Create a HTML page listByName.html to display the results.\nNote:\nThe hradmin user has the privilege to update an employee record. The hrstaff\nuser does not have the privilege to update an employee record.\nDeclare a new method getEmployeeByFn(String) in\nEmployeeBean.java\nTo modify the details of an employee, the hradmin must first search for the employee based\non his\/her first name. The getEmployeeByFn(String) method searches employees based on\ntheir first name.\nClass Name: src\/main\/java\/com\/oracle\/jdbc\/samples\/bean\/EmployeeBean.java\nGithub Location: EmployeeBean.java\n7-1", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"39", + "topic":"Java Employee Management" + } + }, + { + "id":"86357f8a-550d-434e-b2f0-a1202c65da36", + "question":"What should you do if you find errors in the documentation?", + "reference_answer":"If you find any errors, please report them to us in writing.", + "reference_context":"Document 1: Oracle Database Get Started with Java Development, 23ai\nF47017-03\nCopyright \u00a9 2007, 2024, Oracle and\/or its affiliates.\nPrimary Author: Tulika Das\nContributing Authors: Apoorva Srinivas, Tanmay Choudhury\nContributors: Kuassi Mensah, Nirmala Sundarappa\nThis software and related documentation are provided under a license agreement containing restrictions on\nuse and disclosure and are protected by intellectual property laws. Except as expressly permitted in your\nlicense agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license,\ntransmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse\nengineering, disassembly, or decompilation of this software, unless required by law for interoperability, is\nprohibited.\nThe information contained herein is subject to change without notice and is not warranted to be error-free. If\nyou find any errors, please report them to us in writing.\nIf this is software, software documentation, data (as defined in the Federal Acquisition Regulation), or related\ndocumentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S.\nGovernment, then the following notice is applicable:\nU.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software,\nany programs embedded, installed, or activated on delivered hardware, and modifications of such programs)\nand Oracle computer documentation or other Oracle data delivered to or accessed by U.S. Government end\nusers are \"commercial computer software,\" \"commercial computer software documentation,\" or \"limited rights\ndata\" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental\nregulations. As such, the use, reproduction, duplication, release, display, disclosure, modification, preparation\nof derivative works, and\/or adaptation of i) Oracle programs (including any operating system, integrated\nsoftware, any programs embedded, installed, or activated on delivered hardware, and modifications of such\nprograms), ii) Oracle computer documentation and\/or iii) other Oracle data, is subject to the rights and\nlimitations specified in the license contained in the applicable contract. The terms governing the U.S.\nGovernment's use of Oracle cloud services are defined by the applicable contract for such services. No other\nrights are granted to the U.S. Government.\nThis software or hardware is developed for general use in a variety of information management applications.\nIt is not developed or intended for use in any inherently dangerous applications, including applications that\nmay create a risk of personal injury. If you use this software or hardware in dangerous applications, then you\nshall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its\nsafe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this\nsoftware or hardware in dangerous applications.\nOracle\u00ae, Java, MySQL and NetSuite are registered trademarks of Oracle and\/or its affiliates. Other names\nmay be trademarks of their respective owners.\nIntel and Intel Inside are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are\nused under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Epyc,\nand the AMD logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered\ntrademark of The Open Group.\nThis software or hardware and documentation may provide access to or information about content, products,\nand services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly\ndisclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise\nset forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be\nresponsible for any loss, costs, or damages incurred due to your access to or use of third-party content,\nproducts, or services, except as set forth in an applicable agreement between you and Oracle.", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"1", + "topic":"Java Web Application Development" + } + }, + { + "id":"ce5fd492-f8c5-4454-90dd-207e331146ab", + "question":"What is the purpose of the variable FN_KEY in the WebController class?", + "reference_answer":"The variable FN_KEY is declared to capture the first name of the employee and is a global variable, hence, it is declared outside the method processRequest() but within the WebController class.", + "reference_context":"Document 43: 2. Declare a variable FN_KEY to capture first name of the employee. This is a global variable,\nhence, declare it outside the method processRequest() but within the WebController\nclass.\nprivate static final String FN_KEY = \"firstName\";\n3. The method processRequest() is already created in the ListAll feature. Now, we add the\ncode to implement Update an Employee functionality. Add an ELSEIF condition to\nhandle the new functionality. Get the employee id entered by the user and invoke the\nmethod getEmployee(int) to verify if the employee record exists.\nif ((value = request.getParameter(ID_KEY)) != null) {\n int empId = Integer.valueOf(value).intValue();\n employeeList = employeeBean.getEmployee(empId);\n } \n\/* New code added below *\/\nelse if ((value = request.getParameter(FN_KEY)) != null) {\n employeeList = jdbcBean.getEmployeeByFn(value);\n}\nelse { \n \/* Previously used getEmployees() method for Listall feature *\/\n employeeList = employeeBean.getEmployees();\n }\nCreate a New HTML for Search by Employee Id\nA HTML page that shows an input placeholder for the user to enter the employee first name.\nIf the employee record is found, then the details of the employee is displayed on the page,\notherwise, an error message will be displayed.\nClass Name:src\/main\/webapp\/listById.html\nGithub Location: listByName.html\nSteps to create the HTML page:\n1. Create the title, stylesheet, and body for the HTML page.\n\n\n\n\nList Employee by Id<\/title>\n<!-- Specify the stylesheet here -->\n<link rel=\"stylesheet\" type=\"text\/css\" href=\"css\/app.css\" >\n<!-- Bootstrap JS for the UI -->\n<link rel=\"stylesheet\" \nhref=\"http:\/\/maxcdn.bootstrapcdn.com\/bootstrap\/3.3.6\/css\/\nbootstrap.min.css\">\n<script src=\"https:\/\/ajax.googleapis.com\/ajax\/libs\/jquery\/1.12.2\/\njquery.min.js\"><\/script>\n<\/head>\nChapter 7\nCreate a New HTML for Search by Employee Id\n7-5\n\nDocument 36: 2. Declare a variable ID_KEY to capture the employee id. This is a global variable, hence,\ndeclare it outside the method processRequest() but within the WebController class.\nprivate static final String ID_KEY = \"id\";\n3. The method processRequest() is already created in the ListAll feature. Now, we add the\ncode to implement Search by employee id functionality. Declare a variable value of the\nString type to capture the input from the user.\nString value = null;\n4. Add an IF condition to handle the new functionality. Get the employee id entered by the\nuser and invoke the method getEmployee(int) to verify if the employee record exists.\nif ((value = request.getParameter(ID_KEY)) != null) {\n int empId = Integer.valueOf(value).intValue();\n employeeList = employeeBean.getEmployee(empId);\n } else { \n \/\/ Previously used getEmployees() method for Listall feature\n employeeList = employeeBean.getEmployees();\n }\nCreate a New HTML for Search by Employee Id\nA HTML page that shows an input placeholder for the user to enter the employee id. If the\nemployee record is found, then the details of the employee is displayed on the page,\notherwise, an error message will be displayed.\nClass Name:src\/main\/webapp\/listById.html\nGithub Location: listById.html\nSteps to create the HTML page:\n1. Create the title, stylesheet, and body for the HTML page.\n<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n<meta charset=\"UTF-8\">\n<title>List Employee by Id<\/title>\n<!-- Specify the stylesheet here -->\n<link rel=\"stylesheet\" type=\"text\/css\" href=\"css\/app.css\" >\n<!-- Bootstrap JS for the UI -->\n<link rel=\"stylesheet\" \nhref=\"http:\/\/maxcdn.bootstrapcdn.com\/bootstrap\/3.3.6\/css\/\nbootstrap.min.css\">\n<\/head>\n2. Start the <body> tag and a <input> tag for capturing the employee id.\n<body>\n<div><label>Employee Id: <\/label>\n<input id=\"empId\" type=\"textfield\"\nChapter 6\nCreate a New HTML for Search by Employee Id\n6-3", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"43", + "topic":"Java Employee Management" + } + }, + { + "id":"2618274b-ab53-4011-9052-2d9b8a07a583", + "question":"What can you do through the increment salary tab in the HR application?", + "reference_answer":"You can alter (increase or decrease) the percentage of salary for hike.", + "reference_context":"Document 15: \u2022 Increment Salary\nThrough the increment salary tab, you can alter (increase or decrease) the\npercentage of salary for hike.\n\u2022 About\nThis page provides an overview of the HR Application and explains the various\nfunctionalities it offers.\nChapter 3\nFunctionalities of the HR Web Application\n3-2", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"15", + "topic":"Java Employee Management" + } + }, + { + "id":"ccce862e-223c-43ea-8f90-ccb510be884b", + "question":"What is the purpose of the variable ID_KEY in the WebController class?", + "reference_answer":"The variable ID_KEY is declared to capture the employee id and is a global variable, hence, it is declared outside the method processRequest() but within the WebController class.", + "reference_context":"Document 36: 2. Declare a variable ID_KEY to capture the employee id. This is a global variable, hence,\ndeclare it outside the method processRequest() but within the WebController class.\nprivate static final String ID_KEY = \"id\";\n3. The method processRequest() is already created in the ListAll feature. Now, we add the\ncode to implement Search by employee id functionality. Declare a variable value of the\nString type to capture the input from the user.\nString value = null;\n4. Add an IF condition to handle the new functionality. Get the employee id entered by the\nuser and invoke the method getEmployee(int) to verify if the employee record exists.\nif ((value = request.getParameter(ID_KEY)) != null) {\n int empId = Integer.valueOf(value).intValue();\n employeeList = employeeBean.getEmployee(empId);\n } else { \n \/\/ Previously used getEmployees() method for Listall feature\n employeeList = employeeBean.getEmployees();\n }\nCreate a New HTML for Search by Employee Id\nA HTML page that shows an input placeholder for the user to enter the employee id. If the\nemployee record is found, then the details of the employee is displayed on the page,\notherwise, an error message will be displayed.\nClass Name:src\/main\/webapp\/listById.html\nGithub Location: listById.html\nSteps to create the HTML page:\n1. Create the title, stylesheet, and body for the HTML page.\n<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n<meta charset=\"UTF-8\">\n<title>List Employee by Id<\/title>\n<!-- Specify the stylesheet here -->\n<link rel=\"stylesheet\" type=\"text\/css\" href=\"css\/app.css\" >\n<!-- Bootstrap JS for the UI -->\n<link rel=\"stylesheet\" \nhref=\"http:\/\/maxcdn.bootstrapcdn.com\/bootstrap\/3.3.6\/css\/\nbootstrap.min.css\">\n<\/head>\n2. Start the <body> tag and a <input> tag for capturing the employee id.\n<body>\n<div><label>Employee Id: <\/label>\n<input id=\"empId\" type=\"textfield\"\nChapter 6\nCreate a New HTML for Search by Employee Id\n6-3\n\nDocument 43: 2. Declare a variable FN_KEY to capture first name of the employee. This is a global variable,\nhence, declare it outside the method processRequest() but within the WebController\nclass.\nprivate static final String FN_KEY = \"firstName\";\n3. The method processRequest() is already created in the ListAll feature. Now, we add the\ncode to implement Update an Employee functionality. Add an ELSEIF condition to\nhandle the new functionality. Get the employee id entered by the user and invoke the\nmethod getEmployee(int) to verify if the employee record exists.\nif ((value = request.getParameter(ID_KEY)) != null) {\n int empId = Integer.valueOf(value).intValue();\n employeeList = employeeBean.getEmployee(empId);\n } \n\/* New code added below *\/\nelse if ((value = request.getParameter(FN_KEY)) != null) {\n employeeList = jdbcBean.getEmployeeByFn(value);\n}\nelse { \n \/* Previously used getEmployees() method for Listall feature *\/\n employeeList = employeeBean.getEmployees();\n }\nCreate a New HTML for Search by Employee Id\nA HTML page that shows an input placeholder for the user to enter the employee first name.\nIf the employee record is found, then the details of the employee is displayed on the page,\notherwise, an error message will be displayed.\nClass Name:src\/main\/webapp\/listById.html\nGithub Location: listByName.html\nSteps to create the HTML page:\n1. Create the title, stylesheet, and body for the HTML page.\n<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n<meta charset=\"UTF-8\">\n<title>List Employee by Id<\/title>\n<!-- Specify the stylesheet here -->\n<link rel=\"stylesheet\" type=\"text\/css\" href=\"css\/app.css\" >\n<!-- Bootstrap JS for the UI -->\n<link rel=\"stylesheet\" \nhref=\"http:\/\/maxcdn.bootstrapcdn.com\/bootstrap\/3.3.6\/css\/\nbootstrap.min.css\">\n<script src=\"https:\/\/ajax.googleapis.com\/ajax\/libs\/jquery\/1.12.2\/\njquery.min.js\"><\/script>\n<\/head>\nChapter 7\nCreate a New HTML for Search by Employee Id\n7-5", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"36", + "topic":"Java Employee Management" + } + }, + { + "id":"866b8f03-5582-4d03-8549-6fd40f696107", + "question":"Could you explain the specific functionality and intended outcome of the getEmployees() method within the JavaBeanImpl class, particularly in relation to its interaction with the EMPLOYEES table and error handling mechanisms?", + "reference_answer":"The getEmployees() method is used to retrieve a list of employees from the EMPLOYEES table.", + "reference_context":"Document 28: 2. Create a method getEmployees() to retrieve a list of employees from the EMPLOYEES\ntable. Update the SELECT query to be used by choosing the columns that you want from\nthe EMPLOYEES table.\na. Inside the JavaBeanImpl class, declare the method getEmployees().\nb. Use try and catch blocks to establish a database connection and fetch the\nemployee details using a SQL SELECT statement.\nc. Store the employee details in an array returnValue.\nd. Catch the SQLException and log the message in logger.\npublic List<Employee> getEmployees() {\n List<Employee> returnValue = new ArrayList<>();\n try (Connection connection = getConnection()) {\n try (Statement statement = connection.createStatement()) {\n try (ResultSet resultSet = statement.executeQuery(\"\n SELECT Employee_Id, First_Name, Last_Name, Email, \nPhone_Number, Job_Id, Salary \n FROM EMPLOYEES\")) {\n while(resultSet.next()) {\n returnValue.add(new Employee(resultSet));\n }\n }\n }\n } catch (SQLException ex) {\nlogger.log(Level.SEVERE, null, ex);\n ex.printStackTrace();\n }\nreturn returnValue;\n}\nNote:\nThis topic describes how to add the implementation method of List All functionality.\nSimilarly, you learn to add getEmployee, updateEmployee, getEmployeeByFn and\nincrementSalary implementation methods for other functionalities of the HR Web\nApplication in the upcoming chapters.\nCreating a Servlet to Process the Request\nIn this section, you learn to create a Servlet to process a request.\nClass Name: src\/main\/java\/com\/oracle\/jdbc\/samples\/web\/WebController.java\nGithub Location: WebController.java\nDescription: This is the main servlet that controls all the flows of the application. For every\nnew functionality of the application, we will be adding the code to handle the new requests\nand responses in doPost() and processResponse() respectively.\nChapter 5\nCreating a Servlet to Process the Request\n5-5", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"28", + "topic":"Java Employee Management" + } + }, + { + "id":"0a15bfa3-9bd9-40db-a6fc-a5412c878436", + "question":"Could you elaborate on the specific functionality and intended outcome of the incrementSalary(int) method, particularly in the context of how it interacts with employee salary adjustments based on a given percentage input?", + "reference_answer":"The incrementSalary(int) method updates the salary value of all employees by incrementing the value according to a given percentage.", + "reference_context":"Document 47: 8\nIncrement Salary\nThe Increment Salary functionality modifies the salaries of all employees by incrementing the\nvalues according to the input percentage.\nEnter a percentage for salary hike in the placeholder on the web page. Click confirm to\nmodify the salaries of all employees in the database table. You can verify the changes by\nclicking on the List All tab.\nIn this chapter, you learn how to add code required to build the Increment Salary\nfunctionality. You will learn how to:\n1. Declare a new method incrementSalary(int) in JavaBean.java.\n2. Implement a new method incrementSalary(int) in JavaBeanImpl.java.\n3. Add new code to WebController.java to process the request and response.\n4. Create a HTML page incrementSalary.html to display the results.\nDeclare a new method incrementSalary(int)\nThe incrementSalary(int) method updates the salary value of all employees by\nincrementing the value according to a given percentage.\nClass Name: src\/main\/java\/com\/oracle\/jdbc\/samples\/bean\/JavaBean.java.\nGithub Location: JavaBean.java\nSteps to declare a new method:\n1. Open the JdbcBean.java file in IntelliJ. To create the JdbcBean.java class, refer to \nCreating a Java Bean Interface for a JDBC Connection. Use the same class and declare\nnew methods for each one of the functionalities.\n2. Declare a method incrementSalary(int) that takes an integer for percentage as an\ninput parameter.\npublic List<Employee> incrementSalary(int incrementPct);\n8-1\n\nDocument 48: Implement a new method incrementSalary(int)\nThe incrementSalary(int) method enables you to increment the salary of all\nemployees according to a given percentage.\nClass Name: src\/main\/java\/com\/oracle\/jdbc\/samples\/bean\/JavaBeanImpl.java\nGithub Location: JavaBeanImpl.java\nSteps to Implement a new method:\n1. Open the JdbcBeanImpl.java file in IntelliJ. To create the JdbcBeanImpl.java\nclass, refer to Creating a Java Bean Implementation for a JDBC Connection. Use\nthe same class and add new implementation methods for each one of the\nfunctionalities.\n2. Add the following code snippet to implement the incrementSalary(int) method:\npublic List<Employee> incrementSalary (int incrementPct) {\n List<Employee> returnValue = new ArrayList<>();\n \n\/* Get the database connection*\/\n try (Connection connection = getConnection()) {\n try (CallableStatement callableStatement = \n connection.prepareCall(\"begin ? := \nrefcur_pkg.incrementsalary(?); end;\")) {\n callableStatement.registerOutParameter(1, \nOracleTypes.CURSOR);\n callableStatement.setInt(2, incrementPct);\n callableStatement.execute();\n try (ResultSet resultSet = (ResultSet) \ncallableStatement.getObject(1)) {\n while (resultSet.next()) {\n returnValue.add(new Employee(resultSet));\n }\n }\n }\n } catch (SQLException ex) {\n\/* Catch the SQLException and log the message in the logger*\/\n logger.log(Level.SEVERE, null, ex);\n ex.printStackTrace();\n }\nreturn returnValue;\n}\nAdd the Code to a Servlet\nAdd the relevant code to WebController.java to give a salary raise to all employees.\nClass Name: src\/main\/java\/com\/oracle\/jdbc\/samples\/web\/WebController.java\nChapter 8\nImplement a new method incrementSalary(int)\n8-2", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"47", + "topic":"Java Employee Management" + } + }, + { + "id":"036c54cd-0da0-4633-a3a7-e4c6ee80047d", + "question":"What are the detailed steps you need to follow in order to successfully initiate the HR Web application after you have deployed the .war file on the Tomcat server, including any necessary configurations or file placements?", + "reference_answer":"Start the tomcat server and then access the HR web application from a browser using the URL http:\/\/localhost:8080\/JdbcWebSamples\/.", + "reference_context":"Document 22: 8. Once the source code is packaged successfully, a war file is generated. Locate the\nJdbcWebSamples.war under the target folder.\nRun the HR Web Application\nThe HR Web application is run using the Tomcat server.\nDeploy the .war file on the Apache server\n1. Navigate to the HRWebApp folder on your local machine. Under the target folder, locate\nJdbcWebSamples.war file.\n2. Place JdbcWebSamples.war file under TOMCAT_HOME\/webapps\/.\n3. Navigate to the HRWebApp folder on your local machine. Locate tomcat-users.xml file.\n4. Place tomcat-users.xml file under TOMCAT_HOME\/conf\/.\n5. Start the tomcat server.\n6. Once the tomcat is started, access the HR web application from a browser using the URL\nhttp:\/\/localhost:8080\/JdbcWebSamples\/.\nVerify the HR Web Application\n1. Login to the application with either hradmin or hrstaff user.\nChapter 4\nRun the HR Web Application\n4-7", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"22", + "topic":"Others" + } + }, + { + "id":"37c21809-488e-4b64-8c35-88950def34ba", + "question":"Could you elaborate on the specific types of information that the about.html file presents regarding the HR Application, including its users and functionalities, as well as any relevant steps for accessing this file?", + "reference_answer":"The about.html file displays information about the HR Application, users, and functionalities.", + "reference_context":"Document 56: <\/security-constraint>\n<login-config>\n<auth-method>FORM<\/auth-method>\n<form-login-config>\n<form-login-page>\/login.html<\/form-login-page>\n<form-error-page>\/login-failed.html<\/form-error-page>\n<\/form-login-config>\n<\/login-config>\n<\/web-app>\nCreate about.html\nThe about.html file displays information about the HR Application, users and functionalities.\nClass Name: src\/main\/webapp\/about.html\nGithub Location: about.html\nSteps to use the HTML page: Download the about.html and use it in your application.\nCreate index.html\nThe index.html file consists of all details about the HR Web Application. It describes in detail\nits users and functionalities.\nClass Name: src\/main\/webapp\/index.html\nGithub Location: index.html\nSteps to create the HTML page:\n1. Create the title, head, and stylesheet for index.html.\n<!DOCTYPE html>\n<html>\n<head>\n<meta charset=\"utf-8\">\n<title>Employee table listing<\/title>\n<link rel=\"stylesheet\" type=\"text\/css\" href=\"css\/app.css\" >\n<style>\niframe:focus {\noutline: none;\n}\niframe[seamless] {\ndisplay: block;\n}\n<\/style>\n<\/head>\n<body>\n2. Create <body> and actions for the features through navigation links and logout.\n<body>\n<div id=\"sideNav\" class=\"sidenav\">\nChapter 9\nCreate about.html\n9-5", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"56", + "topic":"Login and Logout Functionality" + } + }, + { + "id":"bd02d150-d035-42c3-8f42-3b1066752aca", + "question":"What specific details about employees can be retrieved and displayed when utilizing the List All functionality in the HR Web Application, and how does this information correlate with the data stored in the Employees table?", + "reference_answer":"The List All functionality displays employees' details such as Employee_id, First_name, Last_Name, Email, Phone_number, Job_id, Salary, etc.", + "reference_context":"Document 23: Note:\n\u2022 For more information about the hradmin and hrstaff users, see \nOverview of the HR Web Application and Create Login and Logout\nFunctionality.\n\u2022 Refer tomcat-users.xml file for the username and password\ninformation required to login to the HR Web Application.\n2. HR Web Application has several functionalities. List All displays the employees'\ndetails such as Employee_id, First_name, Last_Name, Email, Phone_number,\nJob_id, Salary etc. The details are retrieved from the Employees table and\ndisplayed on a web page. See the screenshot below that shows List All\nfunctionality.\n3. Similarly, verify the remaining functionalities such as Search by ID, Update\nEmployee Record and Increment Salary by giving appropriate inputs.\nChapter 4\nRun the HR Web Application\n4-8", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"23", + "topic":"Others" + } + }, + { + "id":"3fdbecce-bc15-4c6d-9606-9f9b0b458912", + "question":"After deploying the application, what specific errors or messages should I look for in the Tomcat log file located at TOMCAT_HOME\/logs\/catalina.out to effectively troubleshoot any potential issues?", + "reference_answer":"Check TOMCAT_HOME\/logs\/catalina.out for any errors after deploying the application.", + "reference_context":"Document 61: 11\nTroubleshooting and Debugging\n1. Tomcat log file:\nCheck TOMCAT_HOME\/logs\/catalina.out for any errors after deploying the application.\n2. Additional Logging:\nEnable logging in Tomcat to find logging messages in the code.\nNote:\nRefer https:\/\/tomcat.apache.org\/tomcat-8.0-doc\/logging.html for more information\nDebugging UI Related Issues\n1. Browser Version:\nThis application has been tested on Firefox (version 52) and Chrome (version 58)\nsuccessfully.\n2. Browser Console:\nLook for errors in the browser console to find and debug issues.\n3. Monitor Network Traffic:\nTrack network traffic to find out the requests being made, and the responses to the requests.\nA return status higher than 400 indicates errors. If you find errors in the range of 400 t 500\ninspect the Tomcat log files for debugging.\nInspect the JSON responses from various calls to the backend server.\n4. Additional Logging:\nEdit the packaged HTML files, and add console.log(...) to add extra logging.\n11-1", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"61", + "topic":"Others" + } + }, + { + "id":"a145968d-43a5-4aa3-b1a9-df6afee1d0d8", + "question":"Could you please identify the title of Table 1-1 as referenced in Document 5, specifically in relation to the overall architecture of the web application?", + "reference_answer":"Architecture of the Web Application", + "reference_context":"Document 5: List of Tables\n1-1 Architecture of the Web Application 1-2\n1-2 Components Required for the Application 1-2\n4-1 Github Repository Details 4-3\nvi", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"5", + "topic":"Others" + } + }, + { + "id":"83be1d4f-8884-4d4b-a496-fd26dcdc6a0a", + "question":"What specific error message is presented to users on the login-failed.html page in the event that their login attempt is unsuccessful, and how does this message inform them of the issue?", + "reference_answer":"Sorry, login failed!", + "reference_context":"Document 54: td {\n height: 30px;\n }\n<\/style>\n<\/head>\n2. Create the <body> and <form> to submit the login credentials entered by the user.\n<body>\n<div id=\"cent\">\n<form method=\"POST\" action=\"j_security_check\">\n<table>\n<tr>\n<td colspan=\"2\">Login to the Jdbc Web Sample application:<\/td>\n<\/tr>\n<td>Name:<\/td>\n<td><input type=\"text\" name=\"j_username\" \/><\/td>\n<\/tr>\n<tr>\n<td>Password:<\/td>\n<td><input type=\"password\" name=\"j_password\"\/><\/td>\n<\/tr>\n<tr>\n<td colspan=\"2\"><input type=\"submit\" value=\"Go\" \/><\/td>\n<\/tr>\n<\/table>\n<\/form>\n<\/div>\n<\/body>\nCreate login-failed.html\nA html page to display the error messgae if the login is unsuccessful.\nClass Name: src\/main\/webapp\/login-failed.html\nGithub Location: login-failed.html\nSteps to create the HTML page:\n1. Create the login-failed.html as shown below.\n<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n<meta charset=\"UTF-8\">\n<title>Login Failed<\/title>\n<\/head>\n<body>\n<p>\nSorry, login failed!\n<\/p>\nChapter 9\nCreate login-failed.html\n9-3", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"54", + "topic":"Login and Logout Functionality" + } + }, + { + "id":"fd7d5bed-f867-4943-a6f0-87a741f53beb", + "question":"Considering the context of developing an HR web application, what is one of the most effective practices for establishing a database, particularly in terms of utilizing cloud services and built-in schemas?", + "reference_answer":"Use the Oracle Database Service on Cloud (DBCS) to create a database on cloud.", + "reference_context":"Document 60: 10\nBest Practices\n1. Use Database Service on Cloud:\nUse the Oracle Database Service on Cloud (DBCS) to create a database on cloud. DBCS\ncomes with an in-built HR schema and tables that you can use when you build the HR web\napplication.\n2. JDBC Driver, UCP:\nUse the latest 23ai versions of JDBC drivers and UCP.\nNote:\nDownload the latest JDBC drivers and UCP from the JDBC and UCP Downloads\npage\n3. JDK Version\nUse a JDK version that is compatible with Oracle Database Release 23ai. Refer to Version\nCompatibility for Oracle JDBC Drivers for more details.\n4. Auto-Closeable Statements\nStarting JDK7, \u2018Auto-closeable statements\u2019 has been introduced, that close by default without\nan explicit catch statement.\n5. Use PreparedStatement instead of Statement objects:\nStatement in JDBC must be localized to being used for DDL (ALTER, CREATE, GRANT etc)\nsince these commands cannot accept bind variables.\nUse PreparedStatement or CallableStatement for any other type of statement. These\nstatements can accept bind variables.\n10-1", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"60", + "topic":"Java Web Application Development" + } + }, + { + "id":"8b0b433c-12df-41f7-b582-ead979a4b4ec", + "question":"Could you identify the title of the document referenced in the provided context, specifically focusing on its relevance to Oracle Database and Java Development?", + "reference_answer":"Get Started with Java Development", + "reference_context":"Document 0: Oracle\u00ae Database\nGet Started with Java Development\n23ai\nF47017-03\nMay 2024", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"0", + "topic":"Java Web Application Development" + } + }, + { + "id":"41889a0d-2f18-4da8-b377-b7947b12e86d", + "question":"Could you elaborate on the specific role and functionality of the getConnection() method within the JavaBeanImpl class, particularly in terms of how it facilitates database connectivity and handles exceptions?", + "reference_answer":"The getConnection() method registers the driver and establishes the database connection by passing the connection string, the database username, and the database password.", + "reference_context":"Document 27: a. Declare the package for the JavaBean.java class. Import the Employee class\nas it contains the employee details.\npackage com.oracle.jdbc.samples.bean;\nimport com.oracle.jdbc.samples.entity.Employee;\nb. Import the other dependent classes as shown in the following code snippet. If\na particular class is not imported, then IntelliJ displays a message reminding\nyou to import the required package.\nimport java.sql.*;\nimport java.util.ArrayList;\nimport java.util.List;\nimport java.util.logging.Level;\nimport java.util.logging.Logger;\nimport java.sql.PreparedStatement;\nimport oracle.jdbc.OracleStatement;\nimport oracle.jdbc.OracleConnection;\nimport oracle.jdbc.driver.OracleDriver;\nimport oracle.jdbc.OracleTypes;\nimport java.sql.PreparedStatement;\nimport oracle.jdbc.OracleStatement;\nimport oracle.jdbc.OracleConnection;\nimport oracle.ucp.jdbc.PoolDataSourceFactory;\nimport oracle.ucp.jdbc.PoolDataSource;\nc. Declare the JavaBeanImpl class that implements JavaBean.\npublic class JavaBeanImpln implements JavaBean { }\nd. Inside the JavaBeanImpl class, create a logger to log exceptions.\nstatic final Logger logger = \nLogger.getLogger(\"com.oracle.jdbc.samples.bean.JdbcBeanImpl\");\ne. Inside the JavaBeanImpl class, declare a static method getConnection(). The\ngetConection() method registers the driver and establishes the database\nconnection by passing the connection string, the database username and the\ndatabase password as shown in the following code snippet.\npublic static Connection getConnection() throws SQLException {\n DriverManager.registerDriver(new oracle.jdbc.OracleDriver());\n Connection connection = \nDriverManager.getConnection(\"jdbc:oracle:thin:@\/\/myorclhost:5521\/\nmyorcldbservice\", \"hr\", \"hr\");\n return connection;\n}\nChapter 5\nCreating a Java Bean Implementation for a JDBC Connection\n5-4", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"27", + "topic":"Others" + } + }, + { + "id":"789217ff-ba0a-4d97-9358-77af90f16db9", + "question":"Could you elaborate on the specific role and functionality of the constructor within the Employee class, particularly in relation to how it initializes the employee's attributes using a ResultSet and handles potential SQLExceptions?", + "reference_answer":"The constructor for the Employee class takes ResultSet as the input and throws a SQLException. In this constructor, it sets all the values for the attributes of the Employee class.", + "reference_context":"Document 25: Creating a Java Bean Entity for an Employee\nThe Employee class contains the getter and setter methods for all attributes of an\nemployee. For example, the First_name has a getter and a setter method like\ngetFirst_Name and setFirst_Name respectively.\nClass Name: src\/main\/java\/com\/oracle\/jdbc\/samples\/entity\/Employee.java\nGithub Location: Employee.java\nSteps to create Employee.java:\n1. Declare the package for the class Employee.java.\npackage com.oracle.jdbc.samples.entity;\n2. Import the following packages required for the Employee class.\nimport java.sql.ResultSet;\nimport java.sql.SQLException;\nimport java.sql.Timestamp;\n3. Declare an Employee class. Add a pair of parenthesis ({ }). Place the cursor in\nbetween the parenthesis:\npublic class Employee {}\n4. Declare the following variables for each one of the attributes of an employee.\nprivate int Employee_Id;\nprivate String First_Name;\nprivate String Last_Name;\nprivate String Email;\nprivate String Phone_Number;\nprivate String Job_Id;\nprivate int Salary;\n5. Create a constructor for the Employee class which takes ResultSet as the input\nand throws a SQLException. In this constructor, set all the values for the attributes\nof the Employee class.\npublic Employee(ResultSet resultSet) throws SQLException {\n this.Employee_Id = resultSet.getInt(1);\n this.First_Name = resultSet.getString(2);\n this.Last_Name = resultSet.getString(3);\n this.Email = resultSet.getString(4);\n this.Phone_Number = resultSet.getString(5);\n this.Job_Id = resultSet.getString(6);\n this.Salary = resultSet.getInt(7);\n}\n6. Create the Getter and Setter methods, that is, getX and setX methods to get and\nset the values for all attributes of the Employee such as Employee_id, first_name,\nChapter 5\nCreating a Java Bean Entity for an Employee\n5-2", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"25", + "topic":"Java Employee Management" + } + }, + { + "id":"a050d8f5-cd1f-4ae4-9ece-01843fa126aa", + "question":"What specific actions are available to you when utilizing the increment salary tab in the HR Application, particularly regarding the modification of salary percentages for hikes, and how does this feature integrate with the overall functionalities of the application?", + "reference_answer":"You can alter (increase or decrease) the percentage of salary for hike.", + "reference_context":"Document 15: \u2022 Increment Salary\nThrough the increment salary tab, you can alter (increase or decrease) the\npercentage of salary for hike.\n\u2022 About\nThis page provides an overview of the HR Application and explains the various\nfunctionalities it offers.\nChapter 3\nFunctionalities of the HR Web Application\n3-2", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"15", + "topic":"Java Employee Management" + } + }, + { + "id":"a12745b4-e743-47c5-8fd8-20849958de0d", + "question":"Could you elaborate on the primary objectives and key tasks addressed in the chapter titled 'Getting Started with the Application', particularly focusing on the necessary prerequisites and setup processes required for effectively building the application?", + "reference_answer":"The chapter helps you understand the prerequisites for building the application and how to get the environment ready, including signing up for the Oracle Cloud Free Tier or installing the Oracle Database on premise, installing IntelliJ, and downloading tools like Maven.", + "reference_context":"Document 10: 2. Overview of the HR Web Application: This chapter discusses the HR Web\napplication in depth and familiarize you with the flows of the Web application,\npackages and files that you create as a part of the web application.\n3. Getting Started with the Application: In this chapter, you understand the\nprerequisites for building the application and how to get the environment ready. It\nstarts with signing up for the Oracle Cloud Free Tier or installing the Oracle\nDatabase on premise. Later, you install IntelliJ, an IDE to build the application. You\nuse Tomcat Java EE container to deploy and run the application.\nThe chapter also helps you download any other tools, such as Maven, that helps\nyou to build the application.\n4. List All Employees: This chapter helps you to put all the components together\nand build an initial functionality to connect to the Oracle Database, and retrieve\nemployee details from the database.\n5. Search By Employee ID: This chapter provides details on how to implement the\n\u2018Search by Employee ID\u2019 functionality.\n6. Update an Employee Record: In this chapter, you learn how to update employee\nrecords. This is a two step process. Firstly, you search the employee\u2019s records,\nbased on first name. Once you retrieve the required results, you can update the\nsalary, job ID, firstname, lastname and other details.\n7. Delete an Employee Record: In this chapter, you learn how to delete an\nemployee record, in a two-step process.\n8. Increase Salary to All Employees: In this chapter, you understand how to\nprovide an increment to the salary of the employees listed in the table, using \u2018Java\nin the database\u2019.\n9. Creating Application Users: This chapter shows how to create \u2018hradmin\u2019 and\n\u2018hrstaff\u2019 users in Tomcat and IntelliJ.\n10. Summary: This chapter summarizes all that you have learnt so far. It also\nprovides appropriate references and links for enhancing your use of the web\napplication.\nChapter 1\nObjectives and Tasks\n1-4", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"10", + "topic":"Java Web Application Development" + } + }, + { + "id":"f5c4a4c3-2dd6-4c64-b165-5017fd8c077d", + "question":"What is required to create and compile Java applications according to the provided context?", + "reference_answer":"You need the full Java 2 Platform, Standard Edition, Software Development Kit (J2SE SDK), formerly known as the Java Development Kit (JDK).", + "reference_context":"Document 17: See Also:\nProvisioning an ATP Database instance video for instructions\nOption 2: Oracle Database Free Available on OTN\nAs an alternate option, you can install Oracle Database Free on your system.\nSee Also:\nOracle Database Free Installation Guide\nInstall the HR schema\nThe HR web application uses the tables and data from the HR sample schema\nprovided by Oracle. You need to install the HR schema in your database.\nOnce you provision an Autonomous Database or install Oracle Database Free, see \nInstalling HR Schema for detailed instructions to install the sample schema.\nJ2SE or JDK\nTo create and compile Java applications, you need the full Java 2 Platform, Standard\nEdition, Software Development Kit (J2SE SDK), formerly known as the Java\nDevelopment Kit (JDK).\n1. Download and install the Java SE. Refer http:\/\/www.oracle.com\/technetwork\/java\/\njavase\/downloads\/index.html.\n2. Set the PATH environment variable for JDK. Refer JDK Installation Instructions for\nWindows for detailed instructions.\nJDBC Drivers\nYou need to download certain JDBC drivers for running the HR application.\nThis guide uses Maven Central to download the necessary JDBC drivers required for\nthe application. Later in this guide, you learn to add the following dependency to your\nproject's pom.xml file:\n<dependencies>\n <dependency>\n <groupId>com.oracle.database.jdbc<groupId>\n <artifactId>ojdbc8-production<\/artifactId>\n <version>19.7.0.0<\/version>\n <type>pom<\/type>\n <\/dependency>\n<\/dependencies>\nChapter 4\nWhat You Need to Install\n4-2", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"17", + "topic":"Java Web Application Development" + } + }, + { + "id":"b36c40d9-34b2-4827-b686-dc7e36554149", + "question":"What is the purpose of the function processResponse() in the provided code?", + "reference_answer":"The function processResponse() is designed to display the JSON results on an HTML page by parsing the response, creating a table, and populating it with the data.", + "reference_context":"Document 51: }\n else {\n console.log(\"Salary not updated\");\n showStatus(\"alert alert-warning\", \"Updating records, attempt \ncancelled\");\n }\n}\n<\/script>\n3. Create the function processRequest() to display the JSON results on HTML page.\nunction processResponse(response) {\n var arr = JSON.parse(response);\n var i;\n var out = \"<table>\";\nkeys = Object.keys(arr[0]);\n \/* Print headers *\/\n out += \"<tr>\"\nfor(i = 0; i < keys.length; ++i) {\n out += \"<th>\"+keys[i]+\"<\/th>\"\n}\nout += \"<\/tr>\";\n\/* Print values *\/\nfor(j = 0; j < arr.length; j++) {\nout += \"<tr>\"\nfor(i = 0; i < keys.length; ++i) {\nout += \"<td>\"+arr[j][keys[i]]+\"<\/td>\"\n}\nout += \"<\/tr>\"\n}\nout += \"<\/table>\";\ndocument.getElementById(\"id-emp\").innerHTML = out;\n}\nChapter 8\nCreate a new HTML for Increment Salary\n8-5", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"51", + "topic":"Java Employee Management" + } + }, + { + "id":"cd931a92-d06a-4fe0-8975-c71019fcedf6", + "question":"What should be used instead of Statement objects for executing queries that can accept bind variables?", + "reference_answer":"Use PreparedStatement or CallableStatement for any type of statement that can accept bind variables.", + "reference_context":"Document 60: 10\nBest Practices\n1. Use Database Service on Cloud:\nUse the Oracle Database Service on Cloud (DBCS) to create a database on cloud. DBCS\ncomes with an in-built HR schema and tables that you can use when you build the HR web\napplication.\n2. JDBC Driver, UCP:\nUse the latest 23ai versions of JDBC drivers and UCP.\nNote:\nDownload the latest JDBC drivers and UCP from the JDBC and UCP Downloads\npage\n3. JDK Version\nUse a JDK version that is compatible with Oracle Database Release 23ai. Refer to Version\nCompatibility for Oracle JDBC Drivers for more details.\n4. Auto-Closeable Statements\nStarting JDK7, \u2018Auto-closeable statements\u2019 has been introduced, that close by default without\nan explicit catch statement.\n5. Use PreparedStatement instead of Statement objects:\nStatement in JDBC must be localized to being used for DDL (ALTER, CREATE, GRANT etc)\nsince these commands cannot accept bind variables.\nUse PreparedStatement or CallableStatement for any other type of statement. These\nstatements can accept bind variables.\n10-1", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"60", + "topic":"Java Web Application Development" + } + }, + { + "id":"4a6a7958-ff36-47c5-a87d-050d5e64a18d", + "question":"What should be logged if the record could not be updated?", + "reference_answer":"If the record could not be updated, the message 'Unable to update record' should be logged.", + "reference_context":"Document 42: employee id *\/\n \"UPDATE employees SET FIRST_NAME = ?, LAST_NAME = ?, EMAIL \n= ?, PHONE_NUMBER = ?,\n SALARY = ? WHERE EMPLOYEE_ID = ?\")) {\n \/*Set the new values entered by the user for each attribute \n and execute the prepapredStatement *\/\n preparedStatement.setString(1, \nemployee.getFirst_Name());\n preparedStatement.setString(2, \nemployee.getLast_Name());\n preparedStatement.setString(3, employee.getEmail());\n preparedStatement.setString(4, \nemployee.getPhone_Number());\n preparedStatement.setInt(5, employee.getSalary());\n preparedStatement.setInt(6, \nemployee.getEmployee_Id());\n updateCount = preparedStatement.executeUpdate();\n }\n }catch (SQLException ex) { \/* Catch the SQLException and log \nthe message in the logger*\/\n logger.log(Level.SEVERE, \"Unable to update record\", ex);\n throw new SQLException(\"Alert! Record could not be updated, \n\"+ex.getMessage(), ex);\n }\n\/* Log the message with the number of records updated to the logger \n*\/\n logger.fine(\"Update count: \" +updateCount);\n\/* If none of the records were updated, enter an alert message *\/\n if (updateCount != 1) { \n logger.severe(\"Unable to update record\");\n throw new SQLException(\"Alert! Record could not be updated\");\n }\n\/* Return the success message if the record was updated *\/\n return \"Success: Record updated\"; \n}\n \nAdd the code to a Servlet to process the request\nAdd the relevant code to WebController.java to update an employee.\nClass Name: src\/main\/java\/com\/oracle\/jdbc\/samples\/web\/WebController.java\nGithub Location: WebController.java\nSteps to add the code:\n1. Open the WebController.java class. To create the WebController.java, refer to \nCreating a Servlet to Process the Request. Use the same class and add the\nrequired code.\nChapter 7\nAdd the code to a Servlet to process the request\n7-4", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"42", + "topic":"Java Employee Management" + } + }, + { + "id":"c4ef5c88-6eec-498c-8110-1b39ba67d7e7", + "question":"What file must be double-clicked in the Project window to compile the HRWebApp in IntelliJ?", + "reference_answer":"The pom.xml file must be double-clicked in the Project window.", + "reference_context":"Document 19: 2. Navigate to the location where the HRWebApp is downloaded. Select HRWebApp\nand click OK.\n3. A project with all the files required to build the HR Web application is displayed.\nCompile the Application in IntelliJ\nThe code requires a few updates before you compile the application.\nThe HRWebApp must be downloaded and opened in IntelliJ.\n1. In the Project window on the left, double-click on pom.xml file.\nChapter 4\nCompile the Application in IntelliJ\n4-4", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"19", + "topic":"Others" + } + }, + { + "id":"ec27414a-8ff0-4ade-97c8-015cb860f989", + "question":"What message is displayed on the login-failed.html page when the login is unsuccessful?", + "reference_answer":"Sorry, login failed!", + "reference_context":"Document 54: td {\n height: 30px;\n }\n<\/style>\n<\/head>\n2. Create the <body> and <form> to submit the login credentials entered by the user.\n<body>\n<div id=\"cent\">\n<form method=\"POST\" action=\"j_security_check\">\n<table>\n<tr>\n<td colspan=\"2\">Login to the Jdbc Web Sample application:<\/td>\n<\/tr>\n<td>Name:<\/td>\n<td><input type=\"text\" name=\"j_username\" \/><\/td>\n<\/tr>\n<tr>\n<td>Password:<\/td>\n<td><input type=\"password\" name=\"j_password\"\/><\/td>\n<\/tr>\n<tr>\n<td colspan=\"2\"><input type=\"submit\" value=\"Go\" \/><\/td>\n<\/tr>\n<\/table>\n<\/form>\n<\/div>\n<\/body>\nCreate login-failed.html\nA html page to display the error messgae if the login is unsuccessful.\nClass Name: src\/main\/webapp\/login-failed.html\nGithub Location: login-failed.html\nSteps to create the HTML page:\n1. Create the login-failed.html as shown below.\n<!DOCTYPE html>\n<html lang=\"en\">\n<head>\n<meta charset=\"UTF-8\">\n<title>Login Failed<\/title>\n<\/head>\n<body>\n<p>\nSorry, login failed!\n<\/p>\nChapter 9\nCreate login-failed.html\n9-3", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"simple", + "seed_document_id":"54", + "topic":"Login and Logout Functionality" + } + }, + { + "id":"bea4478c-ff3e-455b-a035-0c661e1db6b2", + "question":"In the event that executing the update statement results in no records being modified, what specific steps should you take to troubleshoot this issue, considering the potential logging and exception handling mechanisms outlined in the provided context?", + "reference_answer":"If none of the records were updated, you should log a severe message stating 'Unable to update record' and throw a SQLException with the message 'Alert! Record could not be updated'.", + "reference_context":"Document 42: employee id *\/\n \"UPDATE employees SET FIRST_NAME = ?, LAST_NAME = ?, EMAIL \n= ?, PHONE_NUMBER = ?,\n SALARY = ? WHERE EMPLOYEE_ID = ?\")) {\n \/*Set the new values entered by the user for each attribute \n and execute the prepapredStatement *\/\n preparedStatement.setString(1, \nemployee.getFirst_Name());\n preparedStatement.setString(2, \nemployee.getLast_Name());\n preparedStatement.setString(3, employee.getEmail());\n preparedStatement.setString(4, \nemployee.getPhone_Number());\n preparedStatement.setInt(5, employee.getSalary());\n preparedStatement.setInt(6, \nemployee.getEmployee_Id());\n updateCount = preparedStatement.executeUpdate();\n }\n }catch (SQLException ex) { \/* Catch the SQLException and log \nthe message in the logger*\/\n logger.log(Level.SEVERE, \"Unable to update record\", ex);\n throw new SQLException(\"Alert! Record could not be updated, \n\"+ex.getMessage(), ex);\n }\n\/* Log the message with the number of records updated to the logger \n*\/\n logger.fine(\"Update count: \" +updateCount);\n\/* If none of the records were updated, enter an alert message *\/\n if (updateCount != 1) { \n logger.severe(\"Unable to update record\");\n throw new SQLException(\"Alert! Record could not be updated\");\n }\n\/* Return the success message if the record was updated *\/\n return \"Success: Record updated\"; \n}\n \nAdd the code to a Servlet to process the request\nAdd the relevant code to WebController.java to update an employee.\nClass Name: src\/main\/java\/com\/oracle\/jdbc\/samples\/web\/WebController.java\nGithub Location: WebController.java\nSteps to add the code:\n1. Open the WebController.java class. To create the WebController.java, refer to \nCreating a Servlet to Process the Request. Use the same class and add the\nrequired code.\nChapter 7\nAdd the code to a Servlet to process the request\n7-4", + "conversation_history":[ + + ], + "metadata":{ + "question_type":"complex", + "seed_document_id":"42", + "topic":"Java Employee Management" + } + } +] \ No newline at end of file diff --git a/ai-optimizer/help/help.md b/ai-optimizer/help/help.md new file mode 100644 index 00000000..02743e7e --- /dev/null +++ b/ai-optimizer/help/help.md @@ -0,0 +1,55 @@ +# Need Help? + +We’d love to hear from you! You can contact us in the #ai-optimizer channel in the [Oracle Developers Slack workspace](https://app.slack.com/client/T01KVSQQYP3), or open an issue in [GitHub](https://github.com/oracle-samples/ai-optimizer/issues/new). + +## Troubleshooting common problems + +### Startup Time + +**_Problem_**: +After starting the AI Optimizer, it takes a long time to load the first page. + +**_Solution_**: +This is normally the result of a configured database that is inaccessible. Depending on how you've configured the database, if `retry_count`, and `retry_delay` is set but the database is inaccessible, the AI Optimizer will appear to hang for the duration of `retry_count * retry_delay` during the startup. + +### Embedding Rate Limits + +**_Problem_**: +During embedding, especially when using trial keys, you may experience a failure due to rate limits. For example: + +``` +Operation Failed: Unexpected error: status_code: 429, body: +data=None message='trial token rate limit exceeded, limit is 100000 tokens per minute'. +``` + +**_Solution_**: +Set a rate limit based on the API Key restrictions. + + +### Testbed Evaluation + +**_Problem_**: +During the Evaluation in the **Testbed**, a database error occurs: `DPY-4011: the database or network closed the connection` + +**_Solution_**: +Increase the memory of the vector_memory_size. If this is an Oracle Autonomous Database, scale up the CPU. + +### Autonomous Database behind VPN + +**_Problem_**: +Connection to an Autonomous database while inside a VPN fails. + +**_Solution_**: +Update the database connection string to include a `https_proxy` and `https_proxy_port`. + + For example: + + ```text + myadb_high = ( + description=( + address= + (protocol=tcps)(port=1522) + (https_proxy=<proxy_host>)(https_proxy_port=<proxy_port>) # <-- Add + (host=<adb_host>) + )(connect_data=(service_name=s<service_name>))(security=(ssl_server_dn_match=yes)) + )``` \ No newline at end of file diff --git a/ai-optimizer/other-livelabs/other-livelabs.md b/ai-optimizer/other-livelabs/other-livelabs.md new file mode 100644 index 00000000..6835d049 --- /dev/null +++ b/ai-optimizer/other-livelabs/other-livelabs.md @@ -0,0 +1,7 @@ +# Other LiveLabs you might like + +- [Autonomous Database Dedicated](https://livelabs.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=677) + +- [Manage and Monitor Autonomous Database](https://livelabs.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=553) + +- [Scaling and Performance in the Autonomous Database](https://livelabs.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=608) diff --git a/ai-optimizer/tenancy/cleanup/cleanup.md b/ai-optimizer/tenancy/cleanup/cleanup.md new file mode 100644 index 00000000..a759081a --- /dev/null +++ b/ai-optimizer/tenancy/cleanup/cleanup.md @@ -0,0 +1,27 @@ +# Cleanup the Oracle AI Optimizer and Toolkit installation + +## Introduction + +In the lab you cleanup the deployed Oracle AI Optimizer and Toolkit environment by using Oracle Cloud OCI Resource Manager. + +Estimated Time: 10 minutes + +### Objectives + +* Navigate to the OCI Developer Services +* Destroy the environment + +### Prerequisites + +Ypu have successfully deployed an Oracle AI Optimizer and Toolkit environment by using Oracle CLoud OCI Resource Manager. + +## Task 1: Start the Oracle Resource Manager Destroy job + +To destroy the AI Optimizer infrastructure, in OCI navigate to `Developer Services -> Stacks`. Choose the Compartment the AI Optimizer was deployed into and select the stack Name. Click on the **Destroy** button. + +![Destroy job](images/destroy.png) + +## Acknowledgements + +* **Author** - Andy Tael, July 2025 +* **Last Updated By** - Andy Tael, July 2025 diff --git a/ai-optimizer/tenancy/cleanup/images/destroy.png b/ai-optimizer/tenancy/cleanup/images/destroy.png new file mode 100644 index 00000000..3cf7de72 Binary files /dev/null and b/ai-optimizer/tenancy/cleanup/images/destroy.png differ diff --git a/ai-optimizer/tenancy/deploy/deploy.md b/ai-optimizer/tenancy/deploy/deploy.md new file mode 100644 index 00000000..a325ae04 --- /dev/null +++ b/ai-optimizer/tenancy/deploy/deploy.md @@ -0,0 +1,76 @@ +# Deploy the Oracle AI Optimizer and Toolkit + +## Introduction + +In the lab you will deploy the Oracle AI Optimizer and Toolkit by using infrastructure as code and OCI Resource Manager. + +Estimated Time: 20 minutes + +### Objectives + +* Go to the GitHub page where Oracle AI Optimizer and Toolkit is maintained +* Click on the **Deploy to Cloud button** +* Review the predefined settings +* Get the necessary URLs for the Oracle AI Optimizer and Toolkit + +### Prerequisites + +This lab assumes you have: + +* Completed the *Get Started* Lab +* Access and quota to a GPU shape (the recommended deployment, it will work with a regular CP but, **extremely** slow) + +## Task 1: Start the Oracle Resource Manager job + +1. Got to the [Oracle AI Optimizer and Toolkit GitHub](https://github.com/oracle-samples/ai-optimizer). + +1. Click on the **Deploy to Oracle Cloud** Button. This will take you to the **Create Stack** page in Oracle Cloud. + +## Task 2: Enter the values for the deployment + +1. Review and accept the terms but checking the box and make sure you deploy Oracle AI Optimizer and Toolkit in the correct compartment. Click the **Next** Button. + + ![Accept the terms](images/checkbox.png) + +1. Leave the Application name empty, this will autogenerate an application name. + +1. Choose **VM** as the infrastructure to deploy to. There is an option to deploy to Kubernetes but that is not covered in this Livelab. + +1. Oracle recommends that you deploy to an GPU shape to get decent performance. When deploying to a GPU, as port of the deployment, make available one local Large Language Model and one Embedding Model for use out-of-the-box. Make sure that you have the ability to deploy a GPU Shape. + + LLMs and Embedding Models are designed to use GPUs. This Livelab can work on machines with just CPUs; albeit **extremely** slow! **Note:** A GPU shape has a higher cost. + + If deploying VM on a CPU, you will need to [configure a model](https://oracle-samples.github.io/ai-optimizer/client/configuration/model_config/index.html) for functionality. + + ![GPU Shape](images/appvm.png) + +1. In the Load balancer option Oracle recommends that you add the CIDR block to restrict access for your environment to the **Access Control for Application GUI** and the **Access Control for Application API Server** fields. If not **EVERYONE** can access the environment. + +1. In the Database Options section, option Oracle recommends that you add the CIDR block to restrict access for your environment to the **Access Control for Autonomous Database**. + + ![Loadbalancer and Database options](images/lbdboptions.png) + +1. Click the **Next** Button And review the values. + +1. Click Create. This will create an OCI Resource Manager **Apply** job. + + ![Review](images/lbdboptions.png) + +1. Wait ~5-10 minutes until the **Apply** job is completed. Once the job has **Succeeded**, the AI Optimizer has been deployed! + +1. The **Application Information** tab will provide the URL’s to access the AI Optimizer GUI and API Server. In the “All-in-One” deployment on the VM, the API Server will only become accessible after visiting the GUI at least once. + + ![Access Information](images/appinfo.png) + +1. When accessing the Web GUI using URL provided it should look like this: + + ![Home page](images/homepage.png) + +> **NOTE**: Although the infrastructure is deployed, the AI Optimizer may still be initializing, which can result in a 502 Bad Gateway error when accessing the URLs. Please allow up to 10 minutes for the configuration to complete. + +You may now **proceed to the next lab**. + +## Acknowledgements + +* **Author** - Andy Tael, July 2025 +* **Last Updated By** - Andy Tael, July 2025 diff --git a/ai-optimizer/tenancy/deploy/images/appinfo.png b/ai-optimizer/tenancy/deploy/images/appinfo.png new file mode 100644 index 00000000..16a6e8fb Binary files /dev/null and b/ai-optimizer/tenancy/deploy/images/appinfo.png differ diff --git a/ai-optimizer/tenancy/deploy/images/appvm.png b/ai-optimizer/tenancy/deploy/images/appvm.png new file mode 100644 index 00000000..b92b32cb Binary files /dev/null and b/ai-optimizer/tenancy/deploy/images/appvm.png differ diff --git a/ai-optimizer/tenancy/deploy/images/checkbox.png b/ai-optimizer/tenancy/deploy/images/checkbox.png new file mode 100644 index 00000000..277c0818 Binary files /dev/null and b/ai-optimizer/tenancy/deploy/images/checkbox.png differ diff --git a/ai-optimizer/tenancy/deploy/images/homepage.png b/ai-optimizer/tenancy/deploy/images/homepage.png new file mode 100644 index 00000000..05e96ede Binary files /dev/null and b/ai-optimizer/tenancy/deploy/images/homepage.png differ diff --git a/ai-optimizer/tenancy/deploy/images/lbdboptions.png b/ai-optimizer/tenancy/deploy/images/lbdboptions.png new file mode 100644 index 00000000..b2c3a675 Binary files /dev/null and b/ai-optimizer/tenancy/deploy/images/lbdboptions.png differ diff --git a/ai-optimizer/tenancy/deploy/images/review.png b/ai-optimizer/tenancy/deploy/images/review.png new file mode 100644 index 00000000..0e69abc6 Binary files /dev/null and b/ai-optimizer/tenancy/deploy/images/review.png differ diff --git a/ai-optimizer/tenancy/evaluating/evaluating.md b/ai-optimizer/tenancy/evaluating/evaluating.md new file mode 100644 index 00000000..70271997 --- /dev/null +++ b/ai-optimizer/tenancy/evaluating/evaluating.md @@ -0,0 +1,148 @@ +# Evaluating Performance + +## Introduction + +We are confident that adjusting certain parameters can improve the quality and accuracy of the chatbot’s responses. However, can we be sure that a specific configuration remains reliable when scaled to hundreds or even thousands of different questions? + +In this lab, you will explore the *Testbed* feature. The Testbed allows you to evaluate your chatbot at scale by generating a Q&A test dataset and automatically running it against your current configuration. + +**Note**: The example shown in this lab relies on gpt-4o-mini. Feel free to use your local LLMs (e.g. llama3.1) if you choose to or can't use OpenAI LLMs. + +Estimated Time: 15 minutes + +### Objectives + +In this lab, you will: + +* Explore the *Testbed* tab +* Generate a Q&A Test dataset +* Perform an evaluation on the Q&A Testset + +### Prerequisites + +* All previous labs successfully completed + +## Task 1: Navigate to the Testbed tab + +Access the *Testbed* from the left-hand menu: + +![testbed](./images/testbed.png) + +As a first step, you can either upload an existing Q&A test set—either from a local file or from a saved collection in the database—or generate a new one from a local PDF file. + +## Task 2: Generate a Q&A Test dataset + +The AI Optimizer allows you to generate as many questions and answers as you need, based on a single document from your knowledge base. To enable test dataset generation, simply select the corresponding radio button: + +![generate](./images/generatenew.png) + +1. Upload a document + + Upload the same document that was used to create the vector store. You can easily download it from [this link](https://docs.oracle.com/en/database/oracle/oracle-database/23/vecse/ai-vector-search-users-guide.pdf). + +1. Increase the number of questions to be generated to 10 or more + + Keep in mind that the process can take a significant amount of time, especially if you are using a local LLM without sufficient hardware resources. If you choose to use a remote OpenAI model instead, the generation time will be less affected by the number of Q&A pairs to create. + +1. Leave the default option for: + +* Q&A Language Model: **gpt-4o-mini** +* Q&A Embedding Model: **text-embedding-3-small** + +1. Click on **Generate Q&A** button and wait until the process is over: + + ![patience](./images/patience.png) + +1. Browse the questions and answers generated: + + ![qa-browse](./images/qa-browse.png) + + Note that the **Question** and **Answer** fields are editable, allowing you to modify the proposed Q&A pairs based on the **Context** (which is randomly extracted and not editable) and the **Metadata** generated by the Testbed engine. + + In the *Metadata* field you'll find a **topic** tag that classifies each Q&A pair. The topic list is generated automatically by analyzing the document content and is assigned to each Q&A pair. It will be used in the final report to break down the **Overall Correctness Score** and highlight areas where the chatbot lacks precision. + + You can also export the generated Q&A dataset using the **Download** button. This allows you to edit and review it—e.g., in Visual Studio Code. + + ![qa-json](./images/qa-json.png) + +1. Update the **Test Set Name** + + Replace the automatically generated default name to make it easier to identify the test dataset later, especially when running repeated tests with different chatbot configurations. For example, change it from: + + ![default-test-set](./images/default-test-set.png) + + to something more descriptive, like: + + ![test-rename](./images/test-rename.png) + +## Task 3: Evaluate the Q&A Testset + +Now you are ready to perform an evaluation on the Q&As you generated in the previous step. + +1. In the left-hand menu: + +* Under **Language Model Parameters**, select **gpt-4o-mini** from the **Chat model** dropdown list. +* Ensure **Enable RAG?** is selected (if it wasn't already) +* In the **Select Alias** dropdown list, choose the **TEST2** value. +* Leave all other parameters unchanged + +1. With **gpt-4o-mini** selected as the evaluation model, click the **Start Evaluation** button and wait a few seconds. All questions from your dataset will be submitted to the chatbot using the configuration defined in the left pane: + + ![start-eval](./images/start-eval.png) + +1. Let's examine the result report, starting with the first section: + + ![result](./images/result-topic.png) + + This section displays: + +* The chatbot's **Evaluation Settings**, as configured in the left-hand pane before launching the massive test. +* The **RAG Settings** including the database and vector store used, the name of the embedding **model** used, and all associated parameters (e.g., **chunk size**, **top-k**). +* The **Overall Correctness Score**, representing the percentage of questions for which the LLM judged the chatbot's response as correct compared to the reference answer +* The **Correctness By Topic**, which breaks down the results based on the automatically generated topics assigned to each Q&A pair in the dataset. + + The second section of the report contains details on each question submitted, with a focus on the **Failures** collection and the **Full Report** list. To view all fields, scroll horizontally. In the image below, the second frame has been scrolled to the right: + + ![result](./images/result-question.png) + + The main fields displayed are: + + * **question**: the submitted question + * **reference_answer**: the expected answer used as a benchmark + * **reference_context**: the source document section used to generate the Q&A pair + * **agent_answer**: the response provided by the chatbot based on the current configuration and vector store + * **correctness_reason**: an explanation (if any) of why the response was considered incorrect. If correct, this field will display **None**. + + You can download the results in different formats: + + * Click the **Download Report** button to generate an HTML summary of the *Overall Correctness Score* and *Correctness by Topic* + * To export the **Full Report** and the **Failures** list, download them as .csv files using the download icons shown in the interface: + + ![csv](./images/download-csv.png) + +## Task 4 (optional): Try a different Q&A Testset + +Now let's perform a test using an external saved test dataset, [which you can download here](https://raw.githubusercontent.com/markxnelson/developer/refs/heads/main/ai-optimizer/getting_started-30_testset.json). This file contains 30 pre-generated questions. + +If you wish to remove any Q&A pairs that you consider irrelevant or unhelpful, you can edit the file, save it, and then reload it as a local file—following the steps shown in the screenshot below: + +![load-tests](./images/load-tests.png) + +Next, let’s update the Chat Model parameters by setting the **Temperature** to **0** in the left-hand pane under the **Language Model Parameters** section. +Why? Q&A datasets are typically generated with a low level of creativity to minimize randomness and focus on expressing core concepts clearly—avoiding unnecessary "frills" in the answers. + +Now, repeat the test to see whether there are improvements in the **Overall Correctness Score**. + +* To compare with previous results, open the dropdown under **Previous Evaluations for...** and click on the **View** button to display the associated report. + +![previous](./images/previous.png) + +* You can repeat the tests as many times as needed, changing the **Vector Store**, **Search Type**, and **Top K** parameters to apply the same tuning strategies you've used previously with individual questions—now extended to a full test using curated and reproducible data. + +You may now **proceed to the next lab**. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Andy Tael, July 2025 diff --git a/ai-optimizer/tenancy/evaluating/images/default-test-set.png b/ai-optimizer/tenancy/evaluating/images/default-test-set.png new file mode 100644 index 00000000..11a047ea Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/default-test-set.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/download-csv.png b/ai-optimizer/tenancy/evaluating/images/download-csv.png new file mode 100644 index 00000000..22f87dbb Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/download-csv.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/generatenew-ollama.png b/ai-optimizer/tenancy/evaluating/images/generatenew-ollama.png new file mode 100644 index 00000000..8889862e Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/generatenew-ollama.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/generatenew.png b/ai-optimizer/tenancy/evaluating/images/generatenew.png new file mode 100644 index 00000000..9c1fdc47 Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/generatenew.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/load-tests.png b/ai-optimizer/tenancy/evaluating/images/load-tests.png new file mode 100644 index 00000000..89a38e14 Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/load-tests.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/patience.png b/ai-optimizer/tenancy/evaluating/images/patience.png new file mode 100644 index 00000000..0d9cc381 Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/patience.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/previous.png b/ai-optimizer/tenancy/evaluating/images/previous.png new file mode 100644 index 00000000..7afb2529 Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/previous.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/qa-browse.png b/ai-optimizer/tenancy/evaluating/images/qa-browse.png new file mode 100644 index 00000000..a6df294c Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/qa-browse.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/qa-json.png b/ai-optimizer/tenancy/evaluating/images/qa-json.png new file mode 100644 index 00000000..20a3b6da Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/qa-json.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/result-question.png b/ai-optimizer/tenancy/evaluating/images/result-question.png new file mode 100644 index 00000000..ae9c1bcf Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/result-question.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/result-topic.png b/ai-optimizer/tenancy/evaluating/images/result-topic.png new file mode 100644 index 00000000..2704b259 Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/result-topic.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/start-eval.png b/ai-optimizer/tenancy/evaluating/images/start-eval.png new file mode 100644 index 00000000..13e7a05a Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/start-eval.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/test-rename.png b/ai-optimizer/tenancy/evaluating/images/test-rename.png new file mode 100644 index 00000000..0a516a63 Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/test-rename.png differ diff --git a/ai-optimizer/tenancy/evaluating/images/testbed.png b/ai-optimizer/tenancy/evaluating/images/testbed.png new file mode 100644 index 00000000..59ad8ddd Binary files /dev/null and b/ai-optimizer/tenancy/evaluating/images/testbed.png differ diff --git a/ai-optimizer/tenancy/experimenting/experimenting.md b/ai-optimizer/tenancy/experimenting/experimenting.md new file mode 100644 index 00000000..77588580 --- /dev/null +++ b/ai-optimizer/tenancy/experimenting/experimenting.md @@ -0,0 +1,293 @@ +# Experimenting with Models, Parameters and Techniques + +## Introduction + +In this lab, we will experiment with different configurations, adjusting parameters and using various LLMs to observe how the answers vary. + +> **NOTE**: The answers received by any LLM are based on probabilistic models. Therefore, the answer received in your own testing might differ from the ones shown in this documentation. + +Estimated Lab Time: 15 minutes + +### Objectives + +In this lab, you will: + +* Test different parameter configurations and compare the resulting answers +* Apply various distance metrics to the same set of questions and evaluate how the answers differ +* Experiment with different vector stores and compare the answers + +### Prerequisites + +This lab assumes you have: + +* All previous labs successfully completed + +## Task 1: Play with the Parameters + +Using the **TEST1** vector store, you will experiment with the main parameters configured as shown below to observe how answer quality changes. We will begin by adjusting the **Temperature** parameter. For the first test, set the Temperature to 0 to minimize creativity in the responses. Then, repeat the same question with a different Temperature value to observe how the answers change. + +**IMPORTANT**: After each question—regardless of whether you're repeating the same one with different settings—be sure to clear the history by clicking the **Clear** button. We will start by testing the first question. + +* LLM: llama3.1 +* TopK: 4 +* Search Type: Similarity +* Temperature: 0 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I determine the accuracy of my vector indexes? + +A: The user wants to determine the accuracy of their vector indexes in Oracle Database 23c. According to the provided documentation, there are several ways to achieve this: + +1. Use the DBMS_VECTOR.INDEX_ACCURACY_QUERY procedure: This procedure provides an accuracy report for a top-K index search for a specific query vector and a specific target accuracy. +2. Use the DBMS_VECTOR.INDEX_ACCURACY_REPORT function: This function captures accuracy values achieved by approximate searches using a particular vector index for a certain period of time. +3. Use the INDEX_VECTOR_MEMORY_ADVISOR procedure to determine the vector memory size. +Here is an excerpt from the documentation: +To verify the accuracy of a vector index, you can use the DBMS_VECTOR.INDEX_ACCURACY_QUERY function. This function takes four parameters: OWNER_NAME, INDEX_NAME, QV (query vector), and TOP_K (top K results). +The user can also refer to the Oracle Database AI Vector Search User's Guide for more information on determining the accuracy of their vector indexes. +Document references: +* DBMS_VECTOR.INDEX_ACCURACY_QUERY procedure in the provided text +* DBMS_VECTOR.INDEX_ACCURACY_REPORT function in the provided text +* INDEX_VECTOR_MEMORY_ADVISOR procedure in the provided text +``` + +* LLM: llama3.1 +* TopK: 4 +* Search Type: Similarity +* Temperature: 1 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I determine the accuracy of my vector indexes? + +A: To determine the accuracy of your vector indexes in Oracle Database 23c AI, you can use the DBMS_VECTOR.INDEX_ACCURACY_QUERY procedure. +According to the provided documentation, this procedure provides an accuracy report for a top-K index search for a specific query vector and a specific target accuracy +Here's how you can do it: +Declare the necessary variables: + +declare + q_v VECTOR; + report varchar2(128); + +Initialize the query vector q_v with the desired vector (in this case, [0,1,1,0,0]): + +q_v := to_vector('[0,1,1,0,0]'); + +Execute the DBMS_VECTOR.INDEX_ACCURACY_QUERY procedure: + +exec DBMS_VECTOR.INDEX_ACCURACY_QUERY('OWNER_NAME', 'INDEX_NAME', q_v, TOP_K, target_accuracy); + +Replace 'OWNER_NAME', 'INDEX_NAME', TOP_K, and target_accuracy with your actual values. +This will provide you with an accuracy report for the specified vector index. +Additionally, you can use the DBMS_VECTOR.INDEX_ACCURACY_REPORT function to capture accuracy values achieved by approximate searches using a particular vector index for a certain period of time. +``` + +**COMMENT**: As you can see, setting the Temperature to 1 increases the variability of the output, often resulting in an answer that differs significantly from the previous one. + +We will now experiment with the **TopK** parameter, while keeping Temperature fixed at 0 to minimize output variability. + +**IMPORTANT**: After each question—regardless of whether you're repeating the same one with different settings—be sure to clear the history by clicking the **Clear** button. + +Let's move onto a different question and see the results: + +* LLM: llama3.1 +* TopK: 4 +* Search Type: Similarity +* Temperature: 0 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I convert string text to embeddings using PL/SQL? + +A: To generate a vector embedding from an input text string in Oracle Database 23c, you can use the UTL_TO_EMBEDDING PL/SQL function. This function is designed to take a text string as input and return its equivalent vector embedding. +To convert a text string to an embedding using this function, follow these steps: +The database needs to have access to an external vector embedding model stored either internally in the database or accessed through a third-party BINARY vector embedding. +Construct your query such that it makes a call on UTL_TO_EMBEDDING. This function likely requires at least one parameter which would be the text string to convert into embeddings. +For example, if you have a table with a column named 'text' where each row contains the string for conversion: + +SELECT UTL_TO_EMBEDDING(text) FROM YOUR_TABLE; + +Make sure that your database setup includes the access details required for whatever vector embedding model is being referenced. Without knowing more about how UTL_TO_EMBEDDING function expects setup on such models within Oracle databases, it's not possible to offer more specifics than this. +``` + +* LLM: llama3.1 +* TopK: 10 +* Search Type: Similarity +* Temperature: 0 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I convert string text to embeddings using PL/SQL? + +A: To convert a string text to an embedding using PL/SQL in Oracle Database 23c (not 23ai, as there is no such version), you can use the DBMS_VECTOR package. +Here's a simple example: + +DECLARE + input CLOB; + v VECTOR; + +BEGIN + input := 'Your string here'; + SELECT dbms_vector.utl_to_embedding(input, json('{"provider":"database", "model":"doc_model"}')) INTO v FROM dual; + + DBMS_OUTPUT.PUT_LINE(DBMS_VECTOR.util_serialize(v)); +END; +/ +In this code: +input is the string you want to convert. +"{'provider\":\"database\", \"model\":\"doc_model\"}" is a JSON object that specifies the vectorization model (in this case, an in-database model named 'doc_model'). +The DBMS_OUTPUT.PUT_LINE(DBMS_VECTOR.util_serialize(v)) statement then outputs a JSON representation of the generated embedding. +Note: Make sure you have loaded your ONNX model into Oracle Database first by using dbms_math.load_onnx_model package (which is not shown in this code snippet). +Note that there are similar methods for converting strings to binary embeddings outside the database through APIs, and it requires more parameters depending on which kind of embedding or library you use. +``` + +**COMMENT**: As you can see, increasing the TopK parameter prompts the LLM to retrieve a larger number of vectors from the vector database. This provides the model with more contextual information to rely on when using RAG, resulting in significantly different answers compared to the example with TopK set to 4. + +## Task 2: Use a different Distance Metric + +We will now observe how using a different distance metric can lead to a different answer. For this test, we will set Temperature to 0 and TopK to 4. + +**IMPORTANT**: After each question—regardless of whether you're repeating the same one with different settings—be sure to clear the history by clicking the **Clear** button. + +* LLM: llama3.1 +* TopK: 4 +* Search Type: Similarity +* Temperature: 0 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I determine the accuracy of my vector indexes? + +A: To determine the accuracy of your vector indexes in Oracle Database 23c (not 23ai), you can use the DBMS_VECTOR package. +According to the documentation provided, you have two options: +DBMS_VECTOR.INDEX_ACCURACY_QUERY: This function allows you to verify the accuracy of a vector index for a given query vector, top-K, and target accuracy. You can call this function by passing in the OWNER_NAME, INDEX_NAME, QV (query vector), TOP_K (top K results), and TARGET_ACCURACY as parameters. +DBMS_VECTOR.INDEX_ACCURACY_REPORT: This function captures the accuracy values achieved by approximate searches using a particular vector index for a certain period of time. +Here's an example code snippet that demonstrates how to use DBMS_VECTOR.INDEX_ACCURACY_QUERY: + +DECLARE + q_v VECTOR; + report CLOB; + +BEGIN + q_v := TO_VECTOR('[0,1,1,0,0]'); + + REPORT := DBMS_VECTOR.INDEX_ACCURACY_QUERY( + OWNER_NAME => 'VECTOR_USER', + INDEX_NAME => 'VIDX_HNSW', + QV => q_v, + TOP_K => 10, + TARGET_ACCURACY => 0.8); + + DBMS_OUTPUT.PUT_LINE(report); +END; + +Make sure to replace the placeholder values with your actual OWNER_NAME, INDEX_NAME, and vector values. +I hope this helps! +``` + +* LLM: llama3.1 +* Top K: 4 +* Fetch K: 20 +* Degree of Diversity: 0.5 +* Search Type: Maximal Marginal Relevance +* Temperature: 0 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I determine the accuracy of my vector indexes? + +A: To determine the accuracy of your vector indexes in Oracle Database 23c (note: not 23ai), you can use the DBMS_VECTOR.INDEX_ACCURACY_QUERY procedure. +According to the provided documentation and example usage, you need to: +Declare a query vector (q_v) using the TO_VECTOR function. +Call the INDEX_ACCURACY_QUERY procedure with q_v as a parameter, along with other optional parameters as needed (e.g., target accuracy). +Here's an adapted code snippet from the example usage: + +DECLARE + q_v VECTOR; + report VARCHAR2(128); + +BEGIN + q_v := to_vector('[0,1,1,0,0]'); + + -- Set other parameters as needed, e.g., target accuracy + REPORT := DBMS_VECTOR.INDEX_ACCURACY_QUERY(q_v, <target_accuracy>); +END; +/ +This will provide an accuracy report for the specified query vector and target accuracy. +Alternatively, you can use the TOVECTOR function to insert vector data into your application. +``` + +**COMMENT**: In this case, even though the MMR retrieves 20 chunks and removes results that differ by less than 50% in diversity, ultimately providing 4 top-K results that should offer more varied information, the results are better than with a plain similarity search using top-K = 4. This approach can, for example, increase efficiency in production, as it directly provides the 4 top-K chunks, rather than adding an extra step in the execution process that involves computing and fetching the 4 most diverse chunks from the 20 top-K results retrieved. + +## (Optional) Task 3: Compare two different vector stores + +> **NOTE**: You can perform this task only if you have access to [OpenAI APIs](https://platform.openai.com/settings/organization/api-keys). + +We can also explore how different combinations of **LLMs** and vector stores can produce markedly different answers. To test this, we’ll experiment with various pairings—and introduce a completely new question, purely for exploration purposes. + +**IMPORTANT**: After each question—regardless of whether you're repeating the same one with different settings—be sure to clear the history by clicking the **Clear** button. + +* LLM: llama3.1 +* TopK: 4 +* Search Type: Similarity +* Temperature: 0 +* Vector Store: TEST1 + +```text +Q: In Oracle Database 23ai, how do I create a vector column? + +A: To create a vector column in Oracle Database 23c1, you can use the VECTOR data type. The syntax to declare a vector column is as follows: + +CREATE TABLE table_name ( + id NUMBER, + embedding VECTOR +); + +You don't need to specify the number of dimensions or their format initially. This allows you to enter vectors of different dimensions with various formats. You can use the TO_VECTOR function with specific dimensions and format if needed, as shown in a note in your passage. +Here is an example SQL statement that creates a vector column without specifying dimensions and format (simplified): + +CREATE TABLE my_vectors ( + id NUMBER, + embedding VECTOR +); + +You can then insert data into the embedding column using either a string representation of the vector or the explicit use of the TO_VECTOR function with specific number of dimensions, format, or storage format. +``` + +* LLM: gpt-4o-mini +* TopK: 4 +* Search Type: Similarity +* Temperature: 0 +* Vector Store: TEST2 + +```text +Q: In Oracle Database 23ai, how do I create a vector column? + +A: To create a vector column in Oracle Database 23ai, you can declare a table with a column of the VECTOR data type. Here is a simple example: + +CREATE TABLE my_vectors (id NUMBER, embedding VECTOR); + +In this example, the embedding column is defined as a VECTOR type. You do not need to specify the number of dimensions or their format if you want flexibility. If you want to impose constraints, you can specify them like this: + +CREATE TABLE my_vectors (id NUMBER, embedding VECTOR(768, INT8)); + +In this case, each vector must have 768 dimensions formatted as INT8. For more detailed information, see the section on creating tables using the VECTOR data type in the documentation. +``` + +**COMMENT**: Although the overall content remains essentially the same, we can observe notable differences in both the syntax and structure of the answers. The OpenAI-based configuration produces a more concise and precise response, whereas the Ollama-based output is more verbose—and even contains a typo. These differences are largely due to the distinct chunking sizes used by the two vector stores. + +These few examples offer just a glimpse of the cross-testing possibilities available to users. The range of parameters you can experiment with is much broader. The **AI Optimizer and Toolkit** is specifically designed to help you identify the configuration that best suits your needs. + +## Learn More + +* Cross-test vector stores and models. For example, you can test **llama3.1** while using the **text-embedding-3-small** vector store. +* Try setting the Temperature parameter to its maximum value—you’ll see how creative an LLM can become! +* Experiment with other available parameters across different questions to explore the full range of behavior. + +You may now **proceed to the next lab**. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Andy Tael, July 2025 diff --git a/ai-optimizer/tenancy/experimenting/images/sample1.png b/ai-optimizer/tenancy/experimenting/images/sample1.png new file mode 100644 index 00000000..2d9ad473 Binary files /dev/null and b/ai-optimizer/tenancy/experimenting/images/sample1.png differ diff --git a/ai-optimizer/tenancy/experimenting/images/sample2.png b/ai-optimizer/tenancy/experimenting/images/sample2.png new file mode 100644 index 00000000..145c9d39 Binary files /dev/null and b/ai-optimizer/tenancy/experimenting/images/sample2.png differ diff --git a/ai-optimizer/tenancy/explore/explore.md b/ai-optimizer/tenancy/explore/explore.md new file mode 100644 index 00000000..ff0f92cf --- /dev/null +++ b/ai-optimizer/tenancy/explore/explore.md @@ -0,0 +1,129 @@ +# Explore The Environment + +## Introduction + +In this lab, you will explore the environment that was created in the *Deploy the Oracle AI Optimizer and Toolkit* lab. You will set up the database connection, configure the OCI credentials, add new LLMs and Embedding models and test out the *Chat* feature for the first time. + +Estimated Time: 15 minutes + +### Objectives + +In this lab, you will: + +* Set the connection to your Oracle Database 23ai +* Set your Oracle Cloud Infrastructure credentials for OCI connectivity +* Configure the LLMs and embedding models you will use in the following labs +* Test the *Chat* feature for the first time + +### Prerequisites + +This lab assumes you have: + +* Completed the *Deploy the Oracle AI Optimizer and Toolkit* Lab + +## Task 1: Check the DB connection + +Let's check if the DB is correctly connected. + +1. Navigate to the *Databases* tab on the left side pane: + + ![Navigate to the Databases tab](images/database-navigation.jpg) + +1. To configure the Oracle Database 23ai Free, you will need to enter the credentials: + + * Enter the Database Username + * Enter the Database Connection String + * Enter the Database Password for the database user + * Save + + Since you are following the Tenancy version of this LiveLab, you should find everything set-up already: + + ![Database configuration](images/database-config.png) + +1. Since you are using an Autonomous Database, you can activate the SelectAI functionality. You will need to create a [Select AI](https://www.oracle.com/it/autonomous-database/select-ai/) profile and eventually you'll find the list of active profiles within the tenancy: + + ![Select AI Profiles](images/selectai-profiles.png) + +## Task 2: Configure LLMs + +Let's check the models available for use. You'll need to navigate to the *Models* tab: + + ![models menu](images/models.jpg) + +* The default LLMs for chat completions are: + + ![llms](images/llms.png) + +* The default LLMs for embeddings are: + + ![embeddings](images/emb.png) + + * (optional) If you are willing to use models by OpenAI, you will need to configure your **OPENAI API KEY**. To configure one, click the *Edit* button beside the model you would like to use (e.g., **gpt-4o-mini**) and add your own API key in the corresponding box: + + ![openai-key](images/openai-api.png) + +Now you are all set for using the *Chat* feature with the LLMs you just configured! + +## Task 3: Test the *Chat* feature + +The two LLMs availble could be tested right away, in order to understand their behavior with generic questions. First of all, navigate to the chat GUI + +![chat](images/chat.jpg) + +scroll down the left-side menu to find the **Toolkit** menu: + +![toolkit menu](images/toolkit-menu.png) + +select the **LLM Only** option and then choose the **llama3.1** model: + +![chat models](images/chatmodel.png) + +and ask generic questions like: + +```text +<copy> +In Oracle Database 23ai, how do I determine the accuracy of my vector indexes? +</copy> +``` + +```text +<copy> +In Oracle Database 23ai, how do I convert string text to embedding using PL/SQL? +</copy> +``` + +> **NOTE**: *if you see a **Database has no Vector Stores. Disabling Vector Search.** message above, don't panic! That's because you haven't created a vector store yet and thus you can't use the RAG functionality*. + +As you will probably be able to notice, even if the questions refer to Oracle Database 23ai, the LLM will mention Oracle Database 23c. This is known as **Knowledge Cutoff**, meaning that probably the LLM was trained before Oracle Database 23c was even renamed as 23ai. + +Moreover, the LLM tends to answer in a generic way, with no specific mention to the actual script needed to perform the operations requested. In Lab 3, we will compare these results with the one obtained using Retrieval-Augmented Generation (RAG), where more context is given to the LLMs. + +## (Optional) Task 4: Set OCI Credentials + +The Optimizer lets you configure the connection to your OCI tenant for retrieving objects from the *Object Storage* and accessing LLMs from the OCI GenAI service. In the *OCI* configuration tab, you can add your **Oracle Cloud Infrastructure** (OCI) credentials to authenticate to your OCI tenancy. This will enable access to objects and documents stored in your cloud compartments. + +1. Navigate to the *OCI* tab on the left-hand pane: + + ![Navigate to the Databases tab](images/oci-navigation.jpg) + +1. Insert your OCI credentials. Detailed information on how to get the required credentials is available in the [Oracle Cloud Infrastructure Documentation](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/apisigningkey.htm#Required_Keys_and_OCIDs). + +In this livelab, you will have the credentials fields automatically filled to connect to the tenancy that has been deployed at startup, but you can also connect to any other OCI tenancy you have access to. + +After entering your credentials, click the `Save` button. If the credentials are correct, a green confirmation pop-up will appear, indicating successful authentication to your tenancy. + +![OCI credentials success](images/oci-credentials-success.png) + +## Learn More + +* (optional) Now click the **Clear** button under the **History and Context** section, and choose the other available LLM, **gpt-4o-mini**. Then, ask the same questions and compare the results. Note that the History is enabled by default. The **Clear** button resets the *context window* and starts a fresh interaction with a model. + +* (optional) Play with the **Temperature** parameter (and also the other parameters if you wish to!) and compare the quality of the answers, for each LLM that is available. Clear the history by pressing the **Clear** button after each cycle. + +You may now **proceed to the next lab**. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Andy Tael, July 2025 diff --git a/ai-optimizer/tenancy/explore/images/addllama32.png b/ai-optimizer/tenancy/explore/images/addllama32.png new file mode 100644 index 00000000..89844627 Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/addllama32.png differ diff --git a/ai-optimizer/tenancy/explore/images/chat.jpg b/ai-optimizer/tenancy/explore/images/chat.jpg new file mode 100644 index 00000000..d8f21e75 Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/chat.jpg differ diff --git a/ai-optimizer/tenancy/explore/images/chatmodel.png b/ai-optimizer/tenancy/explore/images/chatmodel.png new file mode 100644 index 00000000..73c6b9fc Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/chatmodel.png differ diff --git a/ai-optimizer/tenancy/explore/images/database-config.png b/ai-optimizer/tenancy/explore/images/database-config.png new file mode 100644 index 00000000..f4a70441 Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/database-config.png differ diff --git a/ai-optimizer/tenancy/explore/images/database-navigation.jpg b/ai-optimizer/tenancy/explore/images/database-navigation.jpg new file mode 100644 index 00000000..93f55303 Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/database-navigation.jpg differ diff --git a/ai-optimizer/tenancy/explore/images/emb.png b/ai-optimizer/tenancy/explore/images/emb.png new file mode 100644 index 00000000..99f00981 Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/emb.png differ diff --git a/ai-optimizer/tenancy/explore/images/llms.png b/ai-optimizer/tenancy/explore/images/llms.png new file mode 100644 index 00000000..b93b7b1d Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/llms.png differ diff --git a/ai-optimizer/tenancy/explore/images/models.jpg b/ai-optimizer/tenancy/explore/images/models.jpg new file mode 100644 index 00000000..a6ad39a3 Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/models.jpg differ diff --git a/ai-optimizer/tenancy/explore/images/oci-credentials-success.png b/ai-optimizer/tenancy/explore/images/oci-credentials-success.png new file mode 100644 index 00000000..b21e8118 Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/oci-credentials-success.png differ diff --git a/ai-optimizer/tenancy/explore/images/oci-navigation.jpg b/ai-optimizer/tenancy/explore/images/oci-navigation.jpg new file mode 100644 index 00000000..09c5cb8c Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/oci-navigation.jpg differ diff --git a/ai-optimizer/tenancy/explore/images/openai-api.png b/ai-optimizer/tenancy/explore/images/openai-api.png new file mode 100644 index 00000000..e72162ba Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/openai-api.png differ diff --git a/ai-optimizer/tenancy/explore/images/selectai-profiles.png b/ai-optimizer/tenancy/explore/images/selectai-profiles.png new file mode 100644 index 00000000..a8ea36bb Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/selectai-profiles.png differ diff --git a/ai-optimizer/tenancy/explore/images/toolkit-menu.png b/ai-optimizer/tenancy/explore/images/toolkit-menu.png new file mode 100644 index 00000000..bd9a6792 Binary files /dev/null and b/ai-optimizer/tenancy/explore/images/toolkit-menu.png differ diff --git a/ai-optimizer/tenancy/get-started/files/db-initialize.sql b/ai-optimizer/tenancy/get-started/files/db-initialize.sql new file mode 100644 index 00000000..d1b2a267 --- /dev/null +++ b/ai-optimizer/tenancy/get-started/files/db-initialize.sql @@ -0,0 +1,11 @@ +alter system set vector_memory_size=512M scope=spfile; + +alter session set container=FREEPDB1; + +CREATE USER "WALKTHROUGH" IDENTIFIED BY OrA_41_OpTIMIZER + DEFAULT TABLESPACE "USERS" + TEMPORARY TABLESPACE "TEMP"; +GRANT "DB_DEVELOPER_ROLE" TO "WALKTHROUGH"; +ALTER USER "WALKTHROUGH" DEFAULT ROLE ALL; +ALTER USER "WALKTHROUGH" QUOTA UNLIMITED ON USERS; +EXIT; \ No newline at end of file diff --git a/ai-optimizer/tenancy/get-started/get-started.md b/ai-optimizer/tenancy/get-started/get-started.md new file mode 100644 index 00000000..8fed88c6 --- /dev/null +++ b/ai-optimizer/tenancy/get-started/get-started.md @@ -0,0 +1,215 @@ +# Get started - Installation and Setup + +## Introduction + +The **AI Optimizer** is available to install in your own environment, which may be a developer’s desktop, on-premises data center environment, or a cloud provider. It can be run either on bare-metal, within a container, or in a Kubernetes Cluster. + +This walkthrough will guide you through a basic installation of the **Oracle AI Optimizer and Toolkit** (the **AI Optimizer**). It will allow you to experiment with GenAI, using Retrieval-Augmented Generation (RAG) with Oracle Database 23ai at the core. + +You will run four container images to establish the “Infrastructure”: + +* On-Premises LLM - **llama3.1** +* On-Premises Embedding Model - **mxbai-embed-large** +* Vector Storage - **Oracle Database 23ai Free** +* The AI Optimizer + +> **NOTE**: The walkthrough will reference podman commands. If applicable to your environment, podman can be substituted with docker. If you are using docker, make the walkthrough easier by aliasing the podman command: + +````bash +<copy> +alias podman=docker +</copy> +```` + +Estimated Time: 60 minutes + +### Objectives + +* Perform a full on-premise installation of the **Oracle AI Optimizer and Toolkit** + +### Prerequisites + +This lab assumes you have: + +* An Integrated Development Editor (like **Visual Studio Code**) +* Python 3.11 +* Container Runtime e.g. docker/podman (for running in a Container) +* Internet Access (docker.io and container-registry.oracle.com) +* **100G** of free disk space. +* **12G** of usable memory. +* Sufficient GPU/CPU resources to run the LLM, embedding model, and database + +## Task 1: LLM - llama3.1 + +To enable the _ChatBot_ functionality, access to a **LLM** is required. This workshop will use [Ollama](https://ollama.com/) to run the _llama3.1_ **LLM**. + +1. Start the *Ollama* container: + + The Container Runtime is native: + + ```bash + <copy> + podman run -d --gpus=all -v ollama:$HOME/.ollama -p 11434:11434 --name ollama docker.io/ollama/ollama + </copy> + ``` + + If you don't have access to a GPU, you will have to omit the '--gpus=all' parameter: + + ```bash + <copy> + podman run -d -v ollama:$HOME/.ollama -p 11434:11434 --name ollama docker.io/ollama/ollama + </copy> + ``` + + > **Note:** + AI Runners like Ollama, LM Studio, etc. will not utilize Apple Silicon's "Metal" GPU when running in a container. This may change as the landscape evolves. You can install and run Ollama natively outside a container and it will take advantage of the "Metal" GPU. Later in the Workshop, when configuring the models, the API URL for the Ollama model will be your hosts IP address. + +2. Pull the **LLM** into the container: + + ```bash + <copy> + podman exec -it ollama ollama pull llama3.1 + </copy> + ``` + +3. Test the **LLM**: + + ```bash + <copy> + curl http://127.0.0.1:11434/api/generate -d '{ + "model": "llama3.1", + "prompt": "Why is the sky blue?", + "stream": false + }' + </copy> + ``` + + Unfortunately, if the above `curl` does not respond within 5-10 minutes, the rest of the workshop will be unbearable. + If this is the case, please consider using different hardware. + +## Task 2: Embedding - mxbai-embed-large + +To enable the **RAG** functionality, access to an embedding model is required. In this workshop you will use [Ollama](https://ollama.com/) to run the _mxbai-embed-large_ embedding model. + +1. Pull the embedding model into the container: + + ```bash + <copy> + podman exec -it ollama ollama pull mxbai-embed-large + </copy> + ``` + +## Task 3: The AI Optimizer + +The **AI Optimizer** provides an easy to use front-end for experimenting with **LLM** parameters and **RAG**. + +1. Download and Unzip the latest version of the **AI Optimizer**: + + ```bash + <copy> + curl -L -o ai-optimizer.tar.gz https://github.com/oracle-samples/ai-optimizer/archive/refs/heads/main.tar.gz + mkdir ai-optimizer + tar zxf ai-optimizer.tar.gz --strip-components=1 -C ai-optimizer + </copy> + ``` + +2. Create and activate a Python Virtual Environment: + + ```bash + <copy> + cd ai-optimizer/src + python3.11 -m venv .venv --copies + source .venv/bin/activate + pip3.11 install --upgrade pip wheel setuptools + </copy> + ``` + +3. Install the Python modules: + + ```bash + <copy> + pip3.11 install -e ".[all]" + source .venv/bin/activate + </copy> + ``` +4. Start the AI Optimizer: + + ```bash + <copy> + streamlit run launch_client.py --server.port 8501 + </copy> + ``` + + If you are running on a remote host, you may need to allow access to the `8501` port. + + For example, in Oracle Linux 8/9 with `firewalld`: + + ```bash + <copy> + firewall-cmd --zone=public --add-port=8501/tcp + </copy> + ``` + +## Task 4: Vector Storage - Oracle Database 23ai Free + +AI Vector Search in Oracle Database 23ai provides the ability to store and query by similarity text and multimedia data. The AI Optimizer uses these capabilities to provide more accurate and relevant **LLM** responses via Retrieval-Augmented Generation (**RAG**). [Oracle Database 23ai Free](https://www.oracle.com/uk/database/free/get-started/) provides an ideal, no-cost vector store for this workshop. + +To start Oracle Database 23ai Free: + +1. Start the container: + + ```bash + <copy> + podman run -d --name ai-optimizer-db -p 1521:1521 container-registry.oracle.com/database/free:latest + </copy> + ``` + +2. Alter the `vector_memory_size` parameter and create a [new database user](../client/configuration/db_config#database-user): + + ```bash + <copy> + podman exec -it ai-optimizer-db sqlplus '/ as sysdba' + </copy> + ``` + + ```sql + alter system set vector_memory_size=512M scope=spfile; + + alter session set container=FREEPDB1; + + CREATE USER "WALKTHROUGH" IDENTIFIED BY OrA_41_OpTIMIZER + DEFAULT TABLESPACE "USERS" + TEMPORARY TABLESPACE "TEMP"; + GRANT "DB_DEVELOPER_ROLE" TO "WALKTHROUGH"; + ALTER USER "WALKTHROUGH" DEFAULT ROLE ALL; + ALTER USER "WALKTHROUGH" QUOTA UNLIMITED ON USERS; + EXIT; + ``` + +3. Bounce the database for the `vector_memory_size` to take effect: + + ```bash + <copy> + podman container restart ai-optimizer-db + </copy> + ``` + +Now you are all set! With the "Infrastructure" in-place, you are ready to configure the AI Optimizer. + +In a web browser, navigate to `http://localhost:8501` . + +![Chatbot](images/chatbot-no-models.png) + +Notice that there are no language models configured to use. We will deal with the configuration in the next Lab. + +> **NOTE**: In the next steps of this lab, you will need to check the items inside your database 23ai. This lab uses the VS Code **SQL Developer** plugin, but feel free to use the tool you prefer: + +![sql-developer-plugin](images/sql-developer-plugin.png) + +You may now **proceed to the next lab**. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes +* **Last Updated By** - Lorenzo De Marchis, June 2025 \ No newline at end of file diff --git a/ai-optimizer/tenancy/get-started/images/chatbot-no-models.png b/ai-optimizer/tenancy/get-started/images/chatbot-no-models.png new file mode 100644 index 00000000..3ebcd554 Binary files /dev/null and b/ai-optimizer/tenancy/get-started/images/chatbot-no-models.png differ diff --git a/ai-optimizer/tenancy/get-started/images/sql-developer-plugin.png b/ai-optimizer/tenancy/get-started/images/sql-developer-plugin.png new file mode 100644 index 00000000..795caa33 Binary files /dev/null and b/ai-optimizer/tenancy/get-started/images/sql-developer-plugin.png differ diff --git a/ai-optimizer/tenancy/introduction/introduction.md b/ai-optimizer/tenancy/introduction/introduction.md new file mode 100644 index 00000000..0efabd42 --- /dev/null +++ b/ai-optimizer/tenancy/introduction/introduction.md @@ -0,0 +1,36 @@ +# Introduction + +## About this Workshop + +The **Oracle AI Optimizer and Toolkit** (the **AI Optimizer**) provides a streamlined environment where developers and data scientists can explore the potential of Generative Artificial Intelligence (**GenAI**) combined with Retrieval-Augmented Generation (**RAG**) capabilities. By integrating Oracle Database AI Vector Search, the AI Optimizer enables users to enhance existing Large Language Models (**LLMs**) through RAG. This method significantly improves the performance and accuracy of AI models, helping to avoid common issues such as knowledge cutoff and hallucinations. + +* **GenAI**: Powers the generation of text, images, or other data based on prompts using pre-trained LLMs. +* **RAG**: Enhances LLMs by retrieving relevant, real-time information allowing models to provide up-to-date and accurate responses. +* **Vector Database**: A database, including Oracle Database 23ai, that can natively store and manage vector embeddings and handle the unstructured data they describe, such as documents, images, video, or audio. + +Estimated Time: 120 minutes + +### Objectives + +In this workshop, you will learn how to: + +* Explore the AI Optimizer environment +* Embed documents in a vector format within **Oracle Database 23ai** +* Use Retrieval Augmented Generation (**RAG**) techniques +* Experiment with different models, parameters and techniques +* Evaluate model performances with the *Testbed* functionality +* Export a RAG-based *Springboot* Microservice +* Use the AI Optimizer as an API Server + +### Prerequisites + +This lab assumes you have: + +* An Integrated Development Editor (like Visual Studio Code) or a regular CLI +* (optional) API Keys for Third-Party Models (e.g., OpenAI API Keys that you can get [here](https://platform.openai.com/settings/organization/api-keys)) + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Andy Tael, 2025 diff --git a/ai-optimizer/tenancy/microservice/images/diff-llm-springai.png b/ai-optimizer/tenancy/microservice/images/diff-llm-springai.png new file mode 100644 index 00000000..7a2aef82 Binary files /dev/null and b/ai-optimizer/tenancy/microservice/images/diff-llm-springai.png differ diff --git a/ai-optimizer/tenancy/microservice/images/download-springai.png b/ai-optimizer/tenancy/microservice/images/download-springai.png new file mode 100644 index 00000000..9b6ddac7 Binary files /dev/null and b/ai-optimizer/tenancy/microservice/images/download-springai.png differ diff --git a/ai-optimizer/tenancy/microservice/images/microservice-started.png b/ai-optimizer/tenancy/microservice/images/microservice-started.png new file mode 100644 index 00000000..d52cd9f8 Binary files /dev/null and b/ai-optimizer/tenancy/microservice/images/microservice-started.png differ diff --git a/ai-optimizer/tenancy/microservice/images/select-test1.png b/ai-optimizer/tenancy/microservice/images/select-test1.png new file mode 100644 index 00000000..126822f9 Binary files /dev/null and b/ai-optimizer/tenancy/microservice/images/select-test1.png differ diff --git a/ai-optimizer/tenancy/microservice/microservice.md b/ai-optimizer/tenancy/microservice/microservice.md new file mode 100644 index 00000000..ec174dba --- /dev/null +++ b/ai-optimizer/tenancy/microservice/microservice.md @@ -0,0 +1,132 @@ +# (Optional) Generating Microservice Code + +## Introduction + +The AI Optimizer allows you to export the configured chatbot as a ready-to-run microservice built using Java, Spring Boot and the Spring AI framework. This microservice runs independently from the AI Optimizer, relying solely on the created vector store table and the selected LLM servers. + +In the current release, only fully self-contained Ollama configurations (embedding + chat model) or OpenAI-based configurations are supported. + +Estimated Time: 10 minutes + +### Objectives + +In this lab, you will: + +* Export a RAG configuration as a *SpringAI* microservice +* Start the Springboot microservice +* Test the RAG-based microservice + +### Prerequisites + +To run the microservice exported you need: + +* JDK 21.x +* Apache Maven 3.8.x +* curl command + +## Task 1: Export the SpringAI Microservice (Ollama version) + +1. navigate to the *Chat* tab, select **llama3.1** as the **Chat model** and choose **TEST1** as the vector store alias: + + ![select-test1](./images/select-test1.png) + + This configuration will use the Ollama LLM server provider for both the embedding model and the chat model. + +1. navigate to the *Settings* tab from the left side menu. Here, you should find the *Download SpringAI* button available. + + If you see a message like this: + + ![notollama](./images/diff-llm-springai.png) + + don't worry — simply select **llama3.1** as the Chat model, and the button will appear. + +1. Click the `Download SpringAI` button to export the Ollama-based microservice: + + ![download-springai](./images/download-springai.png) + +## Task 2: Start the SpringAI microservice + +If you have completed *Task 1* from this lab, you should have downloaded a *spring_ai.zip* file. Follow these steps to start it: + +* Unzip the file in a subdirectory + +* Open a terminal window, navigate to the subdirectory, and set executable permissions on `start.sh` using the command: `chmod 755 ./start.sh`. + +* Now start the microservice with: + + ```bash + <copy> + ./start.sh + </copy> + ``` + + If the Spring Boot application starts successfully, you should see something like this in your terminal logs + + ![microservice-started](./images/microservice-started.png) + +Now you are all set for testing the microservice! + +## Task 3: Test the Microservice + +The microservice you just started exposes a web service that accepts HTTP GET requests at: + +* `http://localhost:9090/v1/chat/completions` — to use RAG via the OpenAI-compatible REST API. +* `http://localhost:9090/v1/service/llm` — to chat directly with the underlying LLM. +* `http://localhost:9090/v1/service/search/`— to search for documents similar to the message provided. + +To test it, run a curl command like this in a new terminal: + + ```bash + <copy> + curl -N http://localhost:9090/v1/chat/completions \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer your_api_key" \ + -d '{ + "model": "server", + "messages": [{"role": "user", "content": "In Oracle Database 23ai, how do I determine the accuracy of my vector indexes?"}], + "stream": false + }' + </copy> + ``` + +The response using RAG with the TEST1 vector store will look like this: + + ``` json + {"id":"chatcmpl-iHfRbDEYySEuGLESa8faQKZbcpVQ","object":"chat.completion","created":"1746805121","model":"llama3.1", + + "choices":[{"message":{"role":"assistant","content":"According to the documentation, you can determine the accuracy of your vector indexes using the following methods:\n\n1. Use the `DBMS_VECTOR.INDEX_ACCURACY_QUERY` function to verify the accuracy of a vector index for a given query vector, top-K, and target accuracy.\n2. Use the `DBMS_VECTOR.INDEX_ACCURACY_REPORT` function to capture from your past workloads, accuracy values achieved by approximate searches using a particular vector index for a certain period of time.\n\nAdditionally, you can use the `GET_INDEX_STATUS` procedure to get information about the current status of a vector index, such as its stage and percentage completion."}, + + "index":0,"finish_reason":null}]} + ``` + +You can see how the microservice retrieves relevant context from the vector store, producing an answer similar to the one previously obtained using the RAG functionality within the AI Optimizer. + +You can also send a curl request without leveraging RAG: + +```bash +<copy> +curl --get --data-urlencode 'message=In Oracle Database 23ai, how do I determine the accuracy of my vector indexes?' localhost:9090/v1/service/llm +</copy> +``` + +and it will produce an ungrounded answer like this: + +```json +{"completion":"There is no such thing as \"Oracle Database 23ai\". The current version of Oracle Database is Oracle Database 21c and Oracle Database 22c.\n\nHowever, if you're using a recent version of Oracle Database (18c or later), you can use the `DBMS_VECTOR_INDEX_STATS` package to gather statistics about your vector indexes. This package provides functions to retrieve information about the accuracy of the vector index, such as:\n\n* `GET_BUCKETS_ACCURACY`: Returns the number of buckets that have an accuracy greater than or equal to a specified threshold.\n* `GET_INVERTED_LIST_ACCURACY`: Returns the accuracy of the inverted list for each bucket.\n\nTo use these functions, you'll need to create a vector index on your table and gather statistics about it using the following steps:\n\n1. Create a vector index on your table:\n```sql\nCREATE INDEX my_vector_index ON my_table (my_column) VECTORIZED;\n```\n2. Gather statistics about the vector index:\n```sql\nBEGIN\n DBMS_VECTOR_INDEX_STATS.GATHER_TABLE_STATS('MY_SCHEMA', 'MY_TABLE');\nEND;\n```\n3. Use the `DBMS_VECTOR_INDEX_STATS` package to retrieve information about the accuracy of the vector index:\n```sql\nSELECT GET_BUCKETS_ACCURACY(my_vector_index, 0.5) FROM DUAL; \n-- Returns the number of buckets with an accuracy greater than or equal to 0.5\n```\nPlease note that this is a simplified example and you may need to adjust the syntax depending on your specific use case.\n\nAlso, keep in mind that vector indexes are a feature introduced in Oracle Database 18c, so if you're using an earlier version of Oracle, you won't have access to these features."} +``` + +You can see that the answer is very generic compared to the RAG-enhanced one. + +## Task 4 (Optional): Execute the OpenAI variant + +> **NOTE**: You can perform this task only if you have access to [OpenAI APIs](https://platform.openai.com/settings/organization/api-keys). + +Proceed as in the previous step, selecting **TEST2** as the vector store alias and **gpt-4o-mini** as the **Chat model**. In the terminal where you run the Spring Boot microservice, make sure that your **OPENAI API KEY** is correctly set as environment variable. + +You may now **proceed to the next lab**. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Andy Tael, July 2025 diff --git a/ai-optimizer/tenancy/prepare/images/adb.png b/ai-optimizer/tenancy/prepare/images/adb.png new file mode 100644 index 00000000..47d5f113 Binary files /dev/null and b/ai-optimizer/tenancy/prepare/images/adb.png differ diff --git a/ai-optimizer/tenancy/prepare/images/cloud-navigation.png b/ai-optimizer/tenancy/prepare/images/cloud-navigation.png new file mode 100644 index 00000000..754dd911 Binary files /dev/null and b/ai-optimizer/tenancy/prepare/images/cloud-navigation.png differ diff --git a/ai-optimizer/tenancy/prepare/images/database-vector-store-list.png b/ai-optimizer/tenancy/prepare/images/database-vector-store-list.png new file mode 100644 index 00000000..7d2fcbd8 Binary files /dev/null and b/ai-optimizer/tenancy/prepare/images/database-vector-store-list.png differ diff --git a/ai-optimizer/tenancy/prepare/images/populate-vector-store.png b/ai-optimizer/tenancy/prepare/images/populate-vector-store.png new file mode 100644 index 00000000..975ed16b Binary files /dev/null and b/ai-optimizer/tenancy/prepare/images/populate-vector-store.png differ diff --git a/ai-optimizer/tenancy/prepare/images/query-vector-db.png b/ai-optimizer/tenancy/prepare/images/query-vector-db.png new file mode 100644 index 00000000..304760fe Binary files /dev/null and b/ai-optimizer/tenancy/prepare/images/query-vector-db.png differ diff --git a/ai-optimizer/tenancy/prepare/images/split-embed.jpg b/ai-optimizer/tenancy/prepare/images/split-embed.jpg new file mode 100644 index 00000000..af3d0bcc Binary files /dev/null and b/ai-optimizer/tenancy/prepare/images/split-embed.jpg differ diff --git a/ai-optimizer/tenancy/prepare/images/text-embedding-3-small.png b/ai-optimizer/tenancy/prepare/images/text-embedding-3-small.png new file mode 100644 index 00000000..f90e82e6 Binary files /dev/null and b/ai-optimizer/tenancy/prepare/images/text-embedding-3-small.png differ diff --git a/ai-optimizer/tenancy/prepare/images/vector-store-populated.png b/ai-optimizer/tenancy/prepare/images/vector-store-populated.png new file mode 100644 index 00000000..9184b700 Binary files /dev/null and b/ai-optimizer/tenancy/prepare/images/vector-store-populated.png differ diff --git a/ai-optimizer/tenancy/prepare/prepare.md b/ai-optimizer/tenancy/prepare/prepare.md new file mode 100644 index 00000000..68d49a31 --- /dev/null +++ b/ai-optimizer/tenancy/prepare/prepare.md @@ -0,0 +1,109 @@ +# Prepare the Vector Store + +## Introduction + +In this lab, we will perform a first vector store ingestion to test that it works. We will split & embed one document and store its vector embeddings inside the Oracle 23ai Database set up earlier. We will then be able to inspect the content of the tables that will be created. + +Estimated Lab Time: 15 minutes + +### Objectives + +In this lab, you will: + +* Split/Embed Documents and create a vector store +* Inspect the Vector Database content + +### Prerequisites + +This lab assumes you have: + +* All previous labs successfully completed + +## Task 1: Split/Embed Documents + +In the *Split/Embed* tab, the framework allows you to upload various types of documents and transform their content into vector embeddings in a format that is interpretable by LLMs. + +1. Navigate to the *Split/Embed* tab + + ![split-embed-interface](./images/split-embed.jpg) + +1. Choose the embedding model and its parameters + + Here, you can choose from the embedding models you selected during the configuration phase we had in Lab 1 using a drop-down menu. + For this example, choose **mxbai-embed-large**. + + The **chunk size** defines the length of each segment into which the document will be split, while the **chunk overlap** represents the percentage of overlap between consecutive chunks relative to the chunk size. + + Additionally, you can select different distance metrics and index types to experiment with various vector representations of the same document, allowing you to identify the configuration that best meets your needs. + +3. Select your document source + + Once the embedding model has been selected, scroll down to the *Load and Split Documents* section to upload the document you wish to store in your **Oracle Database 23ai**. + + ![populate-vector-store](images/populate-vector-store.png) + + You can choose from three different file sources: + +* **OCI**: Navigate through your tenancy to select documents from the Object Storage. Ensure that your OCI credentials are properly configured in advance. +* **Local**: Upload a document directly from your local environment. +* **Web**: Import a document from a publicly accessible web URL. + + In this example, we will embed a document from the web, available at [this link](https://docs.oracle.com/en/database/oracle/oracle-database/23/vecse/ai-vector-search-users-guide.pdf). We will give the alias ***TEST1*** to this vector store. + +1. Populate the Vector Store. + + After selecting your document source, you can click on the **Populate Vector Store** button and start the embedding process. + + Once the process is complete, a green confirmation prompt will appear, indicating the number of chunks that have been generated and successfully stored in the database. + + ![vector-store-populated](images/vector-store-populated.png) + + This means that 3016 vectors representations of the information from the input document have been created and stored. + +## Task 2: Inspect the Vector DB + +Now that you've embedded your document, you can query the content of the Vector Store. In the Cloud tenancy that has been created, open the hamburger menu on the upper left side and navigate to *Oracle Database*: + +![cloud-navigation](images/cloud-navigation.png) + +1. Click on *Autonomous Database* and you will get prompted with the Autonomous Database that you are using within the Optimizer. Click on it and you will get redirected to the Admin page of the Autonomous Database. + + ![adb](images/adb.png) + +1. Click on the *Database Action* dropdown menu and press the *SQL* button. A new window will be opened and you be redirected to the SQL playground. Run this query: + + ```sql + select * from ADMIN.TEST1_MXBAI_EMBED_LARGE_512_103_COSINE_HNSW; + ``` + + And you will then retrieve the rows from the newly created table. + + ![query-vector-db](images/query-vector-db.png) + +What you see in the image above are chunks of text from the input document, which have been transformed into vector format and stored in the Oracle database. Essentially, you’ve replicated the knowledge contained in the document within your database! + +## (Optional) Task 3: Create a different Vector Store + +> **NOTE**: You can perform this task only if you have access to [OpenAI APIs](https://platform.openai.com/settings/organization/api-keys). + +By following the sames steps from Task 1, you can create another vector store using the same document but with a different embedding model, **text-embedding-3-small** from the OpenAI models. Give the alias ***TEST2*** to this vector store. + +In this case, you will get a smaller number of chunks, since the model supports a chunk size of 8191 instead of the 512 given by *mxbai-embed-large*: + +![text-embedding-3-small](images/text-embedding-3-small.png) + +## Learn More + +* (optional) See the list of vector stores: + + You can navigate to the *Database* tab in the navigation menu to see the list of all the vector stores that have been created. If needed, you can easily delete them with a single click. + + ![database-vector-store-list](images/database-vector-store-list.png) + +You may now **proceed to the next lab**. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Andy Tael, July 2025 diff --git a/ai-optimizer/tenancy/rag/images/enable-vector-search.png b/ai-optimizer/tenancy/rag/images/enable-vector-search.png new file mode 100644 index 00000000..4eda62c4 Binary files /dev/null and b/ai-optimizer/tenancy/rag/images/enable-vector-search.png differ diff --git a/ai-optimizer/tenancy/rag/images/rag-query-2.png b/ai-optimizer/tenancy/rag/images/rag-query-2.png new file mode 100644 index 00000000..d944b5d6 Binary files /dev/null and b/ai-optimizer/tenancy/rag/images/rag-query-2.png differ diff --git a/ai-optimizer/tenancy/rag/images/rag-query.png b/ai-optimizer/tenancy/rag/images/rag-query.png new file mode 100644 index 00000000..34412fd9 Binary files /dev/null and b/ai-optimizer/tenancy/rag/images/rag-query.png differ diff --git a/ai-optimizer/tenancy/rag/images/reference.png b/ai-optimizer/tenancy/rag/images/reference.png new file mode 100644 index 00000000..93d82a90 Binary files /dev/null and b/ai-optimizer/tenancy/rag/images/reference.png differ diff --git a/ai-optimizer/tenancy/rag/images/select-vector-store.png b/ai-optimizer/tenancy/rag/images/select-vector-store.png new file mode 100644 index 00000000..6e124128 Binary files /dev/null and b/ai-optimizer/tenancy/rag/images/select-vector-store.png differ diff --git a/ai-optimizer/tenancy/rag/rag.md b/ai-optimizer/tenancy/rag/rag.md new file mode 100644 index 00000000..2ef0f59b --- /dev/null +++ b/ai-optimizer/tenancy/rag/rag.md @@ -0,0 +1,95 @@ +# Using Retrieval Augmented Generation + +## Introduction + +Now that you’ve created the vector stores, you can begin testing the knowledge base built in *Lab 2*. In this Lab, you will test the knowledge stored in the `TEST1` vector store, which serves as an example of an augmented LLM based on publicly available sources. + +> **NOTE**: The answers received by any LLM are based on probabilistic models. Therefore, the answer received in your own testing might differ from the ones shown in this documentation. + +Estimated Time: 5 minutes + +### Objectives + +In this lab, you will: + +* Enable the RAG functionality in the *Chat* window +* Ask questions and receive answers enhanced by relevant context stored in the vector database +* Compare results across different configurations + +### Prerequisites + +This lab assumes you have: + +* All previous labs successfully completed + +## Task 1: Enable RAG + +To enable the *RAG* functionality, follow these steps: + +1. Navigate to the *ChatBot* tab from the left-side menu + +1. Select the **Vector Search** option from the **Toolkit** section and then select **llama3.1** as the chat model. + + ![enable-vector-search](./images/enable-vector-search.png) + +1. In the **Select Alias** dropdown, choose the `TEST1` vector store table. The remaining fields in the **Vector Store** will be automatically populated, as each one represents a search parameter used to configure the selected vector store. + + ![select-vector-store](./images/select-vector-store.png) + + In this case, selecting the alias is sufficient to retrieve the correct store. However, you also get visibility into the parameters that were used to create the chunks and their corresponding embedding vectors. + +## Task 2: Ask questions with RAG enabled + +Now that RAG is enabled, we can test it by asking the same questions submitted in *Lab 1*. Previously, the LLM provided generic answers, as the content was either not part of its training data or not sufficiently addressed in the questions. + +For this guided experiment, perform the following: + +1. Ask the ChatBot: + +```text +<copy> +In Oracle Database 23ai, how do I determine the accuracy of my vector indexes? +</copy> +``` + +![rag-query](images/rag-query.png) + +```text +<copy> +In Oracle Database 23ai, how do I convert string text to embeddings using PL/SQL? +</copy> +``` + +![rag-query-2](images/rag-query-2.png) + +You can see how the answers differ from the ones we received earlier. The responses should be different and include references to DBMS_VECTOR and links to the embedded documentation where this information can be found. + +You can also click on one of the *Reference* buttons, to see where the information was retrieved from in the file: + +![reference](images/reference.png) + +## (Optional) Task 3: Repeat Task 2 using OpenAI models + +> **NOTE**: You can perform this task only if you have access to [OpenAI APIs](https://platform.openai.com/settings/organization/api-keys). + +As an additional learning exercise, you can repeat the tests performed in Task 2 of this lab using OpenAI LLMs and vector stores: + +* Select `TEST2` as the vector store to run the questions against +* Choose **gpt-4o-mini** from the **Chat model** dropdown menu to have the same LLM provider. + +afterwards, compare the results to what you got while using *llama3.1*! + +## Learn More + +You can learn more about RAG by playing with more parameters and prompt techniques: + +* Play with the *Temperature* parameter to discover how much the answers provided get enriched. +* Follow up a question with another one asking for “more” to see that the chat history is taken into account when generating a new response. + +You may now **proceed to the next lab**. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Andy Tael, July 2025 diff --git a/ai-optimizer/tenancy/server/images/api-server-activity.png b/ai-optimizer/tenancy/server/images/api-server-activity.png new file mode 100644 index 00000000..28a97b02 Binary files /dev/null and b/ai-optimizer/tenancy/server/images/api-server-activity.png differ diff --git a/ai-optimizer/tenancy/server/images/api-server-config.png b/ai-optimizer/tenancy/server/images/api-server-config.png new file mode 100644 index 00000000..f0db2217 Binary files /dev/null and b/ai-optimizer/tenancy/server/images/api-server-config.png differ diff --git a/ai-optimizer/tenancy/server/images/api-server-settings.png b/ai-optimizer/tenancy/server/images/api-server-settings.png new file mode 100644 index 00000000..47534007 Binary files /dev/null and b/ai-optimizer/tenancy/server/images/api-server-settings.png differ diff --git a/ai-optimizer/tenancy/server/images/curl-response.png b/ai-optimizer/tenancy/server/images/curl-response.png new file mode 100644 index 00000000..a83e46f7 Binary files /dev/null and b/ai-optimizer/tenancy/server/images/curl-response.png differ diff --git a/ai-optimizer/tenancy/server/server.md b/ai-optimizer/tenancy/server/server.md new file mode 100644 index 00000000..34e83885 --- /dev/null +++ b/ai-optimizer/tenancy/server/server.md @@ -0,0 +1,97 @@ +# (Optional) Using the API Server + +## Introduction + +In this lab, you will test the functionality of the API Server. The AI Optimizer uses an API Server to make its features accessible to external clients. You can run the API Server either as part of the AI Optimizer GUI client or as a standalone process. + +Estimated Time: 10 minutes + +### Objectives + +In this lab, you will: + +* Start the AI Optimizer as an API Server +* Review the configuration parameters +* Make an API call to the server + +### Prerequisites + +This lab assumes you have: + +* All previous labs successfully completed + +## Task 1: Start the API Server + +Navigate to the *API Server* tab from the left-hand pane: + +![Server Configuration](images/api-server-config.png) + +You’ll see a `Restart Server` button (in red). This is because the API Server starts automatically with the AI Optimizer. Follow the steps below to edit the server configuration and restart it: + +* Update the API Server Key (e.g., abcd, or any value you prefer) +* Click the Restart Server button to apply the new parameters + +## Task 2: Inspect Server Configuration + +When the API Server starts, it creates a default `server` client with minimal settings. This client is used when calling the API Server externally, outside of the AI Optimizer GUI. + +To copy your current GUI client settings to the server client for external use, click the Copy AI Optimizer Settings button. + +![Server Settings](images/api-server-settings.png) + +You can inspect the configuration of the `server` client by expanding the `{...}` brackets. + +## Task 3: Perform an API call to the server + +Now that the API Server is running, you can perform API calls against it. (You might have to wait a couple minutes after the `Restart Server` performed in task 1) + +* Open a terminal window in your preferred IDE + +* Copy and run the following curl command in the terminal: + + ```bash + <copy> + #VM ip address and Bearer Key to be set accordingly. + curl -X POST "http://your_VM_ip_address:8000/v1/chat/completions" \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer abcd" \ + -H "client: server" \ + -d '{ + "model": "llama3.1", + "messages": [ + { + "role": "user", + "content": "In Oracle Database 23ai, how do I determine the accuracy of my vector indexes?" + } + ] + }' | jq . + </copy> + ``` + + Here, you're essentially repeating the same type of request as in *Lab 3*. You’ve passed the API Server Key you configured in Task 1 along with the question content as parameters. You should receive a response similar to the screenshot below: + + ![curl-response](images/curl-response.png) + + As you can see, the API Server responded with a generic answer—this is expected because RAG was not enabled. You can enable RAG just like you did in the previous labs, and the API Server will provide more context-aware responses. + +## Learn More + +* Enable RAG from the Chatbot tab, then repeat Task 3. Observe how the API Server's response changes with RAG enabled. + +* To explore the Oracle AI Optimizer and Toolkit API documentation, visit: + + ```text + <copy> + http://localhost:8001/v1/docs# + </copy> + ``` + + > **Note**: The port number in the URL (in this case 8001) must match the API Server Port you configured in Task 1 of this lab. + +You may now **proceed to the next lab**. + +## Acknowledgements + +* **Author** - Lorenzo De Marchis, Developer Evangelist, May 2025 +* **Contributors** - Mark Nelson, John Lathouwers, Corrado De Bari, Jorge Ortiz Fuentes, Andy Tael +* **Last Updated By** - Andy Tael, July 2025 diff --git a/ai-optimizer/workshops/desktop/index.html b/ai-optimizer/workshops/desktop/index.html new file mode 100644 index 00000000..e295c943 --- /dev/null +++ b/ai-optimizer/workshops/desktop/index.html @@ -0,0 +1,70 @@ +<!DOCTYPE html> +<html lang="en"> + +<head> + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <meta name="description" content="Oracle LiveLabs gives you access to Oracle's products to run a wide variety of labs and workshops; allowing you to experience our best technology, live!"> + <title>Oracle LiveLabs + + + + + + + + + + + + + +
+
+
+
+
+
+
+
+ + + + + diff --git a/ai-optimizer/workshops/desktop/manifest.json b/ai-optimizer/workshops/desktop/manifest.json new file mode 100644 index 00000000..bdff012a --- /dev/null +++ b/ai-optimizer/workshops/desktop/manifest.json @@ -0,0 +1,52 @@ +{ + "workshoptitle": "Build a Better ChatBot", + "help": "livelabs-help-devdb_us@oracle.com", + "variables": ["../../microservice/variables.json", + "../../variables/microservice-in-another-file.json"], + "tutorials": [ + { + "title": "Introduction", + "description": "The Introduction is always first. The title and contents menu title match for the Introduction.", + "filename": "../../desktop/introduction/introduction.md" + }, + { + "title": "Get Started", + "description": "This is the prerequisites for customers using Free Trial and Paid tenancies, and Always Free accounts (if applicable). The title of the lab and the Contents Menu title (the title above) match for Prerequisite lab. This lab is always first.", + "filename": "../../desktop/get-started/get-started.md" + }, + { + "title": "Lab 1: Explore the Environment", + "description": "Labs that follow the introduction are numbered, starting with Lab 1", + "filename": "../../desktop/explore/explore.md" + }, + { + "title": "Lab 2: Prepare the Vector Store", + "filename": "../../desktop/prepare/prepare.md" + }, + { + "title": "Lab 3: Using Retrieval Augmented Generation", + "filename": "../../desktop/rag/rag.md" + }, + { + "title": "Lab 4: Experimenting with Models, Parameters and Techniques", + "filename": "../../desktop/experimenting/experimenting.md" + }, + { + "title": "Lab 5: Evaluating Performance", + "filename": "../../desktop/evaluating/evaluating.md" + }, + { + "title": "(Optional) Lab 6: Generating Microservice Code", + "filename": "../../desktop/microservice/microservice.md" + }, + { + "title": "(Optional) Lab 7: Using the API Server", + "filename": "../../desktop/server/server.md" + }, + { + "title": "Need Help?", + "description": "Solutions to Common Problems and Directions for Receiving Live Help", + "filename":"../../help/help.md" + } + ] +} diff --git a/ai-optimizer/workshops/tenancy/index.html b/ai-optimizer/workshops/tenancy/index.html new file mode 100644 index 00000000..e295c943 --- /dev/null +++ b/ai-optimizer/workshops/tenancy/index.html @@ -0,0 +1,70 @@ + + + + + + + + + Oracle LiveLabs + + + + + + + + + + + + + +
+
+
+
+
+
+
+
+ + + + + diff --git a/ai-optimizer/workshops/tenancy/manifest.json b/ai-optimizer/workshops/tenancy/manifest.json new file mode 100644 index 00000000..e004e1b6 --- /dev/null +++ b/ai-optimizer/workshops/tenancy/manifest.json @@ -0,0 +1,62 @@ +{ + "workshoptitle": "Build a Better ChatBot", + "help": "livelabs-help-devdb_us@oracle.com", + "variables": ["../../microservice/variables.json", + "../../microservice/variables-in-another-file.json"], + "tutorials": [ + { + "title": "Introduction", + "description": "The Introduction is always first. The title and contents menu title match for the Introduction.", + "filename": "../../tenancy/introduction/introduction.md" + }, + { + "title": "Get Started", + "description": "This is the prerequisites for customers using Free Trial and Paid tenancies, and Always Free accounts (if applicable). The title of the lab and the Contents Menu title (the title above) match for Prerequisite lab. This lab is always first.", + "filename": "https://oracle-livelabs.github.io/common/labs/cloud-login/cloud-login.md" + }, + { + "title": "Lab 1: Deploy the Oracle AI Optimizer and Toolkit", + "description": "Lab where you will deploy the Oracle AI Optimizer and Toolkit using OCI Resource Manager", + "filename": "../../tenancy/deploy/deploy.md" + }, + { + "title": "Lab 2: Explore the Environment", + "description": "Lab where you explore the environment and setup the credentials", + "filename": "../../tenancy/explore/explore.md" + }, + { + "title": "Lab 3: Prepare the Vector Store", + "filename": "../../tenancy/prepare/prepare.md" + }, + { + "title": "Lab 4: Using Retrieval Augmented Generation", + "filename": "../../tenancy/rag/rag.md" + }, + { + "title": "Lab 5: Experimenting with Models, Parameters and Techniques", + "filename": "../../tenancy/experimenting/experimenting.md" + }, + { + "title": "Lab 6: Evaluating Performance", + "filename": "../../tenancy/evaluating/evaluating.md" + }, + { + "title": "(Optional) Lab 7: Generating Microservice Code", + "filename": "../../tenancy/microservice/microservice.md" + }, + { + "title": "(Optional) Lab 8: Using the API Server", + "filename": "../../tenancy/server/server.md" + }, + { + "title": "Lab 9: Cleanup the environment", + "description": "Lab where cleanup the deployed environment", + "filename": "../../tenancy/cleanup/cleanup.md" + }, + { + "title": "Need Help?", + "description": "Solutions to Common Problems and Directions for Receiving Live Help", + "filename":"../../help/help.md" + } + ] +}