diff --git a/dev-ai-data-platform/load-transform/images/click-data-load-button.png b/dev-ai-data-platform/load-transform/images/click-data-load-button.png new file mode 100644 index 00000000..d75b47c1 Binary files /dev/null and b/dev-ai-data-platform/load-transform/images/click-data-load-button.png differ diff --git a/dev-ai-data-platform/load-transform/images/live-feed-wizard-step1.png b/dev-ai-data-platform/load-transform/images/live-feed-wizard-step1.png new file mode 100644 index 00000000..98e3b6db Binary files /dev/null and b/dev-ai-data-platform/load-transform/images/live-feed-wizard-step1.png differ diff --git a/dev-ai-data-platform/load-transform/images/live-feed-wizard-step2.png b/dev-ai-data-platform/load-transform/images/live-feed-wizard-step2.png new file mode 100644 index 00000000..3ef79534 Binary files /dev/null and b/dev-ai-data-platform/load-transform/images/live-feed-wizard-step2.png differ diff --git a/dev-ai-data-platform/load-transform/images/live-feed-wizard-step3.png b/dev-ai-data-platform/load-transform/images/live-feed-wizard-step3.png new file mode 100644 index 00000000..67e11f02 Binary files /dev/null and b/dev-ai-data-platform/load-transform/images/live-feed-wizard-step3.png differ diff --git a/dev-ai-data-platform/load-transform/images/live-feed-wizard-step4.png b/dev-ai-data-platform/load-transform/images/live-feed-wizard-step4.png new file mode 100644 index 00000000..e4bc11ec Binary files /dev/null and b/dev-ai-data-platform/load-transform/images/live-feed-wizard-step4.png differ diff --git a/dev-ai-data-platform/load-transform/images/run-live-table-feed.png b/dev-ai-data-platform/load-transform/images/run-live-table-feed.png new file mode 100644 index 00000000..e8e909aa Binary files /dev/null and b/dev-ai-data-platform/load-transform/images/run-live-table-feed.png differ diff --git a/dev-ai-data-platform/load-transform/images/select-cloud-storage-location.png b/dev-ai-data-platform/load-transform/images/select-cloud-storage-location.png new file mode 100644 index 00000000..5a64ab07 Binary files /dev/null and b/dev-ai-data-platform/load-transform/images/select-cloud-storage-location.png differ diff --git a/dev-ai-data-platform/load-transform/images/select-feed-data.png b/dev-ai-data-platform/load-transform/images/select-feed-data.png new file mode 100644 index 00000000..c06a3be5 Binary files /dev/null and b/dev-ai-data-platform/load-transform/images/select-feed-data.png differ diff --git a/dev-ai-data-platform/load-transform/images/start-demo-2.png b/dev-ai-data-platform/load-transform/images/start-demo-2.png new file mode 100644 index 00000000..119373b6 Binary files /dev/null and b/dev-ai-data-platform/load-transform/images/start-demo-2.png differ diff --git a/dev-ai-data-platform/load-transform/load-transform-freetier.md b/dev-ai-data-platform/load-transform/load-transform-freetier.md deleted file mode 100644 index 6652bec2..00000000 --- a/dev-ai-data-platform/load-transform/load-transform-freetier.md +++ /dev/null @@ -1,186 +0,0 @@ -# 🏗️ Build your Data Pipeline: Load & Transform Data (optional lab) - -Estimated Lab Time: 45 minutes - -## Introduction - -In this lab, you’ll practice setting up the live feed capabilities, that can be used to load data that is continuously collected into cloud object storage. When a load job is enabled for live feed, it is connected to the OCI event notification and subscription mechanism, so that every time a new object is created in the object store bucket, it triggers the live feed, loading the contents to the database - -<**NOTE:** To complete this lab, you will alternate between **Oracle Cloud Infrastructure Console** pages and **Oracle Database Actions** pages. Selecting **Database Actions** will open a new tab. - -## Task 1: Create a Notifications Service Subscription Topic - - 1. Open the OCI Console at cloud.oracle.com, then click the icon in the upper left corner to open the **Navigation Menu**. - - ![Click Navigation Menu](./images/task-1-scrn-0.png) - - 2. Select **Developer Services** from the Navigation Menu - - ![Click Developer Service](./images/task-1-scrn-2.png) - - 3. Click **Notifications** under the **Application Integration** heading. - - ![Click Notifications](./images/task-1-scrn-3.png) - - 4. Click **Create Topic** - - ![Click Create Topic](./images/task-1-scrn-4.png) - - 5. Enter details, then click the **Create** button. - - ![Create Topic](./images/task-1-scrn-5.png) - -## Task 2: Create a Events Service Rule. - -1. Open the **Navigation Menu** Click the icon in the upper left corner to - - ![Click Navigation Menu](./images/task-2-scrn-1.png) - -2. Select **Observations & Management** from the Navigation Menu - - ![Select O&M](./images/task-2-scrn-2.png) - -3. Click **Rules** under the **Events Services** heading. - - ![Select Rules](./images/task-2-scrn-3.png) - -4. Click **Create Rule** and enter details. - - ![Create Rule](./images/lab7-task2-step4.png) - -5. Enter Rule details. - - ![Create Rule](./images/task-2-scrn-5.png) - - * Enter the following Details under **Rule Conditions**: - * **Condition:** Event Type - * **Service Name:** Object Storage - * **Event Type:** Object – Create - * **Action Type:** Notifications - - * Click **Create Rule**. - -## Task 3: Create a Live Table Feed and Copy the notification URL - - >**NOTE:** If continuing from lab 4, click on **Database Actions** link and skip to step 6. - -1. Navigate to your Assigned ADB Instance - - \* Open Navigation Menu - \* Select Oracle Database - \* Select Autonomous Database - - ![Navigate to Autonomous Database in OCI](./images/navigate-to-adb.png) - -2. Select the name of your assigned database. - - ![Navigate to Autonomous Database in OCI](./images/oci-adb-select.png) - -3. Select **Database Actions** then View all database actions. - - ![Create Data Product Share](./images/task1-scrn-3.png "Create Data Product Share") - -4. On the **Database Actions Launchpad**, click the user icon in the top right corner. - - \* Select **Sign Off** from the drop-down options. Enter credentials in the popup window shown below, to login as the LOAN user. - - >**NOTE:** Use the same **ADMIN** password as shown on View Lab Info page - - ![Create Data Product Share](./images/task1-scrn-5.png "Create Data Product Share") - - \* Enter LOAN user credentials. - \* Press Sign-In button. - -5. Select **Feed Data** from the options listed at top of page. - - ![Select Feed Data](./images/task-3-scrn-3.png) - -6. Click the **Create Live Table Feed** button to enter the **Create Live Feed** wizard. - - ![Create Live Feed](./images/task-3-scrn-4.png) - -7. Select desired Cloud Store location, then click **Next**. - - ![SElect Cloud Store Location](./images/task-3-scrn-5.png) - -8. Enter desired Table Settings, then click **Next**. - - ![Load Data from Object Storage](./images/task-3-scrn-6a.png) - -9. Review the information shown on the Preview page, then click **Next**. - - ![Load Data from Object Storage](./images/task-3-scrn-7.png) - -10. Complete the creation of the Live Table Feed - - \* Take the following actions \... - \* Enter the **Live Table Feed Name:** - \* Check box to **Enable for Notification** - \* Uncheck box to **Enable for Scheduling** - \* Click **Create** - - ![Load Data from Object Storage](./images/task-3-scrn-8.png) - -11. When the popup box appears, select **Yes** to run the Live Feed. - - ![Load Data from Object Storage](./images/task-3-scrn-9.png) - -12. **Review** the details for the newly created Live Feed. Then click the hamburger button in the upper right corner of the panel. - - ![Load Data from Object Storage](./images/task-3-scrn-10.png) - -13. Select **Show Notification URL** from the dropdown list. - - ![Load Data from Object Storage](./images/task-3-scrn-11.png) - -14. Copy the notification URL for the live table feed and click OK to proceed to next task. - - ![Load Data from Object Storage](./images/task-3-scrn-12.png) - - >**NOTE:** The notification URL will be used later in Task #4. - -## Task 4: Create a Notifications Service Subscription - - 1. Click on the **Navigation Menu**, then select **Developer Services**. - - ![Access the Object Storage Bucket](./images/task-1-scrn-2.png) - - 2. Click **Notifications** under the **Application Integration** heading. - - ![Access the Object Storage Bucket](./images/task-1-scrn-3.png) - - 3. Select **Subscriptions** (on the left side of the page, just below Topics). The status will be **Active**. - - ![Access the Object Storage Bucket](./images/lab7-task4-step3.png) - - 4. Click **Create Subscription**. - - 5. Enter the Subscription details. - - ![Access the Object Storage Bucket](./images/lab7-task4-step5.png) - - * Provide the following: - * **Subscription topic:** Select the subscription topic you created in Task 2 - * **Protocol:** Email - * **URL** Paste in the URL you copied in Task 3 - - * Click **Create** - - 6. Switch to the Database Actions tab to review the card for the live table feed you are configuring for a notification-based feed. It should reflect an **Active** notification status.. - - ![Load Data from Object Storage](./images/task-5-scrn-4.png) - - * You will receive email notifications when specific live feed events occur and any new files uploaded to the bucket will automatically be loaded into the live feed table. - -## Learn More - -* [The Catalog Tool](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/catalog-entities.html) -* [Autonomous Database](https://docs.oracle.com/en/cloud/paas/autonomous-database/index.html) - -## Acknowledgements - -* **Authors** - Eddie Ambler, Otis Barr, Matt Kowalik -* **Contributors** - Mike Matthews, Marty Gubar, Francis Regalado, Ramona Magadan -* **Last Updated By/Date** - 04-28-2025 - -Copyright (C) Oracle Corporation. diff --git a/dev-ai-data-platform/load-transform/load-transform-mvp.md b/dev-ai-data-platform/load-transform/load-transform-mvp.md deleted file mode 100644 index 186d8aa6..00000000 --- a/dev-ai-data-platform/load-transform/load-transform-mvp.md +++ /dev/null @@ -1,189 +0,0 @@ -# 🏗️ Build your Data Pipeline: Load & Transform Data (optional lab) - -## Introduction - -In this lab, you’ll practice setting up the live feed capabilities, that can be used to load data that is continuously collected into cloud object storage. When a load job is enabled for live feed, it is connected to the OCI event notification and subscription mechanism, so that every time a new object is created in the object store bucket, it triggers the live feed, loading the contents to the database - -## Estimated Lab Time: 45 minutes - -## Objectives - * Login to LiveLabs Sandbox - * Find your LiveLabs Sandbox reservations - -## Prerequisites - This lab assumes you have: - * Completed the Get Started lab - * Successfully launched the hands-on lab environment - -## Task 1: Create a Notifications Service Subscription Topic - - 1. Open the OCI Console at cloud.oracle.com, then click the icon in the upper left corner to open the **Navigation Menu**. - - ![Click Navigation Menu](./images/task-1-scrn-0.png) - - 2. Select **Developer Services** from the Navigation Menu - - ![Click Developer Service](./images/task-1-scrn-2.png) - - 3. Click **Notifications** under the **Application Integration** heading. - - ![Click Notifications](./images/task-1-scrn-3.png) - - 4. Click **Create Topic** - - ![Click Create Topic](./images/task-1-scrn-4.png) - - 5. Enter details, then click the **Create** button. - - ![Create Topic](./images/task-1-scrn-5.png) - -## Task 2: Create a Events Service Rule. - -1. Open the **Navigation Menu** Click the icon in the upper left corner to - - ![Click Navigation Menu](./images/task-2-scrn-1.png) - -2. Select **Observations & Management** from the Navigation Menu - - ![Select O&M](./images/task-2-scrn-2.png) - -3. Click **Rules** under the **Events Services** heading. - - ![Select Rules](./images/task-2-scrn-3.png) - -4. Click **Create Rule** and enter details. - - ![Create Rule](./images/lab7-task2-step4.png) - -5. Enter Rule details, as shown below: - - ![Create Rule](./images/task-2-scrn-5.png) - - Click **Create Rule**. - -## Task 3: Access Local Data Catalog - -1. Navigate to your assigned ADB Instance. - - a. Open Navigation Menu - b. Select Oracle Database - c. Select Autonomous Database - - ![Navigate to Autonomous Database in OCI](./images/navigate-to-adb.png " ") - -1. Select the name of your assigned database. - - ![Navigate to Autonomous Database in OCI](./images/oci-adb-select.png " ") - -1. Click on Database Actions, then select Data Load from the drop-down list - - ![ADB Details | DB Actions](./images/adb-details--dbactions.png " ") - -1. Click on **Catalog** on left rail for the **ADB Catalog** to be displayed - - ![Access Local Data Catalog](./images/access-local-data-catalog-4.png "Access Local Data Catalog") - - **Congratulations you can now see your data objects from your Catalog!** - - - - - - -## Task 4: Create a Live Table Feed and Copy the notification URL - - - -5. Select **Feed Data** from the options listed at top of page. - - ![Select Feed Data](./images/task-3-scrn-3.png) - -6. Click the **Create Live Table Feed** button to enter the **Create Live Feed** wizard. - - ![Create Live Feed](./images/task-3-scrn-4.png) - -7. Select desired Cloud Store location, then click **Next**. - - ![SElect Cloud Store Location](./images/task-3-scrn-5.png) - -8. Enter desired Table Settings, then click **Next**. - - ![Load Data from Object Storage](./images/task-3-scrn-6a.png) - -9. Review the information shown on the Preview page, then click **Next**. - - ![Load Data from Object Storage](./images/task-3-scrn-7.png) - -10. Complete the creation of the Live Table Feed - - \* Take the following actions \... - \* Enter the **Live Table Feed Name:** - \* Check box to **Enable for Notification** - \* Uncheck box to **Enable for Scheduling** - \* Click **Create** - - ![Load Data from Object Storage](./images/task-3-scrn-8.png) - -11. When the popup box appears, select **Yes** to run the Live Feed. - - ![Load Data from Object Storage](./images/task-3-scrn-9.png) - -12. **Review** the details for the newly created Live Feed. Then click the hamburger button in the upper right corner of the panel. - - ![Load Data from Object Storage](./images/task-3-scrn-10.png) - -13. Select **Show Notification URL** from the dropdown list. - - ![Load Data from Object Storage](./images/task-3-scrn-11.png) - -14. Copy the notification URL for the live table feed and click OK to proceed to next task. - - ![Load Data from Object Storage](./images/task-3-scrn-12.png) - - >**NOTE:** The notification URL will be used later in Task #4. - -## Task 4: Create a Notifications Service Subscription - - 1. Click on the **Navigation Menu**, then select **Developer Services**. - - ![Access the Object Storage Bucket](./images/task-1-scrn-2.png) - - 2. Click **Notifications** under the **Application Integration** heading. - - ![Access the Object Storage Bucket](./images/task-1-scrn-3.png) - - 3. Select **Subscriptions** (on the left side of the page, just below Topics). The status will be **Active**. - - ![Access the Object Storage Bucket](./images/lab7-task4-step3.png) - - 4. Click **Create Subscription**. - - 5. Enter the Subscription details. - - ![Access the Object Storage Bucket](./images/lab7-task4-step5.png) - - * Provide the following: - * **Subscription topic:** Select the subscription topic you created in Task 2 - * **Protocol:** Email - * **URL** Paste in the URL you copied in Task 3 - - * Click **Create** - - 6. Switch to the Database Actions tab to review the card for the live table feed you are configuring for a notification-based feed. It should reflect an **Active** notification status.. - - ![Load Data from Object Storage](./images/task-5-scrn-4.png) - - * You will receive email notifications when specific live feed events occur and any new files uploaded to the bucket will automatically be loaded into the live feed table. - -## Learn More - -* [The Catalog Tool](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/catalog-entities.html) -* [Autonomous Database](https://docs.oracle.com/en/cloud/paas/autonomous-database/index.html) - -## Acknowledgements - -* **Authors** - Eddie Ambler, Otis Barr, Matt Kowalik -* **Contributors** - Mike Matthews, Marty Gubar, Francis Regalado, Ramona Magadan -* **Last Updated By/Date** - 04-28-2025 - -Copyright (C) Oracle Corporation. diff --git a/dev-ai-data-platform/load-transform/load-transform.md b/dev-ai-data-platform/load-transform/load-transform.md index 59871d8f..c0285e3e 100644 --- a/dev-ai-data-platform/load-transform/load-transform.md +++ b/dev-ai-data-platform/load-transform/load-transform.md @@ -2,210 +2,74 @@ ## Introduction -As SeerEquities scales its lending operations, speed is becoming a competitive edge. Executives want delinquency alerts as they happen, risk teams want real-time updates on loan applications, and data analysts want to eliminate the delays caused by batch pipelines. +In this lab, you’ll practice setting up the live feed capabilities, that can be used to load data that is continuously collected into cloud object storage. When a load job is enabled for live feed, it can be connected to the OCI event notification and subscription mechanism, so that every time a new object is created in the object store bucket, it triggers the live feed, loading the contents to the database. -But SeerEquities’ current approach—periodic data loads and manual triggers—can’t keep up. Delays in data ingestion mean missed insights and late decisions. +## Estimated Lab Time: 45 minutes -**That’s where live feeds come in**. In this lab, you’ll bring SeerEquities one step closer to a real-time data ecosystem by enabling continuous data ingestion using Oracle’s event-driven architecture. As new data lands in cloud object storage, it will flow automatically into the Autonomous Database—no waiting, no manual refreshes. +## Objectives + * Login to LiveLabs Sandbox + * Find your LiveLabs Sandbox reservations -By implementing this live feed, you're not just optimizing a pipeline—you’re unlocking faster reporting, proactive fraud detection, and just-in-time decision making. It’s a vital step in transforming SeerEquities’ data platform from reactive to real-time. +## Prerequisites + This lab assumes you have: + * Completed the Get Started lab + * Successfully launched the hands-on lab environment -Estimated Time: 45 minutes +## Task 1: Prepare to Build a Data Pipeline -### Objectives - -* Set up a Notifications Service Topic to receive event messages from Object Storage - -* Create an Event Service Rule that monitors new object uploads - -* Configure a Live Feed-enabled load job that triggers on new data - -* Automate ingestion of new files from cloud object storage into the database - -* Enable continuous, real-time data availability for downstream analytics and reporting - - -> **NOTE:** To complete this lab, you will alternate between **Oracle Cloud Infrastructure Console** pages and **Oracle Database Actions** pages. Selecting **Database Actions** will open a new tab. - - -## Task 1: Create a Notifications Service Subscription Topic - - 1. First we will have to access OCI. Right above the workshop instruction you will find **View Login Info**. - - ![Click View Login Info](./images/view-login-info.png) - - 2. Click **View Login Info** to see detailed reservation information such as user and password. Click **Launch OCI** - - ![Click On OCI](./images/launch-oci.png) - - 3. Follow the instructions provided to log in to your Oracle Cloud account, change your password, if prompted, and complete your login to Oracle Cloud. - - a. Check the tenancy. If it is different, click on change tenancy. - - ![Check Tenancy](./images/check-tenancy2.png) - - b. Add the user name and the password from the reservation information. - - ![CheckUserPassword](./images/user-password.png) - - c. Change the password, then click on Reset Password. - - ![Password](./images/change-password.png) - - 4. Congratulations! You are now connected to an Oracle Cloud Infrastructure tenancy. You can now execute the different tasks and steps in this LiveLabs workshop. - - ![RedwoodOCIConsole](./images/redwood-oci-console.png) - - 5. Now, let's navigate to **Developer Services**. Click on the **hamburger menu** on the top left. - - ![HamburgerMenuOCI](./images/hamburger-menu-oci-console.png) - - 6. Select **Developer Services** from the Navigation Menu - - ![Click Developer Service](./images/task-1-scrn-2.png) - - 7. Click **Notifications** under the **Application Integration** heading. - - ![Click Notifications](./images/task-1-scrn-3.png) - - 8. Click **Create Topic** - - ![Click Create Topic](./images/task-1-scrn-4.png) - - 9. You can use the values below, or plug in your own. Just make sure to remember what you choose—you’ll need it again later in the lab! + 1. Click **View Login Info**. Copy your **DB ADMIN Password**, and click the **SQL Worksheet** link. - * **Name**: Live\_Feed\_for_Funding - * **Description**: This is to demo LiveFeed for Funding as a topic. + ![Access Data Catalog](./images/start-demo-2.png "Access Local Data Catalog") - Click the **Create** button. - - ![Create Topic](./images/topic-livefeed.png) - -You’ve now successfully created a Notification Service topic—laying the foundation for event-driven data pipelines. Next, you’ll define the Events Service rule to detect when new data arrives and trigger the live feed automatically. - -## Task 2: Create a Events Service Rule. - -1. Open the **Navigation Menu** Click the icon in the upper left corner to - - ![Click Navigation Menu](./images/task-2-scrn-1.png) - -2. Select **Observations & Management** from the Navigation Menu - - ![Select O&M](./images/task-2-scrn-2.png) - -3. Click **Rules** under the **Events Services** heading. - - ![Select Rules](./images/task-2-scrn-3.png) - -4. Click **Create Rule** and enter details. - - ![Create Rule](./images/lab7-task2-step4.png) - -5. Enter Rule details. - - ![Create Rule](./images/task-2-scrn-5.png) - - * Enter the following Details under **Rule Conditions**: - * **Condition:** Event Type - * **Service Name:** Object Storage - * **Event Type:** Object – Create - * **Action Type:** Notifications - - * Click **Create Rule**. - -## Task 3: Create a Live Table Feed and Copy the notification URL - -1. Click **View Login Info**. Click the **SQL Worksheet** link. - - ![Access Data Catalog](./images/sql-worksheet.png "Access Local Data Catalog") - - >**NOTE:** Use the same **ADMIN** password as shown on View Lab Info page - - ![Create Data Product Share](./images/task1-scrn-5.png "Create Data Product Share") - - \* Enter LOAN user credentials. - \* Press Sign-In button. - -5. Select **Feed Data** from the options listed at top of page. - - ![Select Feed Data](./images/task-3-scrn-3.png) - -6. Click the **Create Live Table Feed** button to enter the **Create Live Feed** wizard. - - ![Create Live Feed](./images/task-3-scrn-4.png) - -7. Select desired Cloud Store location, then click **Next**. - - ![SElect Cloud Store Location](./images/task-3-scrn-5.png) - -8. Enter desired Table Settings, then click **Next**. - - ![Load Data from Object Storage](./images/task-3-scrn-6a.png) - -9. Review the information shown on the Preview page, then click **Next**. - - ![Load Data from Object Storage](./images/task-3-scrn-7.png) - -10. Complete the creation of the Live Table Feed + 2. For your Username enter **LOAN**. Paste in the password you copied in the previous step - \* Take the following actions \... - \* Enter the **Live Table Feed Name:** - \* Check box to **Enable for Notification** - \* Uncheck box to **Enable for Scheduling** - \* Click **Create** - - ![Load Data from Object Storage](./images/task-3-scrn-8.png) - -11. When the popup box appears, select **Yes** to run the Live Feed. + ![Create Data Product Share](./images/task1-scrn-5.png "Create Data Product Share") - ![Load Data from Object Storage](./images/task-3-scrn-9.png) + Click **Sign-in** button -12. **Review** the details for the newly created Live Feed. Then click the hamburger button in the upper right corner of the panel. + 3. Click **Data Load** button - ![Load Data from Object Storage](./images/task-3-scrn-10.png) - -13. Select **Show Notification URL** from the dropdown list. + ![Click Data Load Button](./images/click-data-load-button.png "") - ![Load Data from Object Storage](./images/task-3-scrn-11.png) + 4. Select **Feed Data** from the options listed at top of page -14. Copy the notification URL for the live table feed and click OK to proceed to next task. + ![Click Data Load Button](./images/select-feed-data.png "") - ![Load Data from Object Storage](./images/task-3-scrn-12.png) +## Task 2: Create a Events Service Rule - >**NOTE:** The notification URL will be used later in Task #4. + 1. Click the **Create Live Table Feed** button to enter the Create Live Feed wizard -## Task 4: Create a Notifications Service Subscription + ![Live Feed Wizard - step 1](./images/live-feed-wizard-step1.png "") - 1. Click on the **Navigation Menu**, then select **Developer Services**. + 2. Select desired **Cloud Store location**, then click **Next** - ![Access the Object Storage Bucket](./images/task-1-scrn-2.png) + ![Live Feed Wizard - step 2](./images/select-cloud-storage-location.png "") - 2. Click **Notifications** under the **Application Integration** heading. + 3. Accept the default Table Settings, then click **Next** - ![Access the Object Storage Bucket](./images/task-1-scrn-3.png) + ![Live Feed Wizard - step 2](./images/live-feed-wizard-step2.png "") - 3. Select **Subscriptions** (on the left side of the page, just below Topics). The status will be **Active**. + 4. Review the information shown on the Preview page, then click **Next** - ![Access the Object Storage Bucket](./images/lab7-task4-step3.png) + ![Live Feed Wizard - step 3](./images/live-feed-wizard-step3.png "") - 4. Click **Create Subscription**. + 5. Complete the creation of the Live Table Table - 5. Enter the Subscription details. + a. Enter the **Live Table Feed Name** + b. Check box to **Enable for Scheduling**. + c. Select every **2 minutes** for the polling time interval - ![Access the Object Storage Bucket](./images/lab7-task4-step5.png) + ![Live Feed Wizard - step 4](./images/live-feed-wizard-step4.png "") - * Provide the following: - * **Subscription topic:** Select the subscription topic you created in Task 2 - * **Protocol:** Email - * **URL** Paste in the URL you copied in Task 3 + Click **Create** - * Click **Create** + 6. When the popup box appears, select **Yes** to run the Live Feed. - 6. Switch to the Database Actions tab to review the card for the live table feed you are configuring for a notification-based feed. It should reflect an **Active** notification status.. + ![Run Live Table Feed](./images/run-live-table-feed.png) - ![Load Data from Object Storage](./images/task-5-scrn-4.png) + 7. **Review** the details for the newly created Live Feed. - * You will receive email notifications when specific live feed events occur and any new files uploaded to the bucket will automatically be loaded into the live feed table. + ![Load Data from Object Storage](./images/task-3-scrn-10.png) ## Learn More @@ -213,7 +77,9 @@ You’ve now successfully created a Notification Service topic—laying the foun * [Autonomous Database](https://docs.oracle.com/en/cloud/paas/autonomous-database/index.html) ## Acknowledgements -* **Authors** - Eddie Ambler, Otis Barr -* **Last Updated By/Date** - June 2025, Otis Barr + +* **Authors** - Eddie Ambler, Otis Barr, Matt Kowalik +* **Contributors** - Mike Matthews, Marty Gubar, Francis Regalado, Ramona Magadan +* **Last Updated By/Date** - 04-28-2025 Copyright (C) Oracle Corporation. diff --git a/dev-ai-data-platform/workshops/sandbox_mvp/manifest.json b/dev-ai-data-platform/workshops/sandbox_mvp/manifest.json index 747baaee..aee6ab1b 100644 --- a/dev-ai-data-platform/workshops/sandbox_mvp/manifest.json +++ b/dev-ai-data-platform/workshops/sandbox_mvp/manifest.json @@ -1,5 +1,5 @@ { - "workshoptitle": "Data Engineering: Build a Data Platform with Autonomous Database", + "workshoptitle": "Oracle Data Platform — Finance Edition", "help": "livelabs-help-database_us@oracle.com", "tutorials": [ { @@ -28,6 +28,10 @@ "title": "Lab 4: Subscribe to Data Product", "filename": "../../mktplace-subscribe/mktplace-subscribe.md" }, + { + "title": "Lab 5: Build Your Data Pipeline: Load & Transform Data", + "filename": "../../load-transform/load-transform.md" + }, { "title": "Need Help?", "description": "Solutions to Common Problems and Directions for Receiving Live Help",