diff --git a/ai-vector-halloween/images/README.md b/ai-vector-halloween/images/README.md new file mode 100644 index 00000000..379e284c --- /dev/null +++ b/ai-vector-halloween/images/README.md @@ -0,0 +1,56 @@ +# Images for "The Great Pumpkin Caper" Workshop + +This directory contains placeholder references for images used throughout the workshop. In a production environment, you would need to source or create the following images: + +## Required Images + +### Introduction (`../intro/intro.md`) +- `museum-heist.png` - Dramatic scene of a museum heist at night, with shadowy figures and security lights + +### Lab 1 (`../labs/setup-adb/setup-adb.md`) +- `holmes-examining-clues.png` - Sherlock Holmes at his desk examining various clues and documents +- `autonomous-database-setup.png` - Screenshot or diagram of Oracle Autonomous Database provisioning interface + +### Lab 2 (`../labs/vectorize-clues/vectorize-clues.md`) +- `vector-embeddings-brain.png` - Visualization of how AI embeddings work, perhaps showing text being converted to vectors +- `vector-space-diagram.png` - Diagram showing similar concepts clustering together in vector space + +### Lab 3 (`../labs/traditional-query/traditional-query.md`) +- `traditional-vs-ai-search.png` - Side-by-side comparison showing traditional keyword search vs. AI semantic search +- `keyword-search-problems.png` - Diagram illustrating limitations of keyword-based search (synonyms, context, etc.) + +### Lab 4 (`../labs/vector-search/vector-search.md`) +- `holmes-solving-mystery.png` - Sherlock Holmes in a triumphant pose, having solved a complex mystery +- `vector-search-connections.png` - Visualization showing clues being connected through vector similarity + +## Image Specifications + +- **Format**: PNG or JPG +- **Resolution**: Minimum 800x600 pixels, preferably 1200x900 or higher +- **Style**: Consistent Victorian/steampunk aesthetic to match the Sherlock Holmes theme +- **Colors**: Autumnal palette (oranges, browns, golds) mixed with Victorian colors (deep reds, navy blues) + +## Sourcing Images + +For a production workshop, you could: + +1. **Commission illustrations** from artists specializing in Victorian/steampunk themes +2. **Use public domain images** of Sherlock Holmes (many illustrations are public domain) +3. **Create diagrams** using tools like draw.io, Lucidchart, or Adobe Illustrator +4. **License stock images** from sites like Shutterstock or Getty Images (search for "Sherlock Holmes", "autumn harvest", "database diagrams") + +## Alternative Approach + +If sourcing images is not feasible, you could: + +1. **Remove image references** and rely on text descriptions +2. **Use ASCII art** or text-based diagrams +3. **Create simple SVG diagrams** inline in the markdown + +## Halloween Theme Consistency + +Ensure all images maintain the Halloween/Sherlock Holmes crossover theme: +- Victorian London setting +- Mysterious, foggy atmosphere +- Autumn/Halloween elements (pumpkins, leaves, etc.) +- Detective/mystery solving motifs diff --git a/ai-vector-halloween/images/access-sql-tool.png b/ai-vector-halloween/images/access-sql-tool.png new file mode 100644 index 00000000..f10898f9 Binary files /dev/null and b/ai-vector-halloween/images/access-sql-tool.png differ diff --git a/ai-vector-halloween/images/copy-paste-code.png b/ai-vector-halloween/images/copy-paste-code.png new file mode 100644 index 00000000..cfc73e97 Binary files /dev/null and b/ai-vector-halloween/images/copy-paste-code.png differ diff --git a/ai-vector-halloween/images/embedding-model-select.png b/ai-vector-halloween/images/embedding-model-select.png new file mode 100644 index 00000000..1ea7f927 Binary files /dev/null and b/ai-vector-halloween/images/embedding-model-select.png differ diff --git a/ai-vector-halloween/images/museum-heist.jpeg b/ai-vector-halloween/images/museum-heist.jpeg new file mode 100644 index 00000000..9f615557 Binary files /dev/null and b/ai-vector-halloween/images/museum-heist.jpeg differ diff --git a/ai-vector-halloween/images/run-sql.png b/ai-vector-halloween/images/run-sql.png new file mode 100644 index 00000000..4eb8d4a2 Binary files /dev/null and b/ai-vector-halloween/images/run-sql.png differ diff --git a/ai-vector-halloween/images/running-list-of-sql.png b/ai-vector-halloween/images/running-list-of-sql.png new file mode 100644 index 00000000..1adde097 Binary files /dev/null and b/ai-vector-halloween/images/running-list-of-sql.png differ diff --git a/ai-vector-halloween/images/sql-results.png b/ai-vector-halloween/images/sql-results.png new file mode 100644 index 00000000..8bb6d231 Binary files /dev/null and b/ai-vector-halloween/images/sql-results.png differ diff --git a/ai-vector-halloween/images/vector-embeddings-explained.png b/ai-vector-halloween/images/vector-embeddings-explained.png new file mode 100644 index 00000000..f6d43306 Binary files /dev/null and b/ai-vector-halloween/images/vector-embeddings-explained.png differ diff --git a/ai-vector-halloween/images/vector-embeddings-model.png b/ai-vector-halloween/images/vector-embeddings-model.png new file mode 100644 index 00000000..58325609 Binary files /dev/null and b/ai-vector-halloween/images/vector-embeddings-model.png differ diff --git a/ai-vector-halloween/intro/intro.md b/ai-vector-halloween/intro/intro.md new file mode 100644 index 00000000..b59b44a3 --- /dev/null +++ b/ai-vector-halloween/intro/intro.md @@ -0,0 +1,86 @@ +# The Adventure of the Stolen Pumpkin: A Sherlock Holmes Mystery + +Sherlock Holmes Mystery + +## The Opening Scene + +All Hallow's Eve of 1887 served as a cold start to England's Autumn. With no interest in the Halloween festivities, Holmes was catching up on this morning's newspaper and I was enjoying a quiet moment by the fire in our rooms at 221B Baker Street when I heard a commotion in the hallway. Suddenly, the door burst open with Inspector Lestrade exclaiming, "Holmes! The British Museum has been robbed! Professor Cavendish's original hydrogen research - stolen from the most secure vault in England!" + +Holmes's eyes gleamed. "The legendary Pumpkin Papers." + +"Pumpkin Papers? What in blazes are those?" I asked. + +"Cavendish was rumored to have hidden his most revolutionary hydrogen research in - of all things - a decorative ceramic pumpkin to keep them from falling into the wrong hands," Holmes explained. "When he died in 1810, his estate donated his scientific papers and belongings to the British Museum for preservation. Most were catalogued and made public, but these particular notes - supposedly containing revolutionary insights into hydrogen combustion and potential weaponization - were deemed too dangerous. The museum locked them in their most secure vault. They're worth a fortune to any nation seeking new sources of energy… or weaponry." + +"The theft defies explanation, quite extraordinary" Lestrade continued. "The vault was locked, guards at every entrance. Yet at midnight, strange lights appeared, guards reported screams and apparitions, and by dawn - the papers had vanished." + +"Suspects?" Holmes inquired. + +"Three possibilities. Professor Moriarty's criminal network - they've been after scientific secrets. Foreign agents - German and French spies are in London. Or..." Lestrade paused. "Dr. Cornelius Pumpkin." + +"The mad chemist and stage magician?" I exclaimed. "Why, I read in yesterdays post that he was expelled from the Royal Society?" + +Holmes nodded. "And, Watson, he is none other than Henry Cavendish's great - nephew. Family connection gives him a motive." + +"Watson, the game is afoot! The gleam in his eyes warning of a hunter in search of his prey." + + +## Solve the Mystery + +Join Sherlock Holmes and Dr. Watson in this thrilling Halloween mystery! The famous decorative ceramic pumpkin has been stolen from the British Museum, and suspicions point to Dr. Pumpkin, the town's eccentric inventor. Use Oracle Autonomous AI Database's AI Vector Search to help Holmes uncover the truth behind this bizarre theft! + +**What happened:** Top secret hydrogen fuel cell research was stolen from the British Museum vault! + + +**The suspects:** Three possibilities - Professor Moriarty's agents, foreign spies, or Dr. Pumpkin. But who is the real thief? + +**The clues:** +After examining the crime scene at the British Museum, Inspector Lestrade gathered what evidence he could find and returned to 221B Baker Street. + +"Holmes, Watson," Lestrade began, pulling out his notebook, "we've collected witness reports and physical evidence from the scene. I need your help making sense of it all." + +"Let me have a look at the clues," I said eagerly, reaching for the notes. + +| ID | Description | Evidence Type | +|----|-------------|---------------| +| 1 | Strange glowing lights formed intricate patterns in the air above the gallery, creating an eerie atmosphere | Witness Report | +| 2 | German chemistry textbooks left open to pages about phosphorescent compounds and luminescent substances | Physical Evidence | +| 3 | Someone was heard discussing "family legacy" and "rightful inheritance" near the vault | Witness Report | +| 4 | Luminescent paint residue was found on the cobblestones outside the museum | Physical Evidence | +| 5 | The vault alarm system was disabled using a timing device with unusual symbols etched into its casing | Technical Evidence | +| 6 | Strange glowing patterns resembling chemical formulas were seen dancing in the fog outside | Witness Report | +| 7 | A foreign accent was heard whispering about hydrogen secrets near the vault entrance | Witness Report | +| 8 | Professional stage equipment found abandoned near the scene, including portable lighting rigs and theatrical props | Physical Evidence | +| 9 | Test tubes with glowing residue found near the vault entrance, suggesting chemistry expertise | Physical Evidence | +| 10 | Footprints in the dust that suddenly vanish at the vault door, suggesting theatrical trickery | Physical Evidence | + + +Estimated Workshop Time: 60 minutes + +### About Oracle Autonomous AI Database + +Oracle Autonomous AI Database includes powerful AI features like AI Vector Search which can find meaningful connections between pieces of information, even when they use different words. + +### Objectives + +In this workshop, you will: +* Create a simple database table with mystery clues +* Learn how to "vectorize" text clues (turn them into searchable numbers) +* Compare regular searches with AI-powered vector searches +* Help Holmes solve the mystery by finding hidden connections + +### Prerequisites + +This workshop assumes you have: +* An Oracle Cloud account +* Basic knowledge of databases + + +## Learn More + +* [Oracle Autonomous Database 23ai Documentation](https://docs.oracle.com/en/cloud/paas/autonomous-database/index.html) +* [AI Vector Search in Oracle Database](https://docs.oracle.com/en/database/oracle/oracle-database/23/vecse/) + +## Acknowledgements +* **Author** - Pat Shepherd, Senior Principal Database Product Manager +* **Last Updated By/Date** - Pat Shepherd, October 2025 diff --git a/ai-vector-halloween/labs/setup-adb/setup-adb.md b/ai-vector-halloween/labs/setup-adb/setup-adb.md new file mode 100644 index 00000000..1c76cb46 --- /dev/null +++ b/ai-vector-halloween/labs/setup-adb/setup-adb.md @@ -0,0 +1,143 @@ +# Lab 1: Setting Up the Database and Clue Table + +## Introduction + +In this lab, you'll create a simple database and table to store our mystery clues. This gives us a place to keep all the evidence organized. + +Estimated Lab Time: 15 minutes + + +## Task 1: Create Your Database + +1. Go to Oracle Cloud Console and navigate to **Autonomous Database** +2. Click **Create Autonomous Database** +3. Use these settings: + - **Display name**: `PumpkinMysteryDB` + - **Database name**: `pumpkin` + - **Workload type**: Transaction Processing + - **Deployment**: Shared Infrastructure + - **Always Free**: Check if you want free tier +4. Set a strong password for the ADMIN user +5. Click **Create** and wait for it to finish (5-10 minutes) + + +## Task 2: Create the Clues Table + +For all of the tasks in this lab you will be entering SQL code into SQL Developer Web and running those SQL statement. + +Open the SQL Developer Web tool from your Autonomous AI Database's Database Actions dropdown: +Access SQL Tool + +You will be copying the supplied code from the lab and pasting it into the SQL worksheet: +RCopy and Paste un SQL Statement + + +To run the SQL, highlight only the pasted code and click the green "Run statement" button: +Run SQL Statement + +The output will be visible down below: +Results of SQL Statement + +Pro Tip: you can keep a running list of SQL you have entered. Just remember to highlight the statement you want to run before clicking the green "Run statement" button: +Runing List of SQL Statement + +**Note:** if you ever find yourself wanting to start over, here are commands to run to reset the database back to the instructions before you create your mystery_clues table. Do not run these now, but only when you want to start over: + +```sql + +DELETE FROM mystery_clues; +DROP INDEX idx_clue_embedding; +DROP TABLE mystery_clues CASCADE CONSTRAINTS PURGE; +EXEC DBMS_VECTOR.DROP_ONNX_MODEL(model_name => 'ALL_MINILM_L12_V2'); + +``` + +OK - lets create a simple table to store our mystery clues: + +```sql + +CREATE TABLE mystery_clues ( + clue_id NUMBER PRIMARY KEY, + clue_text VARCHAR2(500) NOT NULL, + clue_type VARCHAR2(50), + location_found VARCHAR2(100), + embedding VECTOR(384) -- For AI Vector Search +); + +``` + +This table stores: +- `clue_id`: A number to identify each clue +- `clue_text`: The actual clue description +- `clue_type`: What kind of clue it is +- `location_found`: Where the clue was discovered +- `embedding`: Special column for AI search (we'll fill this later) + +## Task 3: Add the Mystery Clues + +Let's add 10 simple clues about the museum theft: + +```sql + +INSERT INTO mystery_clues (clue_id, clue_type, location_found, clue_text) +VALUES (1, 'Witness Report', 'Main Gallery', + 'Strange glowing lights formed intricate patterns in the air above the gallery, creating an eerie atmosphere'); + +INSERT INTO mystery_clues (clue_id, clue_type, location_found, clue_text) +VALUES (2, 'Physical Evidence', 'Study Room', + 'German chemistry textbooks left open to pages about phosphorescent compounds and luminescent substances'); + +INSERT INTO mystery_clues (clue_id, clue_type, location_found, clue_text) +VALUES (3, 'Witness Report', 'Museum Corridor', + 'Someone was heard discussing "family legacy" and "rightful inheritance" near the vault'); + +INSERT INTO mystery_clues (clue_id, clue_type, location_found, clue_text) +VALUES (4, 'Physical Evidence', 'Street Outside', + 'Luminescent paint residue was found on the cobblestones outside the museum'); + +INSERT INTO mystery_clues (clue_id, clue_type, location_found, clue_text) +VALUES (5, 'Technical Evidence', 'Security Panel', + 'The vault alarm system was disabled using a timing device with unusual symbols etched into its casing'); + +INSERT INTO mystery_clues (clue_id, clue_type, location_found, clue_text) +VALUES (6, 'Witness Report', 'Near Museum', + 'Strange glowing patterns resembling chemical formulas were seen dancing in the fog outside'); + +INSERT INTO mystery_clues (clue_id, clue_type, location_found, clue_text) +VALUES (7, 'Witness Report', 'Near Vault', + 'A foreign accent was heard whispering about hydrogen secrets near the vault entrance'); + +INSERT INTO mystery_clues (clue_id, clue_type, location_found, clue_text) +VALUES (8, 'Physical Evidence', 'Near Museum', + 'Professional stage equipment found abandoned near the scene, including portable lighting rigs and theatrical props'); + +INSERT INTO mystery_clues (clue_id, clue_type, location_found, clue_text) +VALUES (9, 'Physical Evidence', 'Vault Entrance', + 'Test tubes with glowing residue found near the vault entrance, suggesting chemistry expertise'); + +INSERT INTO mystery_clues (clue_id, clue_type, location_found, clue_text) +VALUES (10, 'Physical Evidence', 'Vault Entrance', + 'Footprints in the dust that suddenly vanish at the vault door, suggesting theatrical trickery'); + +COMMIT; + + +``` + +## Task 4: Check Your Data + +Verify the clues were added correctly: + +```sql + +SELECT clue_id, clue_text, clue_type +FROM mystery_clues +ORDER BY clue_id; + +``` + +You should see all 10 clues listed. + +## Summary + +Great! You've set up your database and added the mystery clues. Now we have evidence to investigate. In the next lab, we'll learn how to use AI to find connections between these clues that regular searches might miss. diff --git a/ai-vector-halloween/labs/traditional-query/traditional-query.md b/ai-vector-halloween/labs/traditional-query/traditional-query.md new file mode 100644 index 00000000..b033342f --- /dev/null +++ b/ai-vector-halloween/labs/traditional-query/traditional-query.md @@ -0,0 +1,107 @@ +# Lab 3: Watson Attempts a Traditional Search + +## Task 1: Watson's Investigation - Traditional Keyword Search + +"Watson, why don't you take the first pass? I shall observe your methodology." + +"Excellent idea!" I exclaimed. "I'll search through the evidence systematically using proper investigative techniques." + +"Right then," I said, rolling up my sleeves. "Let's start with what we know. There were theatrical elements to this crime - lighting effects and stage equipment. Let me search for anything stage or performance." + +### Search 1: Looking for Theatrical Elements + +```sql + +-- Watson searches for theatrical evidence +SELECT clue_id, clue_type, location_found, clue_text +FROM mystery_clues +WHERE UPPER(clue_text) LIKE '%STAGE%' + OR UPPER(clue_text) LIKE '%PERFORMANCE%'; + +``` + +**Results:** 1 clue found + +| clue_id | clue_type | location_found | clue_text | +|---------|-----------|----------------|-----------| +| 8 | Physical Evidence | Near Museum | Professional stage equipment found abandoned near the scene, including portable lighting rigs and theatrical props | + +"Hmm," I muttered. "Only one clue about stage equipment. Could be from a nearby theatre. Or perhaps street performers? This doesn't tell us much - the equipment could have been abandoned there coincidentally." + +### Search 2: Looking for Known Suspects + +"Wait," I said, suddenly remembering our previous encounters. "Didn't we have that eccentric scientist Dr. Cornelius Pumpkin involved in some peculiar business a while back? The one with all those unusual interests? Let me search for him specifically!" + +```sql + +-- Watson searches for Dr. Pumpkin +SELECT clue_id, clue_type, location_found, clue_text +FROM mystery_clues +WHERE UPPER(clue_text) LIKE '%PUMPKIN%' + OR UPPER(clue_text) LIKE '%CORNELIUS%' + OR UPPER(clue_text) LIKE '%DR. PUMPKIN%'; + +``` + +**Results:** 0 clues found + +"Nothing!" I exclaimed in frustration. "No mention of Dr. Pumpkin anywhere in the evidence. Well, I suppose that rules him out as a suspect. Unless..." I trailed off, uncertain. + +Holmes raised an eyebrow but said nothing. + +### Search 3: Looking for Family Connections + +"Let me try a different approach," I said, determined not to give up. "Lestrade mentioned something about someone discussing 'family legacy.' Let me search for that!" + +```sql + +-- Watson searches for family references +SELECT clue_id, clue_type, location_found, clue_text +FROM mystery_clues +WHERE UPPER(clue_text) LIKE '%FAMILY%' + OR UPPER(clue_text) LIKE '%LEGACY%' + OR UPPER(clue_text) LIKE '%INHERITANCE%'; + +``` + +**Results:** 1 clue found + +| clue_id | clue_type | location_found | clue_text | +|---------|-----------|----------------|-----------| +| 3 | Witness Report | Museum Corridor | Someone was heard discussing "family legacy" and "rightful inheritance" near the vault | + +"Interesting!" I said. "Someone talking about family legacy. But Holmes, this is terribly vague! Whose family? What legacy? This could be anyone with a grievance about inheritance." + + +## Watson's Confusion + +I sat back and reviewed my findings, frustration mounting. + +"Holmes, I haven't the faintest idea what to make of this!" I confessed. "I've found clues about: +- Theatrical equipment (possibly abandoned by performers) +- Someone discussing family legacy (utterly ambiguous) +- And notably, NO evidence connecting to any known suspects like Dr. Cornelius Pumpkin + +The problem is, none of these connect! I specifically searched for Dr. Pumpkin - someone we know has peculiar scientific interests that might fit this crime - but his name doesn't appear anywhere in the evidence. So either he's not involved, or... well, I don't know what the alternative would be!" + +Holmes smiled enigmatically. "Watson, your methodology is sound - you've searched diligently for keywords. But you're searching for *words*, not *meaning*. You're finding trees, my friend, but missing the forest." + +"But what else can I do?" I protested. "I've searched for theatrical terms, for specific suspects, for family connections. The evidence simply doesn't connect, and the one person I thought might be involved doesn't appear in any of the clues!" + +"Ah, but Watson," Holmes said, his eyes twinkling, "therein lies the fundamental limitation of keyword search. You searched for the name 'Pumpkin' and found nothing - but what if the evidence describes him without using his name? What if the clues point to him through his *characteristics* rather than his identity?" + +"I don't understand, Holmes." + +"Tell me - if Dr. Cornelius Pumpkin were involved, what would you expect to find? Theatre expertise? Chemistry knowledge? A connection to Cavendish?" + +"Well, yes, but..." + +"And did you find evidence of those things?" + +I paused, thinking back through my searches. "The theatrical equipment, yes. But I didn't search for chemistry because I didn't connect it to Dr. Pumpkin specifically!" + +"Precisely," Holmes said. "And that, my dear Watson, is why your keyword search has led you to confusion rather than clarity. You cannot find what you don't know to search for. Now then, let me show you how vector search can reveal the connections you've been missing..." + +"Holmes," I said with resignation, "I fear this case is beyond the reach of conventional methods. I cannot find suspects who aren't explicitly named in the evidence!" + +"On the contrary, Watson. You've demonstrated perfectly why modern crimes require modern methods. Your keyword searches found *information*, but they couldn't find *meaning*. You searched for a name and found nothing - but the evidence about the person was there all along, just described differently. Now, let me show you what happens when we search for concepts rather than words..." \ No newline at end of file diff --git a/ai-vector-halloween/labs/vector-search/vector-search.md b/ai-vector-halloween/labs/vector-search/vector-search.md new file mode 100644 index 00000000..ad5bd915 --- /dev/null +++ b/ai-vector-halloween/labs/vector-search/vector-search.md @@ -0,0 +1,108 @@ +# Lab 4: Holmes Applies AI Vector Search to the Problem + +## Holmes's Method: The AI Vector Search Investigation + +"Now then, Watson," Holmes said, settling into his chair with his pipe, "you've demonstrated the fundamental limitation of keyword search - you found scattered clues that could support multiple theories." + +"Exactly my problem, Holmes!" I exclaimed. "The evidence is too ambiguous!" + +"Is it, Watson? Or have you simply been searching for the wrong things?" Holmes's eyes gleamed. "Let me demonstrate the power of semantic analysis through Oracle's AI Vector Search technology. You searched for *words* - I shall search for *meanings*." + +"But Holmes," I protested, "how can you search for meaning when the clues themselves are so vague?" + +"Ah, but that's where vector embeddings come in, my dear fellow! Each clue has been transformed into a mathematical representation - a vector in high-dimensional space - that captures its semantic meaning. When we query using AI Vector Search, we find clues that are *conceptually similar*, even if they use entirely different vocabulary!" + +"But you have the same eleven clues I do," I objected. "How will that help?" + +Holmes smiled mysteriously. "Watson, tell me - in all your keyword searches, did you search for personal monograms or identity markers?" + +I blinked. "Well, no... there were no names in any of the clues!" + +"Precisely. Your keyword search methodology requires you to know what to search for. But what if there's a clue you didn't find because it doesn't contain your search terms?" + +Holmes turned to his computer terminal. "Now, Watson, watch what happens when we perform ONE comprehensive semantic search..." + + + +## Task 1: The Comprehensive AI Vector Search + +"Watson, I'm going to construct ONE search query that searches for a complete criminal profile - someone with theatrical expertise, chemistry knowledge, hydrogen research obsession, and a family connection to Cavendish." + +```sql + +-- Holmes's comprehensive semantic search +SELECT clue_id, clue_type, location_found, + SUBSTR(clue_text, 1, 180) || '...' as clue_preview, + VECTOR_DISTANCE(embedding, + VECTOR_EMBEDDING(ALL_MINILM_L12_V2 USING 'chemistry drawings family obsession stagecraft leaving personal signatures' AS data), + COSINE) as similarity_score +FROM mystery_clues +ORDER BY similarity_score +FETCH FIRST 5 ROWS ONLY; + +``` + +**Holmes's Results:** + +| Rank | Evidence Type | Location | Description | Score | +|------|---------------|----------|-------------|-------| +| 11 | Physical Evidence | Inside Vault | A notebook with chemistry research on renewable energy signed "C.P." found dropped on the floor inside the vault... | 0.626 | +| 9 | Physical Evidence | Vault Entrance | Test tubes with glowing residue found near the vault entrance, suggesting chemistry expertise... | 0.63 | +| 4 | Physical Evidence | Street Outside | Luminescent paint residue was found on the cobblestones outside the museum... | 0.682 | +| 6 | Witness Report | Near Museum | Strange glowing patterns resembling chemical formulas were seen dancing in the fog outside... | 0.694 | +| 3 | Witness Report | Museum Corridor | Someone was heard discussing "family legacy" and "rightful inheritance" near the vault... | 0.712 | + + +*AI Vector Search similarity scores range from 0 to 1, with lower scores indicating closser distance and higher relevance.* + + +## Conclusion: The Solution Revealed + +"Watson, my dear fellow, observe how the evidence clusters around a singular perpetrator. The test tubes with their glowing residue at the vault entrance - similarity score 0.63 - together with that notebook bearing the signature 'C.P.' and detailing chemistry research on renewable energy, found within the vault itself at 0.626. Already we see a pattern emerging." + +He gestures enthusiastically with his pipe before adding, "Then consider the chemistry expertise markers: luminescent paint residue upon the cobblestones, those glowing patterns resembling chemical formulas dancing through the fog like some theatrical performance - 0.682 and 0.694 similarity. And importantly, that discussion of 'family legacy' and 'rightful inheritance' near the vault scene registers at 0.712." + +He leans forward with evident delight. "My AI Vector Search comprehends semantic meaning rather than mere word-matching. It connected that 'C.P.' signed notebook directly to the pattern of chemistry, stagecraft, personal signatures, and family obsession. It revealed that all the highest-ranked clues point to one individual possessing both chemistry expertise and family grievances... Dr. Cornelius Pumpkin." + +He chuckles warmly, "Two methods, two perspectives - but yours helped me appreciate why AI Vector Search proves so remarkably efficient for this sort of puzzle." He sat back with a note of satisfaction, "Elementary, my dear Watson. Elementary." + +## Epilogue: The Phosphorus and the Lost Secret + + +Weeks later, Inspector Lestrade returned to 221B Baker Street, his face grave. + +"Gentlemen, I bring news of Dr. Cornelius Pumpkin." + +"Let me guess," Holmes said, not looking up from his chemistry set. "He attempted to escape to the Americas?" + +Lestrade blinked. "How did you... yes. The Phosphorus out of Southampton. But there's been a tragedy. A terrible storm struck in mid-Atlantic. Witnesses from another ship report the Phosphorus caught fire and sank with all hands. The flames had an unusual greenish glow. Dr. Pumpkin and the stolen hydrogen papers were lost to the depths." + +I set down my newspaper. "Good Lord! The poor man!" + +"A man obsessed with creating light, meeting his end in the darkest depths," Holmes observed. "Consider, Watson: Dr. Pumpkin spent his life trying to reclaim his family's legacy. Yet in stealing it, he ensured both he and the papers would be lost to history entirely. Had he simply published legitimate research, he might have made genuine contributions to science." + +"Speaking of connections," I interjected, "wasn't the ship called the Phosphorus? The very chemical he used in his glowing effects." + +"One might say he went down with his signature element. The universe does enjoy its ironies." + +"Though there's something fitting about a man named Pumpkin meeting his end near All Hallow's Eve," I added. "Thematically speaking." + +Lestrade nodded glumly. "Had Scotland Yard employed Oracle's AI Vector Search from the start, we might have identified him before he fled." + +"That, Lestrade, is precisely the lesson. You look for what's explicitly stated. I look for what's implicitly connected." Holmes raised an eyebrow. "Besides, hydrogen is one of the most abundant elements in the universe. Other scientists will rediscover these principles. The real tragedy is what Dr. Pumpkin could have been." + +After Lestrade departed, Holmes knocked out his pipe. "I'll be remembered for teaching one stubborn inspector that 'meaning' matters more than 'matching.'" Holmes smiled. "Now then - you promised Mrs. Hudson you'd chronicle this case. Do emphasize methodology over melodrama. Educational value, Watson." + +"Of course, Holmes. Though the glowing horses are too good to leave out." + +"Watson," Holmes sighed, "you're incorrigible." + +"Yes, Holmes. But you'd be bored without me." + +"Indubitably." + +--- + +**The End** + +*As chronicled by John H. Watson, M.D., with technical annotations on Oracle AI Vector Search methodology. "The case may be closed, but the lesson remains: In an age of information, those who can find meaning in chaos will always stay one step ahead of those who can only find words." - Sherlock Holmes, 1887* diff --git a/ai-vector-halloween/labs/vectorize-clues/vectorize-clues.md b/ai-vector-halloween/labs/vectorize-clues/vectorize-clues.md new file mode 100644 index 00000000..cba3bdd8 --- /dev/null +++ b/ai-vector-halloween/labs/vectorize-clues/vectorize-clues.md @@ -0,0 +1,208 @@ +# Lab 2: Vectorizing the Evidence - Transforming Clues into Embeddings + + +## Introduction + +"Data! Data! Data!" Holmes would exclaim. "I cannot make bricks without clay." But what if our clay could be transformed into something more powerful - something that captures not just the words, but the very essence and relationships between our clues? + +In this lab, you'll learn how to vectorize your textual clues using AI embeddings. These vector representations will allow us to find semantic similarities between clues that traditional keyword searches might miss - much like Holmes connecting seemingly unrelated facts to solve a case. + +Estimated Lab Time: 20 minutes + +## Task 1: Understand Vector Embeddings + +Before we dive into the technical implementation, let's understand what vector embeddings are and why they're crucial for our investigation. + +Vector embeddings are numerical representations of text that capture semantic meaning. Similar concepts will have similar vector representations, allowing us to find connections based on meaning rather than just keywords. + +Vector Embeddings Explained + +## Task 2: Import the Embedding Model + +Before we can vectorize our clues, we need to import the all-MiniLM-L12-v2 embedding model into our database. This model will convert text into numerical vectors that capture meaning. Take a look at this diagram that explains what we are about to do. +Vector Embeddings Explained + +### Load the all-MiniLM-L12-v2 Model + +Execute this SQL script to import the pre-built all-MiniLM-L12-v2 embedding model: + +```sql + +DECLARE + ONNX_MOD_FILE VARCHAR2(100) := 'all_MiniLM_L12_v2.onnx'; + MODNAME VARCHAR2(500); + LOCATION_URI VARCHAR2(200) := 'https://adwc4pm.objectstorage.us-ashburn-1.oci.customer-oci.com/p/eLddQappgBJ7jNi6Guz9m9LOtYe2u8LWY19GfgU8flFK4N9YgP4kTlrE9Px3pE12/n/adwc4pm/b/OML-Resources/o/'; + +BEGIN + DBMS_OUTPUT.PUT_LINE('ONNX model file name in Object Storage is: '||ONNX_MOD_FILE); +-------------------------------------------- +-- Define a model name for the loaded model +-------------------------------------------- + SELECT UPPER(REGEXP_SUBSTR(ONNX_MOD_FILE, '[^.]+')) INTO MODNAME from dual; + DBMS_OUTPUT.PUT_LINE('Model will be loaded and saved with name: '||MODNAME); + +----------------------------------------------------- +-- Read the ONNX model file from Object Storage into +-- the Autonomous Database data pump directory +----------------------------------------------------- + +BEGIN DBMS_DATA_MINING.DROP_MODEL(model_name => MODNAME); +EXCEPTION WHEN OTHERS THEN NULL; END; + + DBMS_CLOUD.GET_OBJECT( + credential_name => NULL, + directory_name => 'DATA_PUMP_DIR', + object_uri => LOCATION_URI||ONNX_MOD_FILE); + +----------------------------------------- +-- Load the ONNX model to the database +----------------------------------------- + + DBMS_VECTOR.LOAD_ONNX_MODEL( + directory => 'DATA_PUMP_DIR', + file_name => ONNX_MOD_FILE, + model_name => MODNAME); + + DBMS_OUTPUT.PUT_LINE('New model successfully loaded with name: '||MODNAME); +END; + +``` + +**Important:** Make sure you have the necessary privileges to load models. This script downloads the model from Oracle's Object Storage and loads it into your database. + +### Verify the Model Was Loaded + +Check that the model loaded successfully: + +```sql + +SELECT model_name, mining_function, algorithm +FROM user_mining_models +WHERE model_name = 'ALL_MINILM_L12_V2'; + +``` + +You should see the model listed. The all-MiniLM-L12-v2 model creates 384-dimensional vectors that capture semantic meaning, allowing us to find clues with similar concepts even when they use different words. +Vector Embedding Model + +## Task 3: Generate Embeddings for Our Clues + +Now we'll vectorize each clue in our mystery_clues table. This will allow us to perform semantic searches later. + +1. Update the embedding column for all existing clues: + + ```sql + + UPDATE mystery_clues + SET embedding = VECTOR_EMBEDDING(ALL_MINILM_L12_V2 USING clue_text AS data) + WHERE embedding IS NULL; + + ``` + + This command uses Oracle's built-in embedding function to convert each clue_text into a vector representation. + +2. Verify that embeddings were generated: + + ```sql + + SELECT clue_id, clue_type, SUBSTR(clue_text, 1, 50) || '...' as clue_preview, + embedding as vector_dimension + FROM mystery_clues + WHERE embedding IS NOT NULL + ORDER BY clue_id; + + ``` + + You should see that each clue now has a vector representation with the appropriate dimensions. + +## Task 4: Create a Vector Index for Efficient Searching + +To perform fast vector searches, we need to create a vector index on our embedding column. + +1. Create the vector index: + + ```sql + + CREATE VECTOR INDEX idx_clue_embedding + ON mystery_clues (embedding) + ORGANIZATION INMEMORY NEIGHBOR GRAPH + DISTANCE COSINE; + + ``` + + This creates an in-memory neighbor graph index optimized for cosine distance similarity searches. + +## Task 5: Test Basic Vector Similarity + +Let's test our vector setup by finding clues that are semantically similar to a test query. + +1. Try a test with "scientific evidence": + + ```sql + + SELECT clue_id, clue_type, location_found, + SUBSTR(clue_text, 1, 80) || '...' as clue_preview, + ROUND(VECTOR_DISTANCE(embedding, + VECTOR_EMBEDDING(ALL_MINILM_L12_V2 USING 'scientific evidence' AS data), + COSINE), 4) as similarity_score + FROM mystery_clues + ORDER BY similarity_score + FETCH FIRST 5 ROWS ONLY; + + ``` + + Notice how this finds this science-related evidence: + +| Clue ID | Clue Type | Location Found | Clue Description | Similarity Score | +|---------|-----------|----------------|------------------|------------------| +| 9 | Physical Evidence | Vault Entrance | Test tubes with glowing residue found near the vault entrance, suggesting chemistry expertise... | 0.705 | +| 6 | Witness Report | Near Museum | Strange glowing patterns resembling chemical formulas were seen dancing in the fog outside... | 0.772 | +| 7 | Witness Report | Near Vault | A foreign accent was heard whispering about hydrogen secrets near the vault entrance... | 0.828 | +| 10 | Physical Evidence | Vault Entrance | Footprints in the dust that suddenly vanish at the vault door, suggesting theatrical trickery... | 0.830 | +| 1 | Witness Report | Main Gallery | Strange glowing lights formed intricate patterns in the air above the gallery, creating an eerie atmosphere... | 0.858 | + + + + +## Task 6: A Breakthrough! New Evidence Arrives + +Holmes gestured to Lestrade's evidence file. "Inspector, what about that handkerchief the forensics team found snagged on the vault filing cabinet?" +Lestrade rummaged through his papers. "Oh yes! Didn't think it was important - probably just dropped by museum staff during normal operations." + +"Let me see it," Holmes demanded. +Lestrade read from his notes: "A monogrammed silk handkerchief, embroidered with 'C.P' in gold thread, was found snagged on a filing cabinet drawer inside the vault." + +Let's add and vectorize this crucial new evidence: + +1. Insert the breakthrough clue: + +```sql + +-- Adding the crucial 11th clue that Watson's keyword search missed +INSERT INTO mystery_clues (clue_id, clue_type, location_found, clue_text) +VALUES (11, 'Physical Evidence', 'Inside Vault', + 'A notebook with chemistry research on renewable energy signed "C.P" found dropped on the floor inside the vault'); + +COMMIT; + +``` + +2. Vectorize the new clue: + +And, now we update the clue to have a vector embedding like so: +```sql + +UPDATE mystery_clues +SET embedding = VECTOR_EMBEDDING(ALL_MINILM_L12_V2 USING clue_text AS data) +WHERE clue_id = 11; + +``` + +"There," Holmes said. "Now all eleven clues are in the database." + +## Summary + +Brilliant work, detective! You've successfully vectorized your clues, transforming simple text into powerful semantic representations. This is the digital equivalent of Holmes's ability to see connections that others miss. + +In the next lab, we'll see how traditional SQL queries might miss important connections that AI Vector Search can reveal. The mystery deepens, but so do our investigative capabilities! + diff --git a/ai-vector-halloween/workshops/pattsmegalab/index.html b/ai-vector-halloween/workshops/pattsmegalab/index.html new file mode 100644 index 00000000..aebbdda4 --- /dev/null +++ b/ai-vector-halloween/workshops/pattsmegalab/index.html @@ -0,0 +1,63 @@ + + + + + + + + + Oracle LiveLabs + + + + + + + + + + + + + +
+
+
+
+
+
+
+
+ + + + + diff --git a/ai-vector-halloween/workshops/pattsmegalab/manifest.json b/ai-vector-halloween/workshops/pattsmegalab/manifest.json new file mode 100644 index 00000000..8fc83169 --- /dev/null +++ b/ai-vector-halloween/workshops/pattsmegalab/manifest.json @@ -0,0 +1,39 @@ +{ + "workshoptitle": "The Adventure of the Stolen Pumpkin: A Sherlock Holmes Mystery", + "tutorials": [ + { + "title": "Getting Started", + "description": "Prerequisites for LiveLabs (Oracle-owned tenancies). The title of the lab and the Contents Menu title (the title above) match for Prerequisite lab. This lab is always first.", + "filename": "https://oracle-livelabs.github.io/common/labs/cloud-login/cloud-login-livelabs2.md" + }, + { + "title": "Introduction", + "filename": "../../intro/intro.md" + }, + { + "title": "Lab 1: Setting Up the Database and Clue Table", + "description": "Create a simple database and table to store mystery clues about the stolen pumpkin.", + "filename": "../../labs/setup-adb/setup-adb.md" + }, + { + "title": "Lab 2: Vectorizing the Evidence - Transforming Clues into Embeddings", + "description": "Learn how to import and use the all-MiniLM-L12-v2 model to vectorize textual clues for AI-powered search.", + "filename": "../../labs/vectorize-clues/vectorize-clues.md" + }, + { + "title": "Lab 3: Watson Tries His Hand at Deduction - The Keyword Search Investigation", + "description": "See how traditional keyword searches miss hidden connections between clues.", + "filename": "../../labs/traditional-query/traditional-query.md" + }, + { + "title": "Lab 4: Holmes Applies AI Vector Search to the Problem", + "description": "Use AI vector search to discover connections that solve the pumpkin theft mystery.", + "filename": "../../labs/vector-search/vector-search.md" + }, + { + "title": "Need Help?", + "description": "Solutions to Common Problems and Directions for Receiving Live Help", + "filename":"https://oracle-livelabs.github.io/common/labs/need-help/need-help-livelabs.md" + } + ] +}