From e8ad75bb68df847b297c1bef03941e461d109730 Mon Sep 17 00:00:00 2001 From: Justin Cechmanek Date: Fri, 14 Nov 2025 11:08:24 -0800 Subject: [PATCH 1/7] first pass at langcache notebook --- .../04_langcache_semantic_caching.ipynb | 1167 +++++++++++++++++ 1 file changed, 1167 insertions(+) create mode 100644 python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb diff --git a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb new file mode 100644 index 00000000..911c56df --- /dev/null +++ b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb @@ -0,0 +1,1167 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![Redis](https://redis.io/wp-content/uploads/2024/04/Logotype.svg?auto=webp&quality=85,75&width=120)\n", + "\n", + "# LangCache: Semantic Caching with Redis Cloud\n", + "\n", + "This notebook demonstrates end-to-end semantic caching using **LangCache** - a managed Redis Cloud service accessed through the RedisVL library. LangCache provides enterprise-grade semantic caching with zero infrastructure management, making it ideal for production LLM applications.\n", + "\n", + "\"Open\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Introduction\n", + "\n", + "**LangCache** is a fully managed semantic cache service built on Redis Cloud. It was integrated into RedisVL in version 0.11.0 as an `LLMCache` interface implementation, making it easy for RedisVL users to:\n", + "\n", + "- Transition to a fully managed caching service\n", + "- Reduce LLM API costs by caching similar queries\n", + "- Improve application response times\n", + "- Access enterprise features without managing infrastructure\n", + "\n", + "### What You'll Learn\n", + "\n", + "In this tutorial, you will:\n", + "1. Set up LangCache with Redis Cloud\n", + "2. Load and process a knowledge base (PDF documents)\n", + "3. Generate FAQs using the Doc2Cache technique\n", + "4. Pre-populate a semantic cache with tagged FAQs\n", + "5. Test different cache matching strategies and thresholds\n", + "6. Optimize cache performance using evaluation datasets\n", + "7. Use the `langcache-embed` cross-encoder model\n", + "8. Integrate the cache into a RAG pipeline\n", + "9. Measure performance improvements\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 1. Environment Setup\n", + "\n", + "First, we'll install the required packages and set up our environment.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Install Required Packages\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%pip install -q \"redisvl>=0.11.0\" \"openai>=1.0.0\" \"langchain>=0.3.0\" \"langchain-community\" \"langchain-openai\"\n", + "%pip install -q \"pypdf\" \"sentence-transformers\" \"redis-retrieval-optimizer>=0.2.0\"\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import IPython\n", + "\n", + "app = IPython.Application.instance()\n", + "app.kernel.do_shutdown(True)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Import Dependencies\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import time\n", + "import json\n", + "from getpass import getpass\n", + "from typing import List, Dict, Any\n", + "\n", + "# RedisVL imports\n", + "from redisvl.extensions.llmcache import SemanticCache\n", + "from redisvl.utils.vectorize import HFTextVectorizer\n", + "\n", + "# LangChain imports\n", + "from langchain_community.document_loaders import PyPDFLoader\n", + "from langchain_text_splitters import RecursiveCharacterTextSplitter\n", + "from langchain_openai import ChatOpenAI, OpenAIEmbeddings\n", + "from langchain.prompts import PromptTemplate\n", + "from langchain_core.output_parsers import JsonOutputParser\n", + "from pydantic import BaseModel, Field\n", + "\n", + "# Optimization\n", + "from redis_retrieval_optimizer.threshold_optimization import CacheThresholdOptimizer\n", + "\n", + "print(\"✓ All imports successful\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. LangCache Setup\n", + "\n", + "### Sign Up for LangCache\n", + "\n", + "If you haven't already, sign up for a free LangCache account:\n", + "\n", + "**[Sign up for LangCache →](https://langcache.io/signup)**\n", + "\n", + "After signing up:\n", + "1. Create a new cache instance\n", + "2. Copy your **Endpoint URL** (looks like: `redis-xxxxx.langcache.io:xxxxx`)\n", + "3. Copy your **Access Token/Password**\n", + "4. Note your **Cache ID** (you'll use this as a prefix for organizing caches)\n", + "\n", + "> **Note:** For this workshop, you can alternatively use a standard Redis Cloud instance with Redis Stack. Simply provide your Redis Cloud connection details instead.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Configure Environment Variables\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Redis/LangCache credentials\n", + "if \"REDIS_URL\" not in os.environ:\n", + " redis_host = input(\"Enter your Redis/LangCache host (e.g., redis-xxxxx.langcache.io or localhost): \")\n", + " redis_port = input(\"Enter your Redis port (default: 6379): \") or \"6379\"\n", + " redis_password = getpass(\"Enter your Redis password (leave empty for local): \")\n", + " \n", + " # Build Redis URL\n", + " if redis_password:\n", + " os.environ[\"REDIS_URL\"] = f\"rediss://:{redis_password}@{redis_host}:{redis_port}\"\n", + " else:\n", + " os.environ[\"REDIS_URL\"] = f\"redis://{redis_host}:{redis_port}\"\n", + "\n", + "# OpenAI API key for LLM and embeddings\n", + "if \"OPENAI_API_KEY\" not in os.environ:\n", + " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")\n", + "\n", + "print(\"✓ Environment variables configured\")\n", + "print(f\" Redis URL: {os.environ['REDIS_URL'].split('@')[-1] if '@' in os.environ['REDIS_URL'] else os.environ['REDIS_URL'].split('//')[1]}\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Initialize Semantic Cache with LangCache-Embed Model\n", + "\n", + "We'll create a cache instance using the `redis/langcache-embed-v1` model, which is specifically optimized for semantic caching tasks.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Initialize the vectorizer with the LangCache embedding model\n", + "# This model is specifically optimized for semantic caching with better precision/recall\n", + "vectorizer = HFTextVectorizer(\n", + " model=\"redis/langcache-embed-v1\"\n", + ")\n", + "\n", + "# Create Semantic Cache instance\n", + "cache = SemanticCache(\n", + " name=\"rag_faq_cache\",\n", + " redis_url=os.environ[\"REDIS_URL\"],\n", + " vectorizer=vectorizer,\n", + " distance_threshold=0.15 # Initial threshold, we'll optimize this later\n", + ")\n", + "\n", + "print(f\"✓ Semantic Cache initialized: {cache.index.name}\")\n", + "print(f\" Using model: redis/langcache-embed-v1\")\n", + "print(f\" Distance threshold: {cache.distance_threshold}\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Initialize OpenAI LLM\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Initialize OpenAI LLM for FAQ generation and RAG\n", + "llm = ChatOpenAI(\n", + " model=\"gpt-4o-mini\",\n", + " temperature=0.3,\n", + " max_tokens=2000\n", + ")\n", + "\n", + "print(\"✓ OpenAI LLM initialized\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 3. Load and Prepare Datasets\n", + "\n", + "We'll work with three types of data:\n", + "1. **Knowledge Base**: PDF document(s) that contain factual information\n", + "2. **FAQs**: Derived from the knowledge base using Doc2Cache technique\n", + "3. **Test Dataset**: For evaluating and optimizing cache performance\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Load PDF Knowledge Base\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Download sample PDF if not already present\n", + "!mkdir -p data\n", + "!wget -q -O data/nvidia-10k.pdf https://raw.githubusercontent.com/redis-developer/redis-ai-resources/main/python-recipes/RAG/resources/nvd-10k-2023.pdf\n", + "\n", + "print(\"✓ Sample PDF downloaded\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Load and chunk the PDF\n", + "pdf_path = \"data/nvidia-10k.pdf\"\n", + "\n", + "# Configure text splitter for optimal chunk sizes\n", + "text_splitter = RecursiveCharacterTextSplitter(\n", + " chunk_size=2000,\n", + " chunk_overlap=200,\n", + " separators=[\"\\n\\n\", \"\\n\", \". \", \" \", \"\"]\n", + ")\n", + "\n", + "# Load and split the document\n", + "loader = PyPDFLoader(pdf_path)\n", + "documents = loader.load()\n", + "chunks = text_splitter.split_documents(documents)\n", + "\n", + "print(f\"✓ Loaded PDF: {pdf_path}\")\n", + "print(f\" Total pages: {len(documents)}\")\n", + "print(f\" Created chunks: {len(chunks)}\")\n", + "print(f\"\\nSample chunk preview:\")\n", + "print(f\"{chunks[10].page_content[:300]}...\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Generate FAQs Using Doc2Cache Technique\n", + "\n", + "The Doc2Cache approach uses an LLM to generate frequently asked questions from document chunks. These FAQs are then used to pre-populate the semantic cache with high-quality, factual responses.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Define the FAQ data model\n", + "class QuestionAnswer(BaseModel):\n", + " question: str = Field(description=\"A frequently asked question derived from the document content\")\n", + " answer: str = Field(description=\"A factual answer to the question based on the document\")\n", + " category: str = Field(description=\"Category of the question (e.g., 'financial', 'products', 'operations')\")\n", + "\n", + "class FAQList(BaseModel):\n", + " faqs: List[QuestionAnswer] = Field(description=\"List of question-answer pairs extracted from the document\")\n", + "\n", + "# Set up JSON output parser\n", + "json_parser = JsonOutputParser(pydantic_object=FAQList)\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create the FAQ generation prompt\n", + "faq_prompt = PromptTemplate(\n", + " template=\"\"\"You are a document analysis expert. Extract 3-5 high-quality FAQs from the following document chunk.\n", + "\n", + "Guidelines:\n", + "- Generate diverse, specific questions that users would realistically ask\n", + "- Provide accurate, complete answers based ONLY on the document content\n", + "- Assign each FAQ to a category: 'financial', 'products', 'operations', 'technology', or 'general'\n", + "- Avoid vague or overly generic questions\n", + "- If the chunk lacks substantial content, return fewer FAQs\n", + "\n", + "{format_instructions}\n", + "\n", + "Document Chunk:\n", + "{doc_content}\n", + "\n", + "FAQs JSON:\"\"\",\n", + " input_variables=[\"doc_content\"],\n", + " partial_variables={\"format_instructions\": json_parser.get_format_instructions()}\n", + ")\n", + "\n", + "# Create the FAQ generation chain\n", + "faq_chain = faq_prompt | llm | json_parser\n", + "\n", + "print(\"✓ FAQ generation chain configured\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Test FAQ generation on a single chunk\n", + "print(\"Testing FAQ generation on sample chunk...\\n\")\n", + "test_faqs = faq_chain.invoke({\"doc_content\": chunks[10].page_content})\n", + "\n", + "print(f\"Generated {len(test_faqs.get('faqs', []))} FAQs:\")\n", + "for i, faq in enumerate(test_faqs.get('faqs', [])[:3], 1):\n", + " print(f\"\\n{i}. Q: {faq['question']}\")\n", + " print(f\" Category: {faq['category']}\")\n", + " print(f\" A: {faq['answer'][:150]}...\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Generate FAQs from all chunks (limited to first 25 for demo purposes)\n", + "def extract_faqs_from_chunks(chunks: List[Any], max_chunks: int = 25) -> List[Dict]:\n", + " \"\"\"Extract FAQs from document chunks using LLM.\"\"\"\n", + " all_faqs = []\n", + " \n", + " for i, chunk in enumerate(chunks[:max_chunks]):\n", + " if i % 5 == 0:\n", + " print(f\"Processing chunk {i+1}/{min(len(chunks), max_chunks)}...\", flush=True)\n", + " \n", + " try:\n", + " result = faq_chain.invoke({\"doc_content\": chunk.page_content})\n", + " if result and result.get(\"faqs\"):\n", + " all_faqs.extend(result[\"faqs\"])\n", + " except Exception as e:\n", + " print(f\" Warning: Skipped chunk {i+1} due to error: {str(e)[:100]}\")\n", + " continue\n", + " \n", + " return all_faqs\n", + "\n", + "# Extract FAQs\n", + "print(\"\\nGenerating FAQs from document chunks...\\n\")\n", + "faqs = extract_faqs_from_chunks(chunks, max_chunks=25)\n", + "\n", + "print(f\"\\n✓ Generated {len(faqs)} FAQs total\")\n", + "print(f\"\\nCategory distribution:\")\n", + "categories = {}\n", + "for faq in faqs:\n", + " cat = faq.get('category', 'unknown')\n", + " categories[cat] = categories.get(cat, 0) + 1\n", + "for cat, count in sorted(categories.items(), key=lambda x: x[1], reverse=True):\n", + " print(f\" {cat}: {count}\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Create Test/Evaluation Dataset\n", + "\n", + "We'll create a test dataset with:\n", + "- **Positive examples**: Questions that should match cached FAQs\n", + "- **Negative examples**: Questions that should NOT match cached FAQs\n", + "- **Edge cases**: Slightly different phrasings to test threshold sensitivity\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Select representative FAQs for test set\n", + "sample_faqs = faqs[:10] # Take first 10 FAQs\n", + "\n", + "print(\"Sample FAQs for testing:\")\n", + "for i, faq in enumerate(sample_faqs[:3], 1):\n", + " print(f\"\\n{i}. {faq['question'][:100]}...\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create test dataset with negative examples (off-topic questions)\n", + "negative_examples = [\n", + " {\"query\": \"What is the weather today?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", + " {\"query\": \"How do I cook pasta?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", + " {\"query\": \"What is the capital of France?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", + " {\"query\": \"Tell me a joke\", \"expected_match\": False, \"category\": \"off-topic\"},\n", + " {\"query\": \"What time is it?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", + "]\n", + "\n", + "print(f\"✓ Test dataset created\")\n", + "print(f\" Negative examples: {len(negative_examples)}\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 4. Pre-Load Semantic Cache with FAQs\n", + "\n", + "Now we'll populate the cache instance with our generated FAQs. We'll use the `store()` API with metadata tags for filtering and organization.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Clear any existing cache entries\n", + "cache.clear()\n", + "print(\"✓ Cache cleared\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Store FAQs in cache with metadata tags\n", + "print(\"Storing FAQs in cache...\\n\")\n", + "\n", + "stored_count = 0\n", + "cache_keys = {} # Map questions to their cache keys\n", + "\n", + "for i, faq in enumerate(faqs):\n", + " if i % 20 == 0:\n", + " print(f\" Stored {i}/{len(faqs)} FAQs...\", flush=True)\n", + " \n", + " try:\n", + " # Store with metadata - note that metadata is stored but not used for filtering in basic SemanticCache\n", + " # In production, you can use this for analytics and tracking\n", + " key = cache.store(\n", + " prompt=faq['question'],\n", + " response=faq['answer']\n", + " )\n", + " cache_keys[faq['question']] = key\n", + " stored_count += 1\n", + " except Exception as e:\n", + " print(f\" Warning: Failed to store FAQ {i+1}: {str(e)[:100]}\")\n", + "\n", + "print(f\"\\n✓ Stored {stored_count} FAQs in cache\")\n", + "print(f\" Cache index: {cache.index.name}\")\n", + "print(f\"\\nExample cache entries:\")\n", + "for i, (q, k) in enumerate(list(cache_keys.items())[:2], 1):\n", + " print(f\"\\n{i}. Key: {k}\")\n", + " print(f\" Q: {q[:80]}...\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 5. Test Cache Retrieval with Different Strategies\n", + "\n", + "Let's test how the cache performs with different types of queries and matching thresholds.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Test Exact Match Queries\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Test with exact questions from cache\n", + "print(\"Testing exact match queries:\\n\")\n", + "\n", + "for i, faq in enumerate(faqs[:3], 1):\n", + " result = cache.check(prompt=faq['question'])\n", + " \n", + " if result:\n", + " print(f\"{i}. ✓ Cache HIT\")\n", + " print(f\" Query: {faq['question'][:80]}...\")\n", + " print(f\" Answer: {result[0]['response'][:100]}...\\n\")\n", + " else:\n", + " print(f\"{i}. ✗ Cache MISS\")\n", + " print(f\" Query: {faq['question'][:80]}...\\n\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Test Semantic Similarity\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Test with semantically similar queries\n", + "print(\"Testing semantic similarity:\\n\")\n", + "\n", + "similar_queries = [\n", + " \"Tell me about NVIDIA's revenue\",\n", + " \"What products does the company make?\",\n", + " \"How is the company performing financially?\",\n", + "]\n", + "\n", + "for i, query in enumerate(similar_queries, 1):\n", + " result = cache.check(prompt=query, return_fields=[\"prompt\", \"response\", \"distance\"])\n", + " \n", + " if result:\n", + " print(f\"{i}. ✓ Cache HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", + " print(f\" Query: {query}\")\n", + " print(f\" Matched: {result[0]['prompt'][:80]}...\")\n", + " print(f\" Answer: {result[0]['response'][:100]}...\\n\")\n", + " else:\n", + " print(f\"{i}. ✗ Cache MISS\")\n", + " print(f\" Query: {query}\\n\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Test Different Distance Thresholds\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Compare cache behavior at different thresholds\n", + "test_query = \"What is NVIDIA's main business?\"\n", + "thresholds = [0.05, 0.10, 0.15, 0.20, 0.25, 0.30]\n", + "\n", + "print(f\"Testing query at different thresholds: '{test_query}'\\n\")\n", + "print(f\"{'Threshold':<12} {'Hit?':<8} {'Distance':<12} {'Matched Query':<50}\")\n", + "print(\"-\" * 85)\n", + "\n", + "for threshold in thresholds:\n", + " cache.distance_threshold = threshold\n", + " result = cache.check(prompt=test_query, return_fields=[\"prompt\", \"vector_distance\"])\n", + " \n", + " if result:\n", + " hit = \"✓ HIT\"\n", + " distance = f\"{result[0].get('vector_distance', 0):.6f}\"\n", + " matched = result[0]['prompt'][:45] + \"...\"\n", + " else:\n", + " hit = \"✗ MISS\"\n", + " distance = \"N/A\"\n", + " matched = \"(no match)\"\n", + " \n", + " print(f\"{threshold:<12.2f} {hit:<8} {distance:<12} {matched:<50}\")\n", + "\n", + "# Reset to original threshold\n", + "cache.distance_threshold = 0.15\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Test Negative Examples (Should Not Match)\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Test with off-topic queries that should NOT match\n", + "print(\"Testing negative examples (should NOT match):\\n\")\n", + "\n", + "for i, test_case in enumerate(negative_examples, 1):\n", + " result = cache.check(prompt=test_case['query'], return_fields=[\"prompt\", \"vector_distance\"])\n", + " \n", + " if result:\n", + " print(f\"{i}. ⚠️ UNEXPECTED HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", + " print(f\" Query: {test_case['query']}\")\n", + " print(f\" Matched: {result[0]['prompt'][:80]}...\\n\")\n", + " else:\n", + " print(f\"{i}. ✓ Correct MISS\")\n", + " print(f\" Query: {test_case['query']}\\n\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 6. Optimize Cache Threshold\n", + "\n", + "Using the `CacheThresholdOptimizer`, we can automatically find the optimal distance threshold based on our test dataset.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create optimization test data\n", + "# Format: [{\"query\": \"...\", \"query_match\": \"cache_key_or_empty_string\"}, ...]\n", + "\n", + "optimization_test_data = []\n", + "\n", + "# Add positive examples (should match specific cache entries)\n", + "for faq in faqs[:5]:\n", + " if faq['question'] in cache_keys:\n", + " optimization_test_data.append({\n", + " \"query\": faq['question'],\n", + " \"query_match\": cache_keys[faq['question']]\n", + " })\n", + "\n", + "# Add negative examples (should not match anything)\n", + "for neg_example in negative_examples:\n", + " optimization_test_data.append({\n", + " \"query\": neg_example['query'],\n", + " \"query_match\": \"\" # Empty string means it should NOT match\n", + " })\n", + "\n", + "print(f\"✓ Created optimization test data:\")\n", + "print(f\" Total examples: {len(optimization_test_data)}\")\n", + "print(f\" Positive (should match): {sum(1 for x in optimization_test_data if x['query_match'])}\")\n", + "print(f\" Negative (should not match): {sum(1 for x in optimization_test_data if not x['query_match'])}\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Optimize threshold based on F1 score\n", + "print(f\"\\nCurrent distance threshold: {cache.distance_threshold}\")\n", + "print(\"\\nOptimizing threshold...\\n\")\n", + "\n", + "optimizer = CacheThresholdOptimizer(\n", + " cache=cache,\n", + " test_data=optimization_test_data,\n", + " eval_metric=\"f1\" # Can also use \"precision\" or \"recall\"\n", + ")\n", + "\n", + "results = optimizer.optimize()\n", + "\n", + "print(f\"\\n✓ Optimization complete!\")\n", + "print(f\" Original threshold: 0.15\")\n", + "print(f\" Optimized threshold: {cache.distance_threshold:.6f}\")\n", + "if 'f1' in results:\n", + " print(f\" F1 Score: {results['f1']:.4f}\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Re-test with optimized threshold\n", + "print(\"\\nRe-testing negative examples with optimized threshold:\\n\")\n", + "\n", + "for i, test_case in enumerate(negative_examples, 1):\n", + " result = cache.check(prompt=test_case['query'], return_fields=[\"prompt\", \"vector_distance\"])\n", + " \n", + " if result:\n", + " print(f\"{i}. ⚠️ HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", + " print(f\" Query: {test_case['query']}\")\n", + " print(f\" Matched: {result[0]['prompt'][:80]}...\\n\")\n", + " else:\n", + " print(f\"{i}. ✓ MISS (correct)\")\n", + " print(f\" Query: {test_case['query']}\\n\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 7. LangCache-Embed Model Deep Dive\n", + "\n", + "The `redis/langcache-embed-v1` model is specifically optimized for semantic caching. Let's examine its characteristics and performance.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Show information about the langcache-embed model\n", + "print(\"LangCache-Embed Model Information:\")\n", + "print(\"=\"*60)\n", + "print(f\"Model: redis/langcache-embed-v1\")\n", + "print(f\"Purpose: Optimized for semantic caching tasks\")\n", + "print(f\"Dimension: {cache.index.schema.fields[-1].attrs.dims}\")\n", + "print(f\"Distance Metric: {cache.index.schema.fields[-1].attrs.distance_metric}\")\n", + "print(\"\\nKey Features:\")\n", + "print(\" • Trained specifically on query-response pairs\")\n", + "print(\" • Balanced precision/recall for optimal cache hit rates\")\n", + "print(\" • Fast inference time suitable for production\")\n", + "print(\" • Optimized for short-form text (queries/prompts)\")\n", + "print(\"\\nAdvantages for Caching:\")\n", + "print(\" • Better semantic understanding of query intent\")\n", + "print(\" • More robust to paraphrasing and rewording\")\n", + "print(\" • Lower false positive rate compared to general embeddings\")\n", + "print(\" • Optimized threshold ranges for cache decisions\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Compare semantic similarities between related and unrelated questions\n", + "test_questions = [\n", + " \"What is NVIDIA's revenue?\",\n", + " \"Tell me about NVIDIA's earnings\", # Semantically similar\n", + " \"How much money does NVIDIA make?\", # Semantically similar\n", + " \"What is the weather today?\", # Unrelated\n", + "]\n", + "\n", + "print(\"\\nComparing semantic similarities:\\n\")\n", + "print(f\"Base question: {test_questions[0]}\\n\")\n", + "\n", + "base_embedding = vectorizer.embed(test_questions[0])\n", + "\n", + "import numpy as np\n", + "\n", + "for query in test_questions[1:]:\n", + " query_embedding = vectorizer.embed(query)\n", + " \n", + " # Calculate cosine similarity\n", + " similarity = np.dot(base_embedding, query_embedding) / (\n", + " np.linalg.norm(base_embedding) * np.linalg.norm(query_embedding)\n", + " )\n", + " distance = 1 - similarity\n", + " \n", + " print(f\"Query: {query}\")\n", + " print(f\" Similarity: {similarity:.4f}\")\n", + " print(f\" Distance: {distance:.4f}\")\n", + " print(f\" Would cache hit (threshold={cache.distance_threshold:.4f})? {distance < cache.distance_threshold}\\n\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 8. RAG Pipeline Integration\n", + "\n", + "Now let's integrate the semantic cache into a complete RAG pipeline and measure the performance improvements.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Build Simple RAG Chain\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.chains import LLMChain\n", + "from langchain.prompts import ChatPromptTemplate\n", + "\n", + "# Create a simple RAG prompt template\n", + "rag_template = ChatPromptTemplate.from_messages([\n", + " (\"system\", \"You are a helpful assistant answering questions about NVIDIA based on their 10-K filing. Provide accurate, concise answers.\"),\n", + " (\"user\", \"{question}\")\n", + "])\n", + "\n", + "# Create RAG chain\n", + "rag_chain = rag_template | llm\n", + "\n", + "print(\"✓ RAG chain created\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Create Cached RAG Function\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def rag_with_cache(question: str, use_cache: bool = True) -> tuple:\n", + " \"\"\"\n", + " Process a question through RAG pipeline with optional semantic caching.\n", + " \n", + " Returns: (answer, cache_hit, response_time)\n", + " \"\"\"\n", + " start_time = time.time()\n", + " cache_hit = False\n", + " \n", + " # Check cache first if enabled\n", + " if use_cache:\n", + " cached_result = cache.check(prompt=question)\n", + " if cached_result:\n", + " answer = cached_result[0]['response']\n", + " cache_hit = True\n", + " response_time = time.time() - start_time\n", + " return answer, cache_hit, response_time\n", + " \n", + " # Cache miss - use LLM\n", + " answer = rag_chain.invoke({\"question\": question})\n", + " response_time = time.time() - start_time\n", + " \n", + " # Store in cache for future use\n", + " if use_cache and hasattr(answer, 'content'):\n", + " cache.store(prompt=question, response=answer.content)\n", + " elif use_cache:\n", + " cache.store(prompt=question, response=str(answer))\n", + " \n", + " return answer.content if hasattr(answer, 'content') else str(answer), cache_hit, response_time\n", + "\n", + "print(\"✓ Cached RAG function ready\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Performance Comparison: With vs Without Cache\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Test questions for RAG evaluation\n", + "test_questions_rag = [\n", + " \"What is NVIDIA's primary business?\",\n", + " \"How much revenue did NVIDIA generate?\",\n", + " \"What are NVIDIA's main products?\",\n", + "]\n", + "\n", + "print(\"\\n\" + \"=\"*80)\n", + "print(\"PERFORMANCE COMPARISON: With Cache vs Without Cache\")\n", + "print(\"=\"*80)\n", + "\n", + "# First pass - populate cache (cache misses, must call LLM)\n", + "print(\"\\n[FIRST PASS - Populating Cache]\\n\")\n", + "first_pass_times = []\n", + "\n", + "for i, question in enumerate(test_questions_rag, 1):\n", + " answer, cache_hit, response_time = rag_with_cache(question, use_cache=True)\n", + " first_pass_times.append(response_time)\n", + " print(f\"{i}. {question}\")\n", + " print(f\" Cache: {'HIT' if cache_hit else 'MISS'} | Time: {response_time:.3f}s\")\n", + " print(f\" Answer: {answer[:100]}...\\n\")\n", + "\n", + "# Second pass - test cache hits with similar questions\n", + "print(\"\\n[SECOND PASS - Cache Hits with Paraphrased Questions]\\n\")\n", + "second_pass_times = []\n", + "\n", + "similar_questions = [\n", + " \"What does NVIDIA do as a business?\",\n", + " \"Can you tell me NVIDIA's revenue figures?\",\n", + " \"What products does NVIDIA sell?\",\n", + "]\n", + "\n", + "for i, question in enumerate(similar_questions, 1):\n", + " answer, cache_hit, response_time = rag_with_cache(question, use_cache=True)\n", + " second_pass_times.append(response_time)\n", + " print(f\"{i}. {question}\")\n", + " print(f\" Cache: {'HIT ✓' if cache_hit else 'MISS ✗'} | Time: {response_time:.3f}s\")\n", + " print(f\" Answer: {answer[:100]}...\\n\")\n", + "\n", + "# Third pass - without cache (baseline)\n", + "print(\"\\n[THIRD PASS - Without Cache (Baseline)]\\n\")\n", + "no_cache_times = []\n", + "\n", + "for i, question in enumerate(test_questions_rag, 1):\n", + " answer, _, response_time = rag_with_cache(question, use_cache=False)\n", + " no_cache_times.append(response_time)\n", + " print(f\"{i}. {question}\")\n", + " print(f\" Cache: DISABLED | Time: {response_time:.3f}s\\n\")\n", + "\n", + "# Summary\n", + "print(\"\\n\" + \"=\"*80)\n", + "print(\"PERFORMANCE SUMMARY\")\n", + "print(\"=\"*80)\n", + "avg_first = sum(first_pass_times)/len(first_pass_times)\n", + "avg_second = sum(second_pass_times)/len(second_pass_times)\n", + "avg_no_cache = sum(no_cache_times)/len(no_cache_times)\n", + "\n", + "print(f\"Average time - First pass (cache miss): {avg_first:.3f}s\")\n", + "print(f\"Average time - Second pass (cache hit): {avg_second:.3f}s\")\n", + "print(f\"Average time - Without cache: {avg_no_cache:.3f}s\")\n", + "\n", + "if avg_second > 0:\n", + " speedup = avg_first / avg_second\n", + " print(f\"\\n✓ Speedup with cache: {speedup:.1f}x faster\")\n", + "\n", + "cache_hit_count = sum(1 for i, _ in enumerate(similar_questions) if second_pass_times[i] < 0.1)\n", + "cache_hit_rate = cache_hit_count / len(similar_questions)\n", + "print(f\" Cache hit rate: {cache_hit_rate*100:.0f}%\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Cost Savings Analysis\n", + "\n", + "Let's estimate the potential cost savings from implementing semantic caching in a production environment.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Estimate cost savings\n", + "# Assumptions: GPT-4o-mini costs ~$0.15 per 1M input tokens, $0.60 per 1M output tokens\n", + "# Average request: ~200 input tokens, ~150 output tokens\n", + "cost_per_llm_call = (200 * 0.15 / 1_000_000) + (150 * 0.60 / 1_000_000) # USD\n", + "cost_per_cache_check = 0.000001 # Negligible (Redis query)\n", + "\n", + "total_queries = 10000 # Simulate 10K queries per day\n", + "cache_hit_rate = 0.70 # Assume 70% hit rate in production\n", + "\n", + "# Without cache\n", + "cost_without_cache = total_queries * cost_per_llm_call\n", + "\n", + "# With cache\n", + "cache_hits = total_queries * cache_hit_rate\n", + "cache_misses = total_queries * (1 - cache_hit_rate)\n", + "cost_with_cache = (cache_hits * cost_per_cache_check) + (cache_misses * cost_per_llm_call)\n", + "\n", + "savings = cost_without_cache - cost_with_cache\n", + "savings_percent = (savings / cost_without_cache) * 100\n", + "\n", + "print(\"\\n\" + \"=\"*80)\n", + "print(f\"COST SAVINGS ESTIMATE ({total_queries:,} queries/day @ {int(cache_hit_rate*100)}% hit rate)\")\n", + "print(\"=\"*80)\n", + "print(f\"Cost without cache: ${cost_without_cache:.4f}/day\")\n", + "print(f\"Cost with cache: ${cost_with_cache:.6f}/day\")\n", + "print(f\"\\nDaily savings: ${savings:.4f} ({savings_percent:.1f}% reduction)\")\n", + "print(f\"Monthly savings: ${savings * 30:.2f}\")\n", + "print(f\"Annual savings: ${savings * 365:.2f}\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 9. Cache Analytics and Monitoring\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Get cache statistics\n", + "print(\"\\n\" + \"=\"*80)\n", + "print(\"CACHE STATISTICS\")\n", + "print(\"=\"*80)\n", + "\n", + "# Count total entries\n", + "info = cache.index.info()\n", + "print(f\"\\nCache Name: {cache.index.name}\")\n", + "print(f\"Total cached entries: {info.get('num_docs', 'N/A')}\")\n", + "print(f\"Distance threshold: {cache.distance_threshold:.6f}\")\n", + "print(f\"Vectorizer model: redis/langcache-embed-v1\")\n", + "print(f\"Embedding dimensions: {cache.index.schema.fields[-1].attrs.dims}\")\n", + "\n", + "# Category breakdown\n", + "if categories:\n", + " print(f\"\\nCategory breakdown:\")\n", + " for cat, count in sorted(categories.items(), key=lambda x: x[1], reverse=True):\n", + " percentage = (count / len(faqs)) * 100\n", + " print(f\" {cat}: {count} FAQs ({percentage:.1f}%)\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 10. Best Practices and Tips\n", + "\n", + "### Key Takeaways\n", + "\n", + "1. **Threshold Optimization**: Start conservative (0.10-0.15) and optimize based on real usage data\n", + "2. **Doc2Cache**: Pre-populate your cache with high-quality FAQs for immediate benefits\n", + "3. **Monitoring**: Track cache hit rates and adjust thresholds as user patterns emerge\n", + "4. **Model Selection**: The `langcache-embed-v1` model is specifically optimized for caching tasks\n", + "5. **Cost-Performance Balance**: Even a 50% cache hit rate provides significant cost savings\n", + "\n", + "### When to Use Semantic Caching\n", + "\n", + "✅ **Good Use Cases:**\n", + "- High-traffic applications with repeated question patterns\n", + "- Customer support chatbots\n", + "- FAQ systems\n", + "- Documentation Q&A\n", + "- Product information queries\n", + "- Educational content Q&A\n", + "\n", + "❌ **Less Suitable:**\n", + "- Highly dynamic content requiring real-time data\n", + "- Creative writing tasks needing variety\n", + "- Personalized responses based on user-specific context\n", + "- Time-sensitive queries (use TTL if needed)\n", + "\n", + "### Performance Tips\n", + "\n", + "1. **Batch Loading**: Pre-populate cache with Doc2Cache for immediate value\n", + "2. **Monitor Hit Rates**: Track and adjust thresholds based on production metrics\n", + "3. **A/B Testing**: Test different thresholds with a subset of traffic\n", + "4. **Cache Warming**: Regularly update cache with trending topics\n", + "5. **TTL Management**: Set time-to-live for entries that may become stale\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 11. Cleanup\n", + "\n", + "Clean up resources when done.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Option 1: Clear cache contents but keep index\n", + "cache.clear()\n", + "print(\"✓ Cache cleared (index preserved)\")\n", + "\n", + "# Option 2: Delete entire cache index (uncomment if needed)\n", + "# cache.delete()\n", + "# print(\"✓ Cache index deleted\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Clean up temporary files\n", + "import shutil\n", + "\n", + "if os.path.exists('data'):\n", + " shutil.rmtree('data')\n", + " print(\"✓ Temporary files cleaned up\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Summary\n", + "\n", + "Congratulations! You've completed this comprehensive guide on semantic caching with LangCache and RedisVL. \n", + "\n", + "**What You've Learned:**\n", + "- ✅ Set up and configure LangCache with Redis Cloud\n", + "- ✅ Load and process PDF documents into knowledge bases\n", + "- ✅ Generate FAQs using the Doc2Cache technique with LLMs\n", + "- ✅ Pre-populate a semantic cache with tagged entries\n", + "- ✅ Test different cache matching strategies and thresholds\n", + "- ✅ Optimize cache performance using test datasets\n", + "- ✅ Leverage the `redis/langcache-embed-v1` cross-encoder model\n", + "- ✅ Integrate semantic caching into RAG pipelines\n", + "- ✅ Measure performance improvements and cost savings\n", + "\n", + "**Next Steps:**\n", + "- Experiment with different distance thresholds for your use case\n", + "- Try other embedding models and compare performance\n", + "- Implement cache analytics and monitoring in production\n", + "- Explore advanced features like TTL, metadata filtering, and cache warming strategies\n", + "- Scale your semantic cache to handle production traffic\n", + "\n", + "**Resources:**\n", + "- [RedisVL Documentation](https://redis.io/docs/stack/search/redisvl/)\n", + "- [LangCache Sign Up](https://langcache.io/signup)\n", + "- [Redis AI Resources](https://github.com/redis-developer/redis-ai-resources)\n", + "- [Semantic Caching Paper](https://arxiv.org/abs/2504.02268)\n" + ] + } + ], + "metadata": { + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 290fd4d9eeadc2f027bf1e2254b4d20a62da2742 Mon Sep 17 00:00:00 2001 From: Justin Cechmanek Date: Fri, 14 Nov 2025 15:56:22 -0800 Subject: [PATCH 2/7] fixes notebook errors --- .../04_langcache_semantic_caching.ipynb | 758 +++++++++++++++--- 1 file changed, 633 insertions(+), 125 deletions(-) diff --git a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb index 911c56df..beda81ed 100644 --- a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb +++ b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb @@ -60,22 +60,37 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.3\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", + "Note: you may need to restart the kernel to use updated packages.\n", + "\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.3\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", + "Note: you may need to restart the kernel to use updated packages.\n", + "✓ Packages installed. Please restart the kernel if prompted, then continue with the next cell.\n" + ] + } + ], "source": [ "%pip install -q \"redisvl>=0.11.0\" \"openai>=1.0.0\" \"langchain>=0.3.0\" \"langchain-community\" \"langchain-openai\"\n", - "%pip install -q \"pypdf\" \"sentence-transformers\" \"redis-retrieval-optimizer>=0.2.0\"\n" + "%pip install -q \"pypdf\" \"sentence-transformers\" \"redis-retrieval-optimizer>=0.2.0\"\n", + "\n", + "print(\"Packages installed. Please restart the kernel if prompted, then continue with the next cell.\")\n" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ - "import IPython\n", - "\n", - "app = IPython.Application.instance()\n", - "app.kernel.do_shutdown(True)\n" + "# Note: If you see import errors, restart the kernel and continue from this cell\n" ] }, { @@ -89,7 +104,24 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/Users/justin.cechmanek/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n", + " from .autonotebook import tqdm as notebook_tqdm\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "11:13:29 numexpr.utils INFO NumExpr defaulting to 10 threads.\n", + "✓ All imports successful\n" + ] + } + ], "source": [ "import os\n", "import time\n", @@ -98,21 +130,21 @@ "from typing import List, Dict, Any\n", "\n", "# RedisVL imports\n", - "from redisvl.extensions.llmcache import SemanticCache\n", + "from redisvl.extensions.cache.llm import SemanticCache\n", "from redisvl.utils.vectorize import HFTextVectorizer\n", "\n", "# LangChain imports\n", "from langchain_community.document_loaders import PyPDFLoader\n", "from langchain_text_splitters import RecursiveCharacterTextSplitter\n", - "from langchain_openai import ChatOpenAI, OpenAIEmbeddings\n", - "from langchain.prompts import PromptTemplate\n", + "from langchain_openai import ChatOpenAI\n", + "from langchain_core.prompts import PromptTemplate, ChatPromptTemplate\n", "from langchain_core.output_parsers import JsonOutputParser\n", "from pydantic import BaseModel, Field\n", "\n", "# Optimization\n", "from redis_retrieval_optimizer.threshold_optimization import CacheThresholdOptimizer\n", "\n", - "print(\"✓ All imports successful\")\n" + "print(\"All imports successful\")\n" ] }, { @@ -147,7 +179,16 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "✓ Environment variables configured\n", + " Redis URL: :6379\n" + ] + } + ], "source": [ "# Redis/LangCache credentials\n", "if \"REDIS_URL\" not in os.environ:\n", @@ -165,7 +206,7 @@ "if \"OPENAI_API_KEY\" not in os.environ:\n", " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")\n", "\n", - "print(\"✓ Environment variables configured\")\n", + "print(\"Environment variables configured\")\n", "print(f\" Redis URL: {os.environ['REDIS_URL'].split('@')[-1] if '@' in os.environ['REDIS_URL'] else os.environ['REDIS_URL'].split('//')[1]}\")\n" ] }, @@ -182,7 +223,20 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "11:14:02 datasets INFO PyTorch version 2.7.0 available.\n", + "11:14:03 sentence_transformers.SentenceTransformer INFO Use pytorch device_name: mps\n", + "11:14:03 sentence_transformers.SentenceTransformer INFO Load pretrained SentenceTransformer: redis/langcache-embed-v1\n", + "✓ Semantic Cache initialized: rag_faq_cache\n", + " Using model: redis/langcache-embed-v1\n", + " Distance threshold: 0.15\n" + ] + } + ], "source": [ "# Initialize the vectorizer with the LangCache embedding model\n", "# This model is specifically optimized for semantic caching with better precision/recall\n", @@ -198,7 +252,7 @@ " distance_threshold=0.15 # Initial threshold, we'll optimize this later\n", ")\n", "\n", - "print(f\"✓ Semantic Cache initialized: {cache.index.name}\")\n", + "print(f\"Semantic Cache initialized: {cache.index.name}\")\n", "print(f\" Using model: redis/langcache-embed-v1\")\n", "print(f\" Distance threshold: {cache.distance_threshold}\")\n" ] @@ -214,7 +268,15 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "✓ OpenAI LLM initialized\n" + ] + } + ], "source": [ "# Initialize OpenAI LLM for FAQ generation and RAG\n", "llm = ChatOpenAI(\n", @@ -223,7 +285,7 @@ " max_tokens=2000\n", ")\n", "\n", - "print(\"✓ OpenAI LLM initialized\")\n" + "print(\"OpenAI LLM initialized\")\n" ] }, { @@ -249,20 +311,44 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "✓ Sample PDF downloaded\n" + ] + } + ], "source": [ "# Download sample PDF if not already present\n", "!mkdir -p data\n", "!wget -q -O data/nvidia-10k.pdf https://raw.githubusercontent.com/redis-developer/redis-ai-resources/main/python-recipes/RAG/resources/nvd-10k-2023.pdf\n", "\n", - "print(\"✓ Sample PDF downloaded\")\n" + "print(\"Sample PDF downloaded\")\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "✓ Loaded PDF: data/nvidia-10k.pdf\n", + " Total pages: 169\n", + " Created chunks: 388\n", + "\n", + "Sample chunk preview:\n", + "Table of Contents\n", + "The world’s leading cloud service providers, or CSPs, and consumer internet companies use our GPUs and broader data center-scale\n", + "accelerated computing platforms to enable, accelerate or enrich the services they deliver to billions of end-users, including search,\n", + "recommendations, so...\n" + ] + } + ], "source": [ "# Load and chunk the PDF\n", "pdf_path = \"data/nvidia-10k.pdf\"\n", @@ -279,7 +365,7 @@ "documents = loader.load()\n", "chunks = text_splitter.split_documents(documents)\n", "\n", - "print(f\"✓ Loaded PDF: {pdf_path}\")\n", + "print(f\"Loaded PDF: {pdf_path}\")\n", "print(f\" Total pages: {len(documents)}\")\n", "print(f\" Created chunks: {len(chunks)}\")\n", "print(f\"\\nSample chunk preview:\")\n", @@ -297,7 +383,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 9, "metadata": {}, "outputs": [], "source": [ @@ -318,7 +404,15 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "✓ FAQ generation chain configured\n" + ] + } + ], "source": [ "# Create the FAQ generation prompt\n", "faq_prompt = PromptTemplate(\n", @@ -344,14 +438,37 @@ "# Create the FAQ generation chain\n", "faq_chain = faq_prompt | llm | json_parser\n", "\n", - "print(\"✓ FAQ generation chain configured\")\n" + "print(\"FAQ generation chain configured\")\n" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 11, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Testing FAQ generation on sample chunk...\n", + "\n", + "11:14:29 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "Generated 5 FAQs:\n", + "\n", + "1. Q: What industries are leveraging NVIDIA's GPUs and software for automation?\n", + " Category: operations\n", + " A: A rapidly growing number of enterprises and startups across a broad range of industries, including transportation for autonomous driving, healthcare f...\n", + "\n", + "2. Q: What was the reason for the termination of the Arm Share Purchase Agreement between NVIDIA and SoftBank?\n", + " Category: general\n", + " A: The termination of the Share Purchase Agreement was due to significant regulatory challenges that prevented the completion of the transaction....\n", + "\n", + "3. Q: What acquisition termination cost did NVIDIA record in fiscal year 2023?\n", + " Category: financial\n", + " A: NVIDIA recorded an acquisition termination cost of $1.35 billion in fiscal year 2023, reflecting the write-off of the prepayment provided at signing f...\n" + ] + } + ], "source": [ "# Test FAQ generation on a single chunk\n", "print(\"Testing FAQ generation on sample chunk...\\n\")\n", @@ -368,7 +485,56 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Generating FAQs from document chunks...\n", + "\n", + "Processing chunk 1/25...\n", + "11:14:36 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:14:45 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:14:54 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:15:00 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:15:07 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "Processing chunk 6/25...\n", + "11:15:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:15:16 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:15:22 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:15:29 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:15:35 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "Processing chunk 11/25...\n", + "11:15:42 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:15:49 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:15:55 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:16:03 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:16:11 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "Processing chunk 16/25...\n", + "11:16:20 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:16:26 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:16:35 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:16:41 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:16:48 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "Processing chunk 21/25...\n", + "11:16:55 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:17:01 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:17:07 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:17:15 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "11:17:23 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "\n", + "✓ Generated 114 FAQs total\n", + "\n", + "Category distribution:\n", + " technology: 32\n", + " operations: 26\n", + " financial: 22\n", + " products: 21\n", + " general: 13\n" + ] + } + ], "source": [ "# Generate FAQs from all chunks (limited to first 25 for demo purposes)\n", "def extract_faqs_from_chunks(chunks: List[Any], max_chunks: int = 25) -> List[Dict]:\n", @@ -393,7 +559,7 @@ "print(\"\\nGenerating FAQs from document chunks...\\n\")\n", "faqs = extract_faqs_from_chunks(chunks, max_chunks=25)\n", "\n", - "print(f\"\\n✓ Generated {len(faqs)} FAQs total\")\n", + "print(f\"\\nGenerated {len(faqs)} FAQs total\")\n", "print(f\"\\nCategory distribution:\")\n", "categories = {}\n", "for faq in faqs:\n", @@ -417,9 +583,23 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 13, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Sample FAQs for testing:\n", + "\n", + "1. What is the fiscal year end date for NVIDIA Corporation as reported in the 10-K?...\n", + "\n", + "2. What is the trading symbol for NVIDIA Corporation's common stock?...\n", + "\n", + "3. Where is the principal executive office of NVIDIA Corporation located?...\n" + ] + } + ], "source": [ "# Select representative FAQs for test set\n", "sample_faqs = faqs[:10] # Take first 10 FAQs\n", @@ -433,7 +613,16 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "✓ Test dataset created\n", + " Negative examples: 5\n" + ] + } + ], "source": [ "# Create test dataset with negative examples (off-topic questions)\n", "negative_examples = [\n", @@ -444,7 +633,7 @@ " {\"query\": \"What time is it?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", "]\n", "\n", - "print(f\"✓ Test dataset created\")\n", + "print(f\"Test dataset created\")\n", "print(f\" Negative examples: {len(negative_examples)}\")\n" ] }, @@ -461,18 +650,52 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "✓ Cache cleared\n" + ] + } + ], "source": [ "# Clear any existing cache entries\n", "cache.clear()\n", - "print(\"✓ Cache cleared\")\n" + "print(\"Cache cleared\")\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Storing FAQs in cache...\n", + "\n", + " Stored 0/114 FAQs...\n", + " Stored 20/114 FAQs...\n", + " Stored 40/114 FAQs...\n", + " Stored 60/114 FAQs...\n", + " Stored 80/114 FAQs...\n", + " Stored 100/114 FAQs...\n", + "\n", + "✓ Stored 114 FAQs in cache\n", + " Cache index: rag_faq_cache\n", + "\n", + "Example cache entries:\n", + "\n", + "1. Key: rag_faq_cache:abd9b974d9eedebc62332adbfd10ab5ff96e9d65dbd4476a27a487dd82b46002\n", + " Q: What is the fiscal year end date for NVIDIA Corporation as reported in the 10-K?...\n", + "\n", + "2. Key: rag_faq_cache:8aa719b5f3d105fdcd9048d2b6c14e04bd60e8501a27ed80481c97adafb01ea7\n", + " Q: What is the trading symbol for NVIDIA Corporation's common stock?...\n" + ] + } + ], "source": [ "# Store FAQs in cache with metadata tags\n", "print(\"Storing FAQs in cache...\\n\")\n", @@ -496,7 +719,7 @@ " except Exception as e:\n", " print(f\" Warning: Failed to store FAQ {i+1}: {str(e)[:100]}\")\n", "\n", - "print(f\"\\n✓ Stored {stored_count} FAQs in cache\")\n", + "print(f\"\\nStored {stored_count} FAQs in cache\")\n", "print(f\" Cache index: {cache.index.name}\")\n", "print(f\"\\nExample cache entries:\")\n", "for i, (q, k) in enumerate(list(cache_keys.items())[:2], 1):\n", @@ -524,7 +747,28 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Testing exact match queries:\n", + "\n", + "1. ✓ Cache HIT\n", + " Query: What is the fiscal year end date for NVIDIA Corporation as reported in the 10-K?...\n", + " Answer: The fiscal year ended January 29, 2023....\n", + "\n", + "2. ✓ Cache HIT\n", + " Query: What is the trading symbol for NVIDIA Corporation's common stock?...\n", + " Answer: The trading symbol for NVIDIA Corporation's common stock is NVDA....\n", + "\n", + "3. ✓ Cache HIT\n", + " Query: Where is the principal executive office of NVIDIA Corporation located?...\n", + " Answer: The principal executive office of NVIDIA Corporation is located at 2788 San Tomas Expressway, Santa ...\n", + "\n" + ] + } + ], "source": [ "# Test with exact questions from cache\n", "print(\"Testing exact match queries:\\n\")\n", @@ -533,7 +777,7 @@ " result = cache.check(prompt=faq['question'])\n", " \n", " if result:\n", - " print(f\"{i}. ✓ Cache HIT\")\n", + " print(f\"{i}. Cache HIT\")\n", " print(f\" Query: {faq['question'][:80]}...\")\n", " print(f\" Answer: {result[0]['response'][:100]}...\\n\")\n", " else:\n", @@ -552,7 +796,25 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Testing semantic similarity:\n", + "\n", + "1. ✗ Cache MISS\n", + " Query: Tell me about NVIDIA's revenue\n", + "\n", + "2. ✗ Cache MISS\n", + " Query: What products does the company make?\n", + "\n", + "3. ✗ Cache MISS\n", + " Query: How is the company performing financially?\n", + "\n" + ] + } + ], "source": [ "# Test with semantically similar queries\n", "print(\"Testing semantic similarity:\\n\")\n", @@ -567,7 +829,7 @@ " result = cache.check(prompt=query, return_fields=[\"prompt\", \"response\", \"distance\"])\n", " \n", " if result:\n", - " print(f\"{i}. ✓ Cache HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", + " print(f\"{i}. Cache HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", " print(f\" Query: {query}\")\n", " print(f\" Matched: {result[0]['prompt'][:80]}...\")\n", " print(f\" Answer: {result[0]['response'][:100]}...\\n\")\n", @@ -580,40 +842,40 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Test Different Distance Thresholds\n" + "### Test Cache with Sample Query\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Testing query: 'What is NVIDIA's main business?'\n", + "Current threshold: 0.1500\n", + "\n", + "✗ Cache MISS - No match found within threshold\n" + ] + } + ], "source": [ - "# Compare cache behavior at different thresholds\n", + "# Test cache behavior with a sample query\n", "test_query = \"What is NVIDIA's main business?\"\n", - "thresholds = [0.05, 0.10, 0.15, 0.20, 0.25, 0.30]\n", "\n", - "print(f\"Testing query at different thresholds: '{test_query}'\\n\")\n", - "print(f\"{'Threshold':<12} {'Hit?':<8} {'Distance':<12} {'Matched Query':<50}\")\n", - "print(\"-\" * 85)\n", + "print(f\"Testing query: '{test_query}'\")\n", + "print(f\"Current threshold: {cache.distance_threshold:.4f}\\n\")\n", "\n", - "for threshold in thresholds:\n", - " cache.distance_threshold = threshold\n", - " result = cache.check(prompt=test_query, return_fields=[\"prompt\", \"vector_distance\"])\n", - " \n", - " if result:\n", - " hit = \"✓ HIT\"\n", - " distance = f\"{result[0].get('vector_distance', 0):.6f}\"\n", - " matched = result[0]['prompt'][:45] + \"...\"\n", - " else:\n", - " hit = \"✗ MISS\"\n", - " distance = \"N/A\"\n", - " matched = \"(no match)\"\n", - " \n", - " print(f\"{threshold:<12.2f} {hit:<8} {distance:<12} {matched:<50}\")\n", + "result = cache.check(prompt=test_query, return_fields=[\"prompt\", \"vector_distance\"])\n", "\n", - "# Reset to original threshold\n", - "cache.distance_threshold = 0.15\n" + "if result:\n", + " print(f\"Cache HIT\")\n", + " print(f\" Distance: {result[0].get('vector_distance', 0):.6f}\")\n", + " print(f\" Matched: {result[0]['prompt'][:80]}...\")\n", + "else:\n", + " print(f\"✗ Cache MISS - No match found within threshold\")\n" ] }, { @@ -627,7 +889,31 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Testing negative examples (should NOT match):\n", + "\n", + "1. ✓ Correct MISS\n", + " Query: What is the weather today?\n", + "\n", + "2. ✓ Correct MISS\n", + " Query: How do I cook pasta?\n", + "\n", + "3. ✓ Correct MISS\n", + " Query: What is the capital of France?\n", + "\n", + "4. ✓ Correct MISS\n", + " Query: Tell me a joke\n", + "\n", + "5. ✓ Correct MISS\n", + " Query: What time is it?\n", + "\n" + ] + } + ], "source": [ "# Test with off-topic queries that should NOT match\n", "print(\"Testing negative examples (should NOT match):\\n\")\n", @@ -640,7 +926,7 @@ " print(f\" Query: {test_case['query']}\")\n", " print(f\" Matched: {result[0]['prompt'][:80]}...\\n\")\n", " else:\n", - " print(f\"{i}. ✓ Correct MISS\")\n", + " print(f\"{i}. Correct MISS\")\n", " print(f\" Query: {test_case['query']}\\n\")\n" ] }, @@ -657,7 +943,18 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "✓ Created optimization test data:\n", + " Total examples: 10\n", + " Positive (should match): 5\n", + " Negative (should not match): 5\n" + ] + } + ], "source": [ "# Create optimization test data\n", "# Format: [{\"query\": \"...\", \"query_match\": \"cache_key_or_empty_string\"}, ...]\n", @@ -679,7 +976,7 @@ " \"query_match\": \"\" # Empty string means it should NOT match\n", " })\n", "\n", - "print(f\"✓ Created optimization test data:\")\n", + "print(f\"Created optimization test data:\")\n", "print(f\" Total examples: {len(optimization_test_data)}\")\n", "print(f\" Positive (should match): {sum(1 for x in optimization_test_data if x['query_match'])}\")\n", "print(f\" Negative (should not match): {sum(1 for x in optimization_test_data if not x['query_match'])}\")\n" @@ -689,24 +986,40 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Current distance threshold: 0.01\n", + "\n", + "Optimizing threshold...\n", + "\n", + "\n", + "✓ Optimization complete!\n", + " Original threshold: 0.15\n", + " Optimized threshold: 0.010000\n" + ] + } + ], "source": [ "# Optimize threshold based on F1 score\n", "print(f\"\\nCurrent distance threshold: {cache.distance_threshold}\")\n", "print(\"\\nOptimizing threshold...\\n\")\n", "\n", "optimizer = CacheThresholdOptimizer(\n", - " cache=cache,\n", - " test_data=optimization_test_data,\n", + " cache,\n", + " optimization_test_data,\n", " eval_metric=\"f1\" # Can also use \"precision\" or \"recall\"\n", ")\n", "\n", "results = optimizer.optimize()\n", "\n", - "print(f\"\\n✓ Optimization complete!\")\n", + "print(f\"\\nOptimization complete!\")\n", "print(f\" Original threshold: 0.15\")\n", "print(f\" Optimized threshold: {cache.distance_threshold:.6f}\")\n", - "if 'f1' in results:\n", + "if results and 'f1' in results:\n", " print(f\" F1 Score: {results['f1']:.4f}\")\n" ] }, @@ -714,7 +1027,32 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Re-testing negative examples with optimized threshold:\n", + "\n", + "1. ✓ MISS (correct)\n", + " Query: What is the weather today?\n", + "\n", + "2. ✓ MISS (correct)\n", + " Query: How do I cook pasta?\n", + "\n", + "3. ✓ MISS (correct)\n", + " Query: What is the capital of France?\n", + "\n", + "4. ✓ MISS (correct)\n", + " Query: Tell me a joke\n", + "\n", + "5. ✓ MISS (correct)\n", + " Query: What time is it?\n", + "\n" + ] + } + ], "source": [ "# Re-test with optimized threshold\n", "print(\"\\nRe-testing negative examples with optimized threshold:\\n\")\n", @@ -727,7 +1065,7 @@ " print(f\" Query: {test_case['query']}\")\n", " print(f\" Matched: {result[0]['prompt'][:80]}...\\n\")\n", " else:\n", - " print(f\"{i}. ✓ MISS (correct)\")\n", + " print(f\"{i}. MISS (correct)\")\n", " print(f\" Query: {test_case['query']}\\n\")\n" ] }, @@ -742,17 +1080,42 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 28, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "LangCache-Embed Model Information:\n", + "============================================================\n", + "Model: redis/langcache-embed-v1\n", + "Purpose: Optimized for semantic caching tasks\n", + "Dimension: 768\n", + "Distance Metric: cosine\n", + "\n", + "Key Features:\n", + " • Trained specifically on query-response pairs\n", + " • Balanced precision/recall for optimal cache hit rates\n", + " • Fast inference time suitable for production\n", + " • Optimized for short-form text (queries/prompts)\n", + "\n", + "Advantages for Caching:\n", + " • Better semantic understanding of query intent\n", + " • More robust to paraphrasing and rewording\n", + " • Lower false positive rate compared to general embeddings\n", + " • Optimized threshold ranges for cache decisions\n" + ] + } + ], "source": [ "# Show information about the langcache-embed model\n", "print(\"LangCache-Embed Model Information:\")\n", "print(\"=\"*60)\n", "print(f\"Model: redis/langcache-embed-v1\")\n", "print(f\"Purpose: Optimized for semantic caching tasks\")\n", - "print(f\"Dimension: {cache.index.schema.fields[-1].attrs.dims}\")\n", - "print(f\"Distance Metric: {cache.index.schema.fields[-1].attrs.distance_metric}\")\n", + "print(f\"Dimension: 768\")\n", + "print(f\"Distance Metric: cosine\")\n", "print(\"\\nKey Features:\")\n", "print(\" • Trained specifically on query-response pairs\")\n", "print(\" • Balanced precision/recall for optimal cache hit rates\")\n", @@ -767,9 +1130,36 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 29, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "Comparing semantic similarities:\n", + "\n", + "Base question: What is NVIDIA's revenue?\n", + "\n", + "Query: Tell me about NVIDIA's earnings\n", + " Similarity: 0.8725\n", + " Distance: 0.1275\n", + " Would cache hit (threshold=0.0100)? False\n", + "\n", + "Query: How much money does NVIDIA make?\n", + " Similarity: 0.9004\n", + " Distance: 0.0996\n", + " Would cache hit (threshold=0.0100)? False\n", + "\n", + "Query: What is the weather today?\n", + " Similarity: 0.3141\n", + " Distance: 0.6859\n", + " Would cache hit (threshold=0.0100)? False\n", + "\n" + ] + } + ], "source": [ "# Compare semantic similarities between related and unrelated questions\n", "test_questions = [\n", @@ -821,11 +1211,16 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "✓ RAG chain created\n" + ] + } + ], "source": [ - "from langchain.chains import LLMChain\n", - "from langchain.prompts import ChatPromptTemplate\n", - "\n", "# Create a simple RAG prompt template\n", "rag_template = ChatPromptTemplate.from_messages([\n", " (\"system\", \"You are a helpful assistant answering questions about NVIDIA based on their 10-K filing. Provide accurate, concise answers.\"),\n", @@ -835,7 +1230,7 @@ "# Create RAG chain\n", "rag_chain = rag_template | llm\n", "\n", - "print(\"✓ RAG chain created\")\n" + "print(\"RAG chain created\")\n" ] }, { @@ -849,7 +1244,15 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "✓ Cached RAG function ready\n" + ] + } + ], "source": [ "def rag_with_cache(question: str, use_cache: bool = True) -> tuple:\n", " \"\"\"\n", @@ -881,7 +1284,7 @@ " \n", " return answer.content if hasattr(answer, 'content') else str(answer), cache_hit, response_time\n", "\n", - "print(\"✓ Cached RAG function ready\")\n" + "print(\"Cached RAG function ready\")\n" ] }, { @@ -895,7 +1298,81 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "================================================================================\n", + "PERFORMANCE COMPARISON: With Cache vs Without Cache\n", + "================================================================================\n", + "\n", + "[FIRST PASS - Populating Cache]\n", + "\n", + "15:52:18 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "1. What is NVIDIA's primary business?\n", + " Cache: MISS | Time: 2.232s\n", + " Answer: NVIDIA's primary business is the design and manufacture of graphics processing units (GPUs) for gami...\n", + "\n", + "15:52:20 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "2. How much revenue did NVIDIA generate?\n", + " Cache: MISS | Time: 2.188s\n", + " Answer: As of the latest 10-K filing, NVIDIA reported total revenue of $26.91 billion for the fiscal year en...\n", + "\n", + "15:52:23 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "3. What are NVIDIA's main products?\n", + " Cache: MISS | Time: 3.195s\n", + " Answer: NVIDIA's main products include:\n", + "\n", + "1. **Graphics Processing Units (GPUs)**: Primarily for gaming, prof...\n", + "\n", + "\n", + "[SECOND PASS - Cache Hits with Paraphrased Questions]\n", + "\n", + "15:52:25 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "1. What does NVIDIA do as a business?\n", + " Cache: MISS ✗ | Time: 2.069s\n", + " Answer: NVIDIA is primarily a technology company that designs and manufactures graphics processing units (GP...\n", + "\n", + "15:52:28 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "2. Can you tell me NVIDIA's revenue figures?\n", + " Cache: MISS ✗ | Time: 2.518s\n", + " Answer: As of the latest 10-K filing, NVIDIA reported total revenue of $26.91 billion for the fiscal year en...\n", + "\n", + "15:52:34 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "3. What products does NVIDIA sell?\n", + " Cache: MISS ✗ | Time: 5.687s\n", + " Answer: NVIDIA sells a variety of products primarily focused on graphics processing units (GPUs) and related...\n", + "\n", + "\n", + "[THIRD PASS - Without Cache (Baseline)]\n", + "\n", + "15:52:36 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "1. What is NVIDIA's primary business?\n", + " Cache: DISABLED | Time: 1.807s\n", + "\n", + "15:52:38 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "2. How much revenue did NVIDIA generate?\n", + " Cache: DISABLED | Time: 2.035s\n", + "\n", + "15:52:41 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "3. What are NVIDIA's main products?\n", + " Cache: DISABLED | Time: 3.880s\n", + "\n", + "\n", + "================================================================================\n", + "PERFORMANCE SUMMARY\n", + "================================================================================\n", + "Average time - First pass (cache miss): 2.538s\n", + "Average time - Second pass (cache hit): 3.424s\n", + "Average time - Without cache: 2.574s\n", + "\n", + "✓ Speedup with cache: 0.7x faster\n", + " Cache hit rate: 0%\n" + ] + } + ], "source": [ "# Test questions for RAG evaluation\n", "test_questions_rag = [\n", @@ -960,7 +1437,7 @@ "\n", "if avg_second > 0:\n", " speedup = avg_first / avg_second\n", - " print(f\"\\n✓ Speedup with cache: {speedup:.1f}x faster\")\n", + " print(f\"\\nSpeedup with cache: {speedup:.1f}x faster\")\n", "\n", "cache_hit_count = sum(1 for i, _ in enumerate(similar_questions) if second_pass_times[i] < 0.1)\n", "cache_hit_rate = cache_hit_count / len(similar_questions)\n", @@ -978,9 +1455,26 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 34, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "================================================================================\n", + "COST SAVINGS ESTIMATE (10,000 queries/day @ 70% hit rate)\n", + "================================================================================\n", + "Cost without cache: $1.2000/day\n", + "Cost with cache: $0.367000/day\n", + "\n", + "Daily savings: $0.8330 (69.4% reduction)\n", + "Monthly savings: $24.99\n", + "Annual savings: $304.04\n" + ] + } + ], "source": [ "# Estimate cost savings\n", "# Assumptions: GPT-4o-mini costs ~$0.15 per 1M input tokens, $0.60 per 1M output tokens\n", @@ -1021,9 +1515,26 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 35, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "================================================================================\n", + "CACHE STATISTICS\n", + "================================================================================\n", + "\n", + "Cache Name: rag_faq_cache\n", + "Total cached entries: 120\n", + "Distance threshold: 0.010000\n", + "Vectorizer model: redis/langcache-embed-v1\n", + "Embedding dimensions: 768\n" + ] + } + ], "source": [ "# Get cache statistics\n", "print(\"\\n\" + \"=\"*80)\n", @@ -1036,14 +1547,7 @@ "print(f\"Total cached entries: {info.get('num_docs', 'N/A')}\")\n", "print(f\"Distance threshold: {cache.distance_threshold:.6f}\")\n", "print(f\"Vectorizer model: redis/langcache-embed-v1\")\n", - "print(f\"Embedding dimensions: {cache.index.schema.fields[-1].attrs.dims}\")\n", - "\n", - "# Category breakdown\n", - "if categories:\n", - " print(f\"\\nCategory breakdown:\")\n", - " for cat, count in sorted(categories.items(), key=lambda x: x[1], reverse=True):\n", - " percentage = (count / len(faqs)) * 100\n", - " print(f\" {cat}: {count} FAQs ({percentage:.1f}%)\")\n" + "print(f\"Embedding dimensions: 768\")\n" ] }, { @@ -1098,29 +1602,19 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [], - "source": [ - "# Option 1: Clear cache contents but keep index\n", - "cache.clear()\n", - "print(\"✓ Cache cleared (index preserved)\")\n", - "\n", - "# Option 2: Delete entire cache index (uncomment if needed)\n", - "# cache.delete()\n", - "# print(\"✓ Cache index deleted\")\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "✓ Cache cleared (index preserved)\n" + ] + } + ], "source": [ - "# Clean up temporary files\n", - "import shutil\n", - "\n", - "if os.path.exists('data'):\n", - " shutil.rmtree('data')\n", - " print(\"✓ Temporary files cleaned up\")\n" + "# Clear cache contents\n", + "cache.delete()\n", + "print(\"Cache deleted\")" ] }, { @@ -1158,8 +1652,22 @@ } ], "metadata": { + "kernelspec": { + "display_name": "redis-ai-res", + "language": "python", + "name": "python3" + }, "language_info": { - "name": "python" + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.9" } }, "nbformat": 4, From 6ea6365662ef9dfa304884cca6da04d7ac157b93 Mon Sep 17 00:00:00 2001 From: Justin Cechmanek Date: Mon, 17 Nov 2025 21:05:08 -0800 Subject: [PATCH 3/7] forces notebook to use langcache --- .../04_langcache_semantic_caching.ipynb | 852 +++++++----------- 1 file changed, 331 insertions(+), 521 deletions(-) diff --git a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb index beda81ed..28f1fc6c 100644 --- a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb +++ b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb @@ -58,7 +58,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, "metadata": {}, "outputs": [ { @@ -72,25 +72,13 @@ "\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.3\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", - "Note: you may need to restart the kernel to use updated packages.\n", - "✓ Packages installed. Please restart the kernel if prompted, then continue with the next cell.\n" + "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ - "%pip install -q \"redisvl>=0.11.0\" \"openai>=1.0.0\" \"langchain>=0.3.0\" \"langchain-community\" \"langchain-openai\"\n", - "%pip install -q \"pypdf\" \"sentence-transformers\" \"redis-retrieval-optimizer>=0.2.0\"\n", - "\n", - "print(\"Packages installed. Please restart the kernel if prompted, then continue with the next cell.\")\n" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "# Note: If you see import errors, restart the kernel and continue from this cell\n" + "%pip install -q \"redisvl>=0.11.0\" \"openai>=1.0.0\" \"langchain>=0.3.0\" \"langchain-community\" \"langchain-openai\" \"langcache\"\n", + "%pip install -q \"pypdf\" \"sentence-transformers\" \"redis-retrieval-optimizer>=0.2.0\"" ] }, { @@ -102,23 +90,14 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": {}, "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/justin.cechmanek/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n", - " from .autonotebook import tqdm as notebook_tqdm\n" - ] - }, { "name": "stdout", "output_type": "stream", "text": [ - "11:13:29 numexpr.utils INFO NumExpr defaulting to 10 threads.\n", - "✓ All imports successful\n" + "20:59:02 numexpr.utils INFO NumExpr defaulting to 10 threads.\n" ] } ], @@ -126,25 +105,13 @@ "import os\n", "import time\n", "import json\n", - "from getpass import getpass\n", "from typing import List, Dict, Any\n", "\n", "# RedisVL imports\n", - "from redisvl.extensions.cache.llm import SemanticCache\n", - "from redisvl.utils.vectorize import HFTextVectorizer\n", - "\n", - "# LangChain imports\n", - "from langchain_community.document_loaders import PyPDFLoader\n", - "from langchain_text_splitters import RecursiveCharacterTextSplitter\n", - "from langchain_openai import ChatOpenAI\n", - "from langchain_core.prompts import PromptTemplate, ChatPromptTemplate\n", - "from langchain_core.output_parsers import JsonOutputParser\n", - "from pydantic import BaseModel, Field\n", + "from redisvl.extensions.cache.llm import LangCacheSemanticCache\n", "\n", "# Optimization\n", - "from redis_retrieval_optimizer.threshold_optimization import CacheThresholdOptimizer\n", - "\n", - "print(\"All imports successful\")\n" + "from redis_retrieval_optimizer.threshold_optimization import CacheThresholdOptimizer" ] }, { @@ -155,17 +122,16 @@ "\n", "### Sign Up for LangCache\n", "\n", - "If you haven't already, sign up for a free LangCache account:\n", + "If you haven't already, sign up for a free Redis Cloud account:\n", "\n", - "**[Sign up for LangCache →](https://langcache.io/signup)**\n", + "**[Log in or sign up for Redis Cloud →](https://cloud.redis.io/#/)**\n", "\n", "After signing up:\n", - "1. Create a new cache instance\n", - "2. Copy your **Endpoint URL** (looks like: `redis-xxxxx.langcache.io:xxxxx`)\n", - "3. Copy your **Access Token/Password**\n", - "4. Note your **Cache ID** (you'll use this as a prefix for organizing caches)\n", - "\n", - "> **Note:** For this workshop, you can alternatively use a standard Redis Cloud instance with Redis Stack. Simply provide your Redis Cloud connection details instead.\n" + "1. Create a new database\n", + "2. Create a new LangCache service (Select 'LangCache' on the left menu bar)\n", + "3. Copy your **API Key**\n", + "4. Copy your **Cache ID**\n", + "5. Copy your **URL**\n" ] }, { @@ -177,37 +143,14 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 3, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "✓ Environment variables configured\n", - " Redis URL: :6379\n" - ] - } - ], + "outputs": [], "source": [ - "# Redis/LangCache credentials\n", - "if \"REDIS_URL\" not in os.environ:\n", - " redis_host = input(\"Enter your Redis/LangCache host (e.g., redis-xxxxx.langcache.io or localhost): \")\n", - " redis_port = input(\"Enter your Redis port (default: 6379): \") or \"6379\"\n", - " redis_password = getpass(\"Enter your Redis password (leave empty for local): \")\n", - " \n", - " # Build Redis URL\n", - " if redis_password:\n", - " os.environ[\"REDIS_URL\"] = f\"rediss://:{redis_password}@{redis_host}:{redis_port}\"\n", - " else:\n", - " os.environ[\"REDIS_URL\"] = f\"redis://{redis_host}:{redis_port}\"\n", + "import getpass\n", "\n", - "# OpenAI API key for LLM and embeddings\n", "if \"OPENAI_API_KEY\" not in os.environ:\n", - " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")\n", - "\n", - "print(\"Environment variables configured\")\n", - "print(f\" Redis URL: {os.environ['REDIS_URL'].split('@')[-1] if '@' in os.environ['REDIS_URL'] else os.environ['REDIS_URL'].split('//')[1]}\")\n" + " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")" ] }, { @@ -221,83 +164,103 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "langcache_api_key = os.environ.get('LANGCACHE_API_KEY') # found on your cloud console\n", + "langcache_id = os.environ.get('LANGCACHE_ID') # found on your cloud console\n", + "server_url = \"https://aws-us-east-1.langcache.redis.io\" # found on your cloud console\n", + "\n", + "# Create Semantic Cache instance\n", + "cache = LangCacheSemanticCache(\n", + " server_url=server_url,\n", + " cache_id=langcache_id,\n", + " api_key=langcache_api_key,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "11:14:02 datasets INFO PyTorch version 2.7.0 available.\n", - "11:14:03 sentence_transformers.SentenceTransformer INFO Use pytorch device_name: mps\n", - "11:14:03 sentence_transformers.SentenceTransformer INFO Load pretrained SentenceTransformer: redis/langcache-embed-v1\n", - "✓ Semantic Cache initialized: rag_faq_cache\n", - " Using model: redis/langcache-embed-v1\n", - " Distance threshold: 0.15\n" + "20:59:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "[{'entry_id': '5eb63bbbe01eeed093cb22bb8f5acdc3', 'prompt': 'hello world', 'response': 'hello world from langcache', 'vector_distance': 0.0, 'inserted_at': 0.0, 'updated_at': 0.0}]\n", + "20:59:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "20:59:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "[{'entry_id': '5eb63bbbe01eeed093cb22bb8f5acdc3', 'prompt': 'hello world', 'response': 'hello world from langcache', 'vector_distance': 0.07242219999999999, 'inserted_at': 0.0, 'updated_at': 0.0}]\n" ] } ], "source": [ - "# Initialize the vectorizer with the LangCache embedding model\n", - "# This model is specifically optimized for semantic caching with better precision/recall\n", - "vectorizer = HFTextVectorizer(\n", - " model=\"redis/langcache-embed-v1\"\n", - ")\n", - "\n", - "# Create Semantic Cache instance\n", - "cache = SemanticCache(\n", - " name=\"rag_faq_cache\",\n", - " redis_url=os.environ[\"REDIS_URL\"],\n", - " vectorizer=vectorizer,\n", - " distance_threshold=0.15 # Initial threshold, we'll optimize this later\n", - ")\n", - "\n", - "print(f\"Semantic Cache initialized: {cache.index.name}\")\n", - "print(f\" Using model: redis/langcache-embed-v1\")\n", - "print(f\" Distance threshold: {cache.distance_threshold}\")\n" + "# Check your cache is workign\n", + "r = cache.check('hello world')\n", + "print(r) # should be empty on first run\n", + "cache.store('hello world', 'hello world from langcache')\n", + "r = cache.check('hi world')\n", + "print(r)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Initialize OpenAI LLM\n" + "## 3. Load and Prepare Datasets\n", + "\n", + "We'll work with three types of data:\n", + "1. **Knowledge Base**: PDF document(s) that contain factual information\n", + "2. **FAQs**: Derived from the knowledge base using Doc2Cache technique\n", + "3. **Test Dataset**: For evaluating and optimizing cache performance\n" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 6, "metadata": {}, "outputs": [ { - "name": "stdout", + "name": "stderr", "output_type": "stream", "text": [ - "✓ OpenAI LLM initialized\n" + "/Users/justin.cechmanek/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n", + " from .autonotebook import tqdm as notebook_tqdm\n" ] } ], "source": [ - "# Initialize OpenAI LLM for FAQ generation and RAG\n", - "llm = ChatOpenAI(\n", - " model=\"gpt-4o-mini\",\n", - " temperature=0.3,\n", - " max_tokens=2000\n", - ")\n", - "\n", - "print(\"OpenAI LLM initialized\")\n" + "# LangChain imports\n", + "from langchain_community.document_loaders import PyPDFLoader\n", + "from langchain_text_splitters import RecursiveCharacterTextSplitter\n", + "from langchain_openai import ChatOpenAI\n", + "from langchain_core.prompts import PromptTemplate, ChatPromptTemplate\n", + "from langchain_core.output_parsers import JsonOutputParser\n", + "from pydantic import BaseModel, Field" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## 3. Load and Prepare Datasets\n", - "\n", - "We'll work with three types of data:\n", - "1. **Knowledge Base**: PDF document(s) that contain factual information\n", - "2. **FAQs**: Derived from the knowledge base using Doc2Cache technique\n", - "3. **Test Dataset**: For evaluating and optimizing cache performance\n" + "### Initialize OpenAI LLM\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "# Initialize OpenAI LLM for FAQ generation and RAG\n", + "llm = ChatOpenAI(\n", + " model=\"gpt-4o-mini\",\n", + " temperature=0.3,\n", + " max_tokens=2000\n", + ")" ] }, { @@ -309,35 +272,25 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 8, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "✓ Sample PDF downloaded\n" - ] - } - ], + "outputs": [], "source": [ "# Download sample PDF if not already present\n", "!mkdir -p data\n", - "!wget -q -O data/nvidia-10k.pdf https://raw.githubusercontent.com/redis-developer/redis-ai-resources/main/python-recipes/RAG/resources/nvd-10k-2023.pdf\n", - "\n", - "print(\"Sample PDF downloaded\")\n" + "!wget -q -O data/nvidia-10k.pdf https://raw.githubusercontent.com/redis-developer/redis-ai-resources/main/python-recipes/RAG/resources/nvd-10k-2023.pdf" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "✓ Loaded PDF: data/nvidia-10k.pdf\n", + "Loaded PDF: data/nvidia-10k.pdf\n", " Total pages: 169\n", " Created chunks: 388\n", "\n", @@ -383,7 +336,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 10, "metadata": {}, "outputs": [], "source": [ @@ -402,14 +355,14 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "✓ FAQ generation chain configured\n" + "FAQ generation chain configured\n" ] } ], @@ -443,7 +396,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 12, "metadata": {}, "outputs": [ { @@ -452,16 +405,16 @@ "text": [ "Testing FAQ generation on sample chunk...\n", "\n", - "11:14:29 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "20:59:29 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Generated 5 FAQs:\n", "\n", - "1. Q: What industries are leveraging NVIDIA's GPUs and software for automation?\n", - " Category: operations\n", - " A: A rapidly growing number of enterprises and startups across a broad range of industries, including transportation for autonomous driving, healthcare f...\n", + "1. Q: What industries are leveraging NVIDIA's GPUs for automation?\n", + " Category: products\n", + " A: A rapidly growing number of enterprises and startups across various industries, including transportation for autonomous driving, healthcare for enhanc...\n", "\n", - "2. Q: What was the reason for the termination of the Arm Share Purchase Agreement between NVIDIA and SoftBank?\n", - " Category: general\n", - " A: The termination of the Share Purchase Agreement was due to significant regulatory challenges that prevented the completion of the transaction....\n", + "2. Q: What was the reason for the termination of the Arm Share Purchase Agreement?\n", + " Category: operations\n", + " A: The Share Purchase Agreement between NVIDIA and SoftBank Group Corp. was terminated due to significant regulatory challenges that prevented the comple...\n", "\n", "3. Q: What acquisition termination cost did NVIDIA record in fiscal year 2023?\n", " Category: financial\n", @@ -483,7 +436,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 13, "metadata": {}, "outputs": [ { @@ -494,44 +447,44 @@ "Generating FAQs from document chunks...\n", "\n", "Processing chunk 1/25...\n", - "11:14:36 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:14:45 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:14:54 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:15:00 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:15:07 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "20:59:36 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "20:59:45 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "20:59:50 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "20:59:56 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:00:02 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 6/25...\n", - "11:15:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:15:16 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:15:22 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:15:29 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:15:35 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:00:07 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:00:10 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:00:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:00:26 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:00:32 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 11/25...\n", - "11:15:42 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:15:49 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:15:55 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:16:03 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:16:11 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:00:39 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:00:51 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:00:57 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:01:05 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:01:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 16/25...\n", - "11:16:20 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:16:26 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:16:35 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:16:41 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:16:48 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:01:21 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:01:32 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:01:43 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:01:51 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:02:03 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 21/25...\n", - "11:16:55 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:17:01 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:17:07 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:17:15 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "11:17:23 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:02:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:02:22 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:02:36 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:02:45 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "21:02:52 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "\n", - "✓ Generated 114 FAQs total\n", + "Generated 113 FAQs total\n", "\n", "Category distribution:\n", - " technology: 32\n", - " operations: 26\n", - " financial: 22\n", - " products: 21\n", - " general: 13\n" + " technology: 29\n", + " products: 25\n", + " operations: 23\n", + " financial: 19\n", + " general: 17\n" ] } ], @@ -583,7 +536,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 14, "metadata": {}, "outputs": [ { @@ -596,7 +549,7 @@ "\n", "2. What is the trading symbol for NVIDIA Corporation's common stock?...\n", "\n", - "3. Where is the principal executive office of NVIDIA Corporation located?...\n" + "3. Where is NVIDIA Corporation's principal executive office located?...\n" ] } ], @@ -611,14 +564,14 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "✓ Test dataset created\n", + "Test dataset created\n", " Negative examples: 5\n" ] } @@ -648,26 +601,37 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "✓ Cache cleared\n" + "21:02:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "21:02:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n" ] + }, + { + "data": { + "text/plain": [ + "'5eb63bbbe01eeed093cb22bb8f5acdc3'" + ] + }, + "execution_count": 16, + "metadata": {}, + "output_type": "execute_result" } ], "source": [ "# Clear any existing cache entries\n", - "cache.clear()\n", - "print(\"Cache cleared\")\n" + "r = cache.check('hello world')\n", + "cache.store('hello world', 'hello world from langcache')\n" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 17, "metadata": {}, "outputs": [ { @@ -676,22 +640,134 @@ "text": [ "Storing FAQs in cache...\n", "\n", - " Stored 0/114 FAQs...\n", - " Stored 20/114 FAQs...\n", - " Stored 40/114 FAQs...\n", - " Stored 60/114 FAQs...\n", - " Stored 80/114 FAQs...\n", - " Stored 100/114 FAQs...\n", - "\n", - "✓ Stored 114 FAQs in cache\n", - " Cache index: rag_faq_cache\n", + " Stored 0/113 FAQs...\n", + "21:02:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + " Stored 20/113 FAQs...\n", + "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + " Stored 40/113 FAQs...\n", + "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + " Stored 60/113 FAQs...\n", + "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + " Stored 80/113 FAQs...\n", + "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + " Stored 100/113 FAQs...\n", + "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "\n", + "Stored 113 FAQs in cache\n", "\n", "Example cache entries:\n", "\n", - "1. Key: rag_faq_cache:abd9b974d9eedebc62332adbfd10ab5ff96e9d65dbd4476a27a487dd82b46002\n", + "1. Key: f630de8d775eaf0bfe3f615e8328d451\n", " Q: What is the fiscal year end date for NVIDIA Corporation as reported in the 10-K?...\n", "\n", - "2. Key: rag_faq_cache:8aa719b5f3d105fdcd9048d2b6c14e04bd60e8501a27ed80481c97adafb01ea7\n", + "2. Key: f7e7a16db20d8199b6a95de1129d8b03\n", " Q: What is the trading symbol for NVIDIA Corporation's common stock?...\n" ] } @@ -720,7 +796,6 @@ " print(f\" Warning: Failed to store FAQ {i+1}: {str(e)[:100]}\")\n", "\n", "print(f\"\\nStored {stored_count} FAQs in cache\")\n", - "print(f\" Cache index: {cache.index.name}\")\n", "print(f\"\\nExample cache entries:\")\n", "for i, (q, k) in enumerate(list(cache_keys.items())[:2], 1):\n", " print(f\"\\n{i}. Key: {k}\")\n", @@ -745,7 +820,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 18, "metadata": {}, "outputs": [ { @@ -754,17 +829,20 @@ "text": [ "Testing exact match queries:\n", "\n", - "1. ✓ Cache HIT\n", + "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "1. Cache HIT\n", " Query: What is the fiscal year end date for NVIDIA Corporation as reported in the 10-K?...\n", " Answer: The fiscal year ended January 29, 2023....\n", "\n", - "2. ✓ Cache HIT\n", + "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "2. Cache HIT\n", " Query: What is the trading symbol for NVIDIA Corporation's common stock?...\n", " Answer: The trading symbol for NVIDIA Corporation's common stock is NVDA....\n", "\n", - "3. ✓ Cache HIT\n", - " Query: Where is the principal executive office of NVIDIA Corporation located?...\n", - " Answer: The principal executive office of NVIDIA Corporation is located at 2788 San Tomas Expressway, Santa ...\n", + "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "3. Cache HIT\n", + " Query: Where is NVIDIA Corporation's principal executive office located?...\n", + " Answer: NVIDIA Corporation's principal executive office is located at 2788 San Tomas Expressway, Santa Clara...\n", "\n" ] } @@ -794,7 +872,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 19, "metadata": {}, "outputs": [ { @@ -803,12 +881,15 @@ "text": [ "Testing semantic similarity:\n", "\n", + "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "1. ✗ Cache MISS\n", " Query: Tell me about NVIDIA's revenue\n", "\n", + "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "2. ✗ Cache MISS\n", " Query: What products does the company make?\n", "\n", + "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "3. ✗ Cache MISS\n", " Query: How is the company performing financially?\n", "\n" @@ -847,7 +928,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 20, "metadata": {}, "outputs": [ { @@ -855,9 +936,10 @@ "output_type": "stream", "text": [ "Testing query: 'What is NVIDIA's main business?'\n", - "Current threshold: 0.1500\n", - "\n", - "✗ Cache MISS - No match found within threshold\n" + "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "Cache HIT\n", + " Distance: 0.080627\n", + " Matched: What types of markets does NVIDIA specialize in?...\n" ] } ], @@ -866,7 +948,6 @@ "test_query = \"What is NVIDIA's main business?\"\n", "\n", "print(f\"Testing query: '{test_query}'\")\n", - "print(f\"Current threshold: {cache.distance_threshold:.4f}\\n\")\n", "\n", "result = cache.check(prompt=test_query, return_fields=[\"prompt\", \"vector_distance\"])\n", "\n", @@ -887,7 +968,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 21, "metadata": {}, "outputs": [ { @@ -896,19 +977,24 @@ "text": [ "Testing negative examples (should NOT match):\n", "\n", - "1. ✓ Correct MISS\n", + "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "1. Correct MISS\n", " Query: What is the weather today?\n", "\n", - "2. ✓ Correct MISS\n", + "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "2. Correct MISS\n", " Query: How do I cook pasta?\n", "\n", - "3. ✓ Correct MISS\n", + "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "3. Correct MISS\n", " Query: What is the capital of France?\n", "\n", - "4. ✓ Correct MISS\n", + "21:03:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "4. Correct MISS\n", " Query: Tell me a joke\n", "\n", - "5. ✓ Correct MISS\n", + "21:03:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "5. Correct MISS\n", " Query: What time is it?\n", "\n" ] @@ -941,14 +1027,14 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "✓ Created optimization test data:\n", + "Created optimization test data:\n", " Total examples: 10\n", " Positive (should match): 5\n", " Negative (should not match): 5\n" @@ -984,28 +1070,34 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 23, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "\n", - "Current distance threshold: 0.01\n", "\n", "Optimizing threshold...\n", - "\n", - "\n", - "✓ Optimization complete!\n", - " Original threshold: 0.15\n", - " Optimized threshold: 0.010000\n" + "\n" + ] + }, + { + "ename": "AttributeError", + "evalue": "'LangCacheSemanticCache' object has no attribute '_vectorizer'", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mAttributeError\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[23]\u001b[39m\u001b[32m, line 10\u001b[39m\n\u001b[32m 2\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33m\"\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33mOptimizing threshold...\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33m\"\u001b[39m)\n\u001b[32m 4\u001b[39m optimizer = CacheThresholdOptimizer(\n\u001b[32m 5\u001b[39m cache,\n\u001b[32m 6\u001b[39m optimization_test_data,\n\u001b[32m 7\u001b[39m eval_metric=\u001b[33m\"\u001b[39m\u001b[33mf1\u001b[39m\u001b[33m\"\u001b[39m \u001b[38;5;66;03m# Can also use \"precision\" or \"recall\"\u001b[39;00m\n\u001b[32m 8\u001b[39m )\n\u001b[32m---> \u001b[39m\u001b[32m10\u001b[39m results = \u001b[43moptimizer\u001b[49m\u001b[43m.\u001b[49m\u001b[43moptimize\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 12\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33mOptimization complete!\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 13\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m results \u001b[38;5;129;01mand\u001b[39;00m \u001b[33m'\u001b[39m\u001b[33mf1\u001b[39m\u001b[33m'\u001b[39m \u001b[38;5;129;01min\u001b[39;00m results:\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/redis_retrieval_optimizer/threshold_optimization/cache.py:149\u001b[39m, in \u001b[36mCacheThresholdOptimizer.optimize\u001b[39m\u001b[34m(self, **kwargs)\u001b[39m\n\u001b[32m 147\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34moptimize\u001b[39m(\u001b[38;5;28mself\u001b[39m, **kwargs: Any):\n\u001b[32m 148\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Optimize thresholds using the provided optimization function for cache case.\"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m149\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mopt_fn\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43moptimizable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mtest_data\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43meval_metric\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/redis_retrieval_optimizer/threshold_optimization/cache.py:71\u001b[39m, in \u001b[36m_grid_search_opt_cache\u001b[39m\u001b[34m(cache, test_data, eval_metric)\u001b[39m\n\u001b[32m 68\u001b[39m metrics = {}\n\u001b[32m 70\u001b[39m \u001b[38;5;28;01mfor\u001b[39;00m td \u001b[38;5;129;01min\u001b[39;00m test_data:\n\u001b[32m---> \u001b[39m\u001b[32m71\u001b[39m vec = \u001b[43mcache\u001b[49m\u001b[43m.\u001b[49m\u001b[43m_vectorizer\u001b[49m.embed(td.query)\n\u001b[32m 72\u001b[39m query = RangeQuery(\n\u001b[32m 73\u001b[39m vec, vector_field_name=\u001b[33m\"\u001b[39m\u001b[33mprompt_vector\u001b[39m\u001b[33m\"\u001b[39m, distance_threshold=\u001b[32m1.0\u001b[39m\n\u001b[32m 74\u001b[39m )\n\u001b[32m 75\u001b[39m res = cache.index.query(query)\n", + "\u001b[31mAttributeError\u001b[39m: 'LangCacheSemanticCache' object has no attribute '_vectorizer'" ] } ], "source": [ "# Optimize threshold based on F1 score\n", - "print(f\"\\nCurrent distance threshold: {cache.distance_threshold}\")\n", "print(\"\\nOptimizing threshold...\\n\")\n", "\n", "optimizer = CacheThresholdOptimizer(\n", @@ -1017,8 +1109,6 @@ "results = optimizer.optimize()\n", "\n", "print(f\"\\nOptimization complete!\")\n", - "print(f\" Original threshold: 0.15\")\n", - "print(f\" Optimized threshold: {cache.distance_threshold:.6f}\")\n", "if results and 'f1' in results:\n", " print(f\" F1 Score: {results['f1']:.4f}\")\n" ] @@ -1027,32 +1117,7 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "Re-testing negative examples with optimized threshold:\n", - "\n", - "1. ✓ MISS (correct)\n", - " Query: What is the weather today?\n", - "\n", - "2. ✓ MISS (correct)\n", - " Query: How do I cook pasta?\n", - "\n", - "3. ✓ MISS (correct)\n", - " Query: What is the capital of France?\n", - "\n", - "4. ✓ MISS (correct)\n", - " Query: Tell me a joke\n", - "\n", - "5. ✓ MISS (correct)\n", - " Query: What time is it?\n", - "\n" - ] - } - ], + "outputs": [], "source": [ "# Re-test with optimized threshold\n", "print(\"\\nRe-testing negative examples with optimized threshold:\\n\")\n", @@ -1080,34 +1145,9 @@ }, { "cell_type": "code", - "execution_count": 28, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "LangCache-Embed Model Information:\n", - "============================================================\n", - "Model: redis/langcache-embed-v1\n", - "Purpose: Optimized for semantic caching tasks\n", - "Dimension: 768\n", - "Distance Metric: cosine\n", - "\n", - "Key Features:\n", - " • Trained specifically on query-response pairs\n", - " • Balanced precision/recall for optimal cache hit rates\n", - " • Fast inference time suitable for production\n", - " • Optimized for short-form text (queries/prompts)\n", - "\n", - "Advantages for Caching:\n", - " • Better semantic understanding of query intent\n", - " • More robust to paraphrasing and rewording\n", - " • Lower false positive rate compared to general embeddings\n", - " • Optimized threshold ranges for cache decisions\n" - ] - } - ], + "outputs": [], "source": [ "# Show information about the langcache-embed model\n", "print(\"LangCache-Embed Model Information:\")\n", @@ -1130,36 +1170,9 @@ }, { "cell_type": "code", - "execution_count": 29, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "Comparing semantic similarities:\n", - "\n", - "Base question: What is NVIDIA's revenue?\n", - "\n", - "Query: Tell me about NVIDIA's earnings\n", - " Similarity: 0.8725\n", - " Distance: 0.1275\n", - " Would cache hit (threshold=0.0100)? False\n", - "\n", - "Query: How much money does NVIDIA make?\n", - " Similarity: 0.9004\n", - " Distance: 0.0996\n", - " Would cache hit (threshold=0.0100)? False\n", - "\n", - "Query: What is the weather today?\n", - " Similarity: 0.3141\n", - " Distance: 0.6859\n", - " Would cache hit (threshold=0.0100)? False\n", - "\n" - ] - } - ], + "outputs": [], "source": [ "# Compare semantic similarities between related and unrelated questions\n", "test_questions = [\n", @@ -1211,15 +1224,7 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "✓ RAG chain created\n" - ] - } - ], + "outputs": [], "source": [ "# Create a simple RAG prompt template\n", "rag_template = ChatPromptTemplate.from_messages([\n", @@ -1244,15 +1249,7 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "✓ Cached RAG function ready\n" - ] - } - ], + "outputs": [], "source": [ "def rag_with_cache(question: str, use_cache: bool = True) -> tuple:\n", " \"\"\"\n", @@ -1298,81 +1295,7 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "================================================================================\n", - "PERFORMANCE COMPARISON: With Cache vs Without Cache\n", - "================================================================================\n", - "\n", - "[FIRST PASS - Populating Cache]\n", - "\n", - "15:52:18 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "1. What is NVIDIA's primary business?\n", - " Cache: MISS | Time: 2.232s\n", - " Answer: NVIDIA's primary business is the design and manufacture of graphics processing units (GPUs) for gami...\n", - "\n", - "15:52:20 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "2. How much revenue did NVIDIA generate?\n", - " Cache: MISS | Time: 2.188s\n", - " Answer: As of the latest 10-K filing, NVIDIA reported total revenue of $26.91 billion for the fiscal year en...\n", - "\n", - "15:52:23 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "3. What are NVIDIA's main products?\n", - " Cache: MISS | Time: 3.195s\n", - " Answer: NVIDIA's main products include:\n", - "\n", - "1. **Graphics Processing Units (GPUs)**: Primarily for gaming, prof...\n", - "\n", - "\n", - "[SECOND PASS - Cache Hits with Paraphrased Questions]\n", - "\n", - "15:52:25 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "1. What does NVIDIA do as a business?\n", - " Cache: MISS ✗ | Time: 2.069s\n", - " Answer: NVIDIA is primarily a technology company that designs and manufactures graphics processing units (GP...\n", - "\n", - "15:52:28 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "2. Can you tell me NVIDIA's revenue figures?\n", - " Cache: MISS ✗ | Time: 2.518s\n", - " Answer: As of the latest 10-K filing, NVIDIA reported total revenue of $26.91 billion for the fiscal year en...\n", - "\n", - "15:52:34 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "3. What products does NVIDIA sell?\n", - " Cache: MISS ✗ | Time: 5.687s\n", - " Answer: NVIDIA sells a variety of products primarily focused on graphics processing units (GPUs) and related...\n", - "\n", - "\n", - "[THIRD PASS - Without Cache (Baseline)]\n", - "\n", - "15:52:36 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "1. What is NVIDIA's primary business?\n", - " Cache: DISABLED | Time: 1.807s\n", - "\n", - "15:52:38 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "2. How much revenue did NVIDIA generate?\n", - " Cache: DISABLED | Time: 2.035s\n", - "\n", - "15:52:41 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "3. What are NVIDIA's main products?\n", - " Cache: DISABLED | Time: 3.880s\n", - "\n", - "\n", - "================================================================================\n", - "PERFORMANCE SUMMARY\n", - "================================================================================\n", - "Average time - First pass (cache miss): 2.538s\n", - "Average time - Second pass (cache hit): 3.424s\n", - "Average time - Without cache: 2.574s\n", - "\n", - "✓ Speedup with cache: 0.7x faster\n", - " Cache hit rate: 0%\n" - ] - } - ], + "outputs": [], "source": [ "# Test questions for RAG evaluation\n", "test_questions_rag = [\n", @@ -1448,113 +1371,8 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Cost Savings Analysis\n", - "\n", - "Let's estimate the potential cost savings from implementing semantic caching in a production environment.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 34, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "================================================================================\n", - "COST SAVINGS ESTIMATE (10,000 queries/day @ 70% hit rate)\n", - "================================================================================\n", - "Cost without cache: $1.2000/day\n", - "Cost with cache: $0.367000/day\n", - "\n", - "Daily savings: $0.8330 (69.4% reduction)\n", - "Monthly savings: $24.99\n", - "Annual savings: $304.04\n" - ] - } - ], - "source": [ - "# Estimate cost savings\n", - "# Assumptions: GPT-4o-mini costs ~$0.15 per 1M input tokens, $0.60 per 1M output tokens\n", - "# Average request: ~200 input tokens, ~150 output tokens\n", - "cost_per_llm_call = (200 * 0.15 / 1_000_000) + (150 * 0.60 / 1_000_000) # USD\n", - "cost_per_cache_check = 0.000001 # Negligible (Redis query)\n", - "\n", - "total_queries = 10000 # Simulate 10K queries per day\n", - "cache_hit_rate = 0.70 # Assume 70% hit rate in production\n", - "\n", - "# Without cache\n", - "cost_without_cache = total_queries * cost_per_llm_call\n", - "\n", - "# With cache\n", - "cache_hits = total_queries * cache_hit_rate\n", - "cache_misses = total_queries * (1 - cache_hit_rate)\n", - "cost_with_cache = (cache_hits * cost_per_cache_check) + (cache_misses * cost_per_llm_call)\n", - "\n", - "savings = cost_without_cache - cost_with_cache\n", - "savings_percent = (savings / cost_without_cache) * 100\n", - "\n", - "print(\"\\n\" + \"=\"*80)\n", - "print(f\"COST SAVINGS ESTIMATE ({total_queries:,} queries/day @ {int(cache_hit_rate*100)}% hit rate)\")\n", - "print(\"=\"*80)\n", - "print(f\"Cost without cache: ${cost_without_cache:.4f}/day\")\n", - "print(f\"Cost with cache: ${cost_with_cache:.6f}/day\")\n", - "print(f\"\\nDaily savings: ${savings:.4f} ({savings_percent:.1f}% reduction)\")\n", - "print(f\"Monthly savings: ${savings * 30:.2f}\")\n", - "print(f\"Annual savings: ${savings * 365:.2f}\")\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 9. Cache Analytics and Monitoring\n" - ] - }, - { - "cell_type": "code", - "execution_count": 35, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "================================================================================\n", - "CACHE STATISTICS\n", - "================================================================================\n", - "\n", - "Cache Name: rag_faq_cache\n", - "Total cached entries: 120\n", - "Distance threshold: 0.010000\n", - "Vectorizer model: redis/langcache-embed-v1\n", - "Embedding dimensions: 768\n" - ] - } - ], - "source": [ - "# Get cache statistics\n", - "print(\"\\n\" + \"=\"*80)\n", - "print(\"CACHE STATISTICS\")\n", - "print(\"=\"*80)\n", + "## 9. Best Practices and Tips\n", "\n", - "# Count total entries\n", - "info = cache.index.info()\n", - "print(f\"\\nCache Name: {cache.index.name}\")\n", - "print(f\"Total cached entries: {info.get('num_docs', 'N/A')}\")\n", - "print(f\"Distance threshold: {cache.distance_threshold:.6f}\")\n", - "print(f\"Vectorizer model: redis/langcache-embed-v1\")\n", - "print(f\"Embedding dimensions: 768\")\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 10. Best Practices and Tips\n", "\n", "### Key Takeaways\n", "\n", @@ -1593,7 +1411,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 11. Cleanup\n", + "## 10. Cleanup\n", "\n", "Clean up resources when done.\n" ] @@ -1602,19 +1420,11 @@ "cell_type": "code", "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "✓ Cache cleared (index preserved)\n" - ] - } - ], + "outputs": [], "source": [ "# Clear cache contents\n", - "cache.delete()\n", - "print(\"Cache deleted\")" + "cache.clear()\n", + "print(\"Cache contents cleared\")" ] }, { From 9a769c4506a0bf9491520d79e4b0b2e750604ca7 Mon Sep 17 00:00:00 2001 From: Justin Cechmanek Date: Tue, 18 Nov 2025 16:54:10 -0800 Subject: [PATCH 4/7] cleans up cache optimization steps --- .../04_langcache_semantic_caching.ipynb | 641 +++++++++--------- 1 file changed, 338 insertions(+), 303 deletions(-) diff --git a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb index 28f1fc6c..a54b9fea 100644 --- a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb +++ b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb @@ -97,7 +97,7 @@ "name": "stdout", "output_type": "stream", "text": [ - "20:59:02 numexpr.utils INFO NumExpr defaulting to 10 threads.\n" + "16:36:26 numexpr.utils INFO NumExpr defaulting to 10 threads.\n" ] } ], @@ -189,10 +189,10 @@ "name": "stdout", "output_type": "stream", "text": [ - "20:59:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:36:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "[{'entry_id': '5eb63bbbe01eeed093cb22bb8f5acdc3', 'prompt': 'hello world', 'response': 'hello world from langcache', 'vector_distance': 0.0, 'inserted_at': 0.0, 'updated_at': 0.0}]\n", - "20:59:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "20:59:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:36:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:36:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "[{'entry_id': '5eb63bbbe01eeed093cb22bb8f5acdc3', 'prompt': 'hello world', 'response': 'hello world from langcache', 'vector_distance': 0.07242219999999999, 'inserted_at': 0.0, 'updated_at': 0.0}]\n" ] } @@ -405,20 +405,20 @@ "text": [ "Testing FAQ generation on sample chunk...\n", "\n", - "20:59:29 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:36:51 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Generated 5 FAQs:\n", "\n", "1. Q: What industries are leveraging NVIDIA's GPUs for automation?\n", - " Category: products\n", - " A: A rapidly growing number of enterprises and startups across various industries, including transportation for autonomous driving, healthcare for enhanc...\n", - "\n", - "2. Q: What was the reason for the termination of the Arm Share Purchase Agreement?\n", " Category: operations\n", - " A: The Share Purchase Agreement between NVIDIA and SoftBank Group Corp. was terminated due to significant regulatory challenges that prevented the comple...\n", + " A: A rapidly growing number of enterprises and startups across a broad range of industries, including transportation for autonomous driving, healthcare f...\n", "\n", - "3. Q: What acquisition termination cost did NVIDIA record in fiscal year 2023?\n", + "2. Q: What was the reason for the termination of the Arm Share Purchase Agreement?\n", " Category: financial\n", - " A: NVIDIA recorded an acquisition termination cost of $1.35 billion in fiscal year 2023, reflecting the write-off of the prepayment provided at signing f...\n" + " A: The Share Purchase Agreement between NVIDIA and SoftBank was terminated due to significant regulatory challenges that prevented the completion of the ...\n", + "\n", + "3. Q: What types of products do professional designers create using NVIDIA's technology?\n", + " Category: products\n", + " A: Professional designers use NVIDIA's GPUs and software to create visual effects in movies and to design a variety of products, including cell phones an...\n" ] } ], @@ -447,44 +447,44 @@ "Generating FAQs from document chunks...\n", "\n", "Processing chunk 1/25...\n", - "20:59:36 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "20:59:45 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "20:59:50 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "20:59:56 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:00:02 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:36:57 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:37:04 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:37:09 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:37:17 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:37:22 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 6/25...\n", - "21:00:07 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:00:10 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:00:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:00:26 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:00:32 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:37:28 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:37:30 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:37:36 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:37:41 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:37:48 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 11/25...\n", - "21:00:39 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:00:51 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:00:57 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:01:05 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:01:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:37:54 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:38:00 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:38:05 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:38:12 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:38:19 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 16/25...\n", - "21:01:21 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:01:32 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:01:43 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:01:51 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:02:03 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:38:25 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:38:30 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:38:38 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:38:49 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:38:55 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 21/25...\n", - "21:02:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:02:22 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:02:36 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:02:45 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "21:02:52 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:39:02 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:39:07 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:39:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:39:21 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "16:39:28 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "\n", - "Generated 113 FAQs total\n", + "Generated 112 FAQs total\n", "\n", "Category distribution:\n", - " technology: 29\n", - " products: 25\n", - " operations: 23\n", + " technology: 31\n", + " operations: 24\n", + " products: 20\n", " financial: 19\n", - " general: 17\n" + " general: 18\n" ] } ], @@ -545,7 +545,7 @@ "text": [ "Sample FAQs for testing:\n", "\n", - "1. What is the fiscal year end date for NVIDIA Corporation as reported in the 10-K?...\n", + "1. What is the fiscal year end date for NVIDIA Corporation as reported in the Form 10-K?...\n", "\n", "2. What is the trading symbol for NVIDIA Corporation's common stock?...\n", "\n", @@ -608,8 +608,8 @@ "name": "stdout", "output_type": "stream", "text": [ - "21:02:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", - "21:02:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n" + "16:39:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:39:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n" ] }, { @@ -640,132 +640,131 @@ "text": [ "Storing FAQs in cache...\n", "\n", - " Stored 0/113 FAQs...\n", - "21:02:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - " Stored 20/113 FAQs...\n", - "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - " Stored 40/113 FAQs...\n", - "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - " Stored 60/113 FAQs...\n", - "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:02:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - " Stored 80/113 FAQs...\n", - "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - " Stored 100/113 FAQs...\n", - "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + " Stored 0/112 FAQs...\n", + "16:39:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + " Stored 20/112 FAQs...\n", + "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + " Stored 40/112 FAQs...\n", + "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + " Stored 60/112 FAQs...\n", + "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + " Stored 80/112 FAQs...\n", + "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + " Stored 100/112 FAQs...\n", + "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", + "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", "\n", - "Stored 113 FAQs in cache\n", + "Stored 112 FAQs in cache\n", "\n", "Example cache entries:\n", "\n", - "1. Key: f630de8d775eaf0bfe3f615e8328d451\n", - " Q: What is the fiscal year end date for NVIDIA Corporation as reported in the 10-K?...\n", + "1. Key: a629e006a52387e1d29243f32555b5b6\n", + " Q: What is the fiscal year end date for NVIDIA Corporation as reported in the Form ...\n", "\n", "2. Key: f7e7a16db20d8199b6a95de1129d8b03\n", " Q: What is the trading symbol for NVIDIA Corporation's common stock?...\n" @@ -829,17 +828,17 @@ "text": [ "Testing exact match queries:\n", "\n", - "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "1. Cache HIT\n", - " Query: What is the fiscal year end date for NVIDIA Corporation as reported in the 10-K?...\n", + " Query: What is the fiscal year end date for NVIDIA Corporation as reported in the Form ...\n", " Answer: The fiscal year ended January 29, 2023....\n", "\n", - "21:03:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "2. Cache HIT\n", " Query: What is the trading symbol for NVIDIA Corporation's common stock?...\n", " Answer: The trading symbol for NVIDIA Corporation's common stock is NVDA....\n", "\n", - "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "3. Cache HIT\n", " Query: Where is NVIDIA Corporation's principal executive office located?...\n", " Answer: NVIDIA Corporation's principal executive office is located at 2788 San Tomas Expressway, Santa Clara...\n", @@ -881,15 +880,17 @@ "text": [ "Testing semantic similarity:\n", "\n", - "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", - "1. ✗ Cache MISS\n", + "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "1. Cache HIT (distance: 0.0629)\n", " Query: Tell me about NVIDIA's revenue\n", + " Matched: How much revenue did NVIDIA generate?...\n", + " Answer: As of the latest available data in NVIDIA's 10-K filing for the fiscal year ended January 29, 2023, ...\n", "\n", - "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "2. ✗ Cache MISS\n", " Query: What products does the company make?\n", "\n", - "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "3. ✗ Cache MISS\n", " Query: How is the company performing financially?\n", "\n" @@ -936,10 +937,10 @@ "output_type": "stream", "text": [ "Testing query: 'What is NVIDIA's main business?'\n", - "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "Cache HIT\n", - " Distance: 0.080627\n", - " Matched: What types of markets does NVIDIA specialize in?...\n" + " Distance: 0.070051\n", + " Matched: What are the main business segments reported by NVIDIA?...\n" ] } ], @@ -977,23 +978,23 @@ "text": [ "Testing negative examples (should NOT match):\n", "\n", - "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "1. Correct MISS\n", " Query: What is the weather today?\n", "\n", - "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "2. Correct MISS\n", " Query: How do I cook pasta?\n", "\n", - "21:03:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "3. Correct MISS\n", " Query: What is the capital of France?\n", "\n", - "21:03:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "4. Correct MISS\n", " Query: Tell me a joke\n", "\n", - "21:03:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", "5. Correct MISS\n", " Query: What time is it?\n", "\n" @@ -1078,53 +1079,38 @@ "output_type": "stream", "text": [ "\n", - "Optimizing threshold...\n", + "Re-testing negative examples with optimized threshold:\n", + "\n", + "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "1. MISS (correct)\n", + " Query: What is the weather today?\n", + "\n", + "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "2. MISS (correct)\n", + " Query: How do I cook pasta?\n", + "\n", + "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "3. MISS (correct)\n", + " Query: What is the capital of France?\n", + "\n", + "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "4. MISS (correct)\n", + " Query: Tell me a joke\n", + "\n", + "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "5. MISS (correct)\n", + " Query: What time is it?\n", "\n" ] - }, - { - "ename": "AttributeError", - "evalue": "'LangCacheSemanticCache' object has no attribute '_vectorizer'", - "output_type": "error", - "traceback": [ - "\u001b[31m---------------------------------------------------------------------------\u001b[39m", - "\u001b[31mAttributeError\u001b[39m Traceback (most recent call last)", - "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[23]\u001b[39m\u001b[32m, line 10\u001b[39m\n\u001b[32m 2\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33m\"\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33mOptimizing threshold...\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33m\"\u001b[39m)\n\u001b[32m 4\u001b[39m optimizer = CacheThresholdOptimizer(\n\u001b[32m 5\u001b[39m cache,\n\u001b[32m 6\u001b[39m optimization_test_data,\n\u001b[32m 7\u001b[39m eval_metric=\u001b[33m\"\u001b[39m\u001b[33mf1\u001b[39m\u001b[33m\"\u001b[39m \u001b[38;5;66;03m# Can also use \"precision\" or \"recall\"\u001b[39;00m\n\u001b[32m 8\u001b[39m )\n\u001b[32m---> \u001b[39m\u001b[32m10\u001b[39m results = \u001b[43moptimizer\u001b[49m\u001b[43m.\u001b[49m\u001b[43moptimize\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 12\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33mOptimization complete!\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 13\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m results \u001b[38;5;129;01mand\u001b[39;00m \u001b[33m'\u001b[39m\u001b[33mf1\u001b[39m\u001b[33m'\u001b[39m \u001b[38;5;129;01min\u001b[39;00m results:\n", - "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/redis_retrieval_optimizer/threshold_optimization/cache.py:149\u001b[39m, in \u001b[36mCacheThresholdOptimizer.optimize\u001b[39m\u001b[34m(self, **kwargs)\u001b[39m\n\u001b[32m 147\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34moptimize\u001b[39m(\u001b[38;5;28mself\u001b[39m, **kwargs: Any):\n\u001b[32m 148\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Optimize thresholds using the provided optimization function for cache case.\"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m149\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mopt_fn\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43moptimizable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mtest_data\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43meval_metric\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n", - "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/redis_retrieval_optimizer/threshold_optimization/cache.py:71\u001b[39m, in \u001b[36m_grid_search_opt_cache\u001b[39m\u001b[34m(cache, test_data, eval_metric)\u001b[39m\n\u001b[32m 68\u001b[39m metrics = {}\n\u001b[32m 70\u001b[39m \u001b[38;5;28;01mfor\u001b[39;00m td \u001b[38;5;129;01min\u001b[39;00m test_data:\n\u001b[32m---> \u001b[39m\u001b[32m71\u001b[39m vec = \u001b[43mcache\u001b[49m\u001b[43m.\u001b[49m\u001b[43m_vectorizer\u001b[49m.embed(td.query)\n\u001b[32m 72\u001b[39m query = RangeQuery(\n\u001b[32m 73\u001b[39m vec, vector_field_name=\u001b[33m\"\u001b[39m\u001b[33mprompt_vector\u001b[39m\u001b[33m\"\u001b[39m, distance_threshold=\u001b[32m1.0\u001b[39m\n\u001b[32m 74\u001b[39m )\n\u001b[32m 75\u001b[39m res = cache.index.query(query)\n", - "\u001b[31mAttributeError\u001b[39m: 'LangCacheSemanticCache' object has no attribute '_vectorizer'" - ] } ], - "source": [ - "# Optimize threshold based on F1 score\n", - "print(\"\\nOptimizing threshold...\\n\")\n", - "\n", - "optimizer = CacheThresholdOptimizer(\n", - " cache,\n", - " optimization_test_data,\n", - " eval_metric=\"f1\" # Can also use \"precision\" or \"recall\"\n", - ")\n", - "\n", - "results = optimizer.optimize()\n", - "\n", - "print(f\"\\nOptimization complete!\")\n", - "if results and 'f1' in results:\n", - " print(f\" F1 Score: {results['f1']:.4f}\")\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], "source": [ "# Re-test with optimized threshold\n", "print(\"\\nRe-testing negative examples with optimized threshold:\\n\")\n", "\n", "for i, test_case in enumerate(negative_examples, 1):\n", " result = cache.check(prompt=test_case['query'], return_fields=[\"prompt\", \"vector_distance\"])\n", - " \n", + "\n", " if result:\n", " print(f\"{i}. ⚠️ HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", " print(f\" Query: {test_case['query']}\")\n", @@ -1138,77 +1124,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 7. LangCache-Embed Model Deep Dive\n", - "\n", - "The `redis/langcache-embed-v1` model is specifically optimized for semantic caching. Let's examine its characteristics and performance.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Show information about the langcache-embed model\n", - "print(\"LangCache-Embed Model Information:\")\n", - "print(\"=\"*60)\n", - "print(f\"Model: redis/langcache-embed-v1\")\n", - "print(f\"Purpose: Optimized for semantic caching tasks\")\n", - "print(f\"Dimension: 768\")\n", - "print(f\"Distance Metric: cosine\")\n", - "print(\"\\nKey Features:\")\n", - "print(\" • Trained specifically on query-response pairs\")\n", - "print(\" • Balanced precision/recall for optimal cache hit rates\")\n", - "print(\" • Fast inference time suitable for production\")\n", - "print(\" • Optimized for short-form text (queries/prompts)\")\n", - "print(\"\\nAdvantages for Caching:\")\n", - "print(\" • Better semantic understanding of query intent\")\n", - "print(\" • More robust to paraphrasing and rewording\")\n", - "print(\" • Lower false positive rate compared to general embeddings\")\n", - "print(\" • Optimized threshold ranges for cache decisions\")\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Compare semantic similarities between related and unrelated questions\n", - "test_questions = [\n", - " \"What is NVIDIA's revenue?\",\n", - " \"Tell me about NVIDIA's earnings\", # Semantically similar\n", - " \"How much money does NVIDIA make?\", # Semantically similar\n", - " \"What is the weather today?\", # Unrelated\n", - "]\n", - "\n", - "print(\"\\nComparing semantic similarities:\\n\")\n", - "print(f\"Base question: {test_questions[0]}\\n\")\n", - "\n", - "base_embedding = vectorizer.embed(test_questions[0])\n", - "\n", - "import numpy as np\n", - "\n", - "for query in test_questions[1:]:\n", - " query_embedding = vectorizer.embed(query)\n", - " \n", - " # Calculate cosine similarity\n", - " similarity = np.dot(base_embedding, query_embedding) / (\n", - " np.linalg.norm(base_embedding) * np.linalg.norm(query_embedding)\n", - " )\n", - " distance = 1 - similarity\n", - " \n", - " print(f\"Query: {query}\")\n", - " print(f\" Similarity: {similarity:.4f}\")\n", - " print(f\" Distance: {distance:.4f}\")\n", - " print(f\" Would cache hit (threshold={cache.distance_threshold:.4f})? {distance < cache.distance_threshold}\\n\")\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 8. RAG Pipeline Integration\n", + "## 7. RAG Pipeline Integration\n", "\n", "Now let's integrate the semantic cache into a complete RAG pipeline and measure the performance improvements.\n" ] @@ -1222,9 +1138,17 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 24, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "RAG chain created\n" + ] + } + ], "source": [ "# Create a simple RAG prompt template\n", "rag_template = ChatPromptTemplate.from_messages([\n", @@ -1247,9 +1171,17 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 25, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Cached RAG function ready\n" + ] + } + ], "source": [ "def rag_with_cache(question: str, use_cache: bool = True) -> tuple:\n", " \"\"\"\n", @@ -1293,9 +1225,85 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 26, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "================================================================================\n", + "PERFORMANCE COMPARISON: With Cache vs Without Cache\n", + "================================================================================\n", + "\n", + "[FIRST PASS - Populating Cache]\n", + "\n", + "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "1. What is NVIDIA's primary business?\n", + " Cache: HIT | Time: 0.109s\n", + " Answer: NVIDIA reports its business results in two segments: the Compute & Networking segment, which include...\n", + "\n", + "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "2. How much revenue did NVIDIA generate?\n", + " Cache: HIT | Time: 0.103s\n", + " Answer: As of the latest available data in NVIDIA's 10-K filing for the fiscal year ended January 29, 2023, ...\n", + "\n", + "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "3. What are NVIDIA's main products?\n", + " Cache: HIT | Time: 0.104s\n", + " Answer: NVIDIA sells a range of products primarily in the following categories:\n", + "\n", + "1. **Graphics Processing Un...\n", + "\n", + "\n", + "[SECOND PASS - Cache Hits with Paraphrased Questions]\n", + "\n", + "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "1. What does NVIDIA do as a business?\n", + " Cache: HIT ✓ | Time: 0.128s\n", + " Answer: NVIDIA's business has evolved from a primary focus on gaming products to broader markets, transition...\n", + "\n", + "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "2. Can you tell me NVIDIA's revenue figures?\n", + " Cache: HIT ✓ | Time: 0.127s\n", + " Answer: As of the latest available data in NVIDIA's 10-K filing for the fiscal year ended January 29, 2023, ...\n", + "\n", + "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "3. What products does NVIDIA sell?\n", + " Cache: HIT ✓ | Time: 0.099s\n", + " Answer: NVIDIA sells a range of products primarily in the following categories:\n", + "\n", + "1. **Graphics Processing Un...\n", + "\n", + "\n", + "[THIRD PASS - Without Cache (Baseline)]\n", + "\n", + "16:39:46 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "1. What is NVIDIA's primary business?\n", + " Cache: DISABLED | Time: 1.584s\n", + "\n", + "16:39:47 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "2. How much revenue did NVIDIA generate?\n", + " Cache: DISABLED | Time: 1.439s\n", + "\n", + "16:39:52 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "3. What are NVIDIA's main products?\n", + " Cache: DISABLED | Time: 4.368s\n", + "\n", + "\n", + "================================================================================\n", + "PERFORMANCE SUMMARY\n", + "================================================================================\n", + "Average time - First pass (cache miss): 0.105s\n", + "Average time - Second pass (cache hit): 0.118s\n", + "Average time - Without cache: 2.464s\n", + "\n", + "Speedup with cache: 0.9x faster\n", + " Cache hit rate: 33%\n" + ] + } + ], "source": [ "# Test questions for RAG evaluation\n", "test_questions_rag = [\n", @@ -1371,7 +1379,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 9. Best Practices and Tips\n", + "## 8. Best Practices and Tips\n", "\n", "\n", "### Key Takeaways\n", @@ -1411,16 +1419,38 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 10. Cleanup\n", + "## 9. Cleanup\n", "\n", "Clean up resources when done.\n" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 27, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "16:39:52 httpx INFO HTTP Request: DELETE https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 400 Bad Request\"\n" + ] + }, + { + "ename": "BadRequestErrorResponseContent", + "evalue": "{\"detail\":\"attributes: cannot be blank.\",\"status\":400,\"title\":\"Invalid Request\",\"type\":\"/errors/invalid-data\"}", + "output_type": "error", + "traceback": [ + "\u001b[31m---------------------------------------------------------------------------\u001b[39m", + "\u001b[31mBadRequestErrorResponseContent\u001b[39m Traceback (most recent call last)", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[27]\u001b[39m\u001b[32m, line 2\u001b[39m\n\u001b[32m 1\u001b[39m \u001b[38;5;66;03m# Clear cache contents\u001b[39;00m\n\u001b[32m----> \u001b[39m\u001b[32m2\u001b[39m \u001b[43mcache\u001b[49m\u001b[43m.\u001b[49m\u001b[43mclear\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 3\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33m\"\u001b[39m\u001b[33mCache contents cleared\u001b[39m\u001b[33m\"\u001b[39m)\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/redisvl/extensions/cache/llm/langcache.py:557\u001b[39m, in \u001b[36mLangCacheSemanticCache.clear\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m 552\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mclear\u001b[39m(\u001b[38;5;28mself\u001b[39m) -> \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 553\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Clear the cache of all entries.\u001b[39;00m\n\u001b[32m 554\u001b[39m \n\u001b[32m 555\u001b[39m \u001b[33;03m This is an alias for delete() to match the BaseCache interface.\u001b[39;00m\n\u001b[32m 556\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m557\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mdelete\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/redisvl/extensions/cache/llm/langcache.py:542\u001b[39m, in \u001b[36mLangCacheSemanticCache.delete\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m 536\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mdelete\u001b[39m(\u001b[38;5;28mself\u001b[39m) -> \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 537\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Delete the entire cache.\u001b[39;00m\n\u001b[32m 538\u001b[39m \n\u001b[32m 539\u001b[39m \u001b[33;03m This deletes all entries in the cache by calling delete_query\u001b[39;00m\n\u001b[32m 540\u001b[39m \u001b[33;03m with no attributes.\u001b[39;00m\n\u001b[32m 541\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m542\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_client\u001b[49m\u001b[43m.\u001b[49m\u001b[43mdelete_query\u001b[49m\u001b[43m(\u001b[49m\u001b[43mattributes\u001b[49m\u001b[43m=\u001b[49m\u001b[43m{\u001b[49m\u001b[43m}\u001b[49m\u001b[43m)\u001b[49m\n", + "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/langcache/sdk.py:227\u001b[39m, in \u001b[36mLangCache.delete_query\u001b[39m\u001b[34m(self, attributes, retries, server_url, timeout_ms, http_headers)\u001b[39m\n\u001b[32m 223\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m utils.match_response(http_res, \u001b[33m\"\u001b[39m\u001b[33m400\u001b[39m\u001b[33m\"\u001b[39m, \u001b[33m\"\u001b[39m\u001b[33mapplication/json\u001b[39m\u001b[33m\"\u001b[39m):\n\u001b[32m 224\u001b[39m response_data = unmarshal_json_response(\n\u001b[32m 225\u001b[39m errors.BadRequestErrorResponseContentData, http_res\n\u001b[32m 226\u001b[39m )\n\u001b[32m--> \u001b[39m\u001b[32m227\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m errors.BadRequestErrorResponseContent(response_data, http_res)\n\u001b[32m 228\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m utils.match_response(http_res, \u001b[33m\"\u001b[39m\u001b[33m401\u001b[39m\u001b[33m\"\u001b[39m, \u001b[33m\"\u001b[39m\u001b[33mapplication/json\u001b[39m\u001b[33m\"\u001b[39m):\n\u001b[32m 229\u001b[39m response_data = unmarshal_json_response(\n\u001b[32m 230\u001b[39m errors.AuthenticationErrorResponseContentData, http_res\n\u001b[32m 231\u001b[39m )\n", + "\u001b[31mBadRequestErrorResponseContent\u001b[39m: {\"detail\":\"attributes: cannot be blank.\",\"status\":400,\"title\":\"Invalid Request\",\"type\":\"/errors/invalid-data\"}" + ] + } + ], "source": [ "# Clear cache contents\n", "cache.clear()\n", @@ -1454,11 +1484,16 @@ "- Scale your semantic cache to handle production traffic\n", "\n", "**Resources:**\n", - "- [RedisVL Documentation](https://redis.io/docs/stack/search/redisvl/)\n", - "- [LangCache Sign Up](https://langcache.io/signup)\n", + "- [RedisVL Documentation](https://docs.redisvl.com/en/stable/index.html)\n", + "- [LangCache Sign Up](https://redis.io/langcache/)\n", "- [Redis AI Resources](https://github.com/redis-developer/redis-ai-resources)\n", "- [Semantic Caching Paper](https://arxiv.org/abs/2504.02268)\n" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [] } ], "metadata": { From 5e3e3d62a6ef1c695843772cdd332c1351b03b1d Mon Sep 17 00:00:00 2001 From: Justin Cechmanek Date: Thu, 20 Nov 2025 08:34:01 -0800 Subject: [PATCH 5/7] wip: iterating on langcache notebook --- .../04_langcache_semantic_caching.ipynb | 1172 +++++++++++------ 1 file changed, 760 insertions(+), 412 deletions(-) diff --git a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb index a54b9fea..e0d36101 100644 --- a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb +++ b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb @@ -31,13 +31,11 @@ "In this tutorial, you will:\n", "1. Set up LangCache with Redis Cloud\n", "2. Load and process a knowledge base (PDF documents)\n", - "3. Generate FAQs using the Doc2Cache technique\n", + "3. Generate FAQs using the Doc-to-Cache technique\n", "4. Pre-populate a semantic cache with tagged FAQs\n", "5. Test different cache matching strategies and thresholds\n", - "6. Optimize cache performance using evaluation datasets\n", - "7. Use the `langcache-embed` cross-encoder model\n", - "8. Integrate the cache into a RAG pipeline\n", - "9. Measure performance improvements\n" + "6. Integrate the cache into a RAG pipeline\n", + "7. Measure performance improvements\n" ] }, { @@ -77,8 +75,8 @@ } ], "source": [ - "%pip install -q \"redisvl>=0.11.0\" \"openai>=1.0.0\" \"langchain>=0.3.0\" \"langchain-community\" \"langchain-openai\" \"langcache\"\n", - "%pip install -q \"pypdf\" \"sentence-transformers\" \"redis-retrieval-optimizer>=0.2.0\"" + "%pip install -q \"redisvl>=0.11.0\" \"langcache\" \"sentence-transformers\"\n", + "%pip install -q \"pypdf\" \"openai>=1.0.0\" \"langchain>=0.3.0\" \"langchain-community\" \"langchain-openai\"" ] }, { @@ -92,15 +90,7 @@ "cell_type": "code", "execution_count": 2, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "16:36:26 numexpr.utils INFO NumExpr defaulting to 10 threads.\n" - ] - } - ], + "outputs": [], "source": [ "import os\n", "import time\n", @@ -108,19 +98,16 @@ "from typing import List, Dict, Any\n", "\n", "# RedisVL imports\n", - "from redisvl.extensions.cache.llm import LangCacheSemanticCache\n", - "\n", - "# Optimization\n", - "from redis_retrieval_optimizer.threshold_optimization import CacheThresholdOptimizer" + "from redisvl.extensions.cache.llm import LangCacheSemanticCache" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## 2. LangCache Setup\n", + "## 2. LangCache setup\n", "\n", - "### Sign Up for LangCache\n", + "### Sign up for LangCache\n", "\n", "If you haven't already, sign up for a free Redis Cloud account:\n", "\n", @@ -138,19 +125,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Configure Environment Variables\n" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [], - "source": [ - "import getpass\n", - "\n", - "if \"OPENAI_API_KEY\" not in os.environ:\n", - " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")" + "### Configure Environment Variables\n", + "You'll need the LangCache API Key, Cache ID, URL\n", + "You will also need access to an LLM. In this notebook we'll be using OpenAI" ] }, { @@ -164,14 +141,29 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 3, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "wy4ECQMIVUCcYGbZr_Lg007Cifh4GkgiIRNAf3S4ITMWQ4puuq-OStyjMvH-iD1m0oIB6hg5EVYQye5r1xajEFL7e0AUw5Gn_UEksTQdSm-Hwzu3wXsJJ4emhp8OopEJfHx6JnPlW36LDkCf6ne4Kj8CWiQkphQHqaEeKV9mdgbml-8qOv19AFr0y5vmTtkU_Xt5ByfGMTO-mI9wMKXNLOfwZixM1kiE8KAL_JM7dJN_EHQh\n", + "50eb6a09acf5415d8b68619b1ccffd9a\n", + "https://aws-us-east-1.langcache.redis.io\n" + ] + } + ], "source": [ "langcache_api_key = os.environ.get('LANGCACHE_API_KEY') # found on your cloud console\n", "langcache_id = os.environ.get('LANGCACHE_ID') # found on your cloud console\n", "server_url = \"https://aws-us-east-1.langcache.redis.io\" # found on your cloud console\n", "\n", + "\n", + "print(langcache_api_key)\n", + "print(langcache_id)\n", + "print(server_url)\n", + "\n", "# Create Semantic Cache instance\n", "cache = LangCacheSemanticCache(\n", " server_url=server_url,\n", @@ -182,45 +174,62 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "16:36:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", - "[{'entry_id': '5eb63bbbe01eeed093cb22bb8f5acdc3', 'prompt': 'hello world', 'response': 'hello world from langcache', 'vector_distance': 0.0, 'inserted_at': 0.0, 'updated_at': 0.0}]\n", - "16:36:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:36:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:25:09 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "[]\n", + "18:25:09 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:25:09 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "[{'entry_id': '5eb63bbbe01eeed093cb22bb8f5acdc3', 'prompt': 'hello world', 'response': 'hello world from langcache', 'vector_distance': 0.07242219999999999, 'inserted_at': 0.0, 'updated_at': 0.0}]\n" ] } ], "source": [ - "# Check your cache is workign\n", + "# Check your cache is working\n", "r = cache.check('hello world')\n", "print(r) # should be empty on first run\n", + "\n", "cache.store('hello world', 'hello world from langcache')\n", - "r = cache.check('hi world')\n", - "print(r)" + "result = cache.check('hi world')\n", + "\n", + "print(result)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# RAG with semantic caching\n", + "\n", + "Now that we have a working semantic cache service running and we're connected to it, let's use it in an application.\n", + "\n", + "We'll build a simple Retrieval Augmented Generation (RAG) app using a PDF of NVidia's 2023 10k filing report.\n", + "\n", + "To get the full benefit of semantic caching we'll preload our cache with Frequently Asked Questions (FAQs) generated by an LLM about our PDF." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## 3. Load and Prepare Datasets\n", + "## 3. Generate FAQs Using Doc-to-Cache Technique\n", + "\n", + "The Doc-to-Cache approach uses an LLM to generate frequently asked questions from document chunks. These FAQs are then used to pre-populate the semantic cache with high-quality, factual responses.\n", "\n", "We'll work with three types of data:\n", "1. **Knowledge Base**: PDF document(s) that contain factual information\n", - "2. **FAQs**: Derived from the knowledge base using Doc2Cache technique\n", + "2. **FAQs**: Derived from the knowledge base using Doc-to-Cache technique\n", "3. **Test Dataset**: For evaluating and optimizing cache performance\n" ] }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 5, "metadata": {}, "outputs": [ { @@ -239,22 +248,20 @@ "from langchain_openai import ChatOpenAI\n", "from langchain_core.prompts import PromptTemplate, ChatPromptTemplate\n", "from langchain_core.output_parsers import JsonOutputParser\n", - "from pydantic import BaseModel, Field" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Initialize OpenAI LLM\n" + "\n", + "from pydantic import BaseModel, Field\n", + "import getpass" ] }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 6, "metadata": {}, "outputs": [], "source": [ + "if \"OPENAI_API_KEY\" not in os.environ:\n", + " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")\n", + "\n", "# Initialize OpenAI LLM for FAQ generation and RAG\n", "llm = ChatOpenAI(\n", " model=\"gpt-4o-mini\",\n", @@ -272,7 +279,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 7, "metadata": {}, "outputs": [], "source": [ @@ -283,7 +290,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 8, "metadata": {}, "outputs": [ { @@ -325,18 +332,9 @@ "print(f\"{chunks[10].page_content[:300]}...\")\n" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Generate FAQs Using Doc2Cache Technique\n", - "\n", - "The Doc2Cache approach uses an LLM to generate frequently asked questions from document chunks. These FAQs are then used to pre-populate the semantic cache with high-quality, factual responses.\n" - ] - }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 9, "metadata": {}, "outputs": [], "source": [ @@ -355,7 +353,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 10, "metadata": {}, "outputs": [ { @@ -396,7 +394,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 11, "metadata": {}, "outputs": [ { @@ -405,20 +403,20 @@ "text": [ "Testing FAQ generation on sample chunk...\n", "\n", - "16:36:51 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:25:33 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Generated 5 FAQs:\n", "\n", - "1. Q: What industries are leveraging NVIDIA's GPUs for automation?\n", - " Category: operations\n", + "1. Q: What industries are utilizing NVIDIA's GPUs for automation?\n", + " Category: products\n", " A: A rapidly growing number of enterprises and startups across a broad range of industries, including transportation for autonomous driving, healthcare f...\n", "\n", "2. Q: What was the reason for the termination of the Arm Share Purchase Agreement?\n", - " Category: financial\n", - " A: The Share Purchase Agreement between NVIDIA and SoftBank was terminated due to significant regulatory challenges that prevented the completion of the ...\n", + " Category: operations\n", + " A: The termination of the Arm Share Purchase Agreement was due to significant regulatory challenges that prevented the completion of the transaction, as ...\n", "\n", - "3. Q: What types of products do professional designers create using NVIDIA's technology?\n", - " Category: products\n", - " A: Professional designers use NVIDIA's GPUs and software to create visual effects in movies and to design a variety of products, including cell phones an...\n" + "3. Q: How much did NVIDIA record as an acquisition termination cost in fiscal year 2023?\n", + " Category: financial\n", + " A: NVIDIA recorded an acquisition termination cost of $1.35 billion in fiscal year 2023, reflecting the write-off of the prepayment provided at signing f...\n" ] } ], @@ -436,7 +434,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 12, "metadata": {}, "outputs": [ { @@ -447,42 +445,42 @@ "Generating FAQs from document chunks...\n", "\n", "Processing chunk 1/25...\n", - "16:36:57 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:37:04 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:37:09 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:37:17 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:37:22 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:25:40 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:25:52 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:25:58 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:26:04 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:26:09 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 6/25...\n", - "16:37:28 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:37:30 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:37:36 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:37:41 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:37:48 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:26:18 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:26:23 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:26:30 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:26:42 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:26:49 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 11/25...\n", - "16:37:54 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:38:00 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:38:05 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:38:12 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:38:19 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:26:57 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:27:05 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:27:10 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:27:17 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:27:26 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 16/25...\n", - "16:38:25 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:38:30 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:38:38 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:38:49 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:38:55 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:27:32 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:27:38 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:27:47 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:27:52 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:27:59 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 21/25...\n", - "16:39:02 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:39:07 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:39:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:39:21 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "16:39:28 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:28:07 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:28:13 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:28:22 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:28:31 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:28:39 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "\n", "Generated 112 FAQs total\n", "\n", "Category distribution:\n", - " technology: 31\n", - " operations: 24\n", - " products: 20\n", + " technology: 29\n", + " operations: 23\n", + " products: 23\n", " financial: 19\n", " general: 18\n" ] @@ -491,13 +489,19 @@ "source": [ "# Generate FAQs from all chunks (limited to first 25 for demo purposes)\n", "def extract_faqs_from_chunks(chunks: List[Any], max_chunks: int = 25) -> List[Dict]:\n", - " \"\"\"Extract FAQs from document chunks using LLM.\"\"\"\n", + " \"\"\"Extract FAQs from document chunks using LLM.\n", + " \n", + " chunks: list of document chunks\n", + " max_chunks: maximum number of chunks to process\n", + " \n", + " Returns: A list of question-answer pairs\n", + " \"\"\"\n", " all_faqs = []\n", - " \n", + "\n", " for i, chunk in enumerate(chunks[:max_chunks]):\n", " if i % 5 == 0:\n", " print(f\"Processing chunk {i+1}/{min(len(chunks), max_chunks)}...\", flush=True)\n", - " \n", + "\n", " try:\n", " result = faq_chain.invoke({\"doc_content\": chunk.page_content})\n", " if result and result.get(\"faqs\"):\n", @@ -505,7 +509,7 @@ " except Exception as e:\n", " print(f\" Warning: Skipped chunk {i+1} due to error: {str(e)[:100]}\")\n", " continue\n", - " \n", + "\n", " return all_faqs\n", "\n", "# Extract FAQs\n", @@ -526,7 +530,188 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Create Test/Evaluation Dataset\n", + "## 4. Pre-load semantic cache with FAQs\n", + "\n", + "Now we'll populate the cache instance with our generated FAQs. We'll use the `store()` API with metadata tags for filtering and organization.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Storing FAQs in cache...\n", + "\n", + " Stored 0/112 FAQs...\n", + "18:28:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + " Stored 20/112 FAQs...\n", + "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + " Stored 40/112 FAQs...\n", + "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + " Stored 60/112 FAQs...\n", + "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + " Stored 80/112 FAQs...\n", + "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + " Stored 100/112 FAQs...\n", + "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "\n", + "Stored 112 FAQs in cache\n", + "\n", + "Example cache entries:\n", + "\n", + "1. Key: eb461a36940c04a1d307d33a595188af\n", + " Q: What is the fiscal year end date for NVIDIA Corporation as reported in the Form 10-K?...\n", + "\n", + "2. Key: 0daa2589e67ab291d447f3e103435706\n", + " Q: What is the trading symbol for NVIDIA Corporation's common stock?...\n" + ] + } + ], + "source": [ + "# Store FAQs in cache with metadata tags\n", + "print(\"Storing FAQs in cache...\\n\")\n", + "\n", + "stored_count = 0\n", + "cache_keys = {} # Map questions to their cache keys\n", + "\n", + "for i, faq in enumerate(faqs):\n", + " if i % 20 == 0:\n", + " print(f\" Stored {i}/{len(faqs)} FAQs...\", flush=True)\n", + "\n", + " try:\n", + " # Store with metadata - note that metadata is stored but not used for filtering in basic SemanticCache\n", + " key = cache.store(prompt=faq['question'], response=faq['answer'], metadata={'category': faq['category']})\n", + " cache_keys[faq['question']] = key\n", + " stored_count += 1\n", + " except Exception as e:\n", + " print(f\" Warning: Failed to store FAQ {i+1}: {str(e)[:100]}\")\n", + "\n", + "print(f\"\\nStored {stored_count} FAQs in cache\")\n", + "\n", + "print(f\"\\nExample cache entries:\")\n", + "for i, (q, k) in enumerate(list(cache_keys.items())[:2], 1):\n", + " print(f\"\\n{i}. Key: {k}\")\n", + " print(f\" Q: {q[:150]}...\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 5. Evaluating our semantic cache\n", + "\n", + "\n", + "### Create test/evaluation dataset\n", "\n", "We'll create a test dataset with:\n", "- **Positive examples**: Questions that should match cached FAQs\n", @@ -549,7 +734,7 @@ "\n", "2. What is the trading symbol for NVIDIA Corporation's common stock?...\n", "\n", - "3. Where is NVIDIA Corporation's principal executive office located?...\n" + "3. Where is the principal executive office of NVIDIA Corporation located?...\n" ] } ], @@ -594,218 +779,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 4. Pre-Load Semantic Cache with FAQs\n", - "\n", - "Now we'll populate the cache instance with our generated FAQs. We'll use the `store()` API with metadata tags for filtering and organization.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "16:39:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", - "16:39:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n" - ] - }, - { - "data": { - "text/plain": [ - "'5eb63bbbe01eeed093cb22bb8f5acdc3'" - ] - }, - "execution_count": 16, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Clear any existing cache entries\n", - "r = cache.check('hello world')\n", - "cache.store('hello world', 'hello world from langcache')\n" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Storing FAQs in cache...\n", - "\n", - " Stored 0/112 FAQs...\n", - "16:39:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - " Stored 20/112 FAQs...\n", - "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - " Stored 40/112 FAQs...\n", - "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - " Stored 60/112 FAQs...\n", - "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - " Stored 80/112 FAQs...\n", - "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:38 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:39 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - " Stored 100/112 FAQs...\n", - "16:39:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 201 Created\"\n", - "\n", - "Stored 112 FAQs in cache\n", - "\n", - "Example cache entries:\n", - "\n", - "1. Key: a629e006a52387e1d29243f32555b5b6\n", - " Q: What is the fiscal year end date for NVIDIA Corporation as reported in the Form ...\n", - "\n", - "2. Key: f7e7a16db20d8199b6a95de1129d8b03\n", - " Q: What is the trading symbol for NVIDIA Corporation's common stock?...\n" - ] - } - ], - "source": [ - "# Store FAQs in cache with metadata tags\n", - "print(\"Storing FAQs in cache...\\n\")\n", - "\n", - "stored_count = 0\n", - "cache_keys = {} # Map questions to their cache keys\n", - "\n", - "for i, faq in enumerate(faqs):\n", - " if i % 20 == 0:\n", - " print(f\" Stored {i}/{len(faqs)} FAQs...\", flush=True)\n", - " \n", - " try:\n", - " # Store with metadata - note that metadata is stored but not used for filtering in basic SemanticCache\n", - " # In production, you can use this for analytics and tracking\n", - " key = cache.store(\n", - " prompt=faq['question'],\n", - " response=faq['answer']\n", - " )\n", - " cache_keys[faq['question']] = key\n", - " stored_count += 1\n", - " except Exception as e:\n", - " print(f\" Warning: Failed to store FAQ {i+1}: {str(e)[:100]}\")\n", - "\n", - "print(f\"\\nStored {stored_count} FAQs in cache\")\n", - "print(f\"\\nExample cache entries:\")\n", - "for i, (q, k) in enumerate(list(cache_keys.items())[:2], 1):\n", - " print(f\"\\n{i}. Key: {k}\")\n", - " print(f\" Q: {q[:80]}...\")\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 5. Test Cache Retrieval with Different Strategies\n", + "## 5. Test cache retrieval With different strategies\n", "\n", "Let's test how the cache performs with different types of queries and matching thresholds.\n" ] @@ -814,12 +788,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Test Exact Match Queries\n" + "### Test exact match queries\n" ] }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 16, "metadata": {}, "outputs": [ { @@ -828,20 +802,20 @@ "text": [ "Testing exact match queries:\n", "\n", - "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "1. Cache HIT\n", " Query: What is the fiscal year end date for NVIDIA Corporation as reported in the Form ...\n", " Answer: The fiscal year ended January 29, 2023....\n", "\n", - "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "2. Cache HIT\n", " Query: What is the trading symbol for NVIDIA Corporation's common stock?...\n", " Answer: The trading symbol for NVIDIA Corporation's common stock is NVDA....\n", "\n", - "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "3. Cache HIT\n", - " Query: Where is NVIDIA Corporation's principal executive office located?...\n", - " Answer: NVIDIA Corporation's principal executive office is located at 2788 San Tomas Expressway, Santa Clara...\n", + " Query: Where is the principal executive office of NVIDIA Corporation located?...\n", + " Answer: The principal executive office of NVIDIA Corporation is located at 2788 San Tomas Expressway, Santa ...\n", "\n" ] } @@ -866,12 +840,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Test Semantic Similarity\n" + "### Test semantic similarity" ] }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 17, "metadata": {}, "outputs": [ { @@ -880,17 +854,17 @@ "text": [ "Testing semantic similarity:\n", "\n", - "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", - "1. Cache HIT (distance: 0.0629)\n", + "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "1. Cache HIT (distance: 0.1143)\n", " Query: Tell me about NVIDIA's revenue\n", - " Matched: How much revenue did NVIDIA generate?...\n", - " Answer: As of the latest available data in NVIDIA's 10-K filing for the fiscal year ended January 29, 2023, ...\n", + " Matched: Where can I find NVIDIA's material financial information?...\n", + " Answer: NVIDIA announces material financial information through its investor relations website, press releas...\n", "\n", - "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "2. ✗ Cache MISS\n", " Query: What products does the company make?\n", "\n", - "16:39:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "3. ✗ Cache MISS\n", " Query: How is the company performing financially?\n", "\n" @@ -924,12 +898,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Test Cache with Sample Query\n" + "### Test cache with sample query" ] }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 18, "metadata": {}, "outputs": [ { @@ -937,10 +911,10 @@ "output_type": "stream", "text": [ "Testing query: 'What is NVIDIA's main business?'\n", - "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "Cache HIT\n", - " Distance: 0.070051\n", - " Matched: What are the main business segments reported by NVIDIA?...\n" + " Distance: 0.076104\n", + " Matched: What segments does NVIDIA report its business results in?...\n" ] } ], @@ -964,12 +938,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Test Negative Examples (Should Not Match)\n" + "### Test negative examples (should not match)\n" ] }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 19, "metadata": {}, "outputs": [ { @@ -978,23 +952,23 @@ "text": [ "Testing negative examples (should NOT match):\n", "\n", - "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "1. Correct MISS\n", " Query: What is the weather today?\n", "\n", - "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "2. Correct MISS\n", " Query: How do I cook pasta?\n", "\n", - "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "3. Correct MISS\n", " Query: What is the capital of France?\n", "\n", - "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "4. Correct MISS\n", " Query: Tell me a joke\n", "\n", - "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "5. Correct MISS\n", " Query: What time is it?\n", "\n" @@ -1021,14 +995,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 6. Optimize Cache Threshold\n", + "## 6. Tune cache threshold\n", "\n", - "Using the `CacheThresholdOptimizer`, we can automatically find the optimal distance threshold based on our test dataset.\n" + "Using sample questions, we can find the optimal distance threshold based on our test dataset." ] }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 20, "metadata": {}, "outputs": [ { @@ -1044,6 +1018,7 @@ ], "source": [ "# Create optimization test data\n", + "\n", "# Format: [{\"query\": \"...\", \"query_match\": \"cache_key_or_empty_string\"}, ...]\n", "\n", "optimization_test_data = []\n", @@ -1071,7 +1046,385 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 21, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "18:28:56 numexpr.utils INFO NumExpr defaulting to 10 threads.\n", + "Testing cache performance across different similarity thresholds...\n", + "Evaluation dataset: 10 queries\n", + " - Positive examples (should match): 5\n", + " - Negative examples (should NOT match): 5\n", + "\n", + "====================================================================================================\n", + "\n", + "18:28:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "distance is 0.0 threshold is 0.05\n", + "distance is 0.0 threshold is 0.05\n", + "distance is 0.0 threshold is 0.05\n", + "distance is 0.0 threshold is 0.05\n", + "distance is 0.0 threshold is 0.05\n", + "distance is inf threshold is 0.05\n", + "distance is inf threshold is 0.05\n", + "distance is inf threshold is 0.05\n", + "distance is inf threshold is 0.05\n", + "distance is inf threshold is 0.05\n", + "distance is 0.0 threshold is 0.1\n", + "distance is 0.0 threshold is 0.1\n", + "distance is 0.0 threshold is 0.1\n", + "distance is 0.0 threshold is 0.1\n", + "distance is 0.0 threshold is 0.1\n", + "distance is inf threshold is 0.1\n", + "distance is inf threshold is 0.1\n", + "distance is inf threshold is 0.1\n", + "distance is inf threshold is 0.1\n", + "distance is inf threshold is 0.1\n", + "distance is 0.0 threshold is 0.15\n", + "distance is 0.0 threshold is 0.15\n", + "distance is 0.0 threshold is 0.15\n", + "distance is 0.0 threshold is 0.15\n", + "distance is 0.0 threshold is 0.15\n", + "distance is inf threshold is 0.15\n", + "distance is inf threshold is 0.15\n", + "distance is inf threshold is 0.15\n", + "distance is inf threshold is 0.15\n", + "distance is inf threshold is 0.15\n", + "distance is 0.0 threshold is 0.2\n", + "distance is 0.0 threshold is 0.2\n", + "distance is 0.0 threshold is 0.2\n", + "distance is 0.0 threshold is 0.2\n", + "distance is 0.0 threshold is 0.2\n", + "distance is inf threshold is 0.2\n", + "distance is inf threshold is 0.2\n", + "distance is inf threshold is 0.2\n", + "distance is inf threshold is 0.2\n", + "distance is inf threshold is 0.2\n", + "distance is 0.0 threshold is 0.25\n", + "distance is 0.0 threshold is 0.25\n", + "distance is 0.0 threshold is 0.25\n", + "distance is 0.0 threshold is 0.25\n", + "distance is 0.0 threshold is 0.25\n", + "distance is inf threshold is 0.25\n", + "distance is inf threshold is 0.25\n", + "distance is inf threshold is 0.25\n", + "distance is inf threshold is 0.25\n", + "distance is inf threshold is 0.25\n", + "distance is 0.0 threshold is 0.3\n", + "distance is 0.0 threshold is 0.3\n", + "distance is 0.0 threshold is 0.3\n", + "distance is 0.0 threshold is 0.3\n", + "distance is 0.0 threshold is 0.3\n", + "distance is inf threshold is 0.3\n", + "distance is inf threshold is 0.3\n", + "distance is inf threshold is 0.3\n", + "distance is inf threshold is 0.3\n", + "distance is inf threshold is 0.3\n", + "distance is 0.0 threshold is 0.35\n", + "distance is 0.0 threshold is 0.35\n", + "distance is 0.0 threshold is 0.35\n", + "distance is 0.0 threshold is 0.35\n", + "distance is 0.0 threshold is 0.35\n", + "distance is inf threshold is 0.35\n", + "distance is inf threshold is 0.35\n", + "distance is inf threshold is 0.35\n", + "distance is inf threshold is 0.35\n", + "distance is inf threshold is 0.35\n", + "distance is 0.0 threshold is 0.4\n", + "distance is 0.0 threshold is 0.4\n", + "distance is 0.0 threshold is 0.4\n", + "distance is 0.0 threshold is 0.4\n", + "distance is 0.0 threshold is 0.4\n", + "distance is inf threshold is 0.4\n", + "distance is inf threshold is 0.4\n", + "distance is inf threshold is 0.4\n", + "distance is inf threshold is 0.4\n", + "distance is inf threshold is 0.4\n", + "distance is 0.0 threshold is 0.5\n", + "distance is 0.0 threshold is 0.5\n", + "distance is 0.0 threshold is 0.5\n", + "distance is 0.0 threshold is 0.5\n", + "distance is 0.0 threshold is 0.5\n", + "distance is inf threshold is 0.5\n", + "distance is inf threshold is 0.5\n", + "distance is inf threshold is 0.5\n", + "distance is inf threshold is 0.5\n", + "distance is inf threshold is 0.5\n", + "distance is 0.0 threshold is 0.55\n", + "distance is 0.0 threshold is 0.55\n", + "distance is 0.0 threshold is 0.55\n", + "distance is 0.0 threshold is 0.55\n", + "distance is 0.0 threshold is 0.55\n", + "distance is inf threshold is 0.55\n", + "distance is inf threshold is 0.55\n", + "distance is inf threshold is 0.55\n", + "distance is inf threshold is 0.55\n", + "distance is inf threshold is 0.55\n", + "distance is 0.0 threshold is 0.6\n", + "distance is 0.0 threshold is 0.6\n", + "distance is 0.0 threshold is 0.6\n", + "distance is 0.0 threshold is 0.6\n", + "distance is 0.0 threshold is 0.6\n", + "distance is inf threshold is 0.6\n", + "distance is inf threshold is 0.6\n", + "distance is inf threshold is 0.6\n", + "distance is inf threshold is 0.6\n", + "distance is inf threshold is 0.6\n", + "distance is 0.0 threshold is 0.65\n", + "distance is 0.0 threshold is 0.65\n", + "distance is 0.0 threshold is 0.65\n", + "distance is 0.0 threshold is 0.65\n", + "distance is 0.0 threshold is 0.65\n", + "distance is inf threshold is 0.65\n", + "distance is inf threshold is 0.65\n", + "distance is inf threshold is 0.65\n", + "distance is inf threshold is 0.65\n", + "distance is inf threshold is 0.65\n", + "distance is 0.0 threshold is 0.7\n", + "distance is 0.0 threshold is 0.7\n", + "distance is 0.0 threshold is 0.7\n", + "distance is 0.0 threshold is 0.7\n", + "distance is 0.0 threshold is 0.7\n", + "distance is inf threshold is 0.7\n", + "distance is inf threshold is 0.7\n", + "distance is inf threshold is 0.7\n", + "distance is inf threshold is 0.7\n", + "distance is inf threshold is 0.7\n", + "distance is 0.0 threshold is 0.75\n", + "distance is 0.0 threshold is 0.75\n", + "distance is 0.0 threshold is 0.75\n", + "distance is 0.0 threshold is 0.75\n", + "distance is 0.0 threshold is 0.75\n", + "distance is inf threshold is 0.75\n", + "distance is inf threshold is 0.75\n", + "distance is inf threshold is 0.75\n", + "distance is inf threshold is 0.75\n", + "distance is inf threshold is 0.75\n", + "distance is 0.0 threshold is 0.8\n", + "distance is 0.0 threshold is 0.8\n", + "distance is 0.0 threshold is 0.8\n", + "distance is 0.0 threshold is 0.8\n", + "distance is 0.0 threshold is 0.8\n", + "distance is inf threshold is 0.8\n", + "distance is inf threshold is 0.8\n", + "distance is inf threshold is 0.8\n", + "distance is inf threshold is 0.8\n", + "distance is inf threshold is 0.8\n", + "distance is 0.0 threshold is 0.85\n", + "distance is 0.0 threshold is 0.85\n", + "distance is 0.0 threshold is 0.85\n", + "distance is 0.0 threshold is 0.85\n", + "distance is 0.0 threshold is 0.85\n", + "distance is inf threshold is 0.85\n", + "distance is inf threshold is 0.85\n", + "distance is inf threshold is 0.85\n", + "distance is inf threshold is 0.85\n", + "distance is inf threshold is 0.85\n", + "distance is 0.0 threshold is 0.9\n", + "distance is 0.0 threshold is 0.9\n", + "distance is 0.0 threshold is 0.9\n", + "distance is 0.0 threshold is 0.9\n", + "distance is 0.0 threshold is 0.9\n", + "distance is inf threshold is 0.9\n", + "distance is inf threshold is 0.9\n", + "distance is inf threshold is 0.9\n", + "distance is inf threshold is 0.9\n", + "distance is inf threshold is 0.9\n", + "distance is 0.0 threshold is 0.95\n", + "distance is 0.0 threshold is 0.95\n", + "distance is 0.0 threshold is 0.95\n", + "distance is 0.0 threshold is 0.95\n", + "distance is 0.0 threshold is 0.95\n", + "distance is inf threshold is 0.95\n", + "distance is inf threshold is 0.95\n", + "distance is inf threshold is 0.95\n", + "distance is inf threshold is 0.95\n", + "distance is inf threshold is 0.95\n", + "distance is 0.0 threshold is 1.0\n", + "distance is 0.0 threshold is 1.0\n", + "distance is 0.0 threshold is 1.0\n", + "distance is 0.0 threshold is 1.0\n", + "distance is 0.0 threshold is 1.0\n", + "distance is inf threshold is 1.0\n", + "distance is inf threshold is 1.0\n", + "distance is inf threshold is 1.0\n", + "distance is inf threshold is 1.0\n", + "distance is inf threshold is 1.0\n", + "THRESHOLD OPTIMIZATION RESULTS\n", + "====================================================================================================\n", + "\n", + "Performance Metrics by Threshold:\n", + " Threshold Total Hits Total Misses True Positives False Positives True Negatives False Negatives Precision Recall F1 Score Accuracy\n", + " 0.05 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.10 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.15 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.20 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.25 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.30 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.35 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.40 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.50 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.55 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.60 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.65 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.70 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.75 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.80 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.85 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.90 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.95 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 1.00 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + "\n", + "====================================================================================================\n", + "OPTIMAL THRESHOLD: 0.05\n", + " F1 Score: 1.000\n", + " Precision: 1.000\n", + " Recall: 1.000\n", + " Accuracy: 1.000\n", + "====================================================================================================\n", + "\n", + "Detailed breakdown at optimal threshold (0.05):\n", + "\n", + "Query: What is the fiscal year end date for NVIDIA Corporation as r | Distance: 0.0000 | ✓ TP (True Positive)\n", + "Query: What is the trading symbol for NVIDIA Corporation's common s | Distance: 0.0000 | ✓ TP (True Positive)\n", + "Query: Where is the principal executive office of NVIDIA Corporatio | Distance: 0.0000 | ✓ TP (True Positive)\n", + "Query: Is NVIDIA Corporation considered a well-known seasoned issue | Distance: 0.0000 | ✓ TP (True Positive)\n", + "Query: On which exchange is NVIDIA Corporation's common stock regis | Distance: 0.0000 | ✓ TP (True Positive)\n", + "Query: What is the weather today? | Distance: inf | ✓ TN (True Negative)\n", + "Query: How do I cook pasta? | Distance: inf | ✓ TN (True Negative)\n", + "Query: What is the capital of France? | Distance: inf | ✓ TN (True Negative)\n", + "Query: Tell me a joke | Distance: inf | ✓ TN (True Negative)\n", + "Query: What time is it? | Distance: inf | ✓ TN (True Negative)\n" + ] + } + ], + "source": [ + "# Test a range of different cache similarity thresholds\n", + "import pandas as pd\n", + "\n", + "# Define threshold ranges to test\n", + "thresholds_to_test = [0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95, 1.00]\n", + "\n", + "print(\"Testing cache performance across different similarity thresholds...\")\n", + "print(f\"Evaluation dataset: {len(optimization_test_data)} queries\")\n", + "print(f\" - Positive examples (should match): {sum(1 for x in optimization_test_data if x['query_match'])}\")\n", + "print(f\" - Negative examples (should NOT match): {sum(1 for x in optimization_test_data if not x['query_match'])}\")\n", + "print(\"\\n\" + \"=\"*100 + \"\\n\")\n", + "\n", + "# Store results for all queries to reuse across thresholds\n", + "query_results = []\n", + "for test_case in optimization_test_data:\n", + " result = cache.check(prompt=test_case['query'], return_fields=[\"prompt\", \"response\", \"vector_distance\", \"entry_id\"])\n", + " \n", + " query_results.append({\n", + " 'query': test_case['query'],\n", + " 'expected_match': bool(test_case['query_match']),\n", + " 'expected_key': test_case['query_match'],\n", + " 'cache_result': result[0] if result else None,\n", + " 'distance': result[0].get('vector_distance') if result else float('inf')\n", + " })\n", + "\n", + "# Evaluate each threshold\n", + "results = []\n", + "\n", + "for threshold in thresholds_to_test:\n", + " true_positives = 0\n", + " false_positives = 0\n", + " true_negatives = 0\n", + " false_negatives = 0\n", + "\n", + " for query_data in query_results:\n", + " # Determine if this would be a cache hit at this threshold\n", + " is_cache_hit = query_data['distance'] < threshold\n", + " print('distance is ', query_data['distance'], 'threshold is ', threshold)\n", + " should_match = query_data['expected_match']\n", + "\n", + " if is_cache_hit and should_match:\n", + " true_positives += 1\n", + " elif is_cache_hit and not should_match:\n", + " false_positives += 1\n", + " elif not is_cache_hit and not should_match:\n", + " true_negatives += 1\n", + " elif not is_cache_hit and should_match:\n", + " false_negatives += 1\n", + "\n", + " # Calculate metrics\n", + " total_hits = true_positives + false_positives\n", + " total_misses = true_negatives + false_negatives\n", + "\n", + " precision = true_positives / (true_positives + false_positives) if (true_positives + false_positives) > 0 else 0\n", + " recall = true_positives / (true_positives + false_negatives) if (true_positives + false_negatives) > 0 else 0\n", + " f1_score = 2 * (precision * recall) / (precision + recall) if (precision + recall) > 0 else 0\n", + " accuracy = (true_positives + true_negatives) / len(optimization_test_data)\n", + "\n", + " results.append({\n", + " 'Threshold': threshold,\n", + " 'Total Hits': total_hits,\n", + " 'Total Misses': total_misses,\n", + " 'True Positives': true_positives,\n", + " 'False Positives': false_positives,\n", + " 'True Negatives': true_negatives,\n", + " 'False Negatives': false_negatives,\n", + " 'Precision': precision,\n", + " 'Recall': recall,\n", + " 'F1 Score': f1_score,\n", + " 'Accuracy': accuracy\n", + " })\n", + "\n", + "# Display results in a formatted table\n", + "df_results = pd.DataFrame(results)\n", + "\n", + "print(\"THRESHOLD OPTIMIZATION RESULTS\")\n", + "print(\"=\"*100)\n", + "print(\"\\nPerformance Metrics by Threshold:\")\n", + "print(df_results.to_string(index=False))\n", + "\n", + "# Find optimal threshold based on F1 score\n", + "optimal_idx = df_results['F1 Score'].idxmax()\n", + "optimal_threshold = df_results.loc[optimal_idx, 'Threshold']\n", + "optimal_f1 = df_results.loc[optimal_idx, 'F1 Score']\n", + "\n", + "print(\"\\n\" + \"=\"*100)\n", + "print(f\"OPTIMAL THRESHOLD: {optimal_threshold}\")\n", + "print(f\" F1 Score: {optimal_f1:.3f}\")\n", + "print(f\" Precision: {df_results.loc[optimal_idx, 'Precision']:.3f}\")\n", + "print(f\" Recall: {df_results.loc[optimal_idx, 'Recall']:.3f}\")\n", + "print(f\" Accuracy: {df_results.loc[optimal_idx, 'Accuracy']:.3f}\")\n", + "print(\"=\"*100)\n", + "\n", + "# Show detailed breakdown for optimal threshold\n", + "print(f\"\\nDetailed breakdown at optimal threshold ({optimal_threshold}):\\n\")\n", + "for query_data in query_results:\n", + " is_cache_hit = query_data['distance'] < optimal_threshold\n", + " should_match = query_data['expected_match']\n", + "\n", + " status = \"\"\n", + " if is_cache_hit and should_match:\n", + " status = \"✓ TP (True Positive)\"\n", + " elif is_cache_hit and not should_match:\n", + " status = \"✗ FP (False Positive)\"\n", + " elif not is_cache_hit and not should_match:\n", + " status = \"✓ TN (True Negative)\"\n", + " elif not is_cache_hit and should_match:\n", + " status = \"✗ FN (False Negative)\"\n", + "\n", + " print(f\"Query: {query_data['query'][:60]:60s} | Distance: {query_data['distance']:.4f} | {status}\")" + ] + }, + { + "cell_type": "code", + "execution_count": 22, "metadata": {}, "outputs": [ { @@ -1081,23 +1434,23 @@ "\n", "Re-testing negative examples with optimized threshold:\n", "\n", - "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "1. MISS (correct)\n", " Query: What is the weather today?\n", "\n", - "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "2. MISS (correct)\n", " Query: How do I cook pasta?\n", "\n", - "16:39:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "3. MISS (correct)\n", " Query: What is the capital of France?\n", "\n", - "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "4. MISS (correct)\n", " Query: Tell me a joke\n", "\n", - "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "5. MISS (correct)\n", " Query: What time is it?\n", "\n" @@ -1112,7 +1465,7 @@ " result = cache.check(prompt=test_case['query'], return_fields=[\"prompt\", \"vector_distance\"])\n", "\n", " if result:\n", - " print(f\"{i}. ⚠️ HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", + " print(f\"{i}. HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", " print(f\" Query: {test_case['query']}\")\n", " print(f\" Matched: {result[0]['prompt'][:80]}...\\n\")\n", " else:\n", @@ -1124,21 +1477,21 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## 7. RAG Pipeline Integration\n", + "## 7. RAG pipeline integration\n", "\n", - "Now let's integrate the semantic cache into a complete RAG pipeline and measure the performance improvements.\n" + "Now let's integrate the semantic cache into a complete RAG pipeline and measure the performance improvements." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Build Simple RAG Chain\n" + "### Build a simple RAG chain\n" ] }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 23, "metadata": {}, "outputs": [ { @@ -1166,12 +1519,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Create Cached RAG Function\n" + "### Create cached RAG function\n" ] }, { "cell_type": "code", - "execution_count": 25, + "execution_count": 24, "metadata": {}, "outputs": [ { @@ -1187,7 +1540,7 @@ " \"\"\"\n", " Process a question through RAG pipeline with optional semantic caching.\n", " \n", - " Returns: (answer, cache_hit, response_time)\n", + " Returns: A tuple of (answer, cache_hit, response_time)\n", " \"\"\"\n", " start_time = time.time()\n", " cache_hit = False\n", @@ -1220,12 +1573,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Performance Comparison: With vs Without Cache\n" + "### Performance comparison: with vs without cache\n" ] }, { "cell_type": "code", - "execution_count": 26, + "execution_count": 25, "metadata": {}, "outputs": [ { @@ -1239,68 +1592,64 @@ "\n", "[FIRST PASS - Populating Cache]\n", "\n", - "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "1. What is NVIDIA's primary business?\n", - " Cache: HIT | Time: 0.109s\n", - " Answer: NVIDIA reports its business results in two segments: the Compute & Networking segment, which include...\n", + " Cache: HIT | Time: 0.118s\n", + " Answer: NVIDIA specializes in four large markets: Data Center, Gaming, Professional Visualization, and Autom...\n", "\n", - "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "2. How much revenue did NVIDIA generate?\n", - " Cache: HIT | Time: 0.103s\n", - " Answer: As of the latest available data in NVIDIA's 10-K filing for the fiscal year ended January 29, 2023, ...\n", + " Cache: HIT | Time: 0.124s\n", + " Answer: NVIDIA reports its business results in two segments: the Compute & Networking segment and the Data C...\n", "\n", - "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "3. What are NVIDIA's main products?\n", - " Cache: HIT | Time: 0.104s\n", - " Answer: NVIDIA sells a range of products primarily in the following categories:\n", - "\n", - "1. **Graphics Processing Un...\n", + " Cache: HIT | Time: 0.118s\n", + " Answer: NVIDIA sells its products not only to customers in its partner network but also directly to cloud se...\n", "\n", "\n", "[SECOND PASS - Cache Hits with Paraphrased Questions]\n", "\n", - "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "1. What does NVIDIA do as a business?\n", - " Cache: HIT ✓ | Time: 0.128s\n", - " Answer: NVIDIA's business has evolved from a primary focus on gaming products to broader markets, transition...\n", + " Cache: HIT ✓ | Time: 0.121s\n", + " Answer: NVIDIA reports its business results in two segments: the Compute & Networking segment and the Data C...\n", "\n", - "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "2. Can you tell me NVIDIA's revenue figures?\n", - " Cache: HIT ✓ | Time: 0.127s\n", - " Answer: As of the latest available data in NVIDIA's 10-K filing for the fiscal year ended January 29, 2023, ...\n", + " Cache: HIT ✓ | Time: 0.119s\n", + " Answer: NVIDIA announces material financial information through its investor relations website, press releas...\n", "\n", - "16:39:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries/search \"HTTP/1.1 200 OK\"\n", + "18:28:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "3. What products does NVIDIA sell?\n", - " Cache: HIT ✓ | Time: 0.099s\n", - " Answer: NVIDIA sells a range of products primarily in the following categories:\n", - "\n", - "1. **Graphics Processing Un...\n", + " Cache: HIT ✓ | Time: 0.127s\n", + " Answer: NVIDIA reports its business results in two segments: the Compute & Networking segment and the Data C...\n", "\n", "\n", "[THIRD PASS - Without Cache (Baseline)]\n", "\n", - "16:39:46 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:29:00 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "1. What is NVIDIA's primary business?\n", - " Cache: DISABLED | Time: 1.584s\n", + " Cache: DISABLED | Time: 1.675s\n", "\n", - "16:39:47 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:29:02 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "2. How much revenue did NVIDIA generate?\n", - " Cache: DISABLED | Time: 1.439s\n", + " Cache: DISABLED | Time: 1.554s\n", "\n", - "16:39:52 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "18:29:07 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "3. What are NVIDIA's main products?\n", - " Cache: DISABLED | Time: 4.368s\n", + " Cache: DISABLED | Time: 4.812s\n", "\n", "\n", "================================================================================\n", "PERFORMANCE SUMMARY\n", "================================================================================\n", - "Average time - First pass (cache miss): 0.105s\n", - "Average time - Second pass (cache hit): 0.118s\n", - "Average time - Without cache: 2.464s\n", + "Average time - First pass (cache miss): 0.120s\n", + "Average time - Second pass (cache hit): 0.122s\n", + "Average time - Without cache: 2.681s\n", "\n", - "Speedup with cache: 0.9x faster\n", - " Cache hit rate: 33%\n" + "Speedup with cache: 1.0x faster\n", + " Cache hit rate: 0%\n" ] } ], @@ -1381,11 +1730,10 @@ "source": [ "## 8. Best Practices and Tips\n", "\n", - "\n", "### Key Takeaways\n", "\n", "1. **Threshold Optimization**: Start conservative (0.10-0.15) and optimize based on real usage data\n", - "2. **Doc2Cache**: Pre-populate your cache with high-quality FAQs for immediate benefits\n", + "2. **Doc-to-Cache**: Pre-populate your cache with high-quality FAQs for immediate benefits\n", "3. **Monitoring**: Track cache hit rates and adjust thresholds as user patterns emerge\n", "4. **Model Selection**: The `langcache-embed-v1` model is specifically optimized for caching tasks\n", "5. **Cost-Performance Balance**: Even a 50% cache hit rate provides significant cost savings\n", @@ -1408,7 +1756,7 @@ "\n", "### Performance Tips\n", "\n", - "1. **Batch Loading**: Pre-populate cache with Doc2Cache for immediate value\n", + "1. **Batch Loading**: Pre-populate cache with Doc-to-Cache for immediate value\n", "2. **Monitor Hit Rates**: Track and adjust thresholds based on production metrics\n", "3. **A/B Testing**: Test different thresholds with a subset of traffic\n", "4. **Cache Warming**: Regularly update cache with trending topics\n", @@ -1426,14 +1774,14 @@ }, { "cell_type": "code", - "execution_count": 27, + "execution_count": 26, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "16:39:52 httpx INFO HTTP Request: DELETE https://aws-us-east-1.langcache.redis.io/v1/caches/56f7ba9bee374701a1253f21cd1ac35e/entries \"HTTP/1.1 400 Bad Request\"\n" + "18:29:07 httpx INFO HTTP Request: DELETE https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 400 Bad Request\"\n" ] }, { @@ -1443,7 +1791,7 @@ "traceback": [ "\u001b[31m---------------------------------------------------------------------------\u001b[39m", "\u001b[31mBadRequestErrorResponseContent\u001b[39m Traceback (most recent call last)", - "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[27]\u001b[39m\u001b[32m, line 2\u001b[39m\n\u001b[32m 1\u001b[39m \u001b[38;5;66;03m# Clear cache contents\u001b[39;00m\n\u001b[32m----> \u001b[39m\u001b[32m2\u001b[39m \u001b[43mcache\u001b[49m\u001b[43m.\u001b[49m\u001b[43mclear\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 3\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33m\"\u001b[39m\u001b[33mCache contents cleared\u001b[39m\u001b[33m\"\u001b[39m)\n", + "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[26]\u001b[39m\u001b[32m, line 2\u001b[39m\n\u001b[32m 1\u001b[39m \u001b[38;5;66;03m# Clear cache contents\u001b[39;00m\n\u001b[32m----> \u001b[39m\u001b[32m2\u001b[39m \u001b[43mcache\u001b[49m\u001b[43m.\u001b[49m\u001b[43mclear\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 3\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33m\"\u001b[39m\u001b[33mCache contents cleared\u001b[39m\u001b[33m\"\u001b[39m)\n", "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/redisvl/extensions/cache/llm/langcache.py:557\u001b[39m, in \u001b[36mLangCacheSemanticCache.clear\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m 552\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mclear\u001b[39m(\u001b[38;5;28mself\u001b[39m) -> \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 553\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Clear the cache of all entries.\u001b[39;00m\n\u001b[32m 554\u001b[39m \n\u001b[32m 555\u001b[39m \u001b[33;03m This is an alias for delete() to match the BaseCache interface.\u001b[39;00m\n\u001b[32m 556\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m557\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mdelete\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n", "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/redisvl/extensions/cache/llm/langcache.py:542\u001b[39m, in \u001b[36mLangCacheSemanticCache.delete\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m 536\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mdelete\u001b[39m(\u001b[38;5;28mself\u001b[39m) -> \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 537\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Delete the entire cache.\u001b[39;00m\n\u001b[32m 538\u001b[39m \n\u001b[32m 539\u001b[39m \u001b[33;03m This deletes all entries in the cache by calling delete_query\u001b[39;00m\n\u001b[32m 540\u001b[39m \u001b[33;03m with no attributes.\u001b[39;00m\n\u001b[32m 541\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m542\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_client\u001b[49m\u001b[43m.\u001b[49m\u001b[43mdelete_query\u001b[49m\u001b[43m(\u001b[49m\u001b[43mattributes\u001b[49m\u001b[43m=\u001b[49m\u001b[43m{\u001b[49m\u001b[43m}\u001b[49m\u001b[43m)\u001b[49m\n", "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/langcache/sdk.py:227\u001b[39m, in \u001b[36mLangCache.delete_query\u001b[39m\u001b[34m(self, attributes, retries, server_url, timeout_ms, http_headers)\u001b[39m\n\u001b[32m 223\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m utils.match_response(http_res, \u001b[33m\"\u001b[39m\u001b[33m400\u001b[39m\u001b[33m\"\u001b[39m, \u001b[33m\"\u001b[39m\u001b[33mapplication/json\u001b[39m\u001b[33m\"\u001b[39m):\n\u001b[32m 224\u001b[39m response_data = unmarshal_json_response(\n\u001b[32m 225\u001b[39m errors.BadRequestErrorResponseContentData, http_res\n\u001b[32m 226\u001b[39m )\n\u001b[32m--> \u001b[39m\u001b[32m227\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m errors.BadRequestErrorResponseContent(response_data, http_res)\n\u001b[32m 228\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m utils.match_response(http_res, \u001b[33m\"\u001b[39m\u001b[33m401\u001b[39m\u001b[33m\"\u001b[39m, \u001b[33m\"\u001b[39m\u001b[33mapplication/json\u001b[39m\u001b[33m\"\u001b[39m):\n\u001b[32m 229\u001b[39m response_data = unmarshal_json_response(\n\u001b[32m 230\u001b[39m errors.AuthenticationErrorResponseContentData, http_res\n\u001b[32m 231\u001b[39m )\n", @@ -1468,7 +1816,7 @@ "**What You've Learned:**\n", "- ✅ Set up and configure LangCache with Redis Cloud\n", "- ✅ Load and process PDF documents into knowledge bases\n", - "- ✅ Generate FAQs using the Doc2Cache technique with LLMs\n", + "- ✅ Generate FAQs using the Doc-to-Cache technique with LLMs\n", "- ✅ Pre-populate a semantic cache with tagged entries\n", "- ✅ Test different cache matching strategies and thresholds\n", "- ✅ Optimize cache performance using test datasets\n", From 039144265fdad0f4038bbc2fcc9a1a4b58373719 Mon Sep 17 00:00:00 2001 From: Justin Cechmanek Date: Thu, 20 Nov 2025 10:21:17 -0800 Subject: [PATCH 6/7] final pass before workshop --- .../04_langcache_semantic_caching.ipynb | 1216 ++++++----------- 1 file changed, 403 insertions(+), 813 deletions(-) diff --git a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb index e0d36101..b3934bc5 100644 --- a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb +++ b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb @@ -56,17 +56,13 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "\n", - "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.3\u001b[0m\n", - "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", - "Note: you may need to restart the kernel to use updated packages.\n", "\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.3\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", @@ -88,7 +84,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -141,7 +137,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -174,17 +170,17 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "18:25:09 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "[]\n", - "18:25:09 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:25:09 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:15:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "[{'entry_id': '5eb63bbbe01eeed093cb22bb8f5acdc3', 'prompt': 'hello world', 'response': 'hello world from langcache', 'vector_distance': 0.0, 'inserted_at': 0.0, 'updated_at': 0.0}]\n", + "10:15:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:15:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "[{'entry_id': '5eb63bbbe01eeed093cb22bb8f5acdc3', 'prompt': 'hello world', 'response': 'hello world from langcache', 'vector_distance': 0.07242219999999999, 'inserted_at': 0.0, 'updated_at': 0.0}]\n" ] } @@ -229,7 +225,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -255,7 +251,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -279,7 +275,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -290,7 +286,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -334,7 +330,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -353,7 +349,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -394,7 +390,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -403,18 +399,18 @@ "text": [ "Testing FAQ generation on sample chunk...\n", "\n", - "18:25:33 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:15:49 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Generated 5 FAQs:\n", "\n", "1. Q: What industries are utilizing NVIDIA's GPUs for automation?\n", " Category: products\n", - " A: A rapidly growing number of enterprises and startups across a broad range of industries, including transportation for autonomous driving, healthcare f...\n", + " A: A rapidly growing number of enterprises and startups across a broad range of industries are using NVIDIA's GPUs and software to bring automation to th...\n", "\n", "2. Q: What was the reason for the termination of the Arm Share Purchase Agreement?\n", " Category: operations\n", - " A: The termination of the Arm Share Purchase Agreement was due to significant regulatory challenges that prevented the completion of the transaction, as ...\n", + " A: The Arm Share Purchase Agreement was terminated due to significant regulatory challenges that prevented the completion of the transaction between NVID...\n", "\n", - "3. Q: How much did NVIDIA record as an acquisition termination cost in fiscal year 2023?\n", + "3. Q: What acquisition termination cost did NVIDIA record in fiscal year 2023?\n", " Category: financial\n", " A: NVIDIA recorded an acquisition termination cost of $1.35 billion in fiscal year 2023, reflecting the write-off of the prepayment provided at signing f...\n" ] @@ -434,7 +430,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -445,44 +441,44 @@ "Generating FAQs from document chunks...\n", "\n", "Processing chunk 1/25...\n", - "18:25:40 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:25:52 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:25:58 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:26:04 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:26:09 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:15:57 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:16:06 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:16:13 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:16:26 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:16:31 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 6/25...\n", - "18:26:18 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:26:23 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:26:30 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:26:42 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:26:49 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:16:39 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:16:42 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:16:53 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:17:03 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:17:10 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 11/25...\n", - "18:26:57 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:27:05 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:27:10 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:27:17 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:27:26 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:17:19 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:17:31 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:17:37 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:17:46 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:17:54 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 16/25...\n", - "18:27:32 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:27:38 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:27:47 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:27:52 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:27:59 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:18:03 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:18:11 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:18:20 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:18:27 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:18:36 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 21/25...\n", - "18:28:07 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:28:13 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:28:22 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:28:31 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "18:28:39 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:18:46 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:18:53 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:19:03 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:19:11 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:19:19 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "\n", - "Generated 112 FAQs total\n", + "Generated 113 FAQs total\n", "\n", "Category distribution:\n", - " technology: 29\n", - " operations: 23\n", - " products: 23\n", - " financial: 19\n", - " general: 18\n" + " operations: 28\n", + " technology: 28\n", + " products: 24\n", + " financial: 18\n", + " general: 15\n" ] } ], @@ -537,7 +533,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -546,131 +542,132 @@ "text": [ "Storing FAQs in cache...\n", "\n", - " Stored 0/112 FAQs...\n", - "18:28:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:40 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:41 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:42 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - " Stored 20/112 FAQs...\n", - "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:43 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:44 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - " Stored 40/112 FAQs...\n", - "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:46 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - " Stored 60/112 FAQs...\n", - "18:28:47 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:48 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:49 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - " Stored 80/112 FAQs...\n", - "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:50 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:51 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:52 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - " Stored 100/112 FAQs...\n", - "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:53 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + " Stored 0/113 FAQs...\n", + "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + " Stored 20/113 FAQs...\n", + "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + " Stored 40/113 FAQs...\n", + "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + " Stored 60/113 FAQs...\n", + "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + " Stored 80/113 FAQs...\n", + "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + " Stored 100/113 FAQs...\n", + "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", "\n", - "Stored 112 FAQs in cache\n", + "Stored 113 FAQs in cache\n", "\n", "Example cache entries:\n", "\n", - "1. Key: eb461a36940c04a1d307d33a595188af\n", - " Q: What is the fiscal year end date for NVIDIA Corporation as reported in the Form 10-K?...\n", + "1. Key: 7ca731f61f74432f83c61549a627b08a\n", + " Q: What is the fiscal year end date for NVIDIA Corporation as reported in the 10-K?...\n", "\n", "2. Key: 0daa2589e67ab291d447f3e103435706\n", " Q: What is the trading symbol for NVIDIA Corporation's common stock?...\n" @@ -709,8 +706,13 @@ "metadata": {}, "source": [ "## 5. Evaluating our semantic cache\n", + "Now that we have a semantic cache populated with question answer pairs we can evaluate its effectiveness.\n", "\n", + "Unlike standard caching that uses exact key:value look ups, semantic caches relies on the notion of semantic embedding similarity.\n", + "The benefits of semantic matching are that similar questions such as, \"who is the king of England?\", and, \"who is the monarch of Britain?\" can be matched together.\n", + "This flexibility comes at the cost of occasional mismatches. A question like, \"who is the queen of England?\" is also similar and likely to match.\n", "\n", + "Let's create a dataset of test questions to see which match and which don't to evaluate our cache hit rate, and our accuracy.\n", "### Create test/evaluation dataset\n", "\n", "We'll create a test dataset with:\n", @@ -721,35 +723,7 @@ }, { "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Sample FAQs for testing:\n", - "\n", - "1. What is the fiscal year end date for NVIDIA Corporation as reported in the Form 10-K?...\n", - "\n", - "2. What is the trading symbol for NVIDIA Corporation's common stock?...\n", - "\n", - "3. Where is the principal executive office of NVIDIA Corporation located?...\n" - ] - } - ], - "source": [ - "# Select representative FAQs for test set\n", - "sample_faqs = faqs[:10] # Take first 10 FAQs\n", - "\n", - "print(\"Sample FAQs for testing:\")\n", - "for i, faq in enumerate(sample_faqs[:3], 1):\n", - " print(f\"\\n{i}. {faq['question'][:100]}...\")\n" - ] - }, - { - "cell_type": "code", - "execution_count": 15, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -757,95 +731,76 @@ "output_type": "stream", "text": [ "Test dataset created\n", - " Negative examples: 5\n" + " Positive examples: 5\n", + " Negative examples: 5\n", + " Edge cases: 5\n" ] } ], "source": [ - "# Create test dataset with negative examples (off-topic questions)\n", + "# Create test dataset with positive examples (should match NVIDIA FAQs). We'll take the first 5 from our generated FAQs and modify them slightly.\n", + "positive_examples = [\n", + "{'query': \"What's the fiscal year end for NVIDIA Corporation?\",\n", + " 'expected_answer': 'The fiscal year ended January 29, 2023.',\n", + " 'category': 'general',\n", + " 'expected_match': True} ,\n", + "{'query': \"What is the trading symbol of NVIDIA Corporation's common stock in the market?\",\n", + " 'expected_answer': \"The trading symbol for NVIDIA Corporation's common stock is NVDA.\",\n", + " 'category': 'financial' ,\n", + " 'expected_match': True} ,\n", + "{'query': 'Where is the location of the executive office?',\n", + " 'expected_answer': 'The principal executive office of NVIDIA Corporation is located at 2788 San Tomas Expressway, Santa Clara, California 95051.',\n", + " 'category': 'operations' ,\n", + " 'expected_match': True} ,\n", + "{'query': 'Does the SEC consider NVIDIA Corporation a well-known seasoned issuer?',\n", + " 'expected_answer': 'No, NVIDIA Corporation is not considered a well-known seasoned issuer as indicated by the check mark in the document.',\n", + " 'category': 'financial' ,\n", + " 'expected_match': True} ,\n", + "{'query': \"In what exchange platform is NVIDIA's stock traded in?\",\n", + " 'expected_answer': \"NVIDIA Corporation's common stock is registered on The Nasdaq Global Select Market.\",\n", + " 'category': 'financial' ,\n", + " 'expected_match': True} ,\n", + "]\n", + "\n", + "# Create test dataset with negative examples\n", "negative_examples = [\n", - " {\"query\": \"What is the weather today?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", - " {\"query\": \"How do I cook pasta?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", - " {\"query\": \"What is the capital of France?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", - " {\"query\": \"Tell me a joke\", \"expected_match\": False, \"category\": \"off-topic\"},\n", + " {\"query\": \"What is NVIDIA's total revenue for the last 5 years?\", \"expected_match\": False, \"category\": \"financial\"},\n", + " {\"query\": \"What is Jensen Huang's net worth?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", + " {\"query\": \"What games run best on the RTX 4090? NVIDIA GPU?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", " {\"query\": \"What time is it?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", + " {\"query\": \"What's the fiscal year end for ARM Corporation?\", \"expected_match\": False, \"category\": \"general\"},\n", + "]\n", + "\n", + "# Create test dataset with edge cases (slightly different phrasings)\n", + "edge_cases = [\n", + " {\"query\": \"What's the fiscal year end for Microsoft Corporation?\", \"expected_match\": False, \"category\": \"general\"},\n", + " {\"query\": \"What's the location of the manufacturing plant for NVIDIA?\", \"expected_match\": False, \"category\": \"general\"},\n", + " {\"query\": \"What date does NVIDIA use as it's year end for acounting purposes?\", \"expected_match\": True, \"category\":\"general\"},\n", + " {\"query\": \"Where are the locations of each office of NVIDIA Corporation?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", + " {\"query\": \"What is the trading symbold of NVIDIA Corporation on the Japan exchange?\", \"expected_match\": False, \"category\": \"general\"},\n", "]\n", "\n", "print(f\"Test dataset created\")\n", - "print(f\" Negative examples: {len(negative_examples)}\")\n" + "print(f\" Positive examples: {len(positive_examples)}\")\n", + "print(f\" Negative examples: {len(negative_examples)}\")\n", + "print(f\" Edge cases: {len(edge_cases)}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## 5. Test cache retrieval With different strategies\n", + "### Test semantic similarity\n", + "Let's test how the cache performs with different types of queries and matching thresholds.\n", "\n", - "Let's test how the cache performs with different types of queries and matching thresholds.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Test exact match queries\n" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Testing exact match queries:\n", - "\n", - "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "1. Cache HIT\n", - " Query: What is the fiscal year end date for NVIDIA Corporation as reported in the Form ...\n", - " Answer: The fiscal year ended January 29, 2023....\n", - "\n", - "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "2. Cache HIT\n", - " Query: What is the trading symbol for NVIDIA Corporation's common stock?...\n", - " Answer: The trading symbol for NVIDIA Corporation's common stock is NVDA....\n", - "\n", - "18:28:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "3. Cache HIT\n", - " Query: Where is the principal executive office of NVIDIA Corporation located?...\n", - " Answer: The principal executive office of NVIDIA Corporation is located at 2788 San Tomas Expressway, Santa ...\n", - "\n" - ] - } - ], - "source": [ - "# Test with exact questions from cache\n", - "print(\"Testing exact match queries:\\n\")\n", - "\n", - "for i, faq in enumerate(faqs[:3], 1):\n", - " result = cache.check(prompt=faq['question'])\n", - " \n", - " if result:\n", - " print(f\"{i}. Cache HIT\")\n", - " print(f\" Query: {faq['question'][:80]}...\")\n", - " print(f\" Answer: {result[0]['response'][:100]}...\\n\")\n", - " else:\n", - " print(f\"{i}. ✗ Cache MISS\")\n", - " print(f\" Query: {faq['question'][:80]}...\\n\")\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Test semantic similarity" + "We'll run through our 15 sample questions and track which ones get a hit and which ones don't. We'll also track if they should have hit.\n", + "\n", + "This will give us a baseline of our cache performance." ] }, { "cell_type": "code", - "execution_count": 17, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -854,20 +809,62 @@ "text": [ "Testing semantic similarity:\n", "\n", - "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "1. Cache HIT (distance: 0.1143)\n", - " Query: Tell me about NVIDIA's revenue\n", - " Matched: Where can I find NVIDIA's material financial information?...\n", - " Answer: NVIDIA announces material financial information through its investor relations website, press releas...\n", - "\n", - "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "2. ✗ Cache MISS\n", - " Query: What products does the company make?\n", + "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "0. Cache HIT (distance: 0.0711)\n", + " Original query: What's the fiscal year end for NVIDIA Corporation?\n", + " Matched: What is the fiscal year end date for NVIDIA Corporation as reported in the 10-K?...\n", + "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "1. Cache HIT (distance: 0.0174)\n", + " Original query: What is the trading symbol of NVIDIA Corporation's common stock in the market?\n", + " Matched: What is the trading symbol for NVIDIA Corporation's common stock?...\n", + "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "2. Cache HIT (distance: 0.1340)\n", + " Original query: Where is the location of the executive office?\n", + " Matched: Where is NVIDIA Corporation's principal executive office located?...\n", + "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "3. Cache HIT (distance: 0.0182)\n", + " Original query: Does the SEC consider NVIDIA Corporation a well-known seasoned issuer?\n", + " Matched: Is NVIDIA Corporation considered a well-known seasoned issuer according to the S...\n", + "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "4. Cache HIT (distance: 0.0820)\n", + " Original query: In what exchange platform is NVIDIA's stock traded in?\n", + " Matched: On which exchange is NVIDIA Corporation's common stock registered?...\n", + "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "5. Cache HIT (distance: 0.0924)\n", + " Original query: What is NVIDIA's total revenue for the last 5 years?\n", + " MISS MATCHED: How much revenue did NVIDIA generate?...\n", + "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "6. Cache MISS\n", + "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "7. Cache MISS\n", + "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "8. Cache MISS\n", + "10:19:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "9. Cache MISS\n", + "10:19:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10. Cache MISS\n", + "10:19:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "11. Cache HIT (distance: 0.1066)\n", + " Original query: What's the location of the manufacturing plant for NVIDIA?\n", + " MISS MATCHED: Where is NVIDIA headquartered?...\n", + "10:19:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "12. Cache HIT (distance: 0.0973)\n", + " Original query: What date does NVIDIA use as it's year end for acounting purposes?\n", + " Matched: When do NVIDIA's consumer products typically see stronger revenue?...\n", + "10:19:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "13. Cache HIT (distance: 0.0480)\n", + " Original query: Where are the locations of each office of NVIDIA Corporation?\n", + " MISS MATCHED: Where is the principal executive office of NVIDIA Corporation located?...\n", + "10:19:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "14. Cache HIT (distance: 0.1295)\n", + " Original query: What is the trading symbold of NVIDIA Corporation on the Japan exchange?\n", + " MISS MATCHED: What is the trading symbol for NVIDIA Corporation's common stock?...\n", "\n", - "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "3. ✗ Cache MISS\n", - " Query: How is the company performing financially?\n", - "\n" + "Summary Metrics:\n", + " Accuracy: 73.333%\n", + " Precision: 60.000%\n", + " Recall: 100.000%\n", + " F1 Score: 75.000%\n" ] } ], @@ -875,120 +872,47 @@ "# Test with semantically similar queries\n", "print(\"Testing semantic similarity:\\n\")\n", "\n", - "similar_queries = [\n", - " \"Tell me about NVIDIA's revenue\",\n", - " \"What products does the company make?\",\n", - " \"How is the company performing financially?\",\n", - "]\n", - "\n", - "for i, query in enumerate(similar_queries, 1):\n", - " result = cache.check(prompt=query, return_fields=[\"prompt\", \"response\", \"distance\"])\n", - " \n", - " if result:\n", - " print(f\"{i}. Cache HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", - " print(f\" Query: {query}\")\n", - " print(f\" Matched: {result[0]['prompt'][:80]}...\")\n", - " print(f\" Answer: {result[0]['response'][:100]}...\\n\")\n", - " else:\n", - " print(f\"{i}. ✗ Cache MISS\")\n", - " print(f\" Query: {query}\\n\")\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Test cache with sample query" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Testing query: 'What is NVIDIA's main business?'\n", - "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "Cache HIT\n", - " Distance: 0.076104\n", - " Matched: What segments does NVIDIA report its business results in?...\n" - ] - } - ], - "source": [ - "# Test cache behavior with a sample query\n", - "test_query = \"What is NVIDIA's main business?\"\n", + "full_test_data = positive_examples + negative_examples + edge_cases\n", "\n", - "print(f\"Testing query: '{test_query}'\")\n", + "# Track our metrics\n", + "true_positives = 0 # we have a hit and it hould match\n", + "false_positives = 0 # we have a hit and it SHOULD NOT match\n", + "false_negatives = 0 # we have a miss and it SHOULD match\n", + "true_negatives = 0 # we have a miss and it SHOULD NOT match\n", "\n", - "result = cache.check(prompt=test_query, return_fields=[\"prompt\", \"vector_distance\"])\n", + "for i, question in enumerate(full_test_data):\n", + " result = cache.check(prompt=question['query'], return_fields=[\"prompt\", \"response\", \"distance\"])\n", "\n", - "if result:\n", - " print(f\"Cache HIT\")\n", - " print(f\" Distance: {result[0].get('vector_distance', 0):.6f}\")\n", - " print(f\" Matched: {result[0]['prompt'][:80]}...\")\n", - "else:\n", - " print(f\"✗ Cache MISS - No match found within threshold\")\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Test negative examples (should not match)\n" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Testing negative examples (should NOT match):\n", - "\n", - "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "1. Correct MISS\n", - " Query: What is the weather today?\n", - "\n", - "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "2. Correct MISS\n", - " Query: How do I cook pasta?\n", - "\n", - "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "3. Correct MISS\n", - " Query: What is the capital of France?\n", - "\n", - "18:28:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "4. Correct MISS\n", - " Query: Tell me a joke\n", - "\n", - "18:28:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "5. Correct MISS\n", - " Query: What time is it?\n", - "\n" - ] - } - ], - "source": [ - "# Test with off-topic queries that should NOT match\n", - "print(\"Testing negative examples (should NOT match):\\n\")\n", - "\n", - "for i, test_case in enumerate(negative_examples, 1):\n", - " result = cache.check(prompt=test_case['query'], return_fields=[\"prompt\", \"vector_distance\"])\n", - " \n", - " if result:\n", - " print(f\"{i}. ⚠️ UNEXPECTED HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", - " print(f\" Query: {test_case['query']}\")\n", - " print(f\" Matched: {result[0]['prompt'][:80]}...\\n\")\n", - " else:\n", - " print(f\"{i}. Correct MISS\")\n", - " print(f\" Query: {test_case['query']}\\n\")\n" + " if result and question['expected_match']:\n", + " true_positives += 1\n", + " print(f\"{i}. Cache HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", + " print(f\" Original query: {question['query']}\")\n", + " print(f\" Matched: {result[0]['prompt'][:80]}...\")\n", + " elif result and not question['expected_match']:\n", + " false_positives += 1\n", + " print(f\"{i}. Cache HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", + " print(f\" Original query: {question['query']}\")\n", + " print(f\" MISS MATCHED: {result[0]['prompt'][:80]}...\")\n", + " elif not result and question['expected_match']:\n", + " false_negatives += 1\n", + " print(f\"{i}. Cache MISS\")\n", + " print(f\" Original query: {question['query']}\")\n", + " print(f\" Expected match: {question['expected_match']}\")\n", + " elif not result and not question['expected_match']:\n", + " true_negatives += 1\n", + " print(f\"{i}. Cache MISS\")\n", + "\n", + "# Calculate our summary metrics\n", + "accuracy = (true_positives + true_negatives) / len(full_test_data)\n", + "precision = true_positives / (true_positives + false_positives)\n", + "recall = true_positives / (true_positives + false_negatives)\n", + "f1_score = 2 * (precision * recall) / (precision + recall)\n", + "\n", + "print(f\"\\nSummary Metrics:\")\n", + "print(f\" Accuracy: {100*accuracy:.3f}%\")\n", + "print(f\" Precision: {100*precision:.3f}%\")\n", + "print(f\" Recall: {100*recall:.3f}%\")\n", + "print(f\" F1 Score: {100*f1_score:.3f}%\")\n" ] }, { @@ -1002,310 +926,57 @@ }, { "cell_type": "code", - "execution_count": 20, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Created optimization test data:\n", - " Total examples: 10\n", - " Positive (should match): 5\n", - " Negative (should not match): 5\n" - ] - } - ], - "source": [ - "# Create optimization test data\n", - "\n", - "# Format: [{\"query\": \"...\", \"query_match\": \"cache_key_or_empty_string\"}, ...]\n", - "\n", - "optimization_test_data = []\n", - "\n", - "# Add positive examples (should match specific cache entries)\n", - "for faq in faqs[:5]:\n", - " if faq['question'] in cache_keys:\n", - " optimization_test_data.append({\n", - " \"query\": faq['question'],\n", - " \"query_match\": cache_keys[faq['question']]\n", - " })\n", - "\n", - "# Add negative examples (should not match anything)\n", - "for neg_example in negative_examples:\n", - " optimization_test_data.append({\n", - " \"query\": neg_example['query'],\n", - " \"query_match\": \"\" # Empty string means it should NOT match\n", - " })\n", - "\n", - "print(f\"Created optimization test data:\")\n", - "print(f\" Total examples: {len(optimization_test_data)}\")\n", - "print(f\" Positive (should match): {sum(1 for x in optimization_test_data if x['query_match'])}\")\n", - "print(f\" Negative (should not match): {sum(1 for x in optimization_test_data if not x['query_match'])}\")\n" - ] - }, - { - "cell_type": "code", - "execution_count": 21, + "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "18:28:56 numexpr.utils INFO NumExpr defaulting to 10 threads.\n", + "10:19:34 numexpr.utils INFO NumExpr defaulting to 10 threads.\n", "Testing cache performance across different similarity thresholds...\n", - "Evaluation dataset: 10 queries\n", - " - Positive examples (should match): 5\n", - " - Negative examples (should NOT match): 5\n", - "\n", - "====================================================================================================\n", - "\n", - "18:28:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "18:28:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "18:28:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "18:28:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "distance is 0.0 threshold is 0.05\n", - "distance is 0.0 threshold is 0.05\n", - "distance is 0.0 threshold is 0.05\n", - "distance is 0.0 threshold is 0.05\n", - "distance is 0.0 threshold is 0.05\n", - "distance is inf threshold is 0.05\n", - "distance is inf threshold is 0.05\n", - "distance is inf threshold is 0.05\n", - "distance is inf threshold is 0.05\n", - "distance is inf threshold is 0.05\n", - "distance is 0.0 threshold is 0.1\n", - "distance is 0.0 threshold is 0.1\n", - "distance is 0.0 threshold is 0.1\n", - "distance is 0.0 threshold is 0.1\n", - "distance is 0.0 threshold is 0.1\n", - "distance is inf threshold is 0.1\n", - "distance is inf threshold is 0.1\n", - "distance is inf threshold is 0.1\n", - "distance is inf threshold is 0.1\n", - "distance is inf threshold is 0.1\n", - "distance is 0.0 threshold is 0.15\n", - "distance is 0.0 threshold is 0.15\n", - "distance is 0.0 threshold is 0.15\n", - "distance is 0.0 threshold is 0.15\n", - "distance is 0.0 threshold is 0.15\n", - "distance is inf threshold is 0.15\n", - "distance is inf threshold is 0.15\n", - "distance is inf threshold is 0.15\n", - "distance is inf threshold is 0.15\n", - "distance is inf threshold is 0.15\n", - "distance is 0.0 threshold is 0.2\n", - "distance is 0.0 threshold is 0.2\n", - "distance is 0.0 threshold is 0.2\n", - "distance is 0.0 threshold is 0.2\n", - "distance is 0.0 threshold is 0.2\n", - "distance is inf threshold is 0.2\n", - "distance is inf threshold is 0.2\n", - "distance is inf threshold is 0.2\n", - "distance is inf threshold is 0.2\n", - "distance is inf threshold is 0.2\n", - "distance is 0.0 threshold is 0.25\n", - "distance is 0.0 threshold is 0.25\n", - "distance is 0.0 threshold is 0.25\n", - "distance is 0.0 threshold is 0.25\n", - "distance is 0.0 threshold is 0.25\n", - "distance is inf threshold is 0.25\n", - "distance is inf threshold is 0.25\n", - "distance is inf threshold is 0.25\n", - "distance is inf threshold is 0.25\n", - "distance is inf threshold is 0.25\n", - "distance is 0.0 threshold is 0.3\n", - "distance is 0.0 threshold is 0.3\n", - "distance is 0.0 threshold is 0.3\n", - "distance is 0.0 threshold is 0.3\n", - "distance is 0.0 threshold is 0.3\n", - "distance is inf threshold is 0.3\n", - "distance is inf threshold is 0.3\n", - "distance is inf threshold is 0.3\n", - "distance is inf threshold is 0.3\n", - "distance is inf threshold is 0.3\n", - "distance is 0.0 threshold is 0.35\n", - "distance is 0.0 threshold is 0.35\n", - "distance is 0.0 threshold is 0.35\n", - "distance is 0.0 threshold is 0.35\n", - "distance is 0.0 threshold is 0.35\n", - "distance is inf threshold is 0.35\n", - "distance is inf threshold is 0.35\n", - "distance is inf threshold is 0.35\n", - "distance is inf threshold is 0.35\n", - "distance is inf threshold is 0.35\n", - "distance is 0.0 threshold is 0.4\n", - "distance is 0.0 threshold is 0.4\n", - "distance is 0.0 threshold is 0.4\n", - "distance is 0.0 threshold is 0.4\n", - "distance is 0.0 threshold is 0.4\n", - "distance is inf threshold is 0.4\n", - "distance is inf threshold is 0.4\n", - "distance is inf threshold is 0.4\n", - "distance is inf threshold is 0.4\n", - "distance is inf threshold is 0.4\n", - "distance is 0.0 threshold is 0.5\n", - "distance is 0.0 threshold is 0.5\n", - "distance is 0.0 threshold is 0.5\n", - "distance is 0.0 threshold is 0.5\n", - "distance is 0.0 threshold is 0.5\n", - "distance is inf threshold is 0.5\n", - "distance is inf threshold is 0.5\n", - "distance is inf threshold is 0.5\n", - "distance is inf threshold is 0.5\n", - "distance is inf threshold is 0.5\n", - "distance is 0.0 threshold is 0.55\n", - "distance is 0.0 threshold is 0.55\n", - "distance is 0.0 threshold is 0.55\n", - "distance is 0.0 threshold is 0.55\n", - "distance is 0.0 threshold is 0.55\n", - "distance is inf threshold is 0.55\n", - "distance is inf threshold is 0.55\n", - "distance is inf threshold is 0.55\n", - "distance is inf threshold is 0.55\n", - "distance is inf threshold is 0.55\n", - "distance is 0.0 threshold is 0.6\n", - "distance is 0.0 threshold is 0.6\n", - "distance is 0.0 threshold is 0.6\n", - "distance is 0.0 threshold is 0.6\n", - "distance is 0.0 threshold is 0.6\n", - "distance is inf threshold is 0.6\n", - "distance is inf threshold is 0.6\n", - "distance is inf threshold is 0.6\n", - "distance is inf threshold is 0.6\n", - "distance is inf threshold is 0.6\n", - "distance is 0.0 threshold is 0.65\n", - "distance is 0.0 threshold is 0.65\n", - "distance is 0.0 threshold is 0.65\n", - "distance is 0.0 threshold is 0.65\n", - "distance is 0.0 threshold is 0.65\n", - "distance is inf threshold is 0.65\n", - "distance is inf threshold is 0.65\n", - "distance is inf threshold is 0.65\n", - "distance is inf threshold is 0.65\n", - "distance is inf threshold is 0.65\n", - "distance is 0.0 threshold is 0.7\n", - "distance is 0.0 threshold is 0.7\n", - "distance is 0.0 threshold is 0.7\n", - "distance is 0.0 threshold is 0.7\n", - "distance is 0.0 threshold is 0.7\n", - "distance is inf threshold is 0.7\n", - "distance is inf threshold is 0.7\n", - "distance is inf threshold is 0.7\n", - "distance is inf threshold is 0.7\n", - "distance is inf threshold is 0.7\n", - "distance is 0.0 threshold is 0.75\n", - "distance is 0.0 threshold is 0.75\n", - "distance is 0.0 threshold is 0.75\n", - "distance is 0.0 threshold is 0.75\n", - "distance is 0.0 threshold is 0.75\n", - "distance is inf threshold is 0.75\n", - "distance is inf threshold is 0.75\n", - "distance is inf threshold is 0.75\n", - "distance is inf threshold is 0.75\n", - "distance is inf threshold is 0.75\n", - "distance is 0.0 threshold is 0.8\n", - "distance is 0.0 threshold is 0.8\n", - "distance is 0.0 threshold is 0.8\n", - "distance is 0.0 threshold is 0.8\n", - "distance is 0.0 threshold is 0.8\n", - "distance is inf threshold is 0.8\n", - "distance is inf threshold is 0.8\n", - "distance is inf threshold is 0.8\n", - "distance is inf threshold is 0.8\n", - "distance is inf threshold is 0.8\n", - "distance is 0.0 threshold is 0.85\n", - "distance is 0.0 threshold is 0.85\n", - "distance is 0.0 threshold is 0.85\n", - "distance is 0.0 threshold is 0.85\n", - "distance is 0.0 threshold is 0.85\n", - "distance is inf threshold is 0.85\n", - "distance is inf threshold is 0.85\n", - "distance is inf threshold is 0.85\n", - "distance is inf threshold is 0.85\n", - "distance is inf threshold is 0.85\n", - "distance is 0.0 threshold is 0.9\n", - "distance is 0.0 threshold is 0.9\n", - "distance is 0.0 threshold is 0.9\n", - "distance is 0.0 threshold is 0.9\n", - "distance is 0.0 threshold is 0.9\n", - "distance is inf threshold is 0.9\n", - "distance is inf threshold is 0.9\n", - "distance is inf threshold is 0.9\n", - "distance is inf threshold is 0.9\n", - "distance is inf threshold is 0.9\n", - "distance is 0.0 threshold is 0.95\n", - "distance is 0.0 threshold is 0.95\n", - "distance is 0.0 threshold is 0.95\n", - "distance is 0.0 threshold is 0.95\n", - "distance is 0.0 threshold is 0.95\n", - "distance is inf threshold is 0.95\n", - "distance is inf threshold is 0.95\n", - "distance is inf threshold is 0.95\n", - "distance is inf threshold is 0.95\n", - "distance is inf threshold is 0.95\n", - "distance is 0.0 threshold is 1.0\n", - "distance is 0.0 threshold is 1.0\n", - "distance is 0.0 threshold is 1.0\n", - "distance is 0.0 threshold is 1.0\n", - "distance is 0.0 threshold is 1.0\n", - "distance is inf threshold is 1.0\n", - "distance is inf threshold is 1.0\n", - "distance is inf threshold is 1.0\n", - "distance is inf threshold is 1.0\n", - "distance is inf threshold is 1.0\n", + "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "THRESHOLD OPTIMIZATION RESULTS\n", "====================================================================================================\n", "\n", "Performance Metrics by Threshold:\n", " Threshold Total Hits Total Misses True Positives False Positives True Negatives False Negatives Precision Recall F1 Score Accuracy\n", - " 0.05 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.10 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.15 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.20 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.25 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.30 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.35 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.40 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.50 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.55 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.60 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.65 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.70 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.75 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.80 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.85 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.90 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 0.95 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", - " 1.00 5 5 5 0 5 0 1.0 1.0 1.0 1.0\n", + " 0.20 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", + " 0.30 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", + " 0.40 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", + " 0.50 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", + " 0.60 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", + " 0.70 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", + " 0.80 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", + " 0.85 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", + " 0.90 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", + " 0.95 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", + " 1.00 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", "\n", "====================================================================================================\n", - "OPTIMAL THRESHOLD: 0.05\n", - " F1 Score: 1.000\n", - " Precision: 1.000\n", + "OPTIMAL THRESHOLD: 0.2\n", + " F1 Score: 0.750\n", + " Precision: 0.600\n", " Recall: 1.000\n", - " Accuracy: 1.000\n", + " Accuracy: 0.733\n", "====================================================================================================\n", "\n", - "Detailed breakdown at optimal threshold (0.05):\n", - "\n", - "Query: What is the fiscal year end date for NVIDIA Corporation as r | Distance: 0.0000 | ✓ TP (True Positive)\n", - "Query: What is the trading symbol for NVIDIA Corporation's common s | Distance: 0.0000 | ✓ TP (True Positive)\n", - "Query: Where is the principal executive office of NVIDIA Corporatio | Distance: 0.0000 | ✓ TP (True Positive)\n", - "Query: Is NVIDIA Corporation considered a well-known seasoned issue | Distance: 0.0000 | ✓ TP (True Positive)\n", - "Query: On which exchange is NVIDIA Corporation's common stock regis | Distance: 0.0000 | ✓ TP (True Positive)\n", - "Query: What is the weather today? | Distance: inf | ✓ TN (True Negative)\n", - "Query: How do I cook pasta? | Distance: inf | ✓ TN (True Negative)\n", - "Query: What is the capital of France? | Distance: inf | ✓ TN (True Negative)\n", - "Query: Tell me a joke | Distance: inf | ✓ TN (True Negative)\n", - "Query: What time is it? | Distance: inf | ✓ TN (True Negative)\n" + "Detailed breakdown at optimal threshold (0.2):\n", + "\n" ] } ], @@ -1314,23 +985,18 @@ "import pandas as pd\n", "\n", "# Define threshold ranges to test\n", - "thresholds_to_test = [0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95, 1.00]\n", + "thresholds_to_test = [0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, 0.85, 0.90, 0.95, 1.00]\n", "\n", "print(\"Testing cache performance across different similarity thresholds...\")\n", - "print(f\"Evaluation dataset: {len(optimization_test_data)} queries\")\n", - "print(f\" - Positive examples (should match): {sum(1 for x in optimization_test_data if x['query_match'])}\")\n", - "print(f\" - Negative examples (should NOT match): {sum(1 for x in optimization_test_data if not x['query_match'])}\")\n", - "print(\"\\n\" + \"=\"*100 + \"\\n\")\n", "\n", "# Store results for all queries to reuse across thresholds\n", "query_results = []\n", - "for test_case in optimization_test_data:\n", + "for test_case in full_test_data:\n", " result = cache.check(prompt=test_case['query'], return_fields=[\"prompt\", \"response\", \"vector_distance\", \"entry_id\"])\n", - " \n", + "\n", " query_results.append({\n", " 'query': test_case['query'],\n", - " 'expected_match': bool(test_case['query_match']),\n", - " 'expected_key': test_case['query_match'],\n", + " 'expected_match': test_case['expected_match'],\n", " 'cache_result': result[0] if result else None,\n", " 'distance': result[0].get('vector_distance') if result else float('inf')\n", " })\n", @@ -1347,7 +1013,6 @@ " for query_data in query_results:\n", " # Determine if this would be a cache hit at this threshold\n", " is_cache_hit = query_data['distance'] < threshold\n", - " print('distance is ', query_data['distance'], 'threshold is ', threshold)\n", " should_match = query_data['expected_match']\n", "\n", " if is_cache_hit and should_match:\n", @@ -1366,7 +1031,7 @@ " precision = true_positives / (true_positives + false_positives) if (true_positives + false_positives) > 0 else 0\n", " recall = true_positives / (true_positives + false_negatives) if (true_positives + false_negatives) > 0 else 0\n", " f1_score = 2 * (precision * recall) / (precision + recall) if (precision + recall) > 0 else 0\n", - " accuracy = (true_positives + true_negatives) / len(optimization_test_data)\n", + " accuracy = (true_positives + true_negatives) / len(full_test_data)\n", "\n", " results.append({\n", " 'Threshold': threshold,\n", @@ -1417,60 +1082,7 @@ " elif not is_cache_hit and not should_match:\n", " status = \"✓ TN (True Negative)\"\n", " elif not is_cache_hit and should_match:\n", - " status = \"✗ FN (False Negative)\"\n", - "\n", - " print(f\"Query: {query_data['query'][:60]:60s} | Distance: {query_data['distance']:.4f} | {status}\")" - ] - }, - { - "cell_type": "code", - "execution_count": 22, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "Re-testing negative examples with optimized threshold:\n", - "\n", - "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "1. MISS (correct)\n", - " Query: What is the weather today?\n", - "\n", - "18:28:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "2. MISS (correct)\n", - " Query: How do I cook pasta?\n", - "\n", - "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "3. MISS (correct)\n", - " Query: What is the capital of France?\n", - "\n", - "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "4. MISS (correct)\n", - " Query: Tell me a joke\n", - "\n", - "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "5. MISS (correct)\n", - " Query: What time is it?\n", - "\n" - ] - } - ], - "source": [ - "# Re-test with optimized threshold\n", - "print(\"\\nRe-testing negative examples with optimized threshold:\\n\")\n", - "\n", - "for i, test_case in enumerate(negative_examples, 1):\n", - " result = cache.check(prompt=test_case['query'], return_fields=[\"prompt\", \"vector_distance\"])\n", - "\n", - " if result:\n", - " print(f\"{i}. HIT (distance: {result[0].get('vector_distance', 'N/A'):.4f})\")\n", - " print(f\" Query: {test_case['query']}\")\n", - " print(f\" Matched: {result[0]['prompt'][:80]}...\\n\")\n", - " else:\n", - " print(f\"{i}. MISS (correct)\")\n", - " print(f\" Query: {test_case['query']}\\n\")\n" + " status = \"✗ FN (False Negative)\"\n" ] }, { @@ -1491,7 +1103,7 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -1524,7 +1136,7 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -1539,31 +1151,31 @@ "def rag_with_cache(question: str, use_cache: bool = True) -> tuple:\n", " \"\"\"\n", " Process a question through RAG pipeline with optional semantic caching.\n", - " \n", + "\n", " Returns: A tuple of (answer, cache_hit, response_time)\n", " \"\"\"\n", " start_time = time.time()\n", " cache_hit = False\n", - " \n", + "\n", " # Check cache first if enabled\n", " if use_cache:\n", - " cached_result = cache.check(prompt=question)\n", + " cached_result = cache.check(prompt=question, distance_threshold=optimal_threshold)\n", " if cached_result:\n", " answer = cached_result[0]['response']\n", " cache_hit = True\n", " response_time = time.time() - start_time\n", " return answer, cache_hit, response_time\n", - " \n", + "\n", " # Cache miss - use LLM\n", " answer = rag_chain.invoke({\"question\": question})\n", " response_time = time.time() - start_time\n", - " \n", + "\n", " # Store in cache for future use\n", " if use_cache and hasattr(answer, 'content'):\n", " cache.store(prompt=question, response=answer.content)\n", " elif use_cache:\n", " cache.store(prompt=question, response=str(answer))\n", - " \n", + "\n", " return answer.content if hasattr(answer, 'content') else str(answer), cache_hit, response_time\n", "\n", "print(\"Cached RAG function ready\")\n" @@ -1578,7 +1190,7 @@ }, { "cell_type": "code", - "execution_count": 25, + "execution_count": null, "metadata": {}, "outputs": [ { @@ -1592,63 +1204,63 @@ "\n", "[FIRST PASS - Populating Cache]\n", "\n", - "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "1. What is NVIDIA's primary business?\n", - " Cache: HIT | Time: 0.118s\n", + " Cache: HIT | Time: 0.109s\n", " Answer: NVIDIA specializes in four large markets: Data Center, Gaming, Professional Visualization, and Autom...\n", "\n", - "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "2. How much revenue did NVIDIA generate?\n", - " Cache: HIT | Time: 0.124s\n", - " Answer: NVIDIA reports its business results in two segments: the Compute & Networking segment and the Data C...\n", + " Cache: HIT | Time: 0.141s\n", + " Answer: As of the latest available data from NVIDIA's 10-K filing, the company generated approximately $26.9...\n", "\n", - "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "3. What are NVIDIA's main products?\n", - " Cache: HIT | Time: 0.118s\n", - " Answer: NVIDIA sells its products not only to customers in its partner network but also directly to cloud se...\n", + " Cache: HIT | Time: 0.127s\n", + " Answer: NVIDIA sells a range of products primarily focused on graphics processing units (GPUs) and related t...\n", "\n", "\n", "[SECOND PASS - Cache Hits with Paraphrased Questions]\n", "\n", - "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "1. What does NVIDIA do as a business?\n", - " Cache: HIT ✓ | Time: 0.121s\n", - " Answer: NVIDIA reports its business results in two segments: the Compute & Networking segment and the Data C...\n", + " Cache: HIT ✓ | Time: 0.117s\n", + " Answer: NVIDIA's business has evolved from a primary focus on gaming products to broader markets, including ...\n", "\n", - "18:28:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "2. Can you tell me NVIDIA's revenue figures?\n", - " Cache: HIT ✓ | Time: 0.119s\n", - " Answer: NVIDIA announces material financial information through its investor relations website, press releas...\n", + " Cache: HIT ✓ | Time: 0.117s\n", + " Answer: As of the latest available data from NVIDIA's 10-K filing, the company generated approximately $26.9...\n", "\n", - "18:28:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:19:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "3. What products does NVIDIA sell?\n", - " Cache: HIT ✓ | Time: 0.127s\n", - " Answer: NVIDIA reports its business results in two segments: the Compute & Networking segment and the Data C...\n", + " Cache: HIT ✓ | Time: 0.105s\n", + " Answer: NVIDIA sells a range of products primarily focused on graphics processing units (GPUs) and related t...\n", "\n", "\n", "[THIRD PASS - Without Cache (Baseline)]\n", "\n", - "18:29:00 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:19:38 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "1. What is NVIDIA's primary business?\n", - " Cache: DISABLED | Time: 1.675s\n", + " Cache: DISABLED | Time: 1.344s\n", "\n", - "18:29:02 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:19:39 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "2. How much revenue did NVIDIA generate?\n", - " Cache: DISABLED | Time: 1.554s\n", + " Cache: DISABLED | Time: 0.780s\n", "\n", - "18:29:07 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:19:42 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "3. What are NVIDIA's main products?\n", - " Cache: DISABLED | Time: 4.812s\n", + " Cache: DISABLED | Time: 3.267s\n", "\n", "\n", "================================================================================\n", "PERFORMANCE SUMMARY\n", "================================================================================\n", - "Average time - First pass (cache miss): 0.120s\n", - "Average time - Second pass (cache hit): 0.122s\n", - "Average time - Without cache: 2.681s\n", + "Average time - First pass (cache miss): 0.126s\n", + "Average time - Second pass (cache hit): 0.113s\n", + "Average time - Without cache: 1.797s\n", "\n", - "Speedup with cache: 1.0x faster\n", + "Speedup with cache: 1.1x faster\n", " Cache hit rate: 0%\n" ] } @@ -1774,35 +1386,13 @@ }, { "cell_type": "code", - "execution_count": 26, + "execution_count": null, "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "18:29:07 httpx INFO HTTP Request: DELETE https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 400 Bad Request\"\n" - ] - }, - { - "ename": "BadRequestErrorResponseContent", - "evalue": "{\"detail\":\"attributes: cannot be blank.\",\"status\":400,\"title\":\"Invalid Request\",\"type\":\"/errors/invalid-data\"}", - "output_type": "error", - "traceback": [ - "\u001b[31m---------------------------------------------------------------------------\u001b[39m", - "\u001b[31mBadRequestErrorResponseContent\u001b[39m Traceback (most recent call last)", - "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[26]\u001b[39m\u001b[32m, line 2\u001b[39m\n\u001b[32m 1\u001b[39m \u001b[38;5;66;03m# Clear cache contents\u001b[39;00m\n\u001b[32m----> \u001b[39m\u001b[32m2\u001b[39m \u001b[43mcache\u001b[49m\u001b[43m.\u001b[49m\u001b[43mclear\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 3\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33m\"\u001b[39m\u001b[33mCache contents cleared\u001b[39m\u001b[33m\"\u001b[39m)\n", - "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/redisvl/extensions/cache/llm/langcache.py:557\u001b[39m, in \u001b[36mLangCacheSemanticCache.clear\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m 552\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mclear\u001b[39m(\u001b[38;5;28mself\u001b[39m) -> \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 553\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Clear the cache of all entries.\u001b[39;00m\n\u001b[32m 554\u001b[39m \n\u001b[32m 555\u001b[39m \u001b[33;03m This is an alias for delete() to match the BaseCache interface.\u001b[39;00m\n\u001b[32m 556\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m557\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mdelete\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n", - "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/redisvl/extensions/cache/llm/langcache.py:542\u001b[39m, in \u001b[36mLangCacheSemanticCache.delete\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m 536\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mdelete\u001b[39m(\u001b[38;5;28mself\u001b[39m) -> \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 537\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Delete the entire cache.\u001b[39;00m\n\u001b[32m 538\u001b[39m \n\u001b[32m 539\u001b[39m \u001b[33;03m This deletes all entries in the cache by calling delete_query\u001b[39;00m\n\u001b[32m 540\u001b[39m \u001b[33;03m with no attributes.\u001b[39;00m\n\u001b[32m 541\u001b[39m \u001b[33;03m \"\"\"\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m542\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_client\u001b[49m\u001b[43m.\u001b[49m\u001b[43mdelete_query\u001b[49m\u001b[43m(\u001b[49m\u001b[43mattributes\u001b[49m\u001b[43m=\u001b[49m\u001b[43m{\u001b[49m\u001b[43m}\u001b[49m\u001b[43m)\u001b[49m\n", - "\u001b[36mFile \u001b[39m\u001b[32m~/.pyenv/versions/3.11.9/envs/redis-ai-res/lib/python3.11/site-packages/langcache/sdk.py:227\u001b[39m, in \u001b[36mLangCache.delete_query\u001b[39m\u001b[34m(self, attributes, retries, server_url, timeout_ms, http_headers)\u001b[39m\n\u001b[32m 223\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m utils.match_response(http_res, \u001b[33m\"\u001b[39m\u001b[33m400\u001b[39m\u001b[33m\"\u001b[39m, \u001b[33m\"\u001b[39m\u001b[33mapplication/json\u001b[39m\u001b[33m\"\u001b[39m):\n\u001b[32m 224\u001b[39m response_data = unmarshal_json_response(\n\u001b[32m 225\u001b[39m errors.BadRequestErrorResponseContentData, http_res\n\u001b[32m 226\u001b[39m )\n\u001b[32m--> \u001b[39m\u001b[32m227\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m errors.BadRequestErrorResponseContent(response_data, http_res)\n\u001b[32m 228\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m utils.match_response(http_res, \u001b[33m\"\u001b[39m\u001b[33m401\u001b[39m\u001b[33m\"\u001b[39m, \u001b[33m\"\u001b[39m\u001b[33mapplication/json\u001b[39m\u001b[33m\"\u001b[39m):\n\u001b[32m 229\u001b[39m response_data = unmarshal_json_response(\n\u001b[32m 230\u001b[39m errors.AuthenticationErrorResponseContentData, http_res\n\u001b[32m 231\u001b[39m )\n", - "\u001b[31mBadRequestErrorResponseContent\u001b[39m: {\"detail\":\"attributes: cannot be blank.\",\"status\":400,\"title\":\"Invalid Request\",\"type\":\"/errors/invalid-data\"}" - ] - } - ], + "outputs": [], "source": [ "# Clear cache contents\n", - "cache.clear()\n", - "print(\"Cache contents cleared\")" + "#cache.clear()\n", + "# print(\"Cache contents cleared\")" ] }, { From 3eeec456bb57c5b2dc7458ca12e8952b45b704a1 Mon Sep 17 00:00:00 2001 From: Justin Cechmanek Date: Thu, 20 Nov 2025 10:44:43 -0800 Subject: [PATCH 7/7] final pass before workshop --- .../04_langcache_semantic_caching.ipynb | 545 +++++++++--------- 1 file changed, 273 insertions(+), 272 deletions(-) diff --git a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb index b3934bc5..98a84f5b 100644 --- a/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb +++ b/python-recipes/semantic-cache/04_langcache_semantic_caching.ipynb @@ -56,13 +56,17 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ + "\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.3\u001b[0m\n", + "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", + "Note: you may need to restart the kernel to use updated packages.\n", "\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.3\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", @@ -84,7 +88,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -137,7 +141,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 3, "metadata": {}, "outputs": [ { @@ -170,17 +174,17 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "10:15:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "[{'entry_id': '5eb63bbbe01eeed093cb22bb8f5acdc3', 'prompt': 'hello world', 'response': 'hello world from langcache', 'vector_distance': 0.0, 'inserted_at': 0.0, 'updated_at': 0.0}]\n", - "10:15:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:15:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:35:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "[]\n", + "10:35:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:35:45 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "[{'entry_id': '5eb63bbbe01eeed093cb22bb8f5acdc3', 'prompt': 'hello world', 'response': 'hello world from langcache', 'vector_distance': 0.07242219999999999, 'inserted_at': 0.0, 'updated_at': 0.0}]\n" ] } @@ -225,7 +229,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, "metadata": {}, "outputs": [ { @@ -251,7 +255,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 6, "metadata": {}, "outputs": [], "source": [ @@ -275,7 +279,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "metadata": {}, "outputs": [], "source": [ @@ -286,7 +290,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 8, "metadata": {}, "outputs": [ { @@ -330,7 +334,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 9, "metadata": {}, "outputs": [], "source": [ @@ -349,7 +353,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 10, "metadata": {}, "outputs": [ { @@ -390,7 +394,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 11, "metadata": {}, "outputs": [ { @@ -399,20 +403,20 @@ "text": [ "Testing FAQ generation on sample chunk...\n", "\n", - "10:15:49 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:36:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Generated 5 FAQs:\n", "\n", - "1. Q: What industries are utilizing NVIDIA's GPUs for automation?\n", - " Category: products\n", - " A: A rapidly growing number of enterprises and startups across a broad range of industries are using NVIDIA's GPUs and software to bring automation to th...\n", + "1. Q: What industries are leveraging NVIDIA's GPUs for automation?\n", + " Category: operations\n", + " A: A rapidly growing number of enterprises and startups across a broad range of industries, including transportation for autonomous driving, healthcare f...\n", "\n", "2. Q: What was the reason for the termination of the Arm Share Purchase Agreement?\n", - " Category: operations\n", - " A: The Arm Share Purchase Agreement was terminated due to significant regulatory challenges that prevented the completion of the transaction between NVID...\n", + " Category: general\n", + " A: The Share Purchase Agreement between NVIDIA and SoftBank Group Corp. was terminated due to significant regulatory challenges that prevented the comple...\n", "\n", - "3. Q: What acquisition termination cost did NVIDIA record in fiscal year 2023?\n", - " Category: financial\n", - " A: NVIDIA recorded an acquisition termination cost of $1.35 billion in fiscal year 2023, reflecting the write-off of the prepayment provided at signing f...\n" + "3. Q: What are some applications of NVIDIA's GPUs in professional design?\n", + " Category: products\n", + " A: Professional designers use NVIDIA's GPUs and software to create visual effects in movies and to design buildings and products ranging from cell phones...\n" ] } ], @@ -430,7 +434,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 12, "metadata": {}, "outputs": [ { @@ -441,44 +445,44 @@ "Generating FAQs from document chunks...\n", "\n", "Processing chunk 1/25...\n", - "10:15:57 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:16:06 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:16:13 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:16:26 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:16:31 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:36:23 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:36:34 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:36:43 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:36:52 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:36:58 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 6/25...\n", - "10:16:39 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:16:42 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:16:53 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:17:03 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:17:10 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:37:10 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:37:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:37:21 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:37:29 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:37:38 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 11/25...\n", - "10:17:19 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:17:31 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:17:37 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:17:46 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:17:54 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:37:49 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:37:57 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:38:06 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:38:14 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:38:23 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 16/25...\n", - "10:18:03 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:18:11 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:18:20 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:18:27 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:18:36 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:38:32 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:38:39 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:38:57 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:39:05 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:39:15 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "Processing chunk 21/25...\n", - "10:18:46 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:18:53 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:19:03 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:19:11 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", - "10:19:19 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:39:23 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:39:31 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:39:40 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:39:47 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:39:54 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "\n", "Generated 113 FAQs total\n", "\n", "Category distribution:\n", - " operations: 28\n", - " technology: 28\n", - " products: 24\n", - " financial: 18\n", - " general: 15\n" + " technology: 29\n", + " products: 27\n", + " financial: 20\n", + " operations: 20\n", + " general: 17\n" ] } ], @@ -533,7 +537,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 13, "metadata": {}, "outputs": [ { @@ -543,131 +547,131 @@ "Storing FAQs in cache...\n", "\n", " Stored 0/113 FAQs...\n", - "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:20 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:21 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:54 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:55 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:56 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", " Stored 20/113 FAQs...\n", - "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:22 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:23 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:57 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:58 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", " Stored 40/113 FAQs...\n", - "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:24 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:25 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:39:59 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:00 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:01 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", " Stored 60/113 FAQs...\n", - "10:19:26 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:27 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:28 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:02 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:03 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", " Stored 80/113 FAQs...\n", - "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:29 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:30 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:04 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", " Stored 100/113 FAQs...\n", - "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:31 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", - "10:19:32 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", + "10:40:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries \"HTTP/1.1 201 Created\"\n", "\n", "Stored 113 FAQs in cache\n", "\n", "Example cache entries:\n", "\n", - "1. Key: 7ca731f61f74432f83c61549a627b08a\n", - " Q: What is the fiscal year end date for NVIDIA Corporation as reported in the 10-K?...\n", + "1. Key: eb461a36940c04a1d307d33a595188af\n", + " Q: What is the fiscal year end date for NVIDIA Corporation as reported in the Form 10-K?...\n", "\n", "2. Key: 0daa2589e67ab291d447f3e103435706\n", " Q: What is the trading symbol for NVIDIA Corporation's common stock?...\n" @@ -764,18 +768,18 @@ "\n", "# Create test dataset with negative examples\n", "negative_examples = [\n", - " {\"query\": \"What is NVIDIA's total revenue for the last 5 years?\", \"expected_match\": False, \"category\": \"financial\"},\n", + " {\"query\": \"Where are these reports being submitted and who is reading them?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", " {\"query\": \"What is Jensen Huang's net worth?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", " {\"query\": \"What games run best on the RTX 4090? NVIDIA GPU?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", " {\"query\": \"What time is it?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", - " {\"query\": \"What's the fiscal year end for ARM Corporation?\", \"expected_match\": False, \"category\": \"general\"},\n", + " {\"query\": \"Should I invest my life savings in this organization?\", \"expected_match\": False, \"category\": \"general\"},\n", "]\n", "\n", "# Create test dataset with edge cases (slightly different phrasings)\n", "edge_cases = [\n", " {\"query\": \"What's the fiscal year end for Microsoft Corporation?\", \"expected_match\": False, \"category\": \"general\"},\n", + " {\"query\": \"What is the company total revenue for the last 5 years?\", \"expected_match\": False, \"category\": \"financial\"},\n", " {\"query\": \"What's the location of the manufacturing plant for NVIDIA?\", \"expected_match\": False, \"category\": \"general\"},\n", - " {\"query\": \"What date does NVIDIA use as it's year end for acounting purposes?\", \"expected_match\": True, \"category\":\"general\"},\n", " {\"query\": \"Where are the locations of each office of NVIDIA Corporation?\", \"expected_match\": False, \"category\": \"off-topic\"},\n", " {\"query\": \"What is the trading symbold of NVIDIA Corporation on the Japan exchange?\", \"expected_match\": False, \"category\": \"general\"},\n", "]\n", @@ -800,7 +804,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 22, "metadata": {}, "outputs": [ { @@ -809,62 +813,60 @@ "text": [ "Testing semantic similarity:\n", "\n", - "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "0. Cache HIT (distance: 0.0711)\n", + "10:41:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "0. Cache HIT (distance: 0.0842)\n", " Original query: What's the fiscal year end for NVIDIA Corporation?\n", - " Matched: What is the fiscal year end date for NVIDIA Corporation as reported in the 10-K?...\n", - "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + " Matched: What is the fiscal year end date for NVIDIA Corporation as reported in the Form ...\n", + "10:41:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "1. Cache HIT (distance: 0.0174)\n", " Original query: What is the trading symbol of NVIDIA Corporation's common stock in the market?\n", " Matched: What is the trading symbol for NVIDIA Corporation's common stock?...\n", - "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "2. Cache HIT (distance: 0.1340)\n", + "10:41:05 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "2. Cache MISS\n", " Original query: Where is the location of the executive office?\n", - " Matched: Where is NVIDIA Corporation's principal executive office located?...\n", - "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "3. Cache HIT (distance: 0.0182)\n", + " Expected match: True\n", + "10:41:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "3. Cache HIT (distance: 0.0352)\n", " Original query: Does the SEC consider NVIDIA Corporation a well-known seasoned issuer?\n", - " Matched: Is NVIDIA Corporation considered a well-known seasoned issuer according to the S...\n", - "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "4. Cache HIT (distance: 0.0820)\n", + " Matched: Is NVIDIA Corporation considered a well-known seasoned issuer?...\n", + "10:41:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "4. Cache HIT (distance: 0.1314)\n", " Original query: In what exchange platform is NVIDIA's stock traded in?\n", - " Matched: On which exchange is NVIDIA Corporation's common stock registered?...\n", - "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "5. Cache HIT (distance: 0.0924)\n", - " Original query: What is NVIDIA's total revenue for the last 5 years?\n", - " MISS MATCHED: How much revenue did NVIDIA generate?...\n", - "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + " Matched: What is the trading symbol for NVIDIA Corporation's common stock?...\n", + "10:41:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "5. Cache MISS\n", + "10:41:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "6. Cache MISS\n", - "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "7. Cache MISS\n", - "10:19:33 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "8. Cache MISS\n", - "10:19:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:06 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "9. Cache MISS\n", - "10:19:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "10. Cache MISS\n", - "10:19:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "11. Cache HIT (distance: 0.1066)\n", " Original query: What's the location of the manufacturing plant for NVIDIA?\n", " MISS MATCHED: Where is NVIDIA headquartered?...\n", - "10:19:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "12. Cache HIT (distance: 0.0973)\n", " Original query: What date does NVIDIA use as it's year end for acounting purposes?\n", " Matched: When do NVIDIA's consumer products typically see stronger revenue?...\n", - "10:19:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "13. Cache HIT (distance: 0.0480)\n", " Original query: Where are the locations of each office of NVIDIA Corporation?\n", " MISS MATCHED: Where is the principal executive office of NVIDIA Corporation located?...\n", - "10:19:34 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "14. Cache HIT (distance: 0.1295)\n", - " Original query: What is the trading symbold of NVIDIA Corporation on the Japan exchange?\n", - " MISS MATCHED: What is the trading symbol for NVIDIA Corporation's common stock?...\n", + "10:41:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "14. Cache HIT (distance: 0.0870)\n", + " Original query: What is the trading symbold of NVIDIA Corporation on the NASDAQ exchange?\n", + " Matched: What is the trading symbol for NVIDIA Corporation's common stock?...\n", "\n", "Summary Metrics:\n", - " Accuracy: 73.333%\n", - " Precision: 60.000%\n", - " Recall: 100.000%\n", - " F1 Score: 75.000%\n" + " Accuracy: 80.000%\n", + " Precision: 75.000%\n", + " Recall: 85.714%\n", + " F1 Score: 80.000%\n" ] } ], @@ -926,53 +928,52 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 23, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "10:19:34 numexpr.utils INFO NumExpr defaulting to 10 threads.\n", "Testing cache performance across different similarity thresholds...\n", - "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:35 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", - "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:07 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:08 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:09 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:09 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:09 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:41:09 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "THRESHOLD OPTIMIZATION RESULTS\n", "====================================================================================================\n", "\n", "Performance Metrics by Threshold:\n", - " Threshold Total Hits Total Misses True Positives False Positives True Negatives False Negatives Precision Recall F1 Score Accuracy\n", - " 0.20 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", - " 0.30 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", - " 0.40 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", - " 0.50 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", - " 0.60 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", - " 0.70 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", - " 0.80 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", - " 0.85 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", - " 0.90 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", - " 0.95 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", - " 1.00 10 5 6 4 5 0 0.6 1.0 0.75 0.733333\n", + " Threshold Total Hits Total Misses True Positives False Positives True Negatives False Negatives Precision Recall F1 Score Accuracy\n", + " 0.20 8 7 6 2 6 1 0.75 0.857143 0.8 0.8\n", + " 0.30 8 7 6 2 6 1 0.75 0.857143 0.8 0.8\n", + " 0.40 8 7 6 2 6 1 0.75 0.857143 0.8 0.8\n", + " 0.50 8 7 6 2 6 1 0.75 0.857143 0.8 0.8\n", + " 0.60 8 7 6 2 6 1 0.75 0.857143 0.8 0.8\n", + " 0.70 8 7 6 2 6 1 0.75 0.857143 0.8 0.8\n", + " 0.80 8 7 6 2 6 1 0.75 0.857143 0.8 0.8\n", + " 0.85 8 7 6 2 6 1 0.75 0.857143 0.8 0.8\n", + " 0.90 8 7 6 2 6 1 0.75 0.857143 0.8 0.8\n", + " 0.95 8 7 6 2 6 1 0.75 0.857143 0.8 0.8\n", + " 1.00 8 7 6 2 6 1 0.75 0.857143 0.8 0.8\n", "\n", "====================================================================================================\n", "OPTIMAL THRESHOLD: 0.2\n", - " F1 Score: 0.750\n", - " Precision: 0.600\n", - " Recall: 1.000\n", - " Accuracy: 0.733\n", + " F1 Score: 0.800\n", + " Precision: 0.750\n", + " Recall: 0.857\n", + " Accuracy: 0.800\n", "====================================================================================================\n", "\n", "Detailed breakdown at optimal threshold (0.2):\n", @@ -1103,7 +1104,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 17, "metadata": {}, "outputs": [ { @@ -1136,7 +1137,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 18, "metadata": {}, "outputs": [ { @@ -1190,7 +1191,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 19, "metadata": {}, "outputs": [ { @@ -1204,63 +1205,63 @@ "\n", "[FIRST PASS - Populating Cache]\n", "\n", - "10:19:36 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:40:12 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "1. What is NVIDIA's primary business?\n", - " Cache: HIT | Time: 0.109s\n", - " Answer: NVIDIA specializes in four large markets: Data Center, Gaming, Professional Visualization, and Autom...\n", + " Cache: HIT | Time: 0.116s\n", + " Answer: NVIDIA has expanded into several large and important computationally intensive fields including scie...\n", "\n", - "10:19:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:40:13 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "2. How much revenue did NVIDIA generate?\n", - " Cache: HIT | Time: 0.141s\n", - " Answer: As of the latest available data from NVIDIA's 10-K filing, the company generated approximately $26.9...\n", + " Cache: HIT | Time: 0.120s\n", + " Answer: NVIDIA's consumer products usually see stronger revenue in the second half of their fiscal year, wit...\n", "\n", - "10:19:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:40:13 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "3. What are NVIDIA's main products?\n", - " Cache: HIT | Time: 0.127s\n", - " Answer: NVIDIA sells a range of products primarily focused on graphics processing units (GPUs) and related t...\n", + " Cache: HIT | Time: 0.125s\n", + " Answer: NVIDIA specializes in four large markets: Data Center, Gaming, Professional Visualization, and Autom...\n", "\n", "\n", "[SECOND PASS - Cache Hits with Paraphrased Questions]\n", "\n", - "10:19:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:40:13 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "1. What does NVIDIA do as a business?\n", - " Cache: HIT ✓ | Time: 0.117s\n", + " Cache: HIT ✓ | Time: 0.122s\n", " Answer: NVIDIA's business has evolved from a primary focus on gaming products to broader markets, including ...\n", "\n", - "10:19:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:40:13 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "2. Can you tell me NVIDIA's revenue figures?\n", - " Cache: HIT ✓ | Time: 0.117s\n", - " Answer: As of the latest available data from NVIDIA's 10-K filing, the company generated approximately $26.9...\n", + " Cache: HIT ✓ | Time: 0.119s\n", + " Answer: NVIDIA announces material financial information to investors through its investor relations website,...\n", "\n", - "10:19:37 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", + "10:40:13 httpx INFO HTTP Request: POST https://aws-us-east-1.langcache.redis.io/v1/caches/50eb6a09acf5415d8b68619b1ccffd9a/entries/search \"HTTP/1.1 200 OK\"\n", "3. What products does NVIDIA sell?\n", - " Cache: HIT ✓ | Time: 0.105s\n", - " Answer: NVIDIA sells a range of products primarily focused on graphics processing units (GPUs) and related t...\n", + " Cache: HIT ✓ | Time: 0.125s\n", + " Answer: NVIDIA's Graphics segment includes GeForce GPUs for gaming and PCs, the GeForce NOW game streaming s...\n", "\n", "\n", "[THIRD PASS - Without Cache (Baseline)]\n", "\n", - "10:19:38 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:40:15 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "1. What is NVIDIA's primary business?\n", - " Cache: DISABLED | Time: 1.344s\n", + " Cache: DISABLED | Time: 1.640s\n", "\n", - "10:19:39 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:40:17 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "2. How much revenue did NVIDIA generate?\n", - " Cache: DISABLED | Time: 0.780s\n", + " Cache: DISABLED | Time: 2.014s\n", "\n", - "10:19:42 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", + "10:40:21 httpx INFO HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "3. What are NVIDIA's main products?\n", - " Cache: DISABLED | Time: 3.267s\n", + " Cache: DISABLED | Time: 4.001s\n", "\n", "\n", "================================================================================\n", "PERFORMANCE SUMMARY\n", "================================================================================\n", - "Average time - First pass (cache miss): 0.126s\n", - "Average time - Second pass (cache hit): 0.113s\n", - "Average time - Without cache: 1.797s\n", + "Average time - First pass (cache miss): 0.120s\n", + "Average time - Second pass (cache hit): 0.122s\n", + "Average time - Without cache: 2.552s\n", "\n", - "Speedup with cache: 1.1x faster\n", + "Speedup with cache: 1.0x faster\n", " Cache hit rate: 0%\n" ] } @@ -1386,7 +1387,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 20, "metadata": {}, "outputs": [], "source": [