notebooks/llmu/co_aws_ch3_text_generation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/cohere-ai/notebooks/blob/main/notebooks/llmu/co_aws_ch3_text_generation.ipynb\"> <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text Generation Using Cohere Command on Amazon Bedrock"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cohere's Command family of models, available on Amazon Bedrock, are powerful LLMs that offer these capabilities. This includes cutting-edge models like Command R and Command R+\n",
    "\n",
    "In this notebook, we'll explore how to use Cohere's Command R+ model on Amazon Bedrock. We'll use a customer support agent scenario as an example and cover a range of tasks, including text generation, summarization, rewriting, and extraction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's install and import the necessary libraries and set up our Cohere client using the cohere SDK. To use Bedrock, we create a BedrockClient by passing the necessary AWS credentials."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install cohere -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cohere\n",
    "\n",
    "# Create Bedrock client via the native Cohere SDK\n",
    "# Contact your AWS administrator for the credentials\n",
    "co = cohere.BedrockClient(\n",
    "    aws_region=\"YOUR_AWS_REGION\",\n",
    "    aws_access_key=\"YOUR_AWS_ACCESS_KEY_ID\",\n",
    "    aws_secret_key=\"YOUR_AWS_SECRET_ACCESS_KEY\",\n",
    "    aws_session_token=\"YOUR_AWS_SESSION_TOKEN\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating some contextual information"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we begin, let's create some context to use in our text generation tasks. In this example, we'll use a technical support FAQ as our context:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Technical support FAQ\n",
    "faq_tech_support = \"\"\"- Question: How do I set up my new smartphone with my mobile plan?\n",
    "- Answer:\n",
    "  - Insert your SIM card into the device.\n",
    "  - Turn on your phone and follow the on-screen setup instructions.\n",
    "  - Connect to your mobile network and enter your account details when prompted.\n",
    "  - Download and install any necessary apps or updates.\n",
    "  - Contact customer support if you need further assistance.\n",
    "\n",
    "- Question: My internet connection is slow. How can I improve my mobile data speed?\n",
    "- Answer:\n",
    "  - Check your signal strength and move to an area with better coverage.\n",
    "  - Restart your device and try connecting again.\n",
    "  - Ensure your data plan is active and has sufficient data.\n",
    "  - Consider upgrading your plan for faster speeds.\n",
    "\n",
    "- Question: I can't connect to my mobile network. What should I do?\n",
    "- Answer:\n",
    "  - Check your SIM card is inserted correctly and not damaged.\n",
    "  - Restart your device and try connecting again.\n",
    "  - Ensure your account is active and not suspended.\n",
    "  - Check for any network outages in your area.\n",
    "  - Contact customer support for further assistance.\n",
    "\n",
    "- Question: How do I set up my voicemail?\n",
    "- Answer:\n",
    "  - Dial your voicemail access number (usually provided by your carrier).\n",
    "  - Follow the prompts to set up your voicemail greeting and password.\n",
    "  - Record your voicemail greeting and save it.\n",
    "  - Test your voicemail by calling your number and leaving a message.\n",
    "\n",
    "- Question: I'm having trouble sending text messages. What could be the issue?\n",
    "- Answer:\n",
    "  - Check your signal strength and move to an area with better coverage.\n",
    "  - Ensure your account has sufficient credit or an active plan.\n",
    "  - Restart your device and try sending a message again.\n",
    "  - Check your message settings and ensure they are correct.\n",
    "  - Contact customer support if the issue persists.\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Function to generate text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's define a function to generate text using the Command R+ model on Bedrock."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_text(message):\n",
    "    response = co.chat(message=message,\n",
    "                       model=\"cohere.command-r-plus-v1:0\")\n",
    "    return response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text generation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use our technical support FAQ as context and generate a response to a customer inquiry."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Response to customer inquiry: \n",
      "\n",
      "Brief slowdowns in data speed can be frustrating. Here are some quick steps to improve your mobile data performance: \n",
      "\n",
      "1. Check your signal strength: Move to an area with better coverage if needed. \n",
      "2. Restart your device: A simple restart can often improve connectivity. \n",
      "3. Data plan status: Ensure your data plan is active and you have sufficient data allocated for faster speeds. \n",
      "4. Consider a plan upgrade: If the issue persists, consider contacting your provider to discuss faster speed plans. \n",
      "\n",
      "Additionally, checking for any network outages in your area and ensuring your SIM card is inserted correctly can also help maintain a stable connection. \n",
      "\n",
      "If the issue continues, reach out to your mobile provider's customer support for further guidance and troubleshooting.\n"
     ]
    }
   ],
   "source": [
    "inquiry = \"I've noticed some fluctuations in my mobile network's performance recently.The connection seems stable most of the time, but every now and then, I experience brief periods of slow data speeds. It happens a few times a day and is quite inconvenient.\"\n",
    "\n",
    "prompt = f\"\"\"Use the FAQs below to provide a concise response to this customer inquiry.\n",
    "\n",
    "# Customer inquiry\n",
    "{inquiry}\n",
    "\n",
    "# FAQs\n",
    "{faq_tech_support}\"\"\"\n",
    "\n",
    "response = generate_text(prompt)\n",
    "\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text summarization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's summarize the customer inquiry into a single sentence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I experience intermittent periods of slow data speeds a few times a day on my mobile network.\n"
     ]
    }
   ],
   "source": [
    "prompt=f\"\"\"Summarize this customer inquiry into one short sentence.\n",
    "\n",
    "Inquiry: {inquiry}\"\"\"\n",
    "\n",
    "response = generate_text(prompt)\n",
    "\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text Rewriting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's rewrite the generated response into an email format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Subject: Troubleshooting Guide for Slow Data and Network Issues\n",
      "\n",
      "Dear [Customer Name], \n",
      "\n",
      "I hope this email finds you well. I understand that experiencing slow data speeds and network connectivity issues can be frustrating. Here are some detailed troubleshooting steps to help resolve these problems: \n",
      "\n",
      "- Signal Strength: \n",
      "   - Please check your device's signal strength by looking at the signal bars displayed on your screen. \n",
      "   - If you're in an area with weak coverage, try moving to a different location with better reception. You can also try going outdoors or near a window to improve signal strength. \n",
      "\n",
      "- Device Restart: \n",
      "   - Simply restart your device. This step can often resolve minor glitches and connectivity issues. Turn your device off, wait for a few seconds, and then turn it back on. \n",
      "\n",
      "- Account Status: \n",
      "   - Ensure that your account is active and in good standing. Sometimes, a suspended or inactive account can lead to connectivity issues. You can log in to your online account or contact our customer support team to confirm your account status. \n",
      "\n",
      "- Contact Customer Support: \n",
      "   - If the above steps do not resolve the issue, please don't hesitate to reach out to our customer support team for further assistance. Our team is readily available to provide additional guidance and ensure that your issue is thoroughly addressed. This can include updating your plan to improve network performance if necessary. \n",
      "\n",
      "Please let me know if these steps helped resolve your issue. Your satisfaction is important to us, and we want to ensure that you have a seamless network experience. If there is anything further I can assist with, feel free to respond to this email directly, and I will be happy to help. \n",
      "\n",
      "Thank you for choosing [Company Name]. We value your patience and understanding during this process. \n",
      "\n",
      "Best regards, \n",
      "\n",
      "[Your Name] \n",
      "Customer Support Agent \n",
      "[Company Name] \n",
      "\n",
      "Please note: This email is ready to send, but feel free to include a personalized greeting and closing if you wish, along with any additional information specific to the customer's account or issue.\n"
     ]
    }
   ],
   "source": [
    "prompt=f\"\"\"Rewrite this customer support agent response into an email format, ready to send to the customer.\n",
    "\n",
    "If you're experiencing brief periods of slow data speeds or difficulty sending text messages and connecting to your mobile network, here are some troubleshooting steps you can follow:\n",
    "\n",
    "1. Check your signal strength - Move to an area with better coverage.\n",
    "2. Restart your device and try connecting again.\n",
    "3. Ensure your account is active and not suspended.\n",
    "4. Contact customer support for further assistance. (This can include updating your plan for better network performance.)\n",
    "\n",
    "Did these steps help resolve the issue? Let me know if you need further assistance.\"\"\"\n",
    "\n",
    "response = generate_text(prompt)\n",
    "\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text Extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's extract some information from the customer inquiry and response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "```json\n",
      "{\n",
      "  \"category\": \"technical support\",\n",
      "  \"product\": \"mobile\",\n",
      "  \"status\": \"open\"\n",
      "}\n",
      "```\n"
     ]
    }
   ],
   "source": [
    "prompt=f\"\"\"Extract the following as a JSON from the text provided below:\n",
    "- category (Options: technical support, promotions, billing)\n",
    "- product (Options: broadband, mobile, TV)\n",
    "- status (Options: open, closed)\n",
    "\n",
    "# Customer Inquiry\n",
    "{inquiry}\n",
    "\n",
    "# Customer Support Agent's Response\n",
    "If you're experiencing brief periods of slow data speeds or difficulty sending text messages and connecting to your mobile network, here are some troubleshooting steps you can follow:\n",
    "\n",
    "1. Check your signal strength - Move to an area with better coverage.\n",
    "2. Restart your device and try connecting again.\n",
    "3. Ensure your account is active and not suspended.\n",
    "4. Contact customer support for further assistance. (This can include updating your plan for better network performance.)\n",
    "\n",
    "Did these steps help resolve the issue? Let me know if you need further assistance.\"\"\"\n",
    "\n",
    "response = generate_text(prompt)\n",
    "\n",
    "print(response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Building a Chatbot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All our examples so far run on a single-turn interaction. But with the Chat endpoint, we can build a chatbot that keeps a memory of all previous interactions. This allows developers to build chatbot-style applications that maintain the state of a conversation.\n",
    "\n",
    "The following is an implementation of a simple chatbot in a customer support setting, where the chatbot acts as a helpful customer support agent.\n",
    "\n",
    "For this, we introduce a couple of additional parameters to the Chat endpoint:\n",
    "- preamble: A preamble contains instructions to help steer a chatbot’s response toward specific characteristics, such as a persona, style, or format. Here we are using a simple preamble of “You are a helpful customer support agent that assist customers of a mobile network service.”\n",
    "- chat_history: We store the history of a conversation between a user and the chatbot as a list, append every new conversation turn, and pass this information to the next endpoint call."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a preamble\n",
    "preamble = \"\"\"## Task and Context\n",
    "You are a helpful customer support agent that assists customers of a mobile network service.\"\"\"\n",
    "\n",
    "# Run the chatbot\n",
    "def run_chatbot(message, chat_history=[]):\n",
    "    response = co.chat(message=message,\n",
    "                       model=\"cohere.command-r-plus-v1:0\",\n",
    "                       preamble=preamble,\n",
    "                       chat_history=chat_history)\n",
    "    \n",
    "    print(response.text)\n",
    "    \n",
    "    chat_history = response.chat_history\n",
    "            \n",
    "    return chat_history"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And here’s an example conversation that runs over a few turns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I'm sorry to hear that you've been experiencing issues with your mobile network's performance. Can you tell me a little more about the specific problems you've been facing? For example, are you experiencing dropped calls, slow data speeds, or something else entirely? The more information you can provide, the better I'll be able to assist you in troubleshooting the issue.\n"
     ]
    }
   ],
   "source": [
    "chat_history = run_chatbot(\"Hi. I've noticed some fluctuations in my mobile network's performance recently.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I'm sorry to hear that you're experiencing issues with your data speed. Here are a few troubleshooting steps you can try to improve your mobile data performance: \n",
      "\n",
      "- Check your device's signal: Weak or fluctuating signals can lead to poor data speeds. Try moving to a different location or adjusting your device's position to see if that improves your signal strength. \n",
      "\n",
      "- Restart your device: Sometimes, a simple restart can refresh your device's connection to the network and improve data speeds. \n",
      "\n",
      "- Check for network outages: It's possible that the issue is not on your end. Check your network provider's website or social media accounts to see if there are any reported outages or maintenance updates in your area. \n",
      "\n",
      "- Update your device software: Ensure that your device's operating system and apps are up to date. Outdated software can sometimes cause issues with network performance. \n",
      "\n",
      "- Review data usage and plan: Check your data usage to ensure you haven't exceeded your plan limits, which can result in reduced speeds. Also, review your current plan to see if it meets your data needs, and consider upgrading if necessary. \n",
      "\n",
      "- Contact your network provider: If none of the above steps help, it's advisable to contact your mobile network provider's customer support. They can run further diagnostics, check the network towers in your area, and suggest advanced troubleshooting techniques. \n",
      "\n",
      "Remember to keep an eye on your data speed after each troubleshooting step to pinpoint the issue effectively.\n"
     ]
    }
   ],
   "source": [
    "chat_history = run_chatbot(\"At times, the data speed is very poor. What should I do?\", chat_history)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You're welcome! If you have any other questions or concerns, feel free to reach out. I'm here to help.\n"
     ]
    }
   ],
   "source": [
    "chat_history = run_chatbot(\"Thanks, I'll try these.\", chat_history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's what is contained in the chat history after a few turns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Chat history:\n",
      "message=\"Hi. I've noticed some fluctuations in my mobile network's performance recently.\" tool_calls=None role='USER' \n",
      "\n",
      "message=\"I'm sorry to hear that you've been experiencing issues with your mobile network's performance. Can you tell me a little more about the specific problems you've been facing? For example, are you experiencing dropped calls, slow data speeds, or something else entirely? The more information you can provide, the better I'll be able to assist you in troubleshooting the issue.\" tool_calls=None role='CHATBOT' \n",
      "\n",
      "message='At times, the data speed is very poor. What should I do?' tool_calls=None role='USER' \n",
      "\n",
      "message=\"I'm sorry to hear that you're experiencing issues with your data speed. Here are a few troubleshooting steps you can try to improve your mobile data performance: \\n\\n- Check your device's signal: Weak or fluctuating signals can lead to poor data speeds. Try moving to a different location or adjusting your device's position to see if that improves your signal strength. \\n\\n- Restart your device: Sometimes, a simple restart can refresh your device's connection to the network and improve data speeds. \\n\\n- Check for network outages: It's possible that the issue is not on your end. Check your network provider's website or social media accounts to see if there are any reported outages or maintenance updates in your area. \\n\\n- Update your device software: Ensure that your device's operating system and apps are up to date. Outdated software can sometimes cause issues with network performance. \\n\\n- Review data usage and plan: Check your data usage to ensure you haven't exceeded your plan limits, which can result in reduced speeds. Also, review your current plan to see if it meets your data needs, and consider upgrading if necessary. \\n\\n- Contact your network provider: If none of the above steps help, it's advisable to contact your mobile network provider's customer support. They can run further diagnostics, check the network towers in your area, and suggest advanced troubleshooting techniques. \\n\\nRemember to keep an eye on your data speed after each troubleshooting step to pinpoint the issue effectively.\" tool_calls=None role='CHATBOT' \n",
      "\n",
      "message=\"Thanks, I'll try these.\" tool_calls=None role='USER' \n",
      "\n",
      "message=\"You're welcome! If you have any other questions or concerns, feel free to reach out. I'm here to help.\" tool_calls=None role='CHATBOT' \n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\"Chat history:\")\n",
    "for turn in chat_history:\n",
    "    print(turn, \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we explored how to use Command R+ on Amazon Bedrock for various text generation and manipulation tasks. We covered text generation, summarization, rewriting, and extraction, demonstrating how these capabilities can be applied to real-world scenarios.\n",
    "\n",
    "If you want to learn more LLM use cases, visit our LLMU chapter on [use case patterns](https://docs.cohere.com/docs/use-case-patterns)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "base"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}