prompt_engineering/Lab_Prompt_Engineering.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Prompt Engineering\n",
        "\n",
        "### Author: Sharon Zhou <<sharonz@cs.stanford.edu>>. CS Dept Faculty, Stanford University.\n",
        "\n",
        "### This notebook lives [here](https://colab.research.google.com/drive/1P4W-V_JDUs1bOm6c5Bfn6aG5HtDx4QZm).\n",
        "\n",
        "#### In this notebook, you'll get change and play with the data that is inputed to LLMs in a programmatic way.\n",
        "\n",
        "#### _Goal: See how even small amounts of data can make an LLM much more useful!_\n"
      ],
      "metadata": {
        "id": "ZpT3x5hdIMoE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# @title Setup: Authenticate with Google & install the open-source [PowerML library](https://pypi.org/project/powerml-app/0.0.35/) to use LLMs easily\n",
        "%%capture\n",
        "\n",
        "from google.colab import auth\n",
        "import requests\n",
        "\n",
        "def authenticate_powerml():\n",
        "  auth.authenticate_user()\n",
        "  gcloud_token = !gcloud auth print-access-token\n",
        "  powerml_token_response = requests.get('https://api.staging.powerml.co:5002/auth/verify_gcloud_token?token=' + gcloud_token[0])\n",
        "  return powerml_token_response.json()['token']\n",
        "\n",
        "token = authenticate_powerml()\n",
        "!pip install powerml-app==0.0.41\n",
        "\n",
        "config = {\"powerml\": { \"key\": token, \"url\": \"https://api.staging.powerml.co\" }}\n",
        "\n",
        "# print('Ready to play with LLMs!')"
      ],
      "metadata": {
        "id": "woS3HNl5V-Vd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Scenario\n",
        "\n",
        "You have a lot of email you have to write for the different extracurriculars: movie, running, and math clubs. So, you want to use an LLM to help you write them. "
      ],
      "metadata": {
        "id": "4nN3m31TZVH2"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "### Exercise 1\n",
        "\n",
        "You have the following code written, so your LLM can write emails for all your clubs. But it writes a random email. You have no control over what the subject of them will be.\n",
        "\n",
        "**Your task: Adjust the context to enable you to specify the subject of the email.**\n",
        "\n",
        "_Goal: Learn how to build reusable contexts/prompts!_\n"
      ],
      "metadata": {
        "id": "LtDJALVjITTG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from powerml import LLM, ContextTemplate\n",
        "\n",
        "# Build a template for giving context to your LLM; TODO: change the context to take in any subject\n",
        "context = \"Write an email for the {{club_name}} club.\" # double curly braces indicate a variable\n",
        "\n",
        "# Specify the variables in the context as arguments\n",
        "# Helpful reference when you have more arguments **hint**\n",
        "args=[\"club_name\"]\n",
        "\n",
        "# Instantiate the context template\n",
        "template = ContextTemplate(context, args)\n",
        "\n",
        "# Instantiate your LLM\n",
        "llm = LLM(config)\n",
        "\n",
        "# Fit your LLM to the context\n",
        "llm.fit(template)\n",
        "\n",
        "my_club_names = ['movie', 'running', 'math']\n",
        "for my_club_name in my_club_names:\n",
        "  # Run your LLM, looping through your different clubs\n",
        "  output = llm.predict(club_name=my_club_name)\n",
        "  print(output)\n",
        "  print(\"--------------\")"
      ],
      "metadata": {
        "id": "5AqV-F9jQJ0G"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Exercise 2\n",
        "The emails you're seeing don't sound like you. You sound better, funnier, or at least more like _you_! Good thing you have a few examples of emails you've written before for each club.\n",
        "\n",
        "**Your task: Given your past emails (provided for you), build a context template to include your past emails and fit a new LLM. Run and compare results.**\n",
        "\n",
        "_Goal: Observe the impact of more data._\n",
        "\n",
        "How does it do?"
      ],
      "metadata": {
        "id": "YSU80AVATKnx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# @markdown #### Setup: Fetch email files\n",
        "\n",
        "!wget -q -O \"movie_club.txt\" \"https://drive.google.com/uc?export=download&id=1dWKvcCIp-XwNBZ6heb5k2Y1tbdPl3WTE\"\n",
        "!wget -q -O \"running_club.txt\" \"https://drive.google.com/uc?export=download&id=1SxeE-0XP4ggjUNYwMD3VQlm_idCHoG9G\"\n",
        "!wget -q -O \"math_club.txt\" \"https://drive.google.com/uc?export=download&id=12z3EuOauuYc4Y9tdDGENs-_2wmvyErmv\"\n",
        "\n",
        "print('Got all the emails!')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "cellView": "form",
        "id": "Qks1Se35n0ZO",
        "outputId": "78828fb6-edcc-4267-e0b4-d0bfd95ecbd9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Got all the emails!\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "with open('movie_club.txt', 'r') as f:\n",
        "  past_movie_club_emails = f.read()\n",
        "with open('running_club.txt', 'r') as f:\n",
        "  past_running_club_emails = f.read()\n",
        "with open('math_club.txt', 'r') as f:\n",
        "  past_math_club_emails = f.read()\n",
        "\n",
        "# TODO (same code as above, without your changes for subject in Exercise 1)\n",
        "context = \"Write an email for the {{club_name}} club.\"\n",
        "args=[\"club_name\"]\n",
        "template = ContextTemplate(context, args)\n",
        "\n",
        "llm = LLM(config)\n",
        "llm.fit(template)\n",
        "\n",
        "my_club_names = ['movie', 'running', 'math']\n",
        "for my_club_name in my_club_names:\n",
        "  output = llm.predict(club_name=my_club_name)\n",
        "  print(output)\n",
        "  print(\"--------------\")"
      ],
      "metadata": {
        "id": "Nis3KpvfTJjw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Exercise 3\n",
        "\n",
        "So you think you're funny and you prize how funny you are. But what if you can get the LLM to make you funnier?\n",
        "\n",
        "**Your task: Engineer the prompt such that you can sound funnier than you.**\n",
        "\n",
        "_Goal: Play with the model, and see where it's easy to manipulate and where it isn't. And that it is not always intuitive._\n"
      ],
      "metadata": {
        "id": "uK5SFggfXUso"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# TODO (same code as Exercise 2 to get you started, without your changes)\n",
        "\n",
        "with open('movie_club.txt', 'r') as f:\n",
        "  past_movie_club_emails = f.read()\n",
        "with open('running_club.txt', 'r') as f:\n",
        "  past_running_club_emails = f.read()\n",
        "with open('math_club.txt', 'r') as f:\n",
        "  past_math_club_emails = f.read()\n",
        "\n",
        "context = \"Write an email for the {{club_name}} club.\"\n",
        "args=[\"club_name\"]\n",
        "template = ContextTemplate(context, args)\n",
        "\n",
        "llm = LLM(config)\n",
        "llm.fit(template)\n",
        "\n",
        "my_club_names = ['movie', 'running', 'math']\n",
        "for my_club_name in my_club_names:\n",
        "  output = llm.predict(club_name=my_club_name)\n",
        "  print(output)\n",
        "  print(\"--------------\")"
      ],
      "metadata": {
        "id": "zxKl_zy0XUGj"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Great work! Hope you saw how small programmatic changes to the context for the LLM made sometimes drastic changes and improvements to how the LLM behaved. Now off to do more things with LLMs!"
      ],
      "metadata": {
        "id": "5v6XeAkgz6t3"
      }
    }
  ]
}