4. Talk to websites in 5 minutes.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Tf5yiyW8kACy"
      },
      "outputs": [],
      "source": [
        "!pip install requests beautifulsoup4\n",
        "!pip install openai"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UoOLGYiVB22Q"
      },
      "outputs": [],
      "source": [
        "import requests\n",
        "import json\n",
        "from io import BytesIO\n",
        "from openai import OpenAI"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IEaD88iAB47o"
      },
      "outputs": [],
      "source": [
        "### Your FARSPEAK app - log in on FARSPEAK.AI and create an app first\n",
        "FARSPEAK_AppName = \"\"\n",
        "FARSPEAK_Env = \"\"\n",
        "FARSPEAK_token = \"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "POWr32BdkfQN",
        "outputId": "91f1e821-ded5-4f29-e1af-795360711e60"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'headings': ['RESTful AI For Databases', 'RESTful AI For Databases', 'Across databases and datalakes', 'Up-to-date embeddings', 'Unstructured and structured data support', 'NLQ API', 'NLD API', 'RESTful and natural language querying', 'Key Features Of FARSPEAK', 'Who Is FARSPEAK For', 'From our community'], 'paragraphs': [], 'links': [{'text': '', 'url': 'index.html'}, {'text': 'Log In & Get Your API Key', 'url': 'https://dashboard.farspeak.ai'}, {'text': 'Learn More', 'url': '#benefits-section'}, {'text': '', 'url': 'https://colab.research.google.com/drive/1uKmjA_Q7nFW_lfzkBrQlbh4OWMu3JDEv?usp=sharing'}, {'text': 'Log In & Get Your API Key', 'url': ''}, {'text': 'Log In & Get Your API Key', 'url': ''}, {'text': 'Terms & Conditions', 'url': 'tc.html'}, {'text': 'Privacy Policy', 'url': 'pp.html'}]}\n"
          ]
        }
      ],
      "source": [
        "import requests\n",
        "from bs4 import BeautifulSoup\n",
        "\n",
        "# Specify the URL\n",
        "url = \"https://farspeak.ai\"\n",
        "\n",
        "# Fetch the web page\n",
        "response = requests.get(url)\n",
        "response.raise_for_status()  # Ensure the request was successful\n",
        "\n",
        "# Parse the HTML\n",
        "soup = BeautifulSoup(response.text, 'html.parser')\n",
        "\n",
        "# Extract headings and paragraphs\n",
        "data = {\n",
        "    'headings': [h.text.strip() for h in soup.find_all(['h1', 'h2', 'h3'])],\n",
        "    'paragraphs': [p.text.strip() for p in soup.find_all('p')],\n",
        "    'links': [{'text': a.text.strip(), 'url': a['href']} for a in soup.find_all('a', href=True)]\n",
        "}\n",
        "\n",
        "# Output the extracted data\n",
        "print(data)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "giOdrQILkKj-",
        "outputId": "709d37b4-ea01-46fc-f90b-2c38916889a6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[{'response': 'Based on the provided data, it seems to be related to a platform or service called FARSPEAK that offers RESTful AI for databases. Here are some insights based on the information provided:\\n\\n1. **Key Features of FARSPEAK**: The platform offers features such as RESTful AI for databases, up-to-date embeddings, support for unstructured and structured data, NLQ API, NLD API, and RESTful and natural language querying. These features indicate that FARSPEAK focuses on providing advanced AI capabilities for querying and interacting with databases using natural language.\\n\\n2. **Target Audience**: The section \"Who Is FARSPEAK For\" suggests that the platform is designed for a specific target audience. It would be beneficial to explore this section further to understand the specific user personas or industries that FARSPEAK caters to. This information can provide insights into the potential use cases and benefits of the platform.\\n\\n3. **Community Engagement**: The mention of \"From our community\" indicates that FARSPEAK has an active community of users or supporters. Understanding the community feedback, testimonials, or contributions can provide valuable insights into the platform\\'s reputation, user satisfaction, and potential areas for improvement.\\n\\n4. **Call-to-Action Links**: The presence of links such as \"Log In & Get Your API Key\" and \"Learn More\" suggests that FARSPEAK is encouraging users to take specific actions, such as signing up for an API key or exploring more information about the platform\\'s benefits. Analyzing the effectiveness of these call-to-action links can provide insights into user engagement and conversion rates.\\n\\n5. **External Links**: The inclusion of links to external resources such as Google Colab, Terms & Conditions, and Privacy Policy indicates that FARSPEAK is transparent about its policies and provides additional resources for users to explore. Analyzing the relevance and accessibility of these external links can help in understanding the platform\\'s credibility and commitment to user privacy and security.\\n\\nOverall, further exploration of the platform\\'s features, target audience, community engagement, and user interactions can provide deeper insights into FARSPEAK\\'s value proposition, user experience, and potential areas for growth and enhancement.'}]\n"
          ]
        }
      ],
      "source": [
        "OPENAI_API_KEY = ''\n",
        "\n",
        "# Generate insights from extracted content\n",
        "client = OpenAI(api_key= OPENAI_API_KEY)\n",
        "\n",
        "response = client.chat.completions.create(\n",
        "        model=\"gpt-3.5-turbo\",\n",
        "        messages= [\n",
        "    {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
        "    {\"role\": \"user\", \"content\": f\"Analyze the following data and provide insights: {data}\"}\n",
        "  ],\n",
        "        temperature=0,\n",
        "        max_tokens=1000\n",
        "    )\n",
        "\n",
        "# Structure the final JSON\n",
        "structured_json = [\n",
        "    {'response': response.choices[0].message.content.strip()}\n",
        "]\n",
        "\n",
        "# Output the JSON\n",
        "print(structured_json)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UJxm9PmZ-pSc"
      },
      "outputs": [],
      "source": [
        "# Upload a file\n",
        "# Use POST {{url}}/docs/{{app}}/{{env}}\n",
        "\n",
        "url = f\"https://api.farspeak.ai/docs/{FARSPEAK_AppName}/{FARSPEAK_Env}/website\"\n",
        "\n",
        "# Token for authentication (if needed)\n",
        "headers = {\n",
        "    'token': FARSPEAK_token\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZMliMimNndyV",
        "outputId": "1c58f10a-d671-4018-80f3-c009db6667c1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[{\"response\": \"Based on the provided data, it seems to be related to a platform or service called FARSPEAK that offers RESTful AI for databases. Here are some insights based on the information provided:\\n\\n1. **Key Features of FARSPEAK**: The platform offers features such as RESTful AI for databases, up-to-date embeddings, support for unstructured and structured data, NLQ API, NLD API, and RESTful and natural language querying. These features indicate that FARSPEAK focuses on providing advanced AI capabilities for querying and interacting with databases using natural language.\\n\\n2. **Target Audience**: The section \\\"Who Is FARSPEAK For\\\" suggests that the platform is designed for a specific target audience. It would be beneficial to explore this section further to understand the specific user personas or industries that FARSPEAK caters to. This information can provide insights into the potential use cases and benefits of the platform.\\n\\n3. **Community Engagement**: The mention of \\\"From our community\\\" indicates that FARSPEAK has an active community of users or supporters. Understanding the community feedback, testimonials, or contributions can provide valuable insights into the platform's reputation, user satisfaction, and potential areas for improvement.\\n\\n4. **Call-to-Action Links**: The presence of links such as \\\"Log In & Get Your API Key\\\" and \\\"Learn More\\\" suggests that FARSPEAK is encouraging users to take specific actions, such as signing up for an API key or exploring more information about the platform's benefits. Analyzing the effectiveness of these call-to-action links can provide insights into user engagement and conversion rates.\\n\\n5. **External Links**: The inclusion of links to external resources such as Google Colab, Terms & Conditions, and Privacy Policy indicates that FARSPEAK is transparent about its policies and provides additional resources for users to explore. Analyzing the relevance and accessibility of these external links can help in understanding the platform's credibility and commitment to user privacy and security.\\n\\nOverall, further exploration of the platform's features, target audience, community engagement, and user interactions can provide deeper insights into FARSPEAK's value proposition, user experience, and potential areas for growth and enhancement.\"}]\n",
            "{\"ids\":[\"qsvyzaok\"]}\n"
          ]
        }
      ],
      "source": [
        "# Add the first dataset\n",
        "print(json.dumps(structured_json))\n",
        "response = requests.post(f\"https://api.farspeak.ai/apps/{FARSPEAK_AppName}/{FARSPEAK_Env}/website\", headers={'token': f'{FARSPEAK_token}'}, data=json.dumps(structured_json))\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v8R7ER7GixH1",
        "outputId": "0df3fb58-aca3-4d7e-d7c5-cee97a99c539"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{\"answer\":\"Based on the provided data, it seems to be related to a platform or service called FARSPEAK that offers RESTful AI for databases. Here are some insights based on the information provided:\\n\\n1. **Key Features of FARSPEAK**: The platform offers features such as RESTful AI for databases, up-to-date embeddings, support for unstructured and structured data, NLQ API, NLD API, and RESTful and natural language querying. These features indicate that FARSPEAK focuses on providing advanced AI capabilities for querying and interacting with databases using natural language.\\n\\n2. **Target Audience**: The section \\\"Who Is FARSPEAK For\\\" suggests that the platform is designed for a specific target audience. It would be beneficial to explore this section further to understand the specific user personas or industries that FARSPEAK caters to. This information can provide insights into the potential use cases and benefits of the platform.\\n\\n3. **Community Engagement**: The mention of \\\"From our community\\\" indicates that FARSPEAK has an active community of users or supporters. Understanding the community feedback, testimonials, or contributions can provide valuable insights into the platform's reputation, user satisfaction, and potential areas for improvement.\\n\\n4. **Call-to-Action Links**: The presence of links such as \\\"Log In & Get Your API Key\\\" and \\\"Learn More\\\" suggests that FARSPEAK is encouraging users to take specific actions, such as signing up for an API key or exploring more information about the platform's benefits. Analyzing the effectiveness of these call-to-action links can provide insights into user engagement and conversion rates.\\n\\n5. **External Links**: The inclusion of links to external resources such as Google Colab, Terms & Conditions, and Privacy Policy indicates that FARSPEAK is transparent about its policies and provides additional resources for users to explore. Analyzing the relevance and accessibility of these external links can help in understanding the platform's credibility and commitment to user privacy and security.\\n\\nOverall, further exploration of the platform's features, target audience, community engagement, and user interactions can provide deeper insights into FARSPEAK's value proposition, user experience, and potential areas for growth and enhancement.\"}\n"
          ]
        }
      ],
      "source": [
        "# Use your knowledge base\n",
        "\n",
        "response = requests.post(f\"https://api.farspeak.ai/knowledgebase/{FARSPEAK_AppName}/{FARSPEAK_Env}\",\n",
        "                         headers={'token': f'{FARSPEAK_token}'},\n",
        "                         json={'query' : 'What are your insights related to FARSPEAK?'})\n",
        "print(response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iEUMlO8M2h4Y"
      },
      "source": [
        "---"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}