examples/local/colab-pro.ipynb

{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "xc8k8pDJ769n"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/maritaca-ai/maritalk-api/blob/main/examples/local/colab-pro.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
        "\n",
        "# MariTalk Local\n",
        "\n",
        "Este notebook mostra como utilizar a MariTalk Local, que é a versão para deploy on-premises da [MariTalk](https://chat.maritaca.ai/). Para executar o modelo é necessário possuir uma licença, cujas instruções estão [nesta página](https://maritaca.ai/#maritalk-local).\n",
        "\n",
        "É necessária uma GPU com pelo menos 24GB de memória RAM para rodar o modelo Small. Se estiver usando o Google Colab, as GPUs que tem pelo menos 24GB são a L4 e A100, que estão disponíveis apenas na versão \"Pro\" (paga) do Colab."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hu30FsG-hIdv"
      },
      "source": [
        "### Instalação"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "24GmM8RwpQzR"
      },
      "outputs": [],
      "source": [
        "!python -m pip install maritalk -q"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YBUV1UaaEqw8",
        "outputId": "77631173-7f0c-46f3-d132-1afa74b86f4b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Wed Apr 17 16:58:05 2024       \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
            "|-----------------------------------------+----------------------+----------------------+\n",
            "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                                         |                      |               MIG M. |\n",
            "|=========================================+======================+======================|\n",
            "|   0  NVIDIA L4                      Off | 00000000:00:03.0 Off |                    0 |\n",
            "| N/A   42C    P8              11W /  72W |      1MiB / 23034MiB |      0%      Default |\n",
            "|                                         |                      |                  N/A |\n",
            "+-----------------------------------------+----------------------+----------------------+\n",
            "                                                                                         \n",
            "+---------------------------------------------------------------------------------------+\n",
            "| Processes:                                                                            |\n",
            "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
            "|        ID   ID                                                             Usage      |\n",
            "|=======================================================================================|\n",
            "|  No running processes found                                                           |\n",
            "+---------------------------------------------------------------------------------------+\n"
          ]
        }
      ],
      "source": [
        "!nvidia-smi"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zNdG6bJKhQDG"
      },
      "source": [
        "### Geração"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "XxK-ZnAvgt6p"
      },
      "outputs": [],
      "source": [
        "# @title Insira sua licença\n",
        "license = \"AAAAA-BBBBB-CCCCC-DDDDD\" # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RhXMrUDyEsmV",
        "outputId": "d14028eb-30b1-42da-a0f8-c7d9970e7764"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading MariTalk-small (path: /root/bin/maritalk)...\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/root/bin/maritalk: 100%|██████████| 14.6G/14.6G [08:10<00:00, 29.9MB/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Starting MariTalk Local API at http://localhost:9090\n"
          ]
        }
      ],
      "source": [
        "import maritalk\n",
        "client = maritalk.MariTalkLocal(port=9090)\n",
        "client.start_server(license=license)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LaAnOP9wegTo",
        "outputId": "e962f129-8bc7-4b33-eef2-5c204e9c67a6"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'status': 'idle'}"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "client.status()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iLg-Vf0GgDFT",
        "outputId": "964c3e1e-0dba-4531-e4fa-b79519d9b0ee"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "martin, finn e azul.\n"
          ]
        }
      ],
      "source": [
        "messages = [\n",
        "    {\"role\": \"user\", \"content\": \"sugira três nomes para a minha cachorra\"},\n",
        "    {\"role\": \"assistant\", \"content\": \"nina, bela e luna.\"},\n",
        "    {\"role\": \"user\", \"content\": \"e para o meu peixe?\"},\n",
        "]\n",
        "print(client.generate(messages)['output'])"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "L4",
      "machine_shape": "hm",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}