sherpa-onnx/sherpa_onnx_python_non_streaming_websocket_server.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyPxRBzrOZhw4hVrNn9s3eTw",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/k2-fsa/colab/blob/master/sherpa-onnx/sherpa_onnx_python_non_streaming_websocket_server.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Introduction\n",
        "\n",
        "This colab notebook shows how to run the Python non-streaming WebSocket server in [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx)\n",
        "\n",
        "Please refer to https://k2-fsa.github.io/sherpa/onnx/python/non-streaming-websocket-server.html for details.\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "YQiOR3yILLm0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Install sherpa-onnx\n"
      ],
      "metadata": {
        "id": "lXxKs3jCLY0f"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IuU8clt6LJ3u",
        "outputId": "5d921234-e65e-42ad-9011-eec54e947eff"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting sherpa-onnx\n",
            "  Downloading sherpa_onnx-1.7.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m13.8/13.8 MB\u001b[0m \u001b[31m85.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from sherpa-onnx) (1.23.5)\n",
            "Collecting sentencepiece==0.1.96 (from sherpa-onnx)\n",
            "  Downloading sentencepiece-0.1.96-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m51.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: sentencepiece, sherpa-onnx\n",
            "Successfully installed sentencepiece-0.1.96 sherpa-onnx-1.7.2\n",
            "Collecting websockets\n",
            "  Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.9/129.9 kB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: websockets\n",
            "Successfully installed websockets-11.0.3\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 1
        }
      ],
      "source": [
        "%%shell\n",
        "\n",
        "pip install sherpa-onnx\n",
        "pip install websockets"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We use the following types of models for demonstration.\n",
        "\n",
        "|Description|URL|\n",
        "|---|---|\n",
        "|Non-streaming transducer|[csukuangfj/sherpa-onnx-zipformer-en-2023-06-26 (English)](http://localhost:6007/onnx/pretrained_models/offline-transducer/zipformer-transducer-models.html#sherpa-onnx-zipformer-en-2023-06-26-english)|\n",
        "|Non-streaming paraformer|[csukuangfj/sherpa-onnx-paraformer-zh-2023-03-28 (Chinese)](http://localhost:6007/onnx/pretrained_models/offline-paraformer/paraformer-models.html#sherpa-onnx-offline-paraformer-zh-2023-03-28-chinese)|\n",
        "|Non-streaming CTC model from NeMo|[stt_en_conformer_ctc_medium](http://localhost:6007/onnx/pretrained_models/offline-ctc/nemo/english.html#stt-en-conformer-ctc-medium-nemo-sherpa-onnx)|\n",
        "|Non-streaming Whisper tiny.en|[tiny.en](http://localhost:6007/onnx/pretrained_models/whisper/tiny.en.html#whisper-tiny-en-sherpa-onnx)|"
      ],
      "metadata": {
        "id": "iTKhibjrLh3x"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We download the python example code first:"
      ],
      "metadata": {
        "id": "gui4xDCNMEqz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "git clone https://github.com/k2-fsa/sherpa-onnx"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VLgpP87kMHft",
        "outputId": "2a522fec-fe94-4dbd-d989-bdd4a4a046e4"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cloning into 'sherpa-onnx'...\n",
            "remote: Enumerating objects: 2672, done.\u001b[K\n",
            "remote: Counting objects: 100% (1149/1149), done.\u001b[K\n",
            "remote: Compressing objects: 100% (456/456), done.\u001b[K\n",
            "remote: Total 2672 (delta 807), reused 848 (delta 682), pack-reused 1523\u001b[K\n",
            "Receiving objects: 100% (2672/2672), 1.92 MiB | 23.40 MiB/s, done.\n",
            "Resolving deltas: 100% (1539/1539), done.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Non-streaming transducer"
      ],
      "metadata": {
        "id": "tzQXX_ybL_BU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Start the server"
      ],
      "metadata": {
        "id": "ikKZOXtxMCb8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "cd sherpa-onnx\n",
        "\n",
        "GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/csukuangfj/sherpa-onnx-zipformer-en-2023-06-26\n",
        "cd sherpa-onnx-zipformer-en-2023-06-26\n",
        "git lfs pull --include \"*.onnx\"\n",
        "cd ..\n",
        "\n",
        "nohup python3 ./python-api-examples/non_streaming_server.py \\\n",
        "  --encoder ./sherpa-onnx-zipformer-en-2023-06-26/encoder-epoch-99-avg-1.onnx \\\n",
        "  --decoder ./sherpa-onnx-zipformer-en-2023-06-26/decoder-epoch-99-avg-1.onnx \\\n",
        "  --joiner ./sherpa-onnx-zipformer-en-2023-06-26/joiner-epoch-99-avg-1.onnx \\\n",
        "  --tokens ./sherpa-onnx-zipformer-en-2023-06-26/tokens.txt \\\n",
        "  --port 6006 > non-streaming-transducer.log &"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cyWdDYxjLdeg",
        "outputId": "c074d6c3-317b-4dc0-e8f7-8e2801e729fb"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cloning into 'sherpa-onnx-zipformer-en-2023-06-26'...\n",
            "remote: Enumerating objects: 20, done.\u001b[K\n",
            "remote: Total 20 (delta 0), reused 0 (delta 0), pack-reused 20\u001b[K\n",
            "Unpacking objects: 100% (20/20), 666.78 KiB | 11.11 MiB/s, done.\n",
            "nohup: redirecting stderr to stdout\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Start the client"
      ],
      "metadata": {
        "id": "gAg9FN8uMSra"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Decode multiple files in parallel\n",
        "\n"
      ],
      "metadata": {
        "id": "PEmCZfXGMWYU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "# sleep 10 second to wait for the server startup\n",
        "sleep 10\n",
        "\n",
        "cd sherpa-onnx\n",
        "\n",
        "python3 ./python-api-examples/offline-websocket-client-decode-files-paralell.py \\\n",
        "  --server-addr localhost \\\n",
        "  --server-port 6006 \\\n",
        "  ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/0.wav \\\n",
        "  ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/1.wav \\\n",
        "  ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/8k.wav"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cS3mNM2nMRec",
        "outputId": "d960ba77-7d86-4c19-cf35-4f8098924a73"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2023-08-11 10:56:41,931 INFO [offline-websocket-client-decode-files-paralell.py:139] {'server_addr': 'localhost', 'server_port': 6006, 'sound_files': ['./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/0.wav', './sherpa-onnx-zipformer-en-2023-06-26/test_wavs/1.wav', './sherpa-onnx-zipformer-en-2023-06-26/test_wavs/8k.wav']}\n",
            "2023-08-11 10:56:41,953 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/0.wav\n",
            "2023-08-11 10:56:42,000 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/1.wav\n",
            "2023-08-11 10:56:42,174 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/8k.wav\n",
            "2023-08-11 10:56:42,629 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/0.wav\n",
            " AFTER EARLY NIGHTFALL THE YELLOW LAMPS WOULD LIGHT UP HERE AND THERE THE SQUALID QUARTER OF THE BROTHELS\n",
            "2023-08-11 10:56:42,639 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/8k.wav\n",
            " YET THESE THOUGHTS AFFECTED HESTER PRYNNE LESS WITH HOPE THAN APPREHENSION\n",
            "2023-08-11 10:56:43,376 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/1.wav\n",
            " GOD AS A DIRECT CONSEQUENCE OF THE SIN WHICH MAN THUS PUNISHED HAD GIVEN HER A LOVELY CHILD WHOSE PLACE WAS ON THAT SAME DISHONORED BOSOM TO CONNECT HER PARENT FOREVER WITH THE RACE AND DESCENT OF MORTALS AND TO BE FINALLY A BLESSED SOUL IN HEAVEN\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Decode multiple files sequentially\n",
        "\n"
      ],
      "metadata": {
        "id": "gB9oa5VzMjMZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "cd sherpa-onnx\n",
        "\n",
        "python3 ./python-api-examples/offline-websocket-client-decode-files-sequential.py \\\n",
        "  --server-addr localhost \\\n",
        "  --server-port 6006 \\\n",
        "  ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/0.wav \\\n",
        "  ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/1.wav \\\n",
        "  ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/8k.wav"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WlFEuyvsMd2S",
        "outputId": "db24c7ba-f54b-4a6c-b60a-d0d15f01bfd9"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2023-08-11 10:56:43,576 INFO [offline-websocket-client-decode-files-sequential.py:141] {'server_addr': 'localhost', 'server_port': 6006, 'sound_files': ['./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/0.wav', './sherpa-onnx-zipformer-en-2023-06-26/test_wavs/1.wav', './sherpa-onnx-zipformer-en-2023-06-26/test_wavs/8k.wav']}\n",
            "2023-08-11 10:56:43,595 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/0.wav\n",
            " AFTER EARLY NIGHTFALL THE YELLOW LAMPS WOULD LIGHT UP HERE AND THERE THE SQUALID QUARTER OF THE BROTHELS\n",
            "2023-08-11 10:56:43,912 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/1.wav\n",
            " GOD AS A DIRECT CONSEQUENCE OF THE SIN WHICH MAN THUS PUNISHED HAD GIVEN HER A LOVELY CHILD WHOSE PLACE WAS ON THAT SAME DISHONORED BOSOM TO CONNECT HER PARENT FOREVER WITH THE RACE AND DESCENT OF MORTALS AND TO BE FINALLY A BLESSED SOUL IN HEAVEN\n",
            "2023-08-11 10:56:44,758 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-zipformer-en-2023-06-26/test_wavs/8k.wav\n",
            " YET THESE THOUGHTS AFFECTED HESTER PRYNNE LESS WITH HOPE THAN APPREHENSION\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Non-streaming paraformer"
      ],
      "metadata": {
        "id": "zI3zoaV_MyRL"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Start the server"
      ],
      "metadata": {
        "id": "1Enw4AhTM09t"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "cd sherpa-onnx\n",
        "\n",
        "GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/csukuangfj/sherpa-onnx-paraformer-zh-2023-03-28\n",
        "cd sherpa-onnx-paraformer-zh-2023-03-28\n",
        "git lfs pull --include \"*.onnx\"\n",
        "cd ..\n",
        "\n",
        "nohup python3 ./python-api-examples/non_streaming_server.py \\\n",
        "  --paraformer ./sherpa-onnx-paraformer-zh-2023-03-28/model.int8.onnx \\\n",
        "  --tokens ./sherpa-onnx-paraformer-zh-2023-03-28/tokens.txt \\\n",
        "  --port 6007 > non-streaming-paraformer.log &"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JSn9FT1CMn7l",
        "outputId": "133b9fff-b2b4-41b1-8954-9e9038baf813"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cloning into 'sherpa-onnx-paraformer-zh-2023-03-28'...\n",
            "remote: Enumerating objects: 17, done.\u001b[K\n",
            "remote: Total 17 (delta 0), reused 0 (delta 0), pack-reused 17\u001b[K\n",
            "Unpacking objects: 100% (17/17), 491.76 KiB | 15.86 MiB/s, done.\n",
            "nohup: redirecting stderr to stdout\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Start the client"
      ],
      "metadata": {
        "id": "k6nMzSHHM8d8"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Decode multiple files in parallel\n",
        "\n"
      ],
      "metadata": {
        "id": "PyTccyRDM_kj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "sleep 10\n",
        "cd sherpa-onnx\n",
        "\n",
        "python3 ./python-api-examples/offline-websocket-client-decode-files-paralell.py \\\n",
        "  --server-addr localhost \\\n",
        "  --server-port 6007 \\\n",
        "  ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/0.wav \\\n",
        "  ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/1.wav \\\n",
        "  ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/2.wav \\\n",
        "  ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/8k.wav"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8_Gq9956M7Ua",
        "outputId": "a72dfee8-b7ce-40ba-9e1c-83deec3e0939"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2023-08-11 10:57:16,459 INFO [offline-websocket-client-decode-files-paralell.py:139] {'server_addr': 'localhost', 'server_port': 6007, 'sound_files': ['./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/0.wav', './sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/1.wav', './sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/2.wav', './sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/8k.wav']}\n",
            "2023-08-11 10:57:16,482 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/0.wav\n",
            "2023-08-11 10:57:16,522 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/1.wav\n",
            "2023-08-11 10:57:16,569 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/2.wav\n",
            "2023-08-11 10:57:16,632 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/8k.wav\n",
            "2023-08-11 10:57:17,804 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/0.wav\n",
            "对我做了介绍啊那么我想说的是呢大家如果对我的研究感兴趣呢你\n",
            "2023-08-11 10:57:17,805 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/8k.wav\n",
            "甚至出现交易几乎停滞的情况\n",
            "2023-08-11 10:57:17,806 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/1.wav\n",
            "重点呢想谈三个问题首先呢就是这一轮全球金融动荡的表现\n",
            "2023-08-11 10:57:17,806 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/2.wav\n",
            "深入的分析这一次全球金融动荡背后的根源\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Decode multiple files sequentially\n",
        "\n"
      ],
      "metadata": {
        "id": "B0xrze38NFjQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "cd sherpa-onnx\n",
        "\n",
        "python3 ./python-api-examples/offline-websocket-client-decode-files-sequential.py \\\n",
        "  --server-addr localhost \\\n",
        "  --server-port 6007 \\\n",
        "  ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/0.wav \\\n",
        "  ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/1.wav \\\n",
        "  ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/2.wav \\\n",
        "  ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/8k.wav"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6SWSV5qnNE5j",
        "outputId": "6747103d-34d2-4c64-a69b-bb83a833dca6"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2023-08-11 10:57:17,996 INFO [offline-websocket-client-decode-files-sequential.py:141] {'server_addr': 'localhost', 'server_port': 6007, 'sound_files': ['./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/0.wav', './sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/1.wav', './sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/2.wav', './sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/8k.wav']}\n",
            "2023-08-11 10:57:18,015 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/0.wav\n",
            "对我做了介绍啊那么我想说的是呢大家如果对我的研究感兴趣呢你\n",
            "2023-08-11 10:57:18,406 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/1.wav\n",
            "重点呢想谈三个问题首先呢就是这一轮全球金融动荡的表现\n",
            "2023-08-11 10:57:18,746 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/2.wav\n",
            "深入的分析这一次全球金融动荡背后的根源\n",
            "2023-08-11 10:57:19,055 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-paraformer-zh-2023-03-28/test_wavs/8k.wav\n",
            "甚至出现交易几乎停滞的情况\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Non-streaming CTC model from NeMo"
      ],
      "metadata": {
        "id": "nWt4nG1DN4cg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Start the server"
      ],
      "metadata": {
        "id": "LLQ8P2gFN-mb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "cd sherpa-onnx\n",
        "\n",
        "GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/csukuangfj/sherpa-onnx-nemo-ctc-en-conformer-medium\n",
        "cd sherpa-onnx-nemo-ctc-en-conformer-medium\n",
        "git lfs pull --include \"*.onnx\"\n",
        "cd ..\n",
        "\n",
        "nohup python3 ./python-api-examples/non_streaming_server.py \\\n",
        "  --nemo-ctc ./sherpa-onnx-nemo-ctc-en-conformer-medium/model.onnx \\\n",
        "  --tokens ./sherpa-onnx-nemo-ctc-en-conformer-medium/tokens.txt \\\n",
        "  --port 6008 > nemo-ctc.log &"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bjQ-q8G9NJ0y",
        "outputId": "d17594a6-f7d1-4102-9f7f-69c776747c37"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cloning into 'sherpa-onnx-nemo-ctc-en-conformer-medium'...\n",
            "remote: Enumerating objects: 20, done.\u001b[K\n",
            "remote: Total 20 (delta 0), reused 0 (delta 0), pack-reused 20\u001b[K\n",
            "Unpacking objects: 100% (20/20), 671.36 KiB | 15.98 MiB/s, done.\n",
            "nohup: redirecting stderr to stdout\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Start the client"
      ],
      "metadata": {
        "id": "lW24HtOkOEtx"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Decode multiple files in parallel"
      ],
      "metadata": {
        "id": "26k0JhwMOHQH"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "sleep 10\n",
        "cd sherpa-onnx\n",
        "\n",
        "python3 ./python-api-examples/offline-websocket-client-decode-files-paralell.py \\\n",
        "  --server-addr localhost \\\n",
        "  --server-port 6008 \\\n",
        "  ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/0.wav \\\n",
        "  ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/1.wav \\\n",
        "  ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/8k.wav"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wvyrl4duODnV",
        "outputId": "f60c7aca-2940-457b-fdde-1047ab70278c"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2023-08-11 10:57:33,709 INFO [offline-websocket-client-decode-files-paralell.py:139] {'server_addr': 'localhost', 'server_port': 6008, 'sound_files': ['./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/0.wav', './sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/1.wav', './sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/8k.wav']}\n",
            "2023-08-11 10:57:33,733 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/0.wav\n",
            "2023-08-11 10:57:33,783 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/1.wav\n",
            "2023-08-11 10:57:34,026 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/8k.wav\n",
            "2023-08-11 10:57:34,304 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/0.wav\n",
            " after early nightfall the yellow lamps would light up here and there the squalid quarter of the brothels\n",
            "2023-08-11 10:57:35,020 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/8k.wav\n",
            " yet these thoughts affected hester pryne less with hope than apprehension\n",
            "2023-08-11 10:57:35,021 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/1.wav\n",
            " god as a direct consequence of the sin which man thus punished had given her a lovely child whose place was on that same dishonored bosom to connect her parent for ever with the race and descent of mortals and to be finally a blessed soul in heaven\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Decode multiple files sequentially\n",
        "\n"
      ],
      "metadata": {
        "id": "bnTw74ESOYpT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "cd sherpa-onnx\n",
        "\n",
        "python3 ./python-api-examples/offline-websocket-client-decode-files-sequential.py \\\n",
        "  --server-addr localhost \\\n",
        "  --server-port 6008 \\\n",
        "  ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/0.wav \\\n",
        "  ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/1.wav \\\n",
        "  ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/8k.wav"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3rC9G5A6OXhN",
        "outputId": "2c84b1d1-0b23-4257-e64b-3581835be120"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2023-08-11 10:57:35,197 INFO [offline-websocket-client-decode-files-sequential.py:141] {'server_addr': 'localhost', 'server_port': 6008, 'sound_files': ['./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/0.wav', './sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/1.wav', './sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/8k.wav']}\n",
            "2023-08-11 10:57:35,216 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/0.wav\n",
            " after early nightfall the yellow lamps would light up here and there the squalid quarter of the brothels\n",
            "2023-08-11 10:57:35,518 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/1.wav\n",
            " god as a direct consequence of the sin which man thus punished had given her a lovely child whose place was on that same dishonored bosom to connect her parent for ever with the race and descent of mortals and to be finally a blessed soul in heaven\n",
            "2023-08-11 10:57:36,318 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-nemo-ctc-en-conformer-medium/test_wavs/8k.wav\n",
            " yet these thoughts affected hester pryne less with hope than apprehension\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Non-streaming Whisper tiny.en"
      ],
      "metadata": {
        "id": "qMqtZM60Oi6i"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Start the server"
      ],
      "metadata": {
        "id": "aA9cBdyEOlby"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "cd sherpa-onnx\n",
        "GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/csukuangfj/sherpa-onnx-whisper-tiny.en\n",
        "cd sherpa-onnx-whisper-tiny.en\n",
        "git lfs pull --include \"*.onnx\"\n",
        "cd ..\n",
        "\n",
        "nohup python3 ./python-api-examples/non_streaming_server.py \\\n",
        "  --whisper-encoder=./sherpa-onnx-whisper-tiny.en/tiny.en-encoder.onnx \\\n",
        "  --whisper-decoder=./sherpa-onnx-whisper-tiny.en/tiny.en-decoder.onnx \\\n",
        "  --tokens=./sherpa-onnx-whisper-tiny.en/tiny.en-tokens.txt \\\n",
        "  --port 6009  > whisper-tiny.en.log &"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YKV1c82lOezH",
        "outputId": "3235012d-863d-403b-e932-da8ca14c5c5c"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cloning into 'sherpa-onnx-whisper-tiny.en'...\n",
            "remote: Enumerating objects: 30, done.\u001b[K\n",
            "remote: Counting objects:   3% (1/30)\u001b[K\rremote: Counting objects:   6% (2/30)\u001b[K\rremote: Counting objects:  10% (3/30)\u001b[K\rremote: Counting objects:  13% (4/30)\u001b[K\rremote: Counting objects:  16% (5/30)\u001b[K\rremote: Counting objects:  20% (6/30)\u001b[K\rremote: Counting objects:  23% (7/30)\u001b[K\rremote: Counting objects:  26% (8/30)\u001b[K\rremote: Counting objects:  30% (9/30)\u001b[K\rremote: Counting objects:  33% (10/30)\u001b[K\rremote: Counting objects:  36% (11/30)\u001b[K\rremote: Counting objects:  40% (12/30)\u001b[K\rremote: Counting objects:  43% (13/30)\u001b[K\rremote: Counting objects:  46% (14/30)\u001b[K\rremote: Counting objects:  50% (15/30)\u001b[K\rremote: Counting objects:  53% (16/30)\u001b[K\rremote: Counting objects:  56% (17/30)\u001b[K\rremote: Counting objects:  60% (18/30)\u001b[K\rremote: Counting objects:  63% (19/30)\u001b[K\rremote: Counting objects:  66% (20/30)\u001b[K\rremote: Counting objects:  70% (21/30)\u001b[K\rremote: Counting objects:  73% (22/30)\u001b[K\rremote: Counting objects:  76% (23/30)\u001b[K\rremote: Counting objects:  80% (24/30)\u001b[K\rremote: Counting objects:  83% (25/30)\u001b[K\rremote: Counting objects:  86% (26/30)\u001b[K\rremote: Counting objects:  90% (27/30)\u001b[K\rremote: Counting objects:  93% (28/30)\u001b[K\rremote: Counting objects:  96% (29/30)\u001b[K\rremote: Counting objects: 100% (30/30)\u001b[K\rremote: Counting objects: 100% (30/30), done.\u001b[K\n",
            "remote: Compressing objects:   3% (1/29)\u001b[K\rremote: Compressing objects:   6% (2/29)\u001b[K\rremote: Compressing objects:  10% (3/29)\u001b[K\rremote: Compressing objects:  13% (4/29)\u001b[K\rremote: Compressing objects:  17% (5/29)\u001b[K\rremote: Compressing objects:  20% (6/29)\u001b[K\rremote: Compressing objects:  24% (7/29)\u001b[K\rremote: Compressing objects:  27% (8/29)\u001b[K\rremote: Compressing objects:  31% (9/29)\u001b[K\rremote: Compressing objects:  34% (10/29)\u001b[K\rremote: Compressing objects:  37% (11/29)\u001b[K\rremote: Compressing objects:  41% (12/29)\u001b[K\rremote: Compressing objects:  44% (13/29)\u001b[K\rremote: Compressing objects:  48% (14/29)\u001b[K\rremote: Compressing objects:  51% (15/29)\u001b[K\rremote: Compressing objects:  55% (16/29)\u001b[K\rremote: Compressing objects:  58% (17/29)\u001b[K\rremote: Compressing objects:  62% (18/29)\u001b[K\rremote: Compressing objects:  65% (19/29)\u001b[K\rremote: Compressing objects:  68% (20/29)\u001b[K\rremote: Compressing objects:  72% (21/29)\u001b[K\rremote: Compressing objects:  75% (22/29)\u001b[K\rremote: Compressing objects:  79% (23/29)\u001b[K\rremote: Compressing objects:  82% (24/29)\u001b[K\rremote: Compressing objects:  86% (25/29)\u001b[K\rremote: Compressing objects:  89% (26/29)\u001b[K\rremote: Compressing objects:  93% (27/29)\u001b[K\rremote: Compressing objects:  96% (28/29)\u001b[K\rremote: Compressing objects: 100% (29/29)\u001b[K\rremote: Compressing objects: 100% (29/29), done.\u001b[K\n",
            "Unpacking objects:   3% (1/30)\rUnpacking objects:   6% (2/30)\rUnpacking objects:  10% (3/30)\rUnpacking objects:  13% (4/30)\rUnpacking objects:  16% (5/30)\rUnpacking objects:  20% (6/30)\rUnpacking objects:  23% (7/30)\rUnpacking objects:  26% (8/30)\rUnpacking objects:  30% (9/30)\rUnpacking objects:  33% (10/30)\rUnpacking objects:  36% (11/30)\rUnpacking objects:  40% (12/30)\rUnpacking objects:  43% (13/30)\rUnpacking objects:  46% (14/30)\rUnpacking objects:  50% (15/30)\rUnpacking objects:  53% (16/30)\rUnpacking objects:  56% (17/30)\rUnpacking objects:  60% (18/30)\rUnpacking objects:  63% (19/30)\rUnpacking objects:  66% (20/30)\rUnpacking objects:  70% (21/30)\rremote: Total 30 (delta 3), reused 0 (delta 0), pack-reused 0\u001b[K\n",
            "Unpacking objects: 100% (30/30), 1023.85 KiB | 9.06 MiB/s, done.\n",
            "nohup: redirecting stderr to stdout\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Start the client"
      ],
      "metadata": {
        "id": "Cj-DtAiNOxdU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Decode multiple files in parallel\n",
        "\n"
      ],
      "metadata": {
        "id": "Tc8les8iOywn"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "sleep 10\n",
        "cd sherpa-onnx\n",
        "\n",
        "python3 ./python-api-examples/offline-websocket-client-decode-files-paralell.py \\\n",
        "  --server-addr localhost \\\n",
        "  --server-port 6009 \\\n",
        "  ./sherpa-onnx-whisper-tiny.en/test_wavs/0.wav \\\n",
        "  ./sherpa-onnx-whisper-tiny.en/test_wavs/1.wav \\\n",
        "  ./sherpa-onnx-whisper-tiny.en/test_wavs/8k.wav"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NKZ37VkKOwWr",
        "outputId": "f39f99f0-3f96-4ae6-b0a4-e3b81dd61d32"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2023-08-11 10:57:54,399 INFO [offline-websocket-client-decode-files-paralell.py:139] {'server_addr': 'localhost', 'server_port': 6009, 'sound_files': ['./sherpa-onnx-whisper-tiny.en/test_wavs/0.wav', './sherpa-onnx-whisper-tiny.en/test_wavs/1.wav', './sherpa-onnx-whisper-tiny.en/test_wavs/8k.wav']}\n",
            "2023-08-11 10:57:54,423 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-whisper-tiny.en/test_wavs/0.wav\n",
            "2023-08-11 10:57:54,475 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-whisper-tiny.en/test_wavs/1.wav\n",
            "2023-08-11 10:57:55,633 INFO [offline-websocket-client-decode-files-paralell.py:113] Sending ./sherpa-onnx-whisper-tiny.en/test_wavs/8k.wav\n",
            "2023-08-11 10:57:55,659 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-whisper-tiny.en/test_wavs/0.wav\n",
            " After early nightfall, the yellow lamps would light up here and there, the squalid quarter of the brothels.\n",
            "2023-08-11 10:57:57,619 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-whisper-tiny.en/test_wavs/1.wav\n",
            " God, as a direct consequence of the sin which man thus punished, had given her a lovely child, whose place was on that same dishonored bosom to connect her parent forever with the race and descent of mortals, and to be finally a blessed soul in heaven.\n",
            "2023-08-11 10:57:58,412 INFO [offline-websocket-client-decode-files-paralell.py:131] ./sherpa-onnx-whisper-tiny.en/test_wavs/8k.wav\n",
            " Yet these thoughts affected Hester Prin less with hope than apprehension.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Decode multiple files sequentially\n",
        "\n"
      ],
      "metadata": {
        "id": "ZYnR1RAbO3n2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%shell\n",
        "\n",
        "cd sherpa-onnx\n",
        "\n",
        "python3 ./python-api-examples/offline-websocket-client-decode-files-sequential.py \\\n",
        "  --server-addr localhost \\\n",
        "  --server-port 6009 \\\n",
        "  ./sherpa-onnx-whisper-tiny.en/test_wavs/0.wav \\\n",
        "  ./sherpa-onnx-whisper-tiny.en/test_wavs/1.wav \\\n",
        "  ./sherpa-onnx-whisper-tiny.en/test_wavs/8k.wav"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HOQ38jk_O3Fo",
        "outputId": "30705edd-6f46-4eed-83e4-f755508c88df"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2023-08-11 10:57:58,701 INFO [offline-websocket-client-decode-files-sequential.py:141] {'server_addr': 'localhost', 'server_port': 6009, 'sound_files': ['./sherpa-onnx-whisper-tiny.en/test_wavs/0.wav', './sherpa-onnx-whisper-tiny.en/test_wavs/1.wav', './sherpa-onnx-whisper-tiny.en/test_wavs/8k.wav']}\n",
            "2023-08-11 10:57:58,724 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-whisper-tiny.en/test_wavs/0.wav\n",
            " After early nightfall, the yellow lamps would light up here and there, the squalid quarter of the brothels.\n",
            "2023-08-11 10:57:59,992 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-whisper-tiny.en/test_wavs/1.wav\n",
            " God, as a direct consequence of the sin which man thus punished, had given her a lovely child, whose place was on that same dishonored bosom to connect her parent forever with the race and descent of mortals, and to be finally a blessed soul in heaven.\n",
            "2023-08-11 10:58:01,930 INFO [offline-websocket-client-decode-files-sequential.py:114] Sending ./sherpa-onnx-whisper-tiny.en/test_wavs/8k.wav\n",
            " Yet these thoughts affected Hester Prin less with hope than apprehension.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": []
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "iVmtLwW5O_Lx"
      },
      "execution_count": 14,
      "outputs": []
    }
  ]
}