contextual_answering.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BDZhEv4BP76c"
      },
      "source": [
        "## Text extraction"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PHnUy3X0P76d",
        "outputId": "3b7ed061-0429-4a5c-e824-5210e20ec4b6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: pdfminer.six in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (20221105)Note: you may need to restart the kernel to use updated packages.\n",
            "\n",
            "Requirement already satisfied: charset-normalizer>=2.0.0 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from pdfminer.six) (3.1.0)\n",
            "Requirement already satisfied: cryptography>=36.0.0 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from pdfminer.six) (41.0.2)\n",
            "Requirement already satisfied: cffi>=1.12 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from cryptography>=36.0.0->pdfminer.six) (1.15.1)\n",
            "Requirement already satisfied: pycparser in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from cffi>=1.12->cryptography>=36.0.0->pdfminer.six) (2.21)\n"
          ]
        }
      ],
      "source": [
        "%pip install pdfminer.six\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QBtZdBSDP76e",
        "outputId": "4a83a3cb-f334-484f-9012-bf7f48a15c71"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Business Process Operations\n",
            "取扱厳秘\n",
            "§ SAP Business Process Operations comprises the most important application-related operations topics to \n",
            "ensure the smooth and reliable flow of the core business processes, to meet a company’s business \n",
            "requirements. \n",
            "§ SAP Business Process Operations comprises the following processes: \n",
            "q Business process monitoring \n",
            "q Business process improvement\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "q Job management and job monitoring\n",
            "q Data consistency management\n",
            "Page 3\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Management\n",
            "Challenges of Job Management\n",
            "取扱厳秘\n",
            "§\n",
            "In  a  highly  complex  and  distributed  system  landscape,  job  scheduling  management  is  important  to  help  manage  data  and  its \n",
            "processing. \n",
            "Challenge\n",
            "Goal\n",
            "Governance\n",
            "Capability\n",
            "Job Request\n",
            "Visibility\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "ü Provide standardized & web-based documentation\n",
            "ü Avoid unnecessary job processing (“forgotten” jobs)\n",
            "ü Avoid unnecessary slow support processes\n",
            "ü “Living” documentation vs, one-time documentation\n",
            "Job Documentation\n",
            "ü Provide standardized & web-based request form\n",
            "ü Integration with IT service management process\n",
            "ü Integration with change management process\n",
            "ü Integration with documentation, scheduling & monitoring\n",
            "Efficiency\n",
            "Job Scheduling\n",
            "Effectiveness\n",
            "Job Monitoring\n",
            "ü Automate job processing where possible (job chains)\n",
            "ü Schedule across system boundaries\n",
            "ü Event-driven vs time-driven scheduling\n",
            "ü Control end-user jobs (job interception)\n",
            "ü Ensure that jobs start/finish at the right time\n",
            "ü Ensure job processing within given time windows\n",
            "ü Correlate important jobs with Business processes\n",
            "ü Management by exception\n",
            "Page 4\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Management\n",
            "Job Management Administration\n",
            "取扱厳秘\n",
            "§ You can open the Job Management Work Center to access the following capabilities as there are no individual \n",
            "tiles for them.\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "§ Overview\n",
            "This view provides overview information with direct links to the personal object worklist (POWL) queries defined for the work center \n",
            "views.\n",
            "§ External Scheduler Management\n",
            "Provides a view to manage jobs scheduled with a connected, certified external job scheduler.\n",
            "§ SAP Scheduler Management\n",
            "In this view, you can manage jobs scheduled with the SAP Scheduler via the BC-XBP interface.\n",
            "Page 5\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Management\n",
            "Job Requests\n",
            "取扱厳秘\n",
            "§ The job request is the starting point of the job management process. End users create job requests for new \n",
            "jobs, job changes, and job deletion. Job requests undergo an end-to-end approval and implementation \n",
            "process, with the workflow handled by IT Service Management and Change Request Management.\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Page 6\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Management\n",
            "Job Documentation\n",
            "取扱厳秘\n",
            "§ Job documentation is the central object of the Job Management standard. It allows you to centrally manage, \n",
            "schedule, and document in detail the background processing of your entire system landscape.\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Page 7\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Management\n",
            "Job Scheduling Assistant\n",
            "取扱厳秘\n",
            "§ With SAP Solution Manager, you can schedule jobs in your managed systems from job documentation via the \n",
            "SAP scheduler (BC-XBP interface) or an external scheduler (SMSE interface). You can also schedule jobs \n",
            "directly, without job documentation.\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Page 8\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Management\n",
            "Job Management Administration\n",
            "§ Recommended Jobs\n",
            "Provides  a  view  on  background  jobs  in  your  system  landscape, \n",
            "organized by system.\n",
            "取扱厳秘\n",
            "§ Task Inbox\n",
            "Provides  a  list  of  job  documentation  tasks.  If  authorized,  you  can \n",
            "create new tasks and display, edit, and delete existing tasks..\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "to  perform  various  configuration  and \n",
            "§ Administration\n",
            "Provides  you  access \n",
            "management tasks.\n",
            "Page 9\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Monitoring\n",
            "Overview\n",
            "取扱厳秘\n",
            "§ Job monitoring belongs to important central monitoring tasks and can ensure that background jobs and job \n",
            "chains are running as expected.\n",
            "§ SAP Solution Manager provides a complete, centralized job monitoring concept based on the End-to-End \n",
            "Monitoring and Alerting Infrastructure (MAI).\n",
            "§ You can monitor the following types of jobs in a distributed system landscape:\n",
            "ü ABAP jobs\n",
            "ü SAP BW process chains\n",
            "ü SAP BusinessObjects (SBOP) jobs\n",
            "ü SAP Data Services (DS) jobs\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Page 10\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Monitoring\n",
            "Unified Job Monitoring\n",
            "There is a Single Source of Truth for all Job Monitoring Functionalities:\n",
            "取扱厳秘\n",
            "§ Unified Configuration UI:\n",
            "One  configuration  UI  for  all  entry  points.  Allows  reuse  of  monitoring \n",
            "objects. Assign monitoring objects to job documentations.\n",
            "§ Unified Alert Inbox:\n",
            "Using  the  same  alert  Inbox  in  Business  Process  Operations,  Job \n",
            "Management and Solution Manager Configuration.\n",
            "§ Unified Monitoring Application:\n",
            "Using UI5 based Job Monitoring application for all entry points.\n",
            "§ Unified Reporting:\n",
            "Using  same  alert  and \n",
            "monitoring area. \n",
            "independent  of \n",
            "job  reporting \n",
            "functions, \n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Page 11\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Monitoring\n",
            "Scope\n",
            "取扱厳秘\n",
            "§ At the minimum, top business critical jobs and job chains (please note that in SAP BPA job chains are named \n",
            "as process chains) should be monitored and an immediate resolution needs to be found in case of a problem.\n",
            "§ To  identify  whether  the  most  important  jobs  and  job  chains  run  as  scheduled,  with  good  performance  and \n",
            "finish successfully, different metrics are available:\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "ü Time metrics: not-started-on-time, start delay, end time, duration, out of time window\n",
            "ü Range metrics: records processed, data packages processed, rows read\n",
            "ü Job log messages\n",
            "ü Job log content\n",
            "ü Others: Parallel processing and Job active\n",
            "Page 12\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Monitoring\n",
            "Creating Job Monitoring Object\n",
            "取扱厳秘\n",
            "§ Before creating Job Monitoring Object, In Basic BPO Configuration Monitored systems must be configured.\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "§ Via  Job  Management  Step  4  Define  Scope  –  create  a  scenario  and  assign  technical  systems  to  the  it.  A \n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Technical Scenario can comprise several systems with several monitoring objects per system.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Page 13\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Monitoring\n",
            "Creating Job Monitoring Object\n",
            "§ Step 5 – Configure Jobs: To configure monitoring the user must:\n",
            "ü Select a job\n",
            "ü Setup Alert Configuration\n",
            "ü Configure Incidents and Notifications\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "取扱厳秘\n",
            "§ Press  button  ‘New’.  For  Job  identification,  select  either \n",
            "‘Directly’ or ‘From Managed System’ (e.g. SAP ERP)\n",
            "Page 14\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Monitoring\n",
            "Creating Job Monitoring Object\n",
            "1\n",
            "2\n",
            "取扱厳秘\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "3\n",
            "§ After the monitoring object has been added additional job criteria can be maintained:\n",
            "ü Schedule\n",
            "ü Alert Configuration\n",
            "ü Custom Alert Text\n",
            "ü Incidents and Notifications…\n",
            "Page 15\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "Job Monitoring\n",
            "Job Monitoring Tile\n",
            "取扱厳秘\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Page 16\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "DEMO\n",
            "Job Management and Job Monitoring\n",
            "取扱厳秘\n",
            "Evaluation only.\n",
            "Evaluation only.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Created with Aspose.Slides for .NET Standard 2.0 22.7.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Copyright 2004-2022Aspose Pty Ltd.\n",
            "Page 17\n",
            "© 2020 JERA Co., Inc. All Rights Reserved.\n",
            "\n"
          ]
        }
      ],
      "source": [
        "from pdfminer.high_level import extract_pages\n",
        "from pdfminer.layout import LTTextContainer\n",
        "\n",
        "def extract_text_from_pdf_page(file_path):\n",
        "    text = ''\n",
        "    for page_layout in extract_pages(file_path):\n",
        "        if page_layout.pageid > 2:\n",
        "            for element in page_layout:\n",
        "                if isinstance(element, LTTextContainer):\n",
        "                    text += element.get_text()\n",
        "    return text\n",
        "\n",
        "pdf_file = \"monitoring.pdf\"\n",
        "text = extract_text_from_pdf_page(pdf_file)\n",
        "print(text)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iMGkwaxXP76f"
      },
      "source": [
        "## Text cleaning"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "C791FaKMP76f",
        "outputId": "50a39575-907f-4f5a-e860-f27fc94a9db6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Business Process Operations\n",
            "\n",
            "- SAP Business Process Operations comprises the most important application-related operations topics to \n",
            "ensure the smooth and reliable flow of the core business processes, to meet a company’s business \n",
            "re-uirements. \n",
            "- SAP Business Process Operations comprises the following processes: \n",
            "- Business process monitoring \n",
            "- Business process improvement\n",
            "- Job management and job monitoring\n",
            "- Data consistency management\n",
            "Page 3\n",
            "\n",
            "Job Management\n",
            "Challenges of Job Management\n",
            "\n",
            "-\n",
            "In  a  highly  complex  and  distributed  system  landscape,  job  scheduling  management  is  important  to  help  manage  data  and  its \n",
            "processing. \n",
            "Challenge\n",
            "Goal\n",
            "Governance\n",
            "Capability\n",
            "Job Re-uest\n",
            "Visibility\n",
            "- Provide standardized & web-based documentation\n",
            "- Avoid unnecessary job processing (“forgotten” jobs)\n",
            "- Avoid unnecessary slow support processes\n",
            "- “Living” documentation vs, one-time documentation\n",
            "Job Documentation\n",
            "- Provide standardized & web-based re-uest form\n",
            "- Integration with IT service management process\n",
            "- Integration with change management process\n",
            "- Integration with documentation, scheduling & monitoring\n",
            "Efficiency\n",
            "Job Scheduling\n",
            "Effectiveness\n",
            "Job Monitoring\n",
            "- Automate job processing where possible (job chains)\n",
            "- Schedule across system boundaries\n",
            "- Event-driven vs time-driven scheduling\n",
            "- Control end-user jobs (job interception)\n",
            "- Ensure that jobs start/finish at the right time\n",
            "- Ensure job processing within given time windows\n",
            "- Correlate important jobs with Business processes\n",
            "- Management by exception\n",
            "Page 4\n",
            "\n",
            "Job Management\n",
            "Job Management Administration\n",
            "\n",
            "- You can open the Job Management Work Center to access the following capabilities as there are no individual \n",
            "tiles for them.\n",
            "- Overview\n",
            "This view provides overview information with direct links to the personal object worklist (POWL) -ueries defined for the work center \n",
            "views.\n",
            "- External Scheduler Management\n",
            "Provides a view to manage jobs scheduled with a connected, certified external job scheduler.\n",
            "- SAP Scheduler Management\n",
            "In this view, you can manage jobs scheduled with the SAP Scheduler via the BC-XBP interface.\n",
            "Page 5\n",
            "\n",
            "Job Management\n",
            "Job Re-uests\n",
            "\n",
            "- The job re-uest is the starting point of the job management process. End users create job re-uests for new \n",
            "jobs, job changes, and job deletion. Job re-uests undergo an end-to-end approval and implementation \n",
            "process, with the workflow handled by IT Service Management and Change Re-uest Management.\n",
            "Page 6\n",
            "\n",
            "Job Management\n",
            "Job Documentation\n",
            "\n",
            "- Job documentation is the central object of the Job Management standard. It allows you to centrally manage, \n",
            "schedule, and document in detail the background processing of your entire system landscape.\n",
            "Page 7\n",
            "\n",
            "Job Management\n",
            "Job Scheduling Assistant\n",
            "\n",
            "- With SAP Solution Manager, you can schedule jobs in your managed systems from job documentation via the \n",
            "SAP scheduler (BC-XBP interface) or an external scheduler (SMSE interface). You can also schedule jobs \n",
            "directly, without job documentation.\n",
            "Page 8\n",
            "\n",
            "Job Management\n",
            "Job Management Administration\n",
            "- Recommended Jobs\n",
            "Provides  a  view  on  background  jobs  in  your  system  landscape, \n",
            "organized by system.\n",
            "\n",
            "- Task Inbox\n",
            "Provides  a  list  of  job  documentation  tasks.  If  authorized,  you  can \n",
            "create new tasks and display, edit, and delete existing tasks..\n",
            "to  perform  various  configuration  and \n",
            "- Administration\n",
            "Provides  you  access \n",
            "management tasks.\n",
            "Page 9\n",
            "\n",
            "Job Monitoring\n",
            "Overview\n",
            "\n",
            "- Job monitoring belongs to important central monitoring tasks and can ensure that background jobs and job \n",
            "chains are running as expected.\n",
            "- SAP Solution Manager provides a complete, centralized job monitoring concept based on the End-to-End \n",
            "Monitoring and Alerting Infrastructure (MAI).\n",
            "- You can monitor the following types of jobs in a distributed system landscape:\n",
            "- ABAP jobs\n",
            "- SAP BW process chains\n",
            "- SAP BusinessObjects (SBOP) jobs\n",
            "- SAP Data Services (DS) jobs\n",
            "Page 10\n",
            "\n",
            "Job Monitoring\n",
            "Unified Job Monitoring\n",
            "There is a Single Source of Truth for all Job Monitoring Functionalities:\n",
            "\n",
            "- Unified Configuration UI:\n",
            "One  configuration  UI  for  all  entry  points.  Allows  reuse  of  monitoring \n",
            "objects. Assign monitoring objects to job documentations.\n",
            "- Unified Alert Inbox:\n",
            "Using  the  same  alert  Inbox  in  Business  Process  Operations,  Job \n",
            "Management and Solution Manager Configuration.\n",
            "- Unified Monitoring Application:\n",
            "Using UI5 based Job Monitoring application for all entry points.\n",
            "- Unified Reporting:\n",
            "Using  same  alert  and \n",
            "monitoring area. \n",
            "independent  of \n",
            "job  reporting \n",
            "functions, \n",
            "Page 11\n",
            "\n",
            "Job Monitoring\n",
            "Scope\n",
            "\n",
            "- At the minimum, top business critical jobs and job chains (please note that in SAP BPA job chains are named \n",
            "as process chains) should be monitored and an immediate resolution needs to be found in case of a problem.\n",
            "- To  identify  whether  the  most  important  jobs  and  job  chains  run  as  scheduled,  with  good  performance  and \n",
            "finish successfully, different metrics are available:\n",
            "- Time metrics: not-started-on-time, start delay, end time, duration, out of time window\n",
            "- Range metrics: records processed, data packages processed, rows read\n",
            "- Job log messages\n",
            "- Job log content\n",
            "- Others: Parallel processing and Job active\n",
            "Page 12\n",
            "\n",
            "Job Monitoring\n",
            "Creating Job Monitoring Object\n",
            "\n",
            "- Before creating Job Monitoring Object, In Basic BPO Configuration Monitored systems must be configured.- Via  Job  Management  Step  4  Define  Scope  –  create  a  scenario  and  assign  technical  systems  to  the  it.  A Technical Scenario can comprise several systems with several monitoring objects per system.Page 13\n",
            "\n",
            "Job Monitoring\n",
            "Creating Job Monitoring Object\n",
            "- Step 5 – Configure Jobs: To configure monitoring the user must:\n",
            "- Select a job\n",
            "- Setup Alert Configuration\n",
            "- Configure Incidents and Notifications\n",
            "\n",
            "- Press  button  ‘New’.  For  Job  identification,  select  either \n",
            "‘Directly’ or ‘From Managed System’ (e.g. SAP ERP)\n",
            "Page 14\n",
            "\n",
            "Job Monitoring\n",
            "Creating Job Monitoring Object\n",
            "1\n",
            "2\n",
            "\n",
            "3\n",
            "- After the monitoring object has been added additional job criteria can be maintained:\n",
            "- Schedule\n",
            "- Alert Configuration\n",
            "- Custom Alert Text\n",
            "- Incidents and Notifications…\n",
            "Page 15\n",
            "\n",
            "Job Monitoring\n",
            "Job Monitoring Tile\n",
            "\n",
            "Page 16\n",
            "\n",
            "DEMO\n",
            "Job Management and Job Monitoring\n",
            "\n",
            "Page 17\n",
            "\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# Strings to remove\n",
        "strings_to_remove = ['© 2020 JERA Co., Inc. All Rights Reserved.', 'Evaluation only.', '取扱厳秘', 'Created with Aspose.Slides for .NET Standard 2.0 22.7.', 'Copyright 2004-2022Aspose Pty Ltd.','\\n\\n\\n']\n",
        "\n",
        "# Remove all occurrences of the strings\n",
        "for string in strings_to_remove:\n",
        "    text = text.replace(string, '')\n",
        "\n",
        "# Characters to replace\n",
        "characters_to_replace = ['§', 'q','ü']\n",
        "\n",
        "# Replace characters with hyphen\n",
        "for char in characters_to_replace:\n",
        "    text = text.replace(char, '-')\n",
        "\n",
        "# Print the extracted text\n",
        "print(text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "irWcPpAxP76f"
      },
      "source": [
        "## QA with pipeline"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-hyJSYlBP76f",
        "outputId": "196afc02-fccf-41de-8d23-795491e6321f"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "C:\\Users\\hchai\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python311\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
            "  from .autonotebook import tqdm as notebook_tqdm\n",
            "No model was supplied, defaulted to distilbert-base-cased-distilled-squad and revision 626af31 (https://huggingface.co/distilbert-base-cased-distilled-squad).\n",
            "Using a pipeline without specifying a model name and revision in production is not recommended.\n",
            "Downloading (…)lve/main/config.json: 100%|██████████| 473/473 [00:00<?, ?B/s] \n",
            "C:\\Users\\hchai\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python311\\site-packages\\huggingface_hub\\file_download.py:133: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\\Users\\hchai\\.cache\\huggingface\\hub. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.\n",
            "To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development\n",
            "  warnings.warn(message)\n",
            "Downloading model.safetensors: 100%|██████████| 261M/261M [00:27<00:00, 9.64MB/s] \n",
            "Downloading (…)okenizer_config.json: 100%|██████████| 29.0/29.0 [00:00<?, ?B/s]\n",
            "Downloading (…)solve/main/vocab.txt: 100%|██████████| 213k/213k [00:00<00:00, 1.01MB/s]\n",
            "Downloading (…)/main/tokenizer.json: 100%|██████████| 436k/436k [00:00<00:00, 3.66MB/s]\n"
          ]
        }
      ],
      "source": [
        "from transformers import pipeline\n",
        "\n",
        "qa_model = pipeline(\"question-answering\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2AVNBBsuP76f",
        "outputId": "552bc96f-7af9-4e6f-c093-0e85680c5e61"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'score': 0.06344479322433472,\n",
              " 'start': 31,\n",
              " 'end': 45,\n",
              " 'answer': 'Job Management'}"
            ]
          },
          "execution_count": 20,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "question = \"What are the SAP Business Process Operations?\"\n",
        "qa_model(question = question, context = text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EcjSVyKPP76g"
      },
      "source": [
        "## QA with distilbert"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Gdu5m_okP76g"
      },
      "outputs": [],
      "source": [
        "import torch\n",
        "from transformers import DistilBertForQuestionAnswering, DistilBertTokenizer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-PstPRbBP76g",
        "outputId": "3eac9b03-4863-4446-907a-02db219b09d0"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Downloading (…)lve/main/config.json: 100%|██████████| 451/451 [00:00<?, ?B/s] \n",
            "Downloading model.safetensors: 100%|██████████| 265M/265M [00:27<00:00, 9.55MB/s] \n",
            "Downloading (…)solve/main/vocab.txt: 100%|██████████| 232k/232k [00:00<00:00, 412kB/s]\n",
            "Downloading (…)okenizer_config.json: 100%|██████████| 28.0/28.0 [00:00<?, ?B/s]\n"
          ]
        }
      ],
      "source": [
        "# Question to ask\n",
        "question = \"What are the SAP Business Process Operations?\"\n",
        "\n",
        "# Load the pre-trained question-answering model and tokenizer\n",
        "model = DistilBertForQuestionAnswering.from_pretrained('distilbert-base-uncased-distilled-squad')\n",
        "tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased-distilled-squad')\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TE7sxamUP76g",
        "outputId": "1a5fbd19-3c02-4ea9-c823-5e5978c9b2bf"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Be aware, overflowing tokens are not returned for the setting you have chosen, i.e. sequence pairs with the 'longest_first' truncation strategy. So the returned list will always be empty even if some tokens have been removed.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Answer: job management and job monitoringtitle\n"
          ]
        }
      ],
      "source": [
        "# Tokenize the input text and question\n",
        "inputs = tokenizer.encode_plus(question, text, add_special_tokens=True, truncation=True, max_length=512, padding='max_length', return_tensors='pt')\n",
        "\n",
        "# Forward pass through the model\n",
        "outputs = model(**inputs)\n",
        "\n",
        "# Get the start and end logits from the model's output\n",
        "start_logits = outputs.start_logits\n",
        "end_logits = outputs.end_logits\n",
        "\n",
        "# Get the most probable answer using the start and end logits\n",
        "start_index = torch.argmax(start_logits, dim=1).item()\n",
        "end_index = torch.argmax(end_logits, dim=1).item()\n",
        "answer = tokenizer.convert_tokens_to_string(tokenizer.convert_ids_to_tokens(inputs['input_ids'][0, start_index:end_index+1].tolist()))\n",
        "\n",
        "# Print the answer\n",
        "print(\"Answer:\", answer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wHugvJPjP76g"
      },
      "source": [
        "## QA with paragraphs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GduPYyRmP76g"
      },
      "outputs": [],
      "source": [
        "import re\n",
        "\n",
        "def delete_numeric_lines(text):\n",
        "    lines = text.split('\\n')\n",
        "    filtered_lines = []\n",
        "    for line in lines:\n",
        "        # Use regular expression to check if line contains only numbers\n",
        "        if not re.match(r'^\\d+$', line.strip()):\n",
        "            filtered_lines.append(line)\n",
        "    filtered_text = '\\n'.join(filtered_lines)\n",
        "    return filtered_text"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Mw38wx-kP76h",
        "outputId": "c428f1ff-7821-4c5e-890e-d55fda20b655"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Business Process Operations\n",
            "\n",
            "- SAP Business Process Operations comprises the most important application-related operations topics to \n",
            "ensure the smooth and reliable flow of the core business processes, to meet a company’s business \n",
            "re-uirements. \n",
            "- SAP Business Process Operations comprises the following processes: \n",
            "- Business process monitoring \n",
            "- Business process improvement\n",
            "- Job management and job monitoring\n",
            "- Data consistency management\n",
            "\n",
            "--------------------\n",
            "\n",
            "Job Management\n",
            "Challenges of Job Management\n",
            "\n",
            "-\n",
            "In  a  highly  complex  and  distributed  system  landscape,  job  scheduling  management  is  important  to  help  manage  data  and  its \n",
            "processing. \n",
            "Challenge\n",
            "Goal\n",
            "Governance\n",
            "Capability\n",
            "Job Re-uest\n",
            "Visibility\n",
            "- Provide standardized & web-based documentation\n",
            "- Avoid unnecessary job processing (“forgotten” jobs)\n",
            "- Avoid unnecessary slow support processes\n",
            "- “Living” documentation vs, one-time documentation\n",
            "Job Documentation\n",
            "- Provide standardized & web-based re-uest form\n",
            "- Integration with IT service management process\n",
            "- Integration with change management process\n",
            "- Integration with documentation, scheduling & monitoring\n",
            "Efficiency\n",
            "Job Scheduling\n",
            "Effectiveness\n",
            "Job Monitoring\n",
            "- Automate job processing where possible (job chains)\n",
            "- Schedule across system boundaries\n",
            "- Event-driven vs time-driven scheduling\n",
            "- Control end-user jobs (job interception)\n",
            "- Ensure that jobs start/finish at the right time\n",
            "- Ensure job processing within given time windows\n",
            "- Correlate important jobs with Business processes\n",
            "- Management by exception\n",
            "\n",
            "--------------------\n",
            "\n",
            "Job Management\n",
            "Job Management Administration\n",
            "\n",
            "- You can open the Job Management Work Center to access the following capabilities as there are no individual \n",
            "tiles for them.\n",
            "- Overview\n",
            "This view provides overview information with direct links to the personal object worklist (POWL) -ueries defined for the work center \n",
            "views.\n",
            "- External Scheduler Management\n",
            "Provides a view to manage jobs scheduled with a connected, certified external job scheduler.\n",
            "- SAP Scheduler Management\n",
            "In this view, you can manage jobs scheduled with the SAP Scheduler via the BC-XBP interface.\n",
            "\n",
            "--------------------\n",
            "\n",
            "Job Management\n",
            "Job Re-uests\n",
            "\n",
            "- The job re-uest is the starting point of the job management process. End users create job re-uests for new \n",
            "jobs, job changes, and job deletion. Job re-uests undergo an end-to-end approval and implementation \n",
            "process, with the workflow handled by IT Service Management and Change Re-uest Management.\n",
            "\n",
            "--------------------\n",
            "\n",
            "Job Management\n",
            "Job Documentation\n",
            "\n",
            "- Job documentation is the central object of the Job Management standard. It allows you to centrally manage, \n",
            "schedule, and document in detail the background processing of your entire system landscape.\n",
            "\n",
            "--------------------\n",
            "\n",
            "Job Management\n",
            "Job Scheduling Assistant\n",
            "\n",
            "- With SAP Solution Manager, you can schedule jobs in your managed systems from job documentation via the \n",
            "SAP scheduler (BC-XBP interface) or an external scheduler (SMSE interface). You can also schedule jobs \n",
            "directly, without job documentation.\n",
            "\n",
            "--------------------\n",
            "\n",
            "Job Management\n",
            "Job Management Administration\n",
            "- Recommended Jobs\n",
            "Provides  a  view  on  background  jobs  in  your  system  landscape, \n",
            "organized by system.\n",
            "\n",
            "- Task Inbox\n",
            "Provides  a  list  of  job  documentation  tasks.  If  authorized,  you  can \n",
            "create new tasks and display, edit, and delete existing tasks..\n",
            "to  perform  various  configuration  and \n",
            "- Administration\n",
            "Provides  you  access \n",
            "management tasks.\n",
            "\n",
            "--------------------\n",
            "\n",
            "Job Monitoring\n",
            "Overview\n",
            "\n",
            "- Job monitoring belongs to important central monitoring tasks and can ensure that background jobs and job \n",
            "chains are running as expected.\n",
            "- SAP Solution Manager provides a complete, centralized job monitoring concept based on the End-to-End \n",
            "Monitoring and Alerting Infrastructure (MAI).\n",
            "- You can monitor the following types of jobs in a distributed system landscape:\n",
            "- ABAP jobs\n",
            "- SAP BW process chains\n",
            "- SAP BusinessObjects (SBOP) jobs\n",
            "- SAP Data Services (DS) jobs\n",
            "\n",
            "--------------------\n",
            "\n",
            "Job Monitoring\n",
            "Unified Job Monitoring\n",
            "There is a Single Source of Truth for all Job Monitoring Functionalities:\n",
            "\n",
            "- Unified Configuration UI:\n",
            "One  configuration  UI  for  all  entry  points.  Allows  reuse  of  monitoring \n",
            "objects. Assign monitoring objects to job documentations.\n",
            "- Unified Alert Inbox:\n",
            "Using  the  same  alert  Inbox  in  Business  Process  Operations,  Job \n",
            "Management and Solution Manager Configuration.\n",
            "- Unified Monitoring Application:\n",
            "Using UI5 based Job Monitoring application for all entry points.\n",
            "- Unified Reporting:\n",
            "Using  same  alert  and \n",
            "monitoring area. \n",
            "independent  of \n",
            "job  reporting \n",
            "functions, \n",
            "\n",
            "--------------------\n",
            "\n",
            "Job Monitoring\n",
            "Scope\n",
            "\n",
            "- At the minimum, top business critical jobs and job chains (please note that in SAP BPA job chains are named \n",
            "as process chains) should be monitored and an immediate resolution needs to be found in case of a problem.\n",
            "- To  identify  whether  the  most  important  jobs  and  job  chains  run  as  scheduled,  with  good  performance  and \n",
            "finish successfully, different metrics are available:\n",
            "- Time metrics: not-started-on-time, start delay, end time, duration, out of time window\n",
            "- Range metrics: records processed, data packages processed, rows read\n",
            "- Job log messages\n",
            "- Job log content\n",
            "- Others: Parallel processing and Job active\n",
            "\n",
            "--------------------\n",
            "\n",
            "Job Monitoring\n",
            "Creating Job Monitoring Object\n",
            "\n",
            "- Before creating Job Monitoring Object, In Basic BPO Configuration Monitored systems must be configured.- Via  Job  Management  Step  4  Define  Scope  –  create  a  scenario  and  assign  technical  systems  to  the  it.  A Technical Scenario can comprise several systems with several monitoring objects per system.\n",
            "--------------------\n",
            "\n",
            "Job Monitoring\n",
            "Creating Job Monitoring Object\n",
            "- Step 5 – Configure Jobs: To configure monitoring the user must:\n",
            "- Select a job\n",
            "- Setup Alert Configuration\n",
            "- Configure Incidents and Notifications\n",
            "\n",
            "- Press  button  ‘New’.  For  Job  identification,  select  either \n",
            "‘Directly’ or ‘From Managed System’ (e.g. SAP ERP)\n",
            "\n",
            "--------------------\n",
            "\n",
            "Job Monitoring\n",
            "Creating Job Monitoring Object\n",
            "\n",
            "- After the monitoring object has been added additional job criteria can be maintained:\n",
            "- Schedule\n",
            "- Alert Configuration\n",
            "- Custom Alert Text\n",
            "- Incidents and Notifications…\n",
            "\n",
            "--------------------\n",
            "\n",
            "Job Monitoring\n",
            "Job Monitoring Tile\n",
            "\n",
            "\n",
            "--------------------\n",
            "\n",
            "DEMO\n",
            "Job Management and Job Monitoring\n",
            "\n",
            "\n",
            "--------------------\n",
            "\n",
            "\n",
            "--------------------\n"
          ]
        }
      ],
      "source": [
        "from textblob import TextBlob\n",
        "\n",
        "def split_into_paragraphs(text):\n",
        "    blob = TextBlob(text)\n",
        "    paragraphs = blob.raw.split('Page')  # Split based on double newlines\n",
        "    return paragraphs\n",
        "paragraphs = split_into_paragraphs(text)\n",
        "\n",
        "for paragraph in paragraphs:\n",
        "    paragraph = delete_numeric_lines(paragraph)\n",
        "    print(paragraph)\n",
        "    print('-' * 20)  # Separator between paragraphs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1JEZNkY9P76h",
        "outputId": "481c30ae-b9ac-49d5-889d-d8a9d78e0c4e"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "No model was supplied, defaulted to distilbert-base-cased-distilled-squad and revision 626af31 (https://huggingface.co/distilbert-base-cased-distilled-squad).\n",
            "Using a pipeline without specifying a model name and revision in production is not recommended.\n"
          ]
        }
      ],
      "source": [
        "qa_model = pipeline(\"question-answering\")\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rCk8GOqmP76h",
        "outputId": "2502e5f6-eea2-48b8-e9d3-d83fbdaa1807"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'score': 0.13790002465248108, 'start': 4254, 'end': 4259, 'answer': 'Inbox'}"
            ]
          },
          "execution_count": 69,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "question = \"What are the SAP Business Process Operations?\"\n",
        "ans=qa_model(question = question, context = text)\n",
        "ans"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DzobZP8pP76h",
        "outputId": "d81e97a9-6c9e-414e-e6ef-20372033f629"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "best answer : 0.8785384297370911, Job Monitoring\n",
            "Job Monitoring Tile\n"
          ]
        }
      ],
      "source": [
        "best_score = 0\n",
        "best_answer = ''\n",
        "question = \"What are the SAP Business Process Operations?\"\n",
        "\n",
        "for paragraph in paragraphs:\n",
        "    paragraph = delete_numeric_lines(paragraph)\n",
        "    ans = qa_model(question = question, context = paragraph)\n",
        "    #print(ans)\n",
        "    if(ans['score']>best_score):\n",
        "        best_score = ans['score']\n",
        "        best_answer = ans['answer']\n",
        "print(\"best answer : \"+str(best_score)+', '+best_answer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9PrPcXIXP76h",
        "outputId": "eb1b20f7-df6e-4740-becf-e88ad1ac2524"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "best answer : 0.8302633762359619, Job Management\n",
            "Job Management Administration\n"
          ]
        }
      ],
      "source": [
        "best_score = 0\n",
        "best_answer = ''\n",
        "question = \"What's SAP?\"\n",
        "\n",
        "for paragraph in paragraphs:\n",
        "    paragraph = delete_numeric_lines(paragraph)\n",
        "    ans = qa_model(question = question, context = paragraph)\n",
        "    #print(ans)\n",
        "    if(ans['score']>best_score):\n",
        "        best_score = ans['score']\n",
        "        best_answer = ans['answer']\n",
        "print(\"best answer : \"+str(best_score)+', '+best_answer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rI6-zdcnP76h",
        "outputId": "ce47cd3b-0784-45bb-e6da-b509095b923d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "best answer : 0.9529011845588684, Job Monitoring\n"
          ]
        }
      ],
      "source": [
        "best_score = 0\n",
        "best_answer = ''\n",
        "question = \"what can you monitor with SAP Solution Manager?\"\n",
        "\n",
        "for paragraph in paragraphs:\n",
        "    paragraph = delete_numeric_lines(paragraph)\n",
        "    ans = qa_model(question = question, context = paragraph)\n",
        "    #print(ans)\n",
        "    if(ans['score']>best_score):\n",
        "        best_score = ans['score']\n",
        "        best_answer = ans['answer']\n",
        "print(\"best answer : \"+str(best_score)+', '+best_answer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_CrDSa0wP76h"
      },
      "outputs": [],
      "source": [
        "txt_sap='''SAP, short for Systems, Applications, and Products, is a leading enterprise software company that has made a significant impact on businesses worldwide. With its rich history and comprehensive suite of applications, SAP has become synonymous with innovation and efficiency in the corporate world.\n",
        "At its core, SAP offers a wide range of software solutions designed to streamline and automate various business processes. From finance and accounting to human resources, supply chain management, customer relationship management, and more, SAP provides integrated modules that cater to the diverse needs of organizations across industries.\n",
        "One of the key strengths of SAP lies in its ability to centralize and integrate data from different departments and functions within an organization. This centralized approach allows for real-time visibility, data-driven decision-making, and improved operational efficiency. By breaking down silos and enabling seamless data flow, SAP empowers businesses to optimize their processes, reduce costs, and gain a competitive edge.\n",
        "SAP's flagship product, SAP ERP (Enterprise Resource Planning), serves as the backbone of many enterprises. It offers a comprehensive suite of modules that encompass various functional areas, such as finance, procurement, production planning, sales, and distribution. With its robust features and scalability, SAP ERP enables organizations to standardize their processes, automate routine tasks, and achieve operational excellence.\n",
        "In addition to ERP, SAP has expanded its portfolio to include cloud-based solutions with the introduction of SAP S/4HANA. Built on an in-memory computing platform, S/4HANA brings together the power of real-time analytics, artificial intelligence, and machine learning. This modern ERP system enables businesses to leverage advanced technologies, gain actionable insights, and drive digital transformation.\n",
        "Furthermore, SAP has embraced the concept of the intelligent enterprise. By combining its software capabilities with emerging technologies like the Internet of Things (IoT), blockchain, and predictive analytics, SAP enables organizations to unlock new business opportunities. The intelligent enterprise leverages data as a strategic asset, enabling companies to predict customer behavior, optimize operations, and create personalized experiences.\n",
        "SAP's impact extends far beyond traditional enterprise software. The company has built a vast ecosystem of partners, developers, and users, fostering collaboration and innovation. Through its community-driven initiatives, such as SAP Community Network and open-source projects like SAP HANA, SAP actively engages with its stakeholders to share knowledge, drive co-innovation, and address evolving market needs.\n",
        "In conclusion, SAP has revolutionized the way businesses operate with its comprehensive suite of enterprise software solutions. By providing a unified platform, integrating data, and embracing emerging technologies, SAP empowers organizations to transform digitally, stay ahead of the competition, and achieve sustainable growth. With its commitment to innovation, customer success, and global impact, SAP continues to shape the future of the business landscape.'''"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BOmVxMhDP76i",
        "outputId": "3e29e850-6fc1-49b1-e3c3-c3030b03e15c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'finance, procurement, production planning, sales, and distribution'"
            ]
          },
          "execution_count": 107,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "quest=\"What functional areas does SAP ERP cover?\"\n",
        "ans = qa_model(question = quest, context = txt_sap)\n",
        "ans\n",
        "ans['answer']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bE_8M52iP76i"
      },
      "source": [
        "## <u>Sentence generation</u>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7n7vnRImP76i"
      },
      "source": [
        "## Transformers"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mOZKFHZiP76i"
      },
      "outputs": [],
      "source": [
        "from transformers import GPT2LMHeadModel, GPT2Tokenizer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aBzKJvENP76i",
        "outputId": "c5ea85a3-979f-4cae-e15c-07c69813048f"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Downloading (…)lve/main/config.json: 100%|██████████| 762/762 [00:00<?, ?B/s] \n",
            "Downloading model.safetensors: 100%|██████████| 353M/353M [00:59<00:00, 5.94MB/s] \n",
            "Downloading (…)neration_config.json: 100%|██████████| 124/124 [00:00<?, ?B/s] \n",
            "Downloading (…)olve/main/vocab.json: 100%|██████████| 1.04M/1.04M [00:00<00:00, 3.11MB/s]\n",
            "Downloading (…)olve/main/merges.txt: 100%|██████████| 456k/456k [00:00<00:00, 1.81MB/s]\n"
          ]
        }
      ],
      "source": [
        "# Load the pre-trained \"distilgpt2\" model and tokenizer\n",
        "model = GPT2LMHeadModel.from_pretrained('distilgpt2')\n",
        "tokenizer = GPT2Tokenizer.from_pretrained('distilgpt2')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uGHsMFGcP76i",
        "outputId": "b1ceb4f0-cbdb-40ee-ec5e-69f60a75166b"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Generated Sentence: a business magnate and investor.\n",
            "\n",
            "The company is a major player in the global financial sector\n"
          ]
        }
      ],
      "source": [
        "\n",
        "# Set the generated sentence length\n",
        "sentence_length = 20\n",
        "\n",
        "# Generate a sentence based on the answer\n",
        "input_ids = tokenizer.encode(ans[\"answer\"], return_tensors='pt')\n",
        "attention_mask = input_ids.clone().fill_(1)  # Set attention mask to 1 for all input tokens\n",
        "\n",
        "# Generate a sentence based on the answer\n",
        "generated_sentence = model.generate(\n",
        "    input_ids=input_ids,\n",
        "    attention_mask=attention_mask,\n",
        "    max_length=sentence_length,\n",
        "    num_return_sequences=1,\n",
        "    no_repeat_ngram_size=2,\n",
        "    early_stopping=True\n",
        ")\n",
        "\n",
        "# Decode the generated sentence\n",
        "decoded_sentence = tokenizer.decode(generated_sentence[0], skip_special_tokens=True)\n",
        "\n",
        "# Print the generated sentence\n",
        "print(\"Generated Sentence:\", decoded_sentence)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lM_1CYjSP76i"
      },
      "source": [
        "## KeyToText"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HYwY_nC4P76i",
        "outputId": "db6b6d13-1e71-4294-d593-79b5a839947b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collecting keytotextNote: you may need to restart the kernel to use updated packages.\n",
            "\n",
            "  Downloading keytotext-2.3.2-py3-none-any.whl (11 kB)\n",
            "Requirement already satisfied: torch in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from keytotext) (2.0.1)\n",
            "Requirement already satisfied: transformers in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from keytotext) (4.30.2)\n",
            "Requirement already satisfied: sentencepiece in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from keytotext) (0.1.99)\n",
            "Collecting wandb (from keytotext)\n",
            "  Downloading wandb-0.15.5-py3-none-any.whl (2.1 MB)\n",
            "                                              0.0/2.1 MB ? eta -:--:--\n",
            "                                              0.0/2.1 MB ? eta -:--:--\n",
            "                                              0.0/2.1 MB 640.0 kB/s eta 0:00:04\n",
            "                                              0.0/2.1 MB 640.0 kB/s eta 0:00:04\n",
            "                                              0.0/2.1 MB 195.7 kB/s eta 0:00:11\n",
            "     -                                        0.1/2.1 MB 302.7 kB/s eta 0:00:07\n",
            "     -                                        0.1/2.1 MB 392.2 kB/s eta 0:00:06\n",
            "     -                                        0.1/2.1 MB 392.2 kB/s eta 0:00:06\n",
            "     --                                       0.1/2.1 MB 405.9 kB/s eta 0:00:05\n",
            "     ---                                      0.2/2.1 MB 518.8 kB/s eta 0:00:04\n",
            "     -----                                    0.3/2.1 MB 655.2 kB/s eta 0:00:03\n",
            "     -----                                    0.3/2.1 MB 655.2 kB/s eta 0:00:03\n",
            "     --------                                 0.4/2.1 MB 769.9 kB/s eta 0:00:03\n",
            "     ---------                                0.5/2.1 MB 873.0 kB/s eta 0:00:02\n",
            "     ----------                               0.6/2.1 MB 913.7 kB/s eta 0:00:02\n",
            "     ----------                               0.6/2.1 MB 913.7 kB/s eta 0:00:02\n",
            "     ------------                             0.6/2.1 MB 922.8 kB/s eta 0:00:02\n",
            "     --------------                           0.8/2.1 MB 1.0 MB/s eta 0:00:02\n",
            "     ---------------                          0.8/2.1 MB 1.1 MB/s eta 0:00:02\n",
            "     ----------------                         0.8/2.1 MB 989.4 kB/s eta 0:00:02\n",
            "     -----------------                        0.9/2.1 MB 1.0 MB/s eta 0:00:02\n",
            "     -------------------                      1.0/2.1 MB 1.1 MB/s eta 0:00:01\n",
            "     --------------------                     1.1/2.1 MB 1.1 MB/s eta 0:00:01\n",
            "     --------------------                     1.1/2.1 MB 1.1 MB/s eta 0:00:01\n",
            "     ----------------------                   1.2/2.1 MB 1.1 MB/s eta 0:00:01\n",
            "     ------------------------                 1.3/2.1 MB 1.2 MB/s eta 0:00:01\n",
            "     -------------------------                1.3/2.1 MB 1.2 MB/s eta 0:00:01\n",
            "     -------------------------                1.3/2.1 MB 1.1 MB/s eta 0:00:01\n",
            "     ---------------------------              1.4/2.1 MB 1.2 MB/s eta 0:00:01\n",
            "     -----------------------------            1.5/2.1 MB 1.2 MB/s eta 0:00:01\n",
            "     ------------------------------           1.6/2.1 MB 1.2 MB/s eta 0:00:01\n",
            "     ------------------------------           1.6/2.1 MB 1.2 MB/s eta 0:00:01\n",
            "     -------------------------------          1.6/2.1 MB 1.2 MB/s eta 0:00:01\n",
            "     ----------------------------------       1.8/2.1 MB 1.2 MB/s eta 0:00:01\n",
            "     -----------------------------------      1.8/2.1 MB 1.3 MB/s eta 0:00:01\n",
            "     ------------------------------------     1.9/2.1 MB 1.2 MB/s eta 0:00:01\n",
            "     --------------------------------------   2.0/2.1 MB 1.3 MB/s eta 0:00:01\n",
            "     ---------------------------------------  2.1/2.1 MB 1.3 MB/s eta 0:00:01\n",
            "     ---------------------------------------- 2.1/2.1 MB 1.3 MB/s eta 0:00:00\n",
            "Collecting pytorch_lightning (from keytotext)\n",
            "  Downloading pytorch_lightning-2.0.5-py3-none-any.whl (722 kB)\n",
            "                                              0.0/722.4 kB ? eta -:--:--\n",
            "     ------------                           245.8/722.4 kB 7.6 MB/s eta 0:00:01\n",
            "     ------------                           245.8/722.4 kB 7.6 MB/s eta 0:00:01\n",
            "     ------------                           245.8/722.4 kB 7.6 MB/s eta 0:00:01\n",
            "     ----------------                       317.4/722.4 kB 1.8 MB/s eta 0:00:01\n",
            "     --------------------------             512.0/722.4 kB 2.5 MB/s eta 0:00:01\n",
            "     --------------------------             512.0/722.4 kB 2.5 MB/s eta 0:00:01\n",
            "     --------------------------             512.0/722.4 kB 2.5 MB/s eta 0:00:01\n",
            "     -----------------------------------    675.8/722.4 kB 1.9 MB/s eta 0:00:01\n",
            "     -------------------------------------- 722.4/722.4 kB 1.8 MB/s eta 0:00:00\n",
            "Collecting datasets (from keytotext)\n",
            "  Downloading datasets-2.13.1-py3-none-any.whl (486 kB)\n",
            "                                              0.0/486.2 kB ? eta -:--:--\n",
            "     -------------------                    245.8/486.2 kB 7.6 MB/s eta 0:00:01\n",
            "     -------------------                    245.8/486.2 kB 7.6 MB/s eta 0:00:01\n",
            "     -------------------                    245.8/486.2 kB 7.6 MB/s eta 0:00:01\n",
            "     ---------------------                  276.5/486.2 kB 1.5 MB/s eta 0:00:01\n",
            "     -------------------------------------  481.3/486.2 kB 2.3 MB/s eta 0:00:01\n",
            "     -------------------------------------- 486.2/486.2 kB 2.2 MB/s eta 0:00:00\n",
            "Requirement already satisfied: huggingface_hub in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from keytotext) (0.16.4)\n",
            "Collecting keybert (from keytotext)\n",
            "  Downloading keybert-0.7.0.tar.gz (21 kB)\n",
            "  Installing build dependencies: started\n",
            "  Installing build dependencies: finished with status 'done'\n",
            "  Getting requirements to build wheel: started\n",
            "  Getting requirements to build wheel: finished with status 'done'\n",
            "  Preparing metadata (pyproject.toml): started\n",
            "  Preparing metadata (pyproject.toml): finished with status 'done'\n",
            "Requirement already satisfied: numpy>=1.17 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from datasets->keytotext) (1.25.0)\n",
            "Collecting pyarrow>=8.0.0 (from datasets->keytotext)\n",
            "  Downloading pyarrow-12.0.1-cp311-cp311-win_amd64.whl (21.5 MB)\n",
            "                                              0.0/21.5 MB ? eta -:--:--\n",
            "                                              0.3/21.5 MB 5.2 MB/s eta 0:00:05\n",
            "                                              0.3/21.5 MB 5.2 MB/s eta 0:00:05\n",
            "                                              0.3/21.5 MB 5.2 MB/s eta 0:00:05\n",
            "                                              0.3/21.5 MB 1.8 MB/s eta 0:00:12\n",
            "                                              0.5/21.5 MB 2.5 MB/s eta 0:00:09\n",
            "                                              0.5/21.5 MB 2.5 MB/s eta 0:00:09\n",
            "                                              0.5/21.5 MB 2.5 MB/s eta 0:00:09\n",
            "     -                                        0.7/21.5 MB 1.7 MB/s eta 0:00:12\n",
            "     -                                        0.8/21.5 MB 2.0 MB/s eta 0:00:11\n",
            "     -                                        0.8/21.5 MB 2.0 MB/s eta 0:00:11\n",
            "     -                                        0.8/21.5 MB 2.0 MB/s eta 0:00:11\n",
            "     -                                        1.0/21.5 MB 1.9 MB/s eta 0:00:12\n",
            "     -                                        1.0/21.5 MB 1.9 MB/s eta 0:00:12\n",
            "     -                                        1.0/21.5 MB 1.9 MB/s eta 0:00:12\n",
            "     --                                       1.1/21.5 MB 1.6 MB/s eta 0:00:13\n",
            "     --                                       1.3/21.5 MB 1.8 MB/s eta 0:00:12\n",
            "     --                                       1.3/21.5 MB 1.8 MB/s eta 0:00:12\n",
            "     --                                       1.3/21.5 MB 1.8 MB/s eta 0:00:12\n",
            "     --                                       1.4/21.5 MB 1.6 MB/s eta 0:00:13\n",
            "     --                                       1.6/21.5 MB 1.7 MB/s eta 0:00:12\n",
            "     --                                       1.6/21.5 MB 1.7 MB/s eta 0:00:12\n",
            "     --                                       1.6/21.5 MB 1.7 MB/s eta 0:00:12\n",
            "     ---                                      1.6/21.5 MB 1.5 MB/s eta 0:00:13\n",
            "     ---                                      1.8/21.5 MB 1.7 MB/s eta 0:00:12\n",
            "     ---                                      1.8/21.5 MB 1.6 MB/s eta 0:00:13\n",
            "     ---                                      1.8/21.5 MB 1.6 MB/s eta 0:00:13\n",
            "     ---                                      2.0/21.5 MB 1.6 MB/s eta 0:00:13\n",
            "     ---                                      2.1/21.5 MB 1.7 MB/s eta 0:00:12\n",
            "     ---                                      2.1/21.5 MB 1.7 MB/s eta 0:00:12\n",
            "     ---                                      2.1/21.5 MB 1.7 MB/s eta 0:00:12\n",
            "     ----                                     2.3/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     ----                                     2.3/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     ----                                     2.3/21.5 MB 1.6 MB/s eta 0:00:13\n",
            "     ----                                     2.3/21.5 MB 1.6 MB/s eta 0:00:13\n",
            "     ----                                     2.5/21.5 MB 1.6 MB/s eta 0:00:13\n",
            "     ----                                     2.6/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     ----                                     2.6/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     ----                                     2.6/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     -----                                    2.8/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     -----                                    2.9/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     -----                                    2.9/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     -----                                    2.9/21.5 MB 1.5 MB/s eta 0:00:13\n",
            "     -----                                    3.1/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     -----                                    3.1/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     -----                                    3.1/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     -----                                    3.1/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     ------                                   3.4/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     ------                                   3.4/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     ------                                   3.4/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     ------                                   3.4/21.5 MB 1.5 MB/s eta 0:00:12\n",
            "     ------                                   3.6/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     ------                                   3.6/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     ------                                   3.6/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     ------                                   3.7/21.5 MB 1.5 MB/s eta 0:00:12\n",
            "     -------                                  3.9/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     -------                                  3.9/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     -------                                  3.9/21.5 MB 1.6 MB/s eta 0:00:12\n",
            "     -------                                  4.0/21.5 MB 1.5 MB/s eta 0:00:12\n",
            "     -------                                  4.2/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     -------                                  4.2/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     -------                                  4.2/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     -------                                  4.3/21.5 MB 1.5 MB/s eta 0:00:12\n",
            "     --------                                 4.4/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     --------                                 4.4/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     --------                                 4.4/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     --------                                 4.6/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     --------                                 4.7/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     --------                                 4.7/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     --------                                 4.7/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ---------                                4.8/21.5 MB 1.5 MB/s eta 0:00:11\n",
            "     ---------                                5.0/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ---------                                5.0/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ---------                                5.0/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ---------                                5.1/21.5 MB 1.5 MB/s eta 0:00:11\n",
            "     ---------                                5.2/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ---------                                5.2/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ---------                                5.2/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ----------                               5.4/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ----------                               5.5/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ----------                               5.5/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ----------                               5.5/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ----------                               5.7/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ----------                               5.7/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ----------                               5.7/21.5 MB 1.6 MB/s eta 0:00:11\n",
            "     ----------                               5.7/21.5 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------                              6.0/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     -----------                              6.0/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     -----------                              6.0/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     -----------                              6.0/21.5 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------                              6.3/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     -----------                              6.3/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     -----------                              6.3/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     -----------                              6.3/21.5 MB 1.5 MB/s eta 0:00:11\n",
            "     ------------                             6.5/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     ------------                             6.5/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     ------------                             6.5/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     ------------                             6.6/21.5 MB 1.5 MB/s eta 0:00:10\n",
            "     ------------                             6.8/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     ------------                             6.8/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     ------------                             6.8/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     -------------                            7.0/21.5 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------                            7.0/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     -------------                            7.0/21.5 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------                            7.0/21.5 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------                            7.3/21.5 MB 1.6 MB/s eta 0:00:10\n",
            "     -------------                            7.3/21.5 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------                            7.3/21.5 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------                            7.4/21.5 MB 1.5 MB/s eta 0:00:10\n",
            "     --------------                           7.6/21.5 MB 1.6 MB/s eta 0:00:09\n",
            "     --------------                           7.6/21.5 MB 1.5 MB/s eta 0:00:10\n",
            "     --------------                           7.6/21.5 MB 1.5 MB/s eta 0:00:10\n",
            "     --------------                           7.6/21.5 MB 1.5 MB/s eta 0:00:10\n",
            "     --------------                           7.8/21.5 MB 1.6 MB/s eta 0:00:09\n",
            "     --------------                           7.8/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------                           7.8/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------                           7.9/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ---------------                          8.1/21.5 MB 1.6 MB/s eta 0:00:09\n",
            "     ---------------                          8.1/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ---------------                          8.1/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ---------------                          8.1/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ---------------                          8.3/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ---------------                          8.3/21.5 MB 1.6 MB/s eta 0:00:09\n",
            "     ---------------                          8.3/21.5 MB 1.6 MB/s eta 0:00:09\n",
            "     ---------------                          8.4/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ----------------                         8.6/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ----------------                         8.6/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ----------------                         8.6/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ----------------                         8.7/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ----------------                         8.9/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ----------------                         8.9/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ----------------                         8.9/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     ----------------                         9.0/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     -----------------                        9.1/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -----------------                        9.1/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -----------------                        9.1/21.5 MB 1.5 MB/s eta 0:00:09\n",
            "     -----------------                        9.4/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -----------------                        9.4/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -----------------                        9.4/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -----------------                        9.4/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -----------------                        9.6/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -----------------                        9.6/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -----------------                        9.6/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     ------------------                       9.7/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     ------------------                       9.9/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     ------------------                       9.9/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     ------------------                       9.9/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     ------------------                       10.0/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     ------------------                       10.2/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     ------------------                       10.2/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     ------------------                       10.2/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -------------------                      10.3/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -------------------                      10.4/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -------------------                      10.4/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -------------------                      10.4/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -------------------                      10.5/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -------------------                      10.7/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -------------------                      10.7/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     -------------------                      10.7/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     --------------------                     10.9/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     --------------------                     11.0/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     --------------------                     11.0/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     --------------------                     11.0/21.5 MB 1.5 MB/s eta 0:00:08\n",
            "     --------------------                     11.1/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     --------------------                     11.2/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     --------------------                     11.2/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     --------------------                     11.2/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ---------------------                    11.3/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ---------------------                    11.5/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ---------------------                    11.5/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ---------------------                    11.5/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ---------------------                    11.7/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ---------------------                    11.7/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ---------------------                    11.7/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ---------------------                    11.7/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ----------------------                   12.0/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ----------------------                   12.0/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ----------------------                   12.0/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ----------------------                   12.0/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ----------------------                   12.2/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ----------------------                   12.3/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ----------------------                   12.3/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     ----------------------                   12.3/21.5 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------                  12.5/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -----------------------                  12.5/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -----------------------                  12.5/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -----------------------                  12.6/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -----------------------                  12.8/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -----------------------                  12.8/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -----------------------                  12.8/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------                 12.9/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------                 13.1/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------                 13.1/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------                 13.1/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------                 13.2/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------                 13.3/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------                 13.3/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------                 13.3/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------                13.6/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------                13.6/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------                13.6/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------                13.6/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------                13.8/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------                13.8/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------                13.8/21.5 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------                13.9/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------               14.1/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------               14.1/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------               14.1/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------               14.2/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------               14.4/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------               14.4/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------               14.4/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------              14.5/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------              14.6/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------              14.6/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------              14.6/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------              14.9/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------              14.9/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------              14.9/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------              14.9/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ----------------------------             15.1/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ----------------------------             15.1/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ----------------------------             15.1/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ----------------------------             15.3/21.5 MB 1.5 MB/s eta 0:00:05\n",
            "     ----------------------------             15.4/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------             15.4/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------             15.4/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -----------------------------            15.6/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -----------------------------            15.7/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -----------------------------            15.7/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -----------------------------            15.7/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -----------------------------            15.9/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -----------------------------            15.9/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -----------------------------            15.9/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -----------------------------            15.9/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     ------------------------------           16.2/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     ------------------------------           16.2/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     ------------------------------           16.2/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     ------------------------------           16.2/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     ------------------------------           16.4/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     ------------------------------           16.5/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     ------------------------------           16.5/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     ------------------------------           16.5/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -------------------------------          16.7/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -------------------------------          16.7/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -------------------------------          16.7/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -------------------------------          16.8/21.5 MB 1.5 MB/s eta 0:00:04\n",
            "     -------------------------------          17.0/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     -------------------------------          17.0/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     -------------------------------          17.0/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     -------------------------------          17.1/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     --------------------------------         17.2/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     --------------------------------         17.3/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     --------------------------------         17.3/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     --------------------------------         17.4/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     --------------------------------         17.5/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     --------------------------------         17.5/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     --------------------------------         17.5/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     --------------------------------         17.7/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     ---------------------------------        17.8/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     ---------------------------------        17.8/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     ---------------------------------        17.8/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     ---------------------------------        18.0/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     ---------------------------------        18.0/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     ---------------------------------        18.0/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     ---------------------------------        18.0/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     ---------------------------------        18.1/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     ----------------------------------       18.3/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     ----------------------------------       18.3/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     ----------------------------------       18.3/21.5 MB 1.5 MB/s eta 0:00:03\n",
            "     ----------------------------------       18.4/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     ----------------------------------       18.6/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     ----------------------------------       18.6/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     ----------------------------------       18.6/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -----------------------------------      18.8/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -----------------------------------      18.8/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -----------------------------------      18.8/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -----------------------------------      18.8/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -----------------------------------      19.1/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -----------------------------------      19.1/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -----------------------------------      19.1/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -----------------------------------      19.1/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     ------------------------------------     19.3/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     ------------------------------------     19.3/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     ------------------------------------     19.3/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     ------------------------------------     19.4/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     ------------------------------------     19.6/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     ------------------------------------     19.6/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     ------------------------------------     19.6/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     ------------------------------------     19.6/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    19.8/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    19.9/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    19.9/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    19.9/21.5 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    20.1/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     -------------------------------------    20.1/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     -------------------------------------    20.1/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     -------------------------------------    20.2/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     -------------------------------------    20.4/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     -------------------------------------    20.4/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     -------------------------------------    20.4/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   20.4/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   20.6/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   20.6/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   20.6/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   20.8/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   20.9/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   20.9/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   20.9/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  21.0/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  21.2/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  21.2/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  21.2/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  21.3/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  21.4/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  21.4/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  21.5/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  21.5/21.5 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------- 21.5/21.5 MB 1.5 MB/s eta 0:00:00\n",
            "Collecting dill<0.3.7,>=0.3.0 (from datasets->keytotext)\n",
            "  Downloading dill-0.3.6-py3-none-any.whl (110 kB)\n",
            "                                              0.0/110.5 kB ? eta -:--:--\n",
            "     -------------------------------------- 110.5/110.5 kB 3.1 MB/s eta 0:00:00\n",
            "Requirement already satisfied: pandas in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from datasets->keytotext) (2.0.3)\n",
            "Requirement already satisfied: requests>=2.19.0 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from datasets->keytotext) (2.31.0)\n",
            "Requirement already satisfied: tqdm>=4.62.1 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from datasets->keytotext) (4.65.0)\n",
            "Collecting xxhash (from datasets->keytotext)\n",
            "  Downloading xxhash-3.2.0-cp311-cp311-win_amd64.whl (30 kB)\n",
            "Collecting multiprocess (from datasets->keytotext)\n",
            "  Downloading multiprocess-0.70.14-py310-none-any.whl (134 kB)\n",
            "                                              0.0/134.3 kB ? eta -:--:--\n",
            "     -------------------------------------- 134.3/134.3 kB 3.9 MB/s eta 0:00:00\n",
            "Requirement already satisfied: fsspec[http]>=2021.11.1 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from datasets->keytotext) (2023.6.0)\n",
            "Collecting aiohttp (from datasets->keytotext)\n",
            "  Downloading aiohttp-3.8.4-cp311-cp311-win_amd64.whl (317 kB)\n",
            "                                              0.0/317.2 kB ? eta -:--:--\n",
            "     -----------------------------          245.8/317.2 kB 7.6 MB/s eta 0:00:01\n",
            "     -----------------------------          245.8/317.2 kB 7.6 MB/s eta 0:00:01\n",
            "     -----------------------------          245.8/317.2 kB 7.6 MB/s eta 0:00:01\n",
            "     -----------------------------          245.8/317.2 kB 7.6 MB/s eta 0:00:01\n",
            "     -------------------------------------- 317.2/317.2 kB 1.6 MB/s eta 0:00:00\n",
            "Requirement already satisfied: packaging in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from datasets->keytotext) (23.1)\n",
            "Requirement already satisfied: pyyaml>=5.1 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from datasets->keytotext) (6.0)\n",
            "Requirement already satisfied: filelock in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from huggingface_hub->keytotext) (3.12.2)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from huggingface_hub->keytotext) (4.7.1)\n",
            "Requirement already satisfied: sentence-transformers>=0.3.8 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from keybert->keytotext) (2.2.2)\n",
            "Requirement already satisfied: scikit-learn>=0.22.2 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from keybert->keytotext) (1.3.0)\n",
            "Collecting rich>=10.4.0 (from keybert->keytotext)\n",
            "  Downloading rich-13.4.2-py3-none-any.whl (239 kB)\n",
            "                                              0.0/239.4 kB ? eta -:--:--\n",
            "     -------------------------------------- 239.4/239.4 kB 4.9 MB/s eta 0:00:00\n",
            "Collecting torchmetrics>=0.7.0 (from pytorch_lightning->keytotext)\n",
            "  Downloading torchmetrics-1.0.1-py3-none-any.whl (729 kB)\n",
            "                                              0.0/729.2 kB ? eta -:--:--\n",
            "     ------------                           245.8/729.2 kB 7.6 MB/s eta 0:00:01\n",
            "     ------------                           245.8/729.2 kB 7.6 MB/s eta 0:00:01\n",
            "     ------------                           245.8/729.2 kB 7.6 MB/s eta 0:00:01\n",
            "     -------------                          266.2/729.2 kB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------             501.8/729.2 kB 2.4 MB/s eta 0:00:01\n",
            "     --------------------------             501.8/729.2 kB 2.4 MB/s eta 0:00:01\n",
            "     --------------------------             501.8/729.2 kB 2.4 MB/s eta 0:00:01\n",
            "     ----------------------------           542.7/729.2 kB 1.5 MB/s eta 0:00:01\n",
            "     -------------------------------------  727.0/729.2 kB 1.9 MB/s eta 0:00:01\n",
            "     -------------------------------------- 729.2/729.2 kB 1.8 MB/s eta 0:00:00\n",
            "Collecting lightning-utilities>=0.7.0 (from pytorch_lightning->keytotext)\n",
            "  Downloading lightning_utilities-0.9.0-py3-none-any.whl (23 kB)\n",
            "Requirement already satisfied: sympy in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from torch->keytotext) (1.12)\n",
            "Requirement already satisfied: networkx in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from torch->keytotext) (3.1)\n",
            "Requirement already satisfied: jinja2 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from torch->keytotext) (3.1.2)\n",
            "Requirement already satisfied: regex!=2019.12.17 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from transformers->keytotext) (2023.6.3)\n",
            "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from transformers->keytotext) (0.13.3)\n",
            "Requirement already satisfied: safetensors>=0.3.1 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from transformers->keytotext) (0.3.1)\n",
            "Requirement already satisfied: Click!=8.0.0,>=7.1 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from wandb->keytotext) (8.1.4)\n",
            "Collecting GitPython!=3.1.29,>=1.0.0 (from wandb->keytotext)\n",
            "  Downloading GitPython-3.1.32-py3-none-any.whl (188 kB)\n",
            "                                              0.0/188.5 kB ? eta -:--:--\n",
            "     -------------------------------------- 188.5/188.5 kB 3.8 MB/s eta 0:00:00\n",
            "Requirement already satisfied: psutil>=5.0.0 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from wandb->keytotext) (5.9.5)\n",
            "Collecting sentry-sdk>=1.0.0 (from wandb->keytotext)\n",
            "  Downloading sentry_sdk-1.28.1-py2.py3-none-any.whl (214 kB)\n",
            "                                              0.0/214.7 kB ? eta -:--:--\n",
            "     ------------------------------------   204.8/214.7 kB 6.1 MB/s eta 0:00:01\n",
            "     -------------------------------------- 214.7/214.7 kB 4.4 MB/s eta 0:00:00\n",
            "Collecting docker-pycreds>=0.4.0 (from wandb->keytotext)\n",
            "  Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)\n",
            "Collecting pathtools (from wandb->keytotext)\n",
            "  Downloading pathtools-0.1.2.tar.gz (11 kB)\n",
            "  Installing build dependencies: started\n",
            "  Installing build dependencies: finished with status 'done'\n",
            "  Getting requirements to build wheel: started\n",
            "  Getting requirements to build wheel: finished with status 'done'\n",
            "  Preparing metadata (pyproject.toml): started\n",
            "  Preparing metadata (pyproject.toml): finished with status 'done'\n",
            "Collecting setproctitle (from wandb->keytotext)\n",
            "  Downloading setproctitle-1.3.2-cp311-cp311-win_amd64.whl (11 kB)\n",
            "Requirement already satisfied: setuptools in c:\\program files\\windowsapps\\pythonsoftwarefoundation.python.3.11_3.11.1264.0_x64__qbz5n2kfra8p0\\lib\\site-packages (from wandb->keytotext) (65.5.0)\n",
            "Collecting appdirs>=1.4.3 (from wandb->keytotext)\n",
            "  Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)\n",
            "Collecting protobuf!=4.21.0,<5,>=3.19.0 (from wandb->keytotext)\n",
            "  Downloading protobuf-4.23.4-cp310-abi3-win_amd64.whl (422 kB)\n",
            "                                              0.0/422.5 kB ? eta -:--:--\n",
            "     -----------------------                256.0/422.5 kB 7.7 MB/s eta 0:00:01\n",
            "     -----------------------                256.0/422.5 kB 7.7 MB/s eta 0:00:01\n",
            "     -----------------------                256.0/422.5 kB 7.7 MB/s eta 0:00:01\n",
            "     -----------------------------          327.7/422.5 kB 1.8 MB/s eta 0:00:01\n",
            "     -------------------------------------- 422.5/422.5 kB 2.0 MB/s eta 0:00:00\n",
            "Requirement already satisfied: colorama in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from Click!=8.0.0,>=7.1->wandb->keytotext) (0.4.6)\n",
            "Requirement already satisfied: six>=1.4.0 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from docker-pycreds>=0.4.0->wandb->keytotext) (1.16.0)\n",
            "Requirement already satisfied: attrs>=17.3.0 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from aiohttp->datasets->keytotext) (23.1.0)\n",
            "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from aiohttp->datasets->keytotext) (3.1.0)\n",
            "Collecting multidict<7.0,>=4.5 (from aiohttp->datasets->keytotext)\n",
            "  Downloading multidict-6.0.4-cp311-cp311-win_amd64.whl (28 kB)\n",
            "Collecting async-timeout<5.0,>=4.0.0a3 (from aiohttp->datasets->keytotext)\n",
            "  Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)\n",
            "Collecting yarl<2.0,>=1.0 (from aiohttp->datasets->keytotext)\n",
            "  Downloading yarl-1.9.2-cp311-cp311-win_amd64.whl (60 kB)\n",
            "                                              0.0/60.2 kB ? eta -:--:--\n",
            "     ---------------------------------------- 60.2/60.2 kB 3.3 MB/s eta 0:00:00\n",
            "Collecting frozenlist>=1.1.1 (from aiohttp->datasets->keytotext)\n",
            "  Downloading frozenlist-1.4.0-cp311-cp311-win_amd64.whl (44 kB)\n",
            "                                              0.0/44.9 kB ? eta -:--:--\n",
            "     ---------------------------------------- 44.9/44.9 kB 1.1 MB/s eta 0:00:00\n",
            "Collecting aiosignal>=1.1.2 (from aiohttp->datasets->keytotext)\n",
            "  Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)\n",
            "Collecting gitdb<5,>=4.0.1 (from GitPython!=3.1.29,>=1.0.0->wandb->keytotext)\n",
            "  Downloading gitdb-4.0.10-py3-none-any.whl (62 kB)\n",
            "                                              0.0/62.7 kB ? eta -:--:--\n",
            "     ---------------------------------------- 62.7/62.7 kB ? eta 0:00:00\n",
            "Requirement already satisfied: idna<4,>=2.5 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from requests>=2.19.0->datasets->keytotext) (3.4)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from requests>=2.19.0->datasets->keytotext) (2.0.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from requests>=2.19.0->datasets->keytotext) (2023.5.7)\n",
            "Collecting markdown-it-py>=2.2.0 (from rich>=10.4.0->keybert->keytotext)\n",
            "  Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)\n",
            "                                              0.0/87.5 kB ? eta -:--:--\n",
            "     ---------------------------------------- 87.5/87.5 kB ? eta 0:00:00\n",
            "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from rich>=10.4.0->keybert->keytotext) (2.15.1)\n",
            "Requirement already satisfied: scipy>=1.5.0 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from scikit-learn>=0.22.2->keybert->keytotext) (1.11.1)\n",
            "Requirement already satisfied: joblib>=1.1.1 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from scikit-learn>=0.22.2->keybert->keytotext) (1.3.1)\n",
            "Requirement already satisfied: threadpoolctl>=2.0.0 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from scikit-learn>=0.22.2->keybert->keytotext) (3.1.0)\n",
            "Requirement already satisfied: torchvision in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from sentence-transformers>=0.3.8->keybert->keytotext) (0.15.2)\n",
            "Requirement already satisfied: nltk in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from sentence-transformers>=0.3.8->keybert->keytotext) (3.8.1)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from jinja2->torch->keytotext) (2.1.3)\n",
            "Requirement already satisfied: python-dateutil>=2.8.2 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from pandas->datasets->keytotext) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from pandas->datasets->keytotext) (2023.3)\n",
            "Requirement already satisfied: tzdata>=2022.1 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from pandas->datasets->keytotext) (2023.3)\n",
            "Requirement already satisfied: mpmath>=0.19 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from sympy->torch->keytotext) (1.3.0)\n",
            "Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb->keytotext)\n",
            "  Downloading smmap-5.0.0-py3-none-any.whl (24 kB)\n",
            "Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=10.4.0->keybert->keytotext)\n",
            "  Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)\n",
            "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from torchvision->sentence-transformers>=0.3.8->keybert->keytotext) (10.0.0)\n",
            "Building wheels for collected packages: keybert, pathtools\n",
            "  Building wheel for keybert (pyproject.toml): started\n",
            "  Building wheel for keybert (pyproject.toml): finished with status 'done'\n",
            "  Created wheel for keybert: filename=keybert-0.7.0-py3-none-any.whl size=23826 sha256=b7a88155d766100f82fbf8feb53263855043dea8796d03aac21bace8e3e8f518\n",
            "  Stored in directory: c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local\\pip\\cache\\wheels\\ec\\44\\51\\2b76ff5ca28f2f5c41ab58782a685d57612cdce2f55d881158\n",
            "  Building wheel for pathtools (pyproject.toml): started\n",
            "  Building wheel for pathtools (pyproject.toml): finished with status 'done'\n",
            "  Created wheel for pathtools: filename=pathtools-0.1.2-py3-none-any.whl size=8803 sha256=6d7298a0c51a3d0e2774417c5d4f36ed95c03ed16c75c873900a7673e739b462\n",
            "  Stored in directory: c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local\\pip\\cache\\wheels\\ea\\b7\\8b\\84e94095ea418b9442f5abeba4ca7b0ad52d3fe7b69d6238a6\n",
            "Successfully built keybert pathtools\n",
            "Installing collected packages: pathtools, appdirs, xxhash, smmap, setproctitle, sentry-sdk, pyarrow, protobuf, multidict, mdurl, lightning-utilities, frozenlist, docker-pycreds, dill, async-timeout, yarl, multiprocess, markdown-it-py, gitdb, aiosignal, torchmetrics, rich, GitPython, aiohttp, wandb, pytorch_lightning, keybert, datasets, keytotext\n",
            "Successfully installed GitPython-3.1.32 aiohttp-3.8.4 aiosignal-1.3.1 appdirs-1.4.4 async-timeout-4.0.2 datasets-2.13.1 dill-0.3.6 docker-pycreds-0.4.0 frozenlist-1.4.0 gitdb-4.0.10 keybert-0.7.0 keytotext-2.3.2 lightning-utilities-0.9.0 markdown-it-py-3.0.0 mdurl-0.1.2 multidict-6.0.4 multiprocess-0.70.14 pathtools-0.1.2 protobuf-4.23.4 pyarrow-12.0.1 pytorch_lightning-2.0.5 rich-13.4.2 sentry-sdk-1.28.1 setproctitle-1.3.2 smmap-5.0.0 torchmetrics-1.0.1 wandb-0.15.5 xxhash-3.2.0 yarl-1.9.2\n"
          ]
        }
      ],
      "source": [
        "%pip install keytotext"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3osvVJqSP76j",
        "outputId": "461a3a7e-3e30-42b5-ae29-b800a4b2475a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collecting opencv-python\n",
            "  Downloading opencv_python-4.8.0.74-cp37-abi3-win_amd64.whl (38.1 MB)\n",
            "                                              0.0/38.1 MB ? eta -:--:--\n",
            "                                              0.0/38.1 MB ? eta -:--:--\n",
            "                                             0.0/38.1 MB 435.7 kB/s eta 0:01:28\n",
            "                                             0.0/38.1 MB 435.7 kB/s eta 0:01:28\n",
            "                                             0.0/38.1 MB 178.6 kB/s eta 0:03:33\n",
            "                                             0.1/38.1 MB 280.5 kB/s eta 0:02:16\n",
            "                                             0.1/38.1 MB 409.6 kB/s eta 0:01:33\n",
            "                                             0.1/38.1 MB 409.6 kB/s eta 0:01:33\n",
            "                                             0.1/38.1 MB 387.0 kB/s eta 0:01:38\n",
            "                                             0.2/38.1 MB 497.6 kB/s eta 0:01:17\n",
            "                                             0.3/38.1 MB 610.3 kB/s eta 0:01:02\n",
            "                                             0.3/38.1 MB 610.3 kB/s eta 0:01:02\n",
            "                                             0.4/38.1 MB 655.2 kB/s eta 0:00:58\n",
            "                                             0.5/38.1 MB 814.9 kB/s eta 0:00:47\n",
            "                                             0.5/38.1 MB 874.4 kB/s eta 0:00:43\n",
            "                                             0.5/38.1 MB 874.4 kB/s eta 0:00:43\n",
            "                                             0.6/38.1 MB 874.0 kB/s eta 0:00:43\n",
            "                                             0.7/38.1 MB 969.4 kB/s eta 0:00:39\n",
            "                                              0.8/38.1 MB 1.0 MB/s eta 0:00:37\n",
            "                                              0.8/38.1 MB 1.0 MB/s eta 0:00:37\n",
            "                                             0.9/38.1 MB 999.9 kB/s eta 0:00:38\n",
            "     -                                        1.0/38.1 MB 1.1 MB/s eta 0:00:34\n",
            "     -                                        1.1/38.1 MB 1.1 MB/s eta 0:00:34\n",
            "     -                                        1.1/38.1 MB 1.1 MB/s eta 0:00:36\n",
            "     -                                        1.2/38.1 MB 1.1 MB/s eta 0:00:34\n",
            "     -                                        1.3/38.1 MB 1.2 MB/s eta 0:00:32\n",
            "     -                                        1.3/38.1 MB 1.2 MB/s eta 0:00:32\n",
            "     -                                        1.4/38.1 MB 1.1 MB/s eta 0:00:33\n",
            "     -                                        1.5/38.1 MB 1.2 MB/s eta 0:00:31\n",
            "     -                                        1.6/38.1 MB 1.2 MB/s eta 0:00:31\n",
            "     -                                        1.6/38.1 MB 1.2 MB/s eta 0:00:30\n",
            "     -                                        1.7/38.1 MB 1.2 MB/s eta 0:00:31\n",
            "     -                                        1.8/38.1 MB 1.2 MB/s eta 0:00:30\n",
            "     -                                        1.9/38.1 MB 1.2 MB/s eta 0:00:30\n",
            "     -                                        1.9/38.1 MB 1.2 MB/s eta 0:00:30\n",
            "     --                                       1.9/38.1 MB 1.2 MB/s eta 0:00:30\n",
            "     --                                       2.0/38.1 MB 1.3 MB/s eta 0:00:29\n",
            "     --                                       2.1/38.1 MB 1.3 MB/s eta 0:00:29\n",
            "     --                                       2.1/38.1 MB 1.3 MB/s eta 0:00:29\n",
            "     --                                       2.2/38.1 MB 1.3 MB/s eta 0:00:29\n",
            "     --                                       2.3/38.1 MB 1.3 MB/s eta 0:00:28\n",
            "     --                                       2.4/38.1 MB 1.3 MB/s eta 0:00:28\n",
            "     --                                       2.4/38.1 MB 1.3 MB/s eta 0:00:28\n",
            "     --                                       2.5/38.1 MB 1.3 MB/s eta 0:00:28\n",
            "     --                                       2.6/38.1 MB 1.3 MB/s eta 0:00:27\n",
            "     --                                       2.7/38.1 MB 1.3 MB/s eta 0:00:27\n",
            "     --                                       2.7/38.1 MB 1.3 MB/s eta 0:00:27\n",
            "     --                                       2.8/38.1 MB 1.3 MB/s eta 0:00:28\n",
            "     ---                                      2.9/38.1 MB 1.3 MB/s eta 0:00:27\n",
            "     ---                                      2.9/38.1 MB 1.3 MB/s eta 0:00:27\n",
            "     ---                                      3.0/38.1 MB 1.3 MB/s eta 0:00:27\n",
            "     ---                                      3.1/38.1 MB 1.3 MB/s eta 0:00:27\n",
            "     ---                                      3.2/38.1 MB 1.4 MB/s eta 0:00:26\n",
            "     ---                                      3.2/38.1 MB 1.3 MB/s eta 0:00:27\n",
            "     ---                                      3.2/38.1 MB 1.3 MB/s eta 0:00:27\n",
            "     ---                                      3.4/38.1 MB 1.4 MB/s eta 0:00:26\n",
            "     ---                                      3.4/38.1 MB 1.4 MB/s eta 0:00:26\n",
            "     ---                                      3.5/38.1 MB 1.3 MB/s eta 0:00:26\n",
            "     ---                                      3.5/38.1 MB 1.3 MB/s eta 0:00:26\n",
            "     ---                                      3.7/38.1 MB 1.4 MB/s eta 0:00:26\n",
            "     ---                                      3.7/38.1 MB 1.4 MB/s eta 0:00:26\n",
            "     ---                                      3.7/38.1 MB 1.3 MB/s eta 0:00:26\n",
            "     ----                                     3.8/38.1 MB 1.4 MB/s eta 0:00:26\n",
            "     ----                                     3.9/38.1 MB 1.4 MB/s eta 0:00:25\n",
            "     ----                                     3.9/38.1 MB 1.4 MB/s eta 0:00:25\n",
            "     ----                                     4.0/38.1 MB 1.4 MB/s eta 0:00:26\n",
            "     ----                                     4.1/38.1 MB 1.4 MB/s eta 0:00:25\n",
            "     ----                                     4.2/38.1 MB 1.4 MB/s eta 0:00:25\n",
            "     ----                                     4.2/38.1 MB 1.4 MB/s eta 0:00:25\n",
            "     ----                                     4.3/38.1 MB 1.4 MB/s eta 0:00:25\n",
            "     ----                                     4.4/38.1 MB 1.4 MB/s eta 0:00:25\n",
            "     ----                                     4.5/38.1 MB 1.4 MB/s eta 0:00:25\n",
            "     ----                                     4.5/38.1 MB 1.4 MB/s eta 0:00:25\n",
            "     ----                                     4.5/38.1 MB 1.4 MB/s eta 0:00:25\n",
            "     ----                                     4.6/38.1 MB 1.4 MB/s eta 0:00:25\n",
            "     ----                                     4.7/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     ----                                     4.7/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     -----                                    4.8/38.1 MB 1.4 MB/s eta 0:00:25\n",
            "     -----                                    4.9/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     -----                                    5.0/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     -----                                    5.0/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     -----                                    5.1/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     -----                                    5.2/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     -----                                    5.3/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     -----                                    5.3/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     -----                                    5.3/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     -----                                    5.4/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     -----                                    5.5/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     -----                                    5.6/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     -----                                    5.7/38.1 MB 1.4 MB/s eta 0:00:24\n",
            "     ------                                   5.8/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   5.8/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   5.9/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   5.9/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   6.0/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   6.1/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   6.1/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   6.2/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   6.3/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   6.3/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   6.4/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   6.5/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   6.6/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     ------                                   6.6/38.1 MB 1.4 MB/s eta 0:00:23\n",
            "     -------                                  6.7/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  6.8/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  6.8/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  6.9/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  7.0/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  7.1/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  7.1/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  7.2/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  7.3/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  7.3/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  7.4/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  7.5/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  7.5/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     -------                                  7.6/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     --------                                 7.6/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     --------                                 7.7/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     --------                                 7.8/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     --------                                 7.9/38.1 MB 1.4 MB/s eta 0:00:22\n",
            "     --------                                 7.9/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     --------                                 8.0/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     --------                                 8.1/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     --------                                 8.1/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     --------                                 8.2/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     --------                                 8.3/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     --------                                 8.4/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     --------                                 8.4/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     --------                                 8.5/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     ---------                                8.6/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     ---------                                8.6/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     ---------                                8.7/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     ---------                                8.8/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     ---------                                8.9/38.1 MB 1.5 MB/s eta 0:00:21\n",
            "     ---------                                8.9/38.1 MB 1.5 MB/s eta 0:00:21\n",
            "     ---------                                9.0/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     ---------                                9.1/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     ---------                                9.1/38.1 MB 1.5 MB/s eta 0:00:20\n",
            "     ---------                                9.1/38.1 MB 1.4 MB/s eta 0:00:21\n",
            "     ---------                                9.2/38.1 MB 1.4 MB/s eta 0:00:20\n",
            "     ---------                                9.3/38.1 MB 1.4 MB/s eta 0:00:20\n",
            "     ---------                                9.4/38.1 MB 1.5 MB/s eta 0:00:20\n",
            "     ---------                                9.4/38.1 MB 1.4 MB/s eta 0:00:20\n",
            "     ---------                                9.5/38.1 MB 1.4 MB/s eta 0:00:20\n",
            "     ----------                               9.6/38.1 MB 1.5 MB/s eta 0:00:20\n",
            "     ----------                               9.7/38.1 MB 1.5 MB/s eta 0:00:20\n",
            "     ----------                               9.7/38.1 MB 1.4 MB/s eta 0:00:20\n",
            "     ----------                               9.8/38.1 MB 1.5 MB/s eta 0:00:20\n",
            "     ----------                               9.9/38.1 MB 1.5 MB/s eta 0:00:20\n",
            "     ----------                               9.9/38.1 MB 1.5 MB/s eta 0:00:20\n",
            "     ----------                               10.0/38.1 MB 1.4 MB/s eta 0:00:20\n",
            "     ----------                               10.0/38.1 MB 1.4 MB/s eta 0:00:20\n",
            "     ----------                               10.2/38.1 MB 1.5 MB/s eta 0:00:20\n",
            "     ----------                               10.2/38.1 MB 1.5 MB/s eta 0:00:20\n",
            "     ----------                               10.3/38.1 MB 1.5 MB/s eta 0:00:20\n",
            "     ----------                               10.3/38.1 MB 1.5 MB/s eta 0:00:19\n",
            "     ----------                               10.4/38.1 MB 1.5 MB/s eta 0:00:19\n",
            "     ----------                               10.5/38.1 MB 1.5 MB/s eta 0:00:19\n",
            "     -----------                              10.5/38.1 MB 1.5 MB/s eta 0:00:19\n",
            "     -----------                              10.6/38.1 MB 1.5 MB/s eta 0:00:19\n",
            "     -----------                              10.7/38.1 MB 1.5 MB/s eta 0:00:19\n",
            "     -----------                              10.7/38.1 MB 1.5 MB/s eta 0:00:19\n",
            "     -----------                              10.8/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     -----------                              10.9/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     -----------                              11.0/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     -----------                              11.0/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     -----------                              11.0/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     -----------                              11.1/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     -----------                              11.2/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     -----------                              11.2/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     -----------                              11.3/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     -----------                              11.4/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             11.5/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             11.5/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             11.6/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             11.7/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             11.7/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             11.8/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             11.9/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             12.0/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             12.0/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             12.1/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             12.1/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             12.2/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     ------------                             12.2/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     ------------                             12.3/38.1 MB 1.5 MB/s eta 0:00:18\n",
            "     -------------                            12.4/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            12.5/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            12.5/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            12.6/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            12.7/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            12.7/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            12.8/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            12.8/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            12.9/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            13.0/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            13.0/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            13.1/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            13.2/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            13.3/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     -------------                            13.3/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     --------------                           13.4/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     --------------                           13.5/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     --------------                           13.5/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     --------------                           13.5/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     --------------                           13.6/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     --------------                           13.7/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     --------------                           13.8/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     --------------                           13.8/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     --------------                           13.9/38.1 MB 1.5 MB/s eta 0:00:17\n",
            "     --------------                           14.0/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     --------------                           14.1/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     --------------                           14.1/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     --------------                           14.2/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     --------------                           14.2/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          14.3/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          14.3/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          14.4/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          14.5/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          14.6/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          14.6/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          14.7/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          14.7/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          14.8/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          14.9/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          14.9/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          15.0/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          15.1/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ---------------                          15.1/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ----------------                         15.2/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ----------------                         15.3/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     ----------------                         15.4/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     ----------------                         15.4/38.1 MB 1.5 MB/s eta 0:00:16\n",
            "     ----------------                         15.5/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     ----------------                         15.6/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     ----------------                         15.6/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     ----------------                         15.7/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     ----------------                         15.8/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     ----------------                         15.9/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     ----------------                         15.9/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     ----------------                         16.0/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     ----------------                         16.0/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     ----------------                         16.1/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     ----------------                         16.1/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     -----------------                        16.2/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     -----------------                        16.3/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     -----------------                        16.4/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     -----------------                        16.4/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     -----------------                        16.4/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     -----------------                        16.5/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     -----------------                        16.6/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     -----------------                        16.7/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     -----------------                        16.7/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     -----------------                        16.8/38.1 MB 1.5 MB/s eta 0:00:15\n",
            "     -----------------                        16.9/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     -----------------                        16.9/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     -----------------                        16.9/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     -----------------                        17.0/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     -----------------                        17.1/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       17.2/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       17.2/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       17.3/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       17.4/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       17.4/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       17.5/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       17.6/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       17.7/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       17.7/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       17.8/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       17.8/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       17.9/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       18.0/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       18.0/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     ------------------                       18.1/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     -------------------                      18.2/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     -------------------                      18.2/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     -------------------                      18.3/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     -------------------                      18.4/38.1 MB 1.5 MB/s eta 0:00:14\n",
            "     -------------------                      18.4/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     -------------------                      18.5/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     -------------------                      18.5/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     -------------------                      18.6/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     -------------------                      18.7/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     -------------------                      18.7/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     -------------------                      18.8/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     -------------------                      18.8/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     -------------------                      18.9/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     -------------------                      19.0/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.0/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.1/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.2/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.3/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.3/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.4/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.5/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.5/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.6/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.6/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.7/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.8/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.8/38.1 MB 1.5 MB/s eta 0:00:13\n",
            "     --------------------                     19.9/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.0/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.0/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.1/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.2/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.3/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.3/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.4/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.4/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.5/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.5/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.6/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.7/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.8/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.8/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ---------------------                    20.9/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ----------------------                   21.0/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ----------------------                   21.1/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ----------------------                   21.1/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ----------------------                   21.2/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ----------------------                   21.3/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ----------------------                   21.3/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ----------------------                   21.3/38.1 MB 1.5 MB/s eta 0:00:12\n",
            "     ----------------------                   21.5/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     ----------------------                   21.5/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     ----------------------                   21.6/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     ----------------------                   21.6/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     ----------------------                   21.7/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     ----------------------                   21.8/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     ----------------------                   21.9/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  21.9/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.0/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.1/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.1/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.2/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.3/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.4/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.4/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.5/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.5/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.6/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.6/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.7/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     -----------------------                  22.8/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     ------------------------                 22.9/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     ------------------------                 22.9/38.1 MB 1.5 MB/s eta 0:00:11\n",
            "     ------------------------                 23.0/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     ------------------------                 23.1/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     ------------------------                 23.1/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     ------------------------                 23.2/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     ------------------------                 23.3/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     ------------------------                 23.3/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     ------------------------                 23.4/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     ------------------------                 23.4/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     ------------------------                 23.5/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     ------------------------                 23.6/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     ------------------------                 23.7/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     ------------------------                 23.7/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------------------                23.8/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------------------                23.9/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------------------                23.9/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------------------                24.0/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------------------                24.1/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------------------                24.1/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------------------                24.2/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------------------                24.2/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------------------                24.3/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------------------                24.4/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------------------                24.4/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     -------------------------                24.5/38.1 MB 1.5 MB/s eta 0:00:10\n",
            "     -------------------------                24.6/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     -------------------------                24.6/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     -------------------------                24.7/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     -------------------------                24.7/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               24.8/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               24.9/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               25.0/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               25.0/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               25.1/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               25.1/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               25.2/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               25.2/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               25.3/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               25.4/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               25.5/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               25.5/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               25.6/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     --------------------------               25.6/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     ---------------------------              25.7/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     ---------------------------              25.8/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     ---------------------------              25.8/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     ---------------------------              25.9/38.1 MB 1.5 MB/s eta 0:00:09\n",
            "     ---------------------------              26.0/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ---------------------------              26.0/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ---------------------------              26.1/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ---------------------------              26.2/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ---------------------------              26.3/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ---------------------------              26.3/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ---------------------------              26.4/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ---------------------------              26.4/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ---------------------------              26.5/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ---------------------------              26.6/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             26.6/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             26.7/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             26.8/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             26.8/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             26.9/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             27.0/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             27.0/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             27.1/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             27.2/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             27.3/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             27.3/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             27.4/38.1 MB 1.5 MB/s eta 0:00:08\n",
            "     ----------------------------             27.4/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     ----------------------------             27.5/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     ----------------------------             27.5/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            27.6/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            27.7/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            27.8/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            27.8/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            27.9/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            28.0/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            28.0/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            28.1/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            28.1/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            28.2/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            28.3/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            28.3/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            28.4/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     -----------------------------            28.5/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     ------------------------------           28.5/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     ------------------------------           28.6/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     ------------------------------           28.7/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     ------------------------------           28.7/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     ------------------------------           28.8/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     ------------------------------           28.8/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     ------------------------------           28.9/38.1 MB 1.5 MB/s eta 0:00:07\n",
            "     ------------------------------           29.0/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------------           29.1/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------------           29.1/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------------           29.2/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------------           29.3/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------------           29.4/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     ------------------------------           29.4/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          29.5/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          29.5/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          29.6/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          29.7/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          29.8/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          29.8/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          29.9/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          29.9/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          30.0/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          30.1/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          30.1/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          30.2/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          30.3/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          30.3/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          30.4/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     -------------------------------          30.4/38.1 MB 1.5 MB/s eta 0:00:06\n",
            "     --------------------------------         30.5/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------------         30.6/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------------         30.7/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------------         30.7/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------------         30.8/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------------         30.9/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------------         30.9/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------------         31.0/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------------         31.1/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------------         31.1/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------------         31.2/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------------         31.2/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     --------------------------------         31.3/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------------        31.4/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------------        31.4/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------------        31.5/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------------        31.6/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------------        31.7/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------------        31.7/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------------        31.8/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------------        31.9/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------------        31.9/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------------        32.0/38.1 MB 1.5 MB/s eta 0:00:05\n",
            "     ---------------------------------        32.1/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ---------------------------------        32.1/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ---------------------------------        32.2/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ---------------------------------        32.2/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ---------------------------------        32.3/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       32.4/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       32.5/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       32.5/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       32.6/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       32.7/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       32.7/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       32.8/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       32.9/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       33.0/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       33.0/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       33.1/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       33.1/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       33.2/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       33.2/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     ----------------------------------       33.3/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     -----------------------------------      33.4/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     -----------------------------------      33.5/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     -----------------------------------      33.5/38.1 MB 1.5 MB/s eta 0:00:04\n",
            "     -----------------------------------      33.5/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     -----------------------------------      33.7/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     -----------------------------------      33.7/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     -----------------------------------      33.8/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     -----------------------------------      33.8/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     -----------------------------------      33.9/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     -----------------------------------      34.0/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     -----------------------------------      34.0/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     -----------------------------------      34.2/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     -----------------------------------      34.2/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     34.3/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     34.3/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     34.4/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     34.5/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     34.5/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     34.6/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     34.7/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     34.7/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     34.8/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     34.8/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     34.9/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     35.0/38.1 MB 1.5 MB/s eta 0:00:03\n",
            "     ------------------------------------     35.1/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     ------------------------------------     35.1/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    35.2/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    35.3/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    35.3/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    35.4/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    35.5/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    35.5/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    35.6/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    35.6/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    35.8/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    35.8/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    35.9/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    35.9/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    36.0/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    36.1/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    36.1/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     -------------------------------------    36.1/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     --------------------------------------   36.3/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     --------------------------------------   36.3/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     --------------------------------------   36.4/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     --------------------------------------   36.4/38.1 MB 1.5 MB/s eta 0:00:02\n",
            "     --------------------------------------   36.5/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   36.6/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   36.6/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   36.7/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   36.8/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   36.9/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   36.9/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   36.9/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     --------------------------------------   37.1/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.1/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.1/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.2/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.3/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.4/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.4/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.4/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.6/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.6/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.7/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.7/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.8/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.9/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.9/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  37.9/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  38.1/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  38.1/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  38.1/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  38.1/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------  38.1/38.1 MB 1.5 MB/s eta 0:00:01\n",
            "     ---------------------------------------- 38.1/38.1 MB 1.5 MB/s eta 0:00:00\n",
            "Requirement already satisfied: numpy>=1.21.2 in c:\\users\\hchai\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from opencv-python) (1.25.0)\n",
            "Installing collected packages: opencv-python\n",
            "Successfully installed opencv-python-4.8.0.74\n",
            "Note: you may need to restart the kernel to use updated packages.\n"
          ]
        }
      ],
      "source": [
        "%pip install opencv-python\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bPFztGVBP76j",
        "outputId": "2625d626-9d81-48ad-9295-0081dc5c35e2"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Downloading (…)lve/main/config.json: 100%|██████████| 1.20k/1.20k [00:00<?, ?B/s]\n",
            "Downloading (…)/main/tokenizer.json: 100%|██████████| 1.39M/1.39M [00:01<00:00, 961kB/s]\n",
            "Downloading pytorch_model.bin: 100%|██████████| 892M/892M [01:36<00:00, 9.20MB/s] \n",
            "C:\\Users\\hchai\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python311\\site-packages\\transformers\\generation\\utils.py:1353: UserWarning: Using `max_length`'s default (20) to control the generation length. This behaviour is deprecated and will be removed from the config in v5 of Transformers -- we recommend using `max_new_tokens` to control the maximum length of the generation.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The data-driven decision-maker has improved operational efficiency by using improved operation efficiency.\n"
          ]
        }
      ],
      "source": [
        "from keytotext import pipeline\n",
        "nlp = pipeline(\"k2t-base\")  #loading the pre-trained model\n",
        "params = {\"do_sample\":True, \"num_beams\":4, \"no_repeat_ngram_size\":3, \"early_stopping\":True}    #decoding params\n",
        "print (nlp(ans['answer'], **params))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MWqToESTP76j",
        "outputId": "6cc463fe-1305-4f8c-82ef-7d3237c64f81"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "C:\\Users\\hchai\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python311\\site-packages\\transformers\\generation\\utils.py:1353: UserWarning: Using `max_length`'s default (20) to control the generation length. This behaviour is deprecated and will be removed from the config in v5 of Transformers -- we recommend using `max_new_tokens` to control the maximum length of the generation.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The sales of the finance product in the year when the distribution system is SAP is sales.\n"
          ]
        }
      ],
      "source": [
        "params = {\"do_sample\":True, \"num_beams\":4, \"no_repeat_ngram_size\":3, \"early_stopping\":True}    #decoding params\n",
        "print (nlp([ans['answer'],'SAP'], **params))"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.4"
    },
    "orig_nbformat": 4,
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}