-
Notifications
You must be signed in to change notification settings - Fork 192
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Jupyter Notebooks for NeuralChat (#277)
* Jupyter Notebooks for NeuralChat Signed-off-by: Lv, Liang1 <liang1.lv@intel.com> * update build and deploy chatbot Signed-off-by: Lv, Liang1 <liang1.lv@intel.com> * added NeuralChat optimization notebooks. Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com> * Update tts.py * Add Notebooks for finetuning chatbot on various platforms (#309) * fix config Signed-off-by: XuhuiRen <xuhui.ren@intel.com> * add notebook Signed-off-by: XuhuiRen <xuhui.ren@intel.com> --------- Signed-off-by: XuhuiRen <xuhui.ren@intel.com> * fix as suggestions Signed-off-by: XuhuiRen <xuhui.ren@intel.com> * Update tts.py * Update build_chatbot_on_spr.ipynb * Update build_chatbot_on_spr.ipynb * Update tts.py * update notebook Signed-off-by: Lv, Liang1 <liang1.lv@intel.com> * update notebook Signed-off-by: Lv, Liang1 <liang1.lv@intel.com> * fix pylint issue Signed-off-by: Lv, Liang1 <liang1.lv@intel.com> --------- Signed-off-by: Lv, Liang1 <liang1.lv@intel.com> Signed-off-by: Ye, Xinyu <xinyu.ye@intel.com> Signed-off-by: XuhuiRen <xuhui.ren@intel.com> Co-authored-by: Ye, Xinyu <xinyu.ye@intel.com> Co-authored-by: Liangyx2 <106130696+Liangyx2@users.noreply.github.com> Co-authored-by: Haihao Shen <haihao.shen@intel.com> Co-authored-by: XuhuiRen <44249229+XuhuiRen@users.noreply.github.com> Co-authored-by: XuhuiRen <xuhui.ren@intel.com>
- Loading branch information
6 people
committed
Sep 14, 2023
1 parent
276f889
commit 52f9f74
Showing
26 changed files
with
3,293 additions
and
1,176 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
61 changes: 61 additions & 0 deletions
61
...ension_for_transformers/neural_chat/docs/notebooks/amp_optimization_on_habana_gaudi.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# AMP Optimization of Chatbot on Habana's Gaudi processors(HPU)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Prepare Environment" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "**IMPORTANT:** Please note Habana's Gaudi processors(HPU) requires docker environment for running. User needs to manually execute below steps to build docker image and run docker container for inference on Habana HPU. The Jupyter notebook server should be started in the docker container and then run this Jupyter notebook. \n", | ||
| "\n", | ||
| "```bash\n", | ||
| "git clone https://github.com/intel/intel-extension-for-transformers.git\n", | ||
| "cd ./intel-extension-for-transformers/intel_extension_for_transformers/neural_chat/docker/\n", | ||
| "docker build --build-arg UBUNTU_VER=22.04 -f Dockerfile -t neuralchat . --target hpu\n", | ||
| "docker run -it --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --net=host --ipc=host neuralchat:latest\n", | ||
| "```\n" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## BF16 Optimization" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from intel_extension_for_transformers.neural_chat import build_chatbot\n", | ||
| "from intel_extension_for_transformers.neural_chat.config import PipelineConfig, AMPConfig\n", | ||
| "config = PipelineConfig(optimization_config=AMPConfig())\n", | ||
| "chatbot = build_chatbot(config)\n", | ||
| "response = chatbot.predict(query=\"Tell me about Intel Xeon Scalable Processors.\")\n", | ||
| "print(response)" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "language_info": { | ||
| "name": "python" | ||
| }, | ||
| "orig_nbformat": 4 | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |
94 changes: 94 additions & 0 deletions
94
intel_extension_for_transformers/neural_chat/docs/notebooks/amp_optimization_on_spr.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# AMP Optimization of Chatbot on 4th Generation of Intel® Xeon® Scalable Processors Sapphire Rapids" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Prepare Environment" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Install intel extension for transformers:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "!pip install intel-extension-for-transformers" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Install Requirements:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "%cd ../../\n", | ||
| "!pip install -r requirements.txt" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## BF16 Optimization" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from intel_extension_for_transformers.neural_chat import build_chatbot\n", | ||
| "from intel_extension_for_transformers.neural_chat.config import PipelineConfig, AMPConfig\n", | ||
| "config = PipelineConfig(optimization_config=AMPConfig())\n", | ||
| "chatbot = build_chatbot(config)\n", | ||
| "response = chatbot.predict(query=\"Tell me about Intel Xeon Scalable Processors.\")\n", | ||
| "print(response)" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "py39", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.9.16" | ||
| }, | ||
| "orig_nbformat": 4 | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |
Oops, something went wrong.