Merge PEFT feature branch (#11240)

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> Signed-off-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com> Co-authored-by: Ben Wilson <39283302+BenWilson2@users.noreply.github.com>
mlflow · Feb 29, 2024 · 2d23fbe · 2d23fbe
1 parent 1e10df4
commit 2d23fbe
Show file tree

Hide file tree

Showing 29 changed files with 3,193 additions and 835 deletions.
diff --git a/docs/source/llms/custom-pyfunc-for-llms/notebooks/custom-pyfunc-advanced-llm.ipynb b/docs/source/llms/custom-pyfunc-for-llms/notebooks/custom-pyfunc-advanced-llm.ipynb
@@ -4,7 +4,20 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Serving LLMs with MLflow: Leveraging Custom PyFunc\n",
+    "## Serving LLMs with MLflow: Leveraging Custom PyFunc"
+   ]
+  },
+  {
+   "cell_type": "raw",
+   "metadata": {},
+   "source": [
+    "<a href=\"https://raw.githubusercontent.com/mlflow/mlflow/master/docs/source/llms/custom-pyfunc-for-llms/notebooks/custom-pyfunc-advanced-llm.ipynb\" class=\"notebook-download-btn\">Download this Notebook</a>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
     "\n",
     "### Introduction\n",
     "\n",

diff --git a/docs/source/llms/transformers/guide/index.rst b/docs/source/llms/transformers/guide/index.rst
diff --git a/docs/source/llms/transformers/index.rst b/docs/source/llms/transformers/index.rst
diff --git a/docs/source/llms/transformers/tutorials/audio-transcription/whisper.ipynb b/docs/source/llms/transformers/tutorials/audio-transcription/whisper.ipynb
@@ -6,8 +6,20 @@
    "source": [
     "## Introduction to MLflow and OpenAI's Whisper\n",
     "\n",
-    "Discover the integration of [OpenAI's Whisper](https://huggingface.co/openai), an [ASR system](https://en.wikipedia.org/wiki/Speech_recognition), with MLflow in this tutorial.\n",
-    "\n",
+    "Discover the integration of [OpenAI's Whisper](https://huggingface.co/openai), an [ASR system](https://en.wikipedia.org/wiki/Speech_recognition), with MLflow in this tutorial.\n"
+   ]
+  },
+  {
+   "cell_type": "raw",
+   "metadata": {},
+   "source": [
+    "<a href=\"https://raw.githubusercontent.com/mlflow/mlflow/master/docs/source/llms/transformers/tutorials/audio-transcription/whisper.ipynb\" class=\"notebook-download-btn\">Download this Notebook</a>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
     "### What You Will Learn in This Tutorial\n",
     "\n",
     "- Establish an audio transcription **pipeline** using the Whisper model.\n",
@@ -333,6 +345,10 @@
     "        signature=signature,\n",
     "        input_example=audio,\n",
     "        model_config=model_config,\n",
+    "        # Since MLflow 2.11.0, you can save the model in 'reference-only' mode to reduce storage usage by not saving\n",
+    "        # the base model weights but only the reference to the HuggingFace model hub. To enable this, uncomment the\n",
+    "        # following line:\n",
+    "        # save_pretrained=False,\n",
     "    )"
    ]
   },

diff --git a/docs/source/llms/transformers/tutorials/conversational/conversational-model.ipynb b/docs/source/llms/transformers/tutorials/conversational/conversational-model.ipynb
@@ -6,7 +6,20 @@
    "source": [
     "## Introduction to Conversational AI with MLflow and DialoGPT\n",
     "\n",
-    "Welcome to our tutorial on integrating [Microsoft's DialoGPT](https://huggingface.co/microsoft/DialoGPT-medium) with MLflow's transformers flavor to explore conversational AI.\n",
+    "Welcome to our tutorial on integrating [Microsoft's DialoGPT](https://huggingface.co/microsoft/DialoGPT-medium) with MLflow's transformers flavor to explore conversational AI."
+   ]
+  },
+  {
+   "cell_type": "raw",
+   "metadata": {},
+   "source": [
+    "<a href=\"https://raw.githubusercontent.com/mlflow/mlflow/master/docs/source/llms/transformers/tutorials/conversational/conversational-model.ipynb\" class=\"notebook-download-btn\">Download this Notebook</a>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
     "\n",
     "### Learning Objectives\n",
     "\n",

diff --git a/docs/source/llms/transformers/tutorials/fine-tuning/transformers-fine-tuning.ipynb b/docs/source/llms/transformers/tutorials/fine-tuning/transformers-fine-tuning.ipynb
@@ -6,8 +6,20 @@
    "source": [
     "## Fine-Tuning Transformers with MLflow for Enhanced Model Management\n",
     "\n",
-    "Welcome to our in-depth tutorial on fine-tuning Transformers models with enhanced management using MLflow.\n",
-    "\n",
+    "Welcome to our in-depth tutorial on fine-tuning Transformers models with enhanced management using MLflow."
+   ]
+  },
+  {
+   "cell_type": "raw",
+   "metadata": {},
+   "source": [
+    "<a href=\"https://raw.githubusercontent.com/mlflow/mlflow/master/docs/source/llms/transformers/tutorials/fine-tuning/transformers-fine-tuning.ipynb\" class=\"notebook-download-btn\">Download the Fine Tuning Notebook</a>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
     "### What You Will Learn in This Tutorial\n",
     "\n",
     "- Understand the process of fine-tuning a Transformers model.\n",