adding feature request openhackathons-org#14 openhackathons-org#15 op…

…enhackathons-org#17

workspace/LLM-Use-Case.ipynb

@@ -138,7 +138,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3",
    "language": "python",
    "name": "python3"
   },
@@ -152,7 +152,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.12"
+   "version": "3.8.8"
   }
  },
  "nbformat": 4,

workspace/data/filtered/ignore.txt

@@ -0,0 +1 @@
+data.txt

workspace/jupyter_notebook/llm-use-case/challenge.ipynb

@@ -348,7 +348,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "def format_training_prompt(conversation, summary):\n",
+    "def format_inference_prompt(conversation, summary):\n",
     "    return f\"\"\"### Instruction: Write  a summary of the conversation below. ### Input: {conversation.strip()} ### Response: \"\"\".strip()\n"
    ]
   },
@@ -366,7 +366,7 @@
     "        {\n",
     "          \"summary\": row[\"summary\"],\n",
     "            \"conversation\": row[\"conversation\"],\n",
-    "            \"prompt\": format_training_prompt(row[\"conversation\"], row[\"summary\"]),\n",
+    "            \"prompt\": format_inference_prompt(row[\"conversation\"], row[\"summary\"]),\n",
     "        }    \n",
     "    )\n",
     "    \n",
@@ -437,7 +437,9 @@
    "id": "789bce5a",
    "metadata": {},
    "source": [
-    "### Build TensorRT engine with single GPU and FP16"
+    "### Build TensorRT engine with single GPU and FP16\n",
+    "\n",
+    "- Build your model tensorrt engine to this directory: `../../model/challenge/trt_engines/fp16/1-gpu/`"
    ]
   },
   {
@@ -451,12 +453,61 @@
     "\n"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "4fd3a409-9c7a-4062-876a-19f2b59eebcb",
+   "metadata": {},
+   "source": [
+    "#### Run inference"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "82d7ee86-c885-487f-8213-0492ff91783a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def format_infernece_prompt():\n",
+    "    return f\"\"\"### Instruction: Write  a summary of the conversation below. ### Input: user: So neither my iPhone nor my Apple Watch are recording my steps/activity, and Health doesn’t recognise either source anymore for some reason. Any ideas?   please read the above.\n",
+    "agent: Let’s investigate this together. To start, can you tell us the software versions your iPhone and Apple Watch are running currently?\n",
+    "user: My iPhone is on 11.1.2, and my watch is on 4.1.\n",
+    "agent: Thank you. Have you tried restarting both devices since this started happening?\n",
+    "user: I’ve restarted both, also un-paired then re-paired the watch.\n",
+    "agent: Got it. When did you first notice that the two devices were not talking to each other. Do the two devices communicate through other apps such as Messages?\n",
+    "user: Yes, everything seems fine, it’s just Health and activity.\n",
+    "agent: Let’s move to DM and look into this a bit more. When reaching out in DM, let us know when this first started happening please. For example, did it start after an update or after installing a certain app? ### Response: Customer enquired about his Iphone and Apple watch which is not showing his any steps/activity and health activities. Agent is asking to move to DM and look into it.\n",
+    "\"\"\".strip()\n",
+    "\n",
+    "text = format_infernece_prompt()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9f1824d9-6768-4b7c-9b36-520a33199df8",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5c7d8c99-dccd-4624-9e6c-24398706a0f2",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
   {
    "cell_type": "markdown",
    "id": "4abb2e99",
    "metadata": {},
    "source": [
-    "### Deploy Model on Triton Server"
+    "---\n",
+    "### Deploy Model on Triton Server\n",
+    "\n",
+    "- Copy the TRT engine to triton_model_repo/tensorrt_llm/1 : `!cp /workspace/app/model/challenge/trt_engines/fp16/1-gpu/* triton_model_repo/tensorrt_llm/1`\n",
+    "- Remember to modify the config files for model Preprocessing, tensorrt_llm, and Postprocessing. "
    ]
   },
   {
@@ -466,10 +517,17 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "\n",
     "\n"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ff4da40c-26d8-48d0-a9bf-d61f0e151443",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
   {
    "cell_type": "markdown",
    "id": "0d1b52ee",
@@ -502,7 +560,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3",
    "language": "python",
    "name": "python3"
   },
@@ -516,7 +574,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.8.8"
   }
  },
  "nbformat": 4,

workspace/jupyter_notebook/llm-use-case/llama-chat-finetune.ipynb

@@ -1082,7 +1082,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3",
    "language": "python",
    "name": "python3"
   },
@@ -1096,7 +1096,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.8.8"
   }
  },
  "nbformat": 4,