Feature/Add LLMs Integration For Oracle Cloud Infrastructure(OCI) Data Science Model Deployment Endpoint

docs/docs/integrations/llms/oci_model_deployment_endpoint.ipynb

@@ -0,0 +1,131 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# OCI Data Science Model Deployment Endpoint\n",
+    "\n",
+    "[OCI Data Science](https://docs.oracle.com/en-us/iaas/data-science/using/home.htm) is a fully managed and serverless platform for data science teams to build, train, and manage machine learning models in the Oracle Cloud Infrastructure.\n",
+    "\n",
+    "This notebooks goes over how to use an LLM hosted on a [OCI Data Science Model Deployment](https://docs.oracle.com/en-us/iaas/data-science/using/model-dep-about.htm).\n",
+    "\n",
+    "To authenticate, [oracle-ads](https://accelerated-data-science.readthedocs.io/en/latest/user_guide/cli/authentication.html) has been used to automatically load credentials for invoking endpoint."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip3 install oracle-ads"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Prerequisite\n",
+    "\n",
+    "### Deploy model\n",
+    "Check [Oracle GitHub samples repository](https://github.com/oracle-samples/oci-data-science-ai-samples/tree/main/model-deployment/containers/llama2) on how to deploy your llm on OCI Data Science Model deployment.\n",
+    "\n",
+    "### Policies\n",
+    "Make sure to have the required [policies](https://docs.oracle.com/en-us/iaas/data-science/using/model-dep-policies-auth.htm#model_dep_policies_auth__predict-endpoint) to access the OCI Data Science Model Deployment endpoint.\n",
+    "\n",
+    "## Set up\n",
+    "\n",
+    "### vLLM\n",
+    "After having deployed model, you have to set up following required parameters of the `OCIModelDeploymentVLLM` call:\n",
+    "\n",
+    "- **`endpoint`**: The model HTTP endpoint from the deployed model, e.g. `https://<MD_OCID>/predict`. \n",
+    "- **`model`**: The location of the model.\n",
+    "\n",
+    "### Text generation inference (TGI)\n",
+    "You have to set up following required parameters of the `OCIModelDeploymentTGI` call:\n",
+    "\n",
+    "- **`endpoint`**: The model HTTP endpoint from the deployed model, e.g. `https://<MD_OCID>/predict`. \n",
+    "\n",
+    "### Authentication\n",
+    "\n",
+    "You can set authentication through either ads or environment variables. When you are working in OCI Data Science Notebook Session, you can leverage resource principal to access other OCI resources. Check out [here](https://accelerated-data-science.readthedocs.io/en/latest/user_guide/cli/authentication.html) to see more options. \n",
+    "\n",
+    "## Example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import ads\n",
+    "from langchain_community.llms import OCIModelDeploymentVLLM\n",
+    "\n",
+    "# Set authentication through ads\n",
+    "# Use resource principal are operating within a\n",
+    "# OCI service that has resource principal based\n",
+    "# authentication configured\n",
+    "ads.set_auth(\"resource_principal\")\n",
+    "\n",
+    "# Create an instance of OCI Model Deployment Endpoint\n",
+    "# Replace the endpoint uri and model name with your own\n",
+    "llm = OCIModelDeploymentVLLM(endpoint=\"https://<MD_OCID>/predict\", model=\"model_name\")\n",
+    "\n",
+    "# Run the LLM\n",
+    "llm.invoke(\"Who is the first president of United States?\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "from langchain_community.llms import OCIModelDeploymentTGI\n",
+    "\n",
+    "# Set authentication through environment variables\n",
+    "# Use API Key setup when you are working from a local\n",
+    "# workstation or on platform which does not support\n",
+    "# resource principals.\n",
+    "os.environ[\"OCI_IAM_TYPE\"] = \"api_key\"\n",
+    "os.environ[\"OCI_CONFIG_PROFILE\"] = \"default\"\n",
+    "os.environ[\"OCI_CONFIG_LOCATION\"] = \"~/.oci\"\n",
+    "\n",
+    "# Set endpoint through environment variables\n",
+    "# Replace the endpoint uri with your own\n",
+    "os.environ[\"OCI_LLM_ENDPOINT\"] = \"https://<MD_OCID>/predict\"\n",
+    "\n",
+    "# Create an instance of OCI Model Deployment Endpoint\n",
+    "llm = OCIModelDeploymentTGI()\n",
+    "\n",
+    "# Run the LLM\n",
+    "llm.invoke(\"Who is the first president of United States?\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "langchain",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.18"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

libs/community/langchain_community/llms/__init__.py

@@ -318,6 +318,22 @@ def _import_nlpcloud() -> Any:
     return NLPCloud
 
 
+def _import_oci_md_tgi() -> Any:
+    from langchain_community.llms.oci_data_science_model_deployment_endpoint import (
+        OCIModelDeploymentTGI,
+    )
+
+    return OCIModelDeploymentTGI
+
+
+def _import_oci_md_vllm() -> Any:
+    from langchain_community.llms.oci_data_science_model_deployment_endpoint import (
+        OCIModelDeploymentVLLM,
+    )
+
+    return OCIModelDeploymentVLLM
+
+
 def _import_octoai_endpoint() -> Any:
     from langchain_community.llms.octoai_endpoint import OctoAIEndpoint
 
@@ -631,6 +647,10 @@ def __getattr__(name: str) -> Any:
         return _import_mosaicml()
     elif name == "NLPCloud":
         return _import_nlpcloud()
+    elif name == "OCIModelDeploymentTGI":
+        return _import_oci_md_tgi()
+    elif name == "OCIModelDeploymentVLLM":
+        return _import_oci_md_vllm()
     elif name == "OctoAIEndpoint":
         return _import_octoai_endpoint()
     elif name == "Ollama":
@@ -761,6 +781,8 @@ def __getattr__(name: str) -> Any:
     "Nebula",
     "NIBittensorLLM",
     "NLPCloud",
+    "OCIModelDeploymentTGI",
+    "OCIModelDeploymentVLLM",
     "Ollama",
     "OpenAI",
     "OpenAIChat",
@@ -848,6 +870,8 @@ def get_type_to_cls_dict() -> Dict[str, Callable[[], Type[BaseLLM]]]:
         "nebula": _import_symblai_nebula,
         "nibittensor": _import_bittensor,
         "nlpcloud": _import_nlpcloud,
+        "oci_model_deployment_tgi_endpoint": _import_oci_md_tgi,
+        "oci_model_deployment_vllm_endpoint": _import_oci_md_vllm,
         "ollama": _import_ollama,
         "openai": _import_openai,
         "openlm": _import_openlm,