docs/source/getting-connected.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5f14c5a8",
   "metadata": {},
   "source": [
    "Copyright (c) 2021, 2022 Oracle and/or its affiliates.\n",
    "Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "serial-equivalent",
   "metadata": {},
   "source": [
    "## Getting Connected"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "knowing-biotechnology",
   "metadata": {},
   "source": [
    "### Configuring Your Connection"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "signal-genealogy",
   "metadata": {},
   "source": [
    "There are two IAM policy validation options in `ocifs`. The first option is using an identity policy, which is a configuration file. This is the most commonly used policy, and is the default in this documentation. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "silver-passport",
   "metadata": {},
   "outputs": [],
   "source": [
    "from ocifs import OCIFileSystem\n",
    "\n",
    "fs = OCIFileSystem(config=\"~/.oci/config\", profile=\"DEFAULT\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "objective-string",
   "metadata": {},
   "source": [
    "Using Pandas and Dask are just as easy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "blocked-dover",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "pd.read_csv(\"oci://bucket@namespace/path/file.csv\", \n",
    "            storage_options={\"config\": \"~/.oci/config\", \"profile\": \"DEFAULT\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "choice-chocolate",
   "metadata": {},
   "outputs": [],
   "source": [
    "from dask import dataframe as dd\n",
    "\n",
    "dd.read_csv(\"oci://bucket@namespace/path/file.csv\", \n",
    "            storage_options={\"config\": \"~/.oci/config\", \"profile\": \"DEFAULT\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sonic-response",
   "metadata": {},
   "source": [
    "### Resource Principal"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "embedded-confidence",
   "metadata": {},
   "source": [
    "The second policy option is using a resource principal. This policy only works if you're operating within a valid OCI resource, such as an [OCI Data Science notebook session](https://www.oracle.com/data-science/cloud-infrastructure-data-science.html). With this option, your resource token path is set by [global OCI signing variables](https://oracle-cloud-infrastructure-python-sdk.readthedocs.io/en/latest/api/signing.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "median-lying",
   "metadata": {},
   "outputs": [],
   "source": [
    "fs = OCIFileSystem()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "warming-warehouse",
   "metadata": {},
   "source": [
    "And with pandas or dask:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "hollow-confidence",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.read_csv(\"oci://bucket@namespace/path/file.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "psychological-affair",
   "metadata": {},
   "outputs": [],
   "source": [
    "dd.read_csv(\"oci://bucket@namespace/path/file.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "conditional-contribution",
   "metadata": {},
   "source": [
    "### Connecting Using a Signer\n",
    "Any signer can be passed in using the signer argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "distant-chassis",
   "metadata": {},
   "outputs": [],
   "source": [
    "resource_principal_signer = oci.auth.signer.get_resource_principals_signer()\n",
    "fs_rp = OCIFileSystem(signer=resource_principal_signer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "manufactured-poison",
   "metadata": {},
   "outputs": [],
   "source": [
    "instance_principal_signer = oci.auth.signers.InstancePrincipalsSecurityTokenSigner()\n",
    "fs_ip = OCIFileSystem(signer=instance_principal_signer)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "terminal-christmas",
   "metadata": {},
   "source": [
    "And with pandas or dask:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "detected-powder",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.read_csv(\"oci://bucket@namespace/path/file.csv\", \n",
    "            storage_options={\"signer\": resource_principal_signer})\n",
    "\n",
    "dd.read_csv(\"oci://bucket@namespace/path/file.csv\",\n",
    "            storage_options={\"signer\": instance_principal_signer})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "peripheral-muslim",
   "metadata": {},
   "source": [
    "### Connecting to a Different Region"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "owned-pavilion",
   "metadata": {},
   "source": [
    "Each filesystem instance has a home region and won't operate outside of that region. The home region defaults to the region of the IAM policy. With a configuration policy, it is `region`. With a resource principal, the region is derived from the `OCI_REGION_METADATA` environment variable. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "greenhouse-richardson",
   "metadata": {},
   "source": [
    "The `OCIFileSystem` delegates this region set up to the [Object Storage Client __init__ method](https://oracle-cloud-infrastructure-python-sdk.readthedocs.io/en/latest/api/object_storage/client/oci.object_storage.ObjectStorageClient.html#oci.object_storage.ObjectStorageClient.__init__) in the OCI Python SDK. The `region` argument accepts any [valid region identifier](https://docs.oracle.com/en-us/iaas/Content/General/Concepts/regions.htm) and constructs the corresponding service endpoint for the Object Storage Client. The following cell is an example of connecting to the sydney region."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "designing-difference",
   "metadata": {},
   "outputs": [],
   "source": [
    "fs_sydney = OCIFileSystem(config=\"~/.oci/config\", region=\"ap-sydney-1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "monetary-standard",
   "metadata": {},
   "source": [
    "Using Pandas or Dask:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "anticipated-solomon",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(\"oci://bucket@namespace/path/file.csv\", \n",
    "          storage_options = {\"config\": \"~/.oci/config\", region=\"ap-sydney-1\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "adolescent-aggregate",
   "metadata": {},
   "outputs": [],
   "source": [
    "ddf.to_csv(\"oci://bucket@namespace/path/file.csv\", \n",
    "           storage_options = {\"config\": \"~/.oci/config\", region=\"ap-sydney-1\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "robust-douglas",
   "metadata": {},
   "source": [
    "Note: You must ensure that you have valid cross-region permissions before attempting to instantiate a file system in a non-home region, see the list of [valid OCI Region Identifiers](https://docs.oracle.com/en-us/iaas/Content/General/Concepts/regions.htm)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ds",
   "language": "python",
   "name": "ds"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}