AWS/AWS_Get_files_from_S3_bucket.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "af542224",
   "metadata": {
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Get_files_from_S3_bucket.ipynb"
    ]
   },
   "source": [
    "<img width=\"8%\" alt=\"AWS.png\" src=\"https://raw.githubusercontent.com/jupyter-naas/awesome-notebooks/master/.github/assets/logos/AWS.png\" style=\"border-radius: 15%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af8e09e6",
   "metadata": {
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Get_files_from_S3_bucket.ipynb"
    ]
   },
   "source": [
    "# AWS - List objects from S3 bucket\n",
    "<a href=\"https://bit.ly/3JyWIk6\">Give Feedback</a> | <a href=\"https://github.com/jupyter-naas/awesome-notebooks/issues/new?assignees=&labels=bug&template=bug_report.md&title=AWS+-+List+objects+from+S3+bucket:+Error+short+description\">Bug report</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0f40b22-d389-4955-9698-49f31dbc458d",
   "metadata": {
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Get_files_from_S3_bucket.ipynb"
    ]
   },
   "source": [
    "**Tags:** #aws #cloud #storage #S3bucket #operations #snippet #list #objects"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b14b5e6f-803c-43a8-a8b5-83e0afcb7eb3",
   "metadata": {
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Get_files_from_S3_bucket.ipynb"
    ]
   },
   "source": [
    "**Author:** [Maxime Jublou](https://www.linkedin.com/in/maximejublou/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f90e8f76-ea59-4538-95bf-1aa64dee2140",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "**Last update:** 2023-11-20 (Created: 2021-09-20)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "naas-description",
   "metadata": {
    "papermill": {},
    "tags": [
     "description"
    ]
   },
   "source": [
    "**Description:** This notebook retrieves objects from an Amazon Web Services (AWS) S3 bucket, allowing users to easily access their data stored in the cloud."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcb6f62c-c8ad-440f-87d3-06c22035240f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-10-01T09:26:39.742671Z",
     "iopub.status.busy": "2021-10-01T09:26:39.742397Z",
     "iopub.status.idle": "2021-10-01T09:26:39.748138Z",
     "shell.execute_reply": "2021-10-01T09:26:39.747484Z",
     "shell.execute_reply.started": "2021-10-01T09:26:39.742602Z"
    },
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Get_files_from_S3_bucket.ipynb"
    ]
   },
   "source": [
    "## Input"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "464ad9fb-2dd8-4267-9e59-3bcf64b23b97",
   "metadata": {
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Get_files_from_S3_bucket.ipynb"
    ]
   },
   "source": [
    "### Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b0c68564-922e-48fc-ab34-1d711ff0e480",
   "metadata": {
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Get_files_from_S3_bucket.ipynb"
    ]
   },
   "outputs": [],
   "source": [
    "import naas\n",
    "try:\n",
    "    import boto3\n",
    "except:\n",
    "    !pip install boto3 getpass4\n",
    "    import boto3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c5d69da-1e06-4410-97d6-bbe5fc01eb3f",
   "metadata": {
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Get_files_from_S3_bucket.ipynb"
    ]
   },
   "source": [
    "### Setup variables\n",
    "**Mandatory**\n",
    "- `aws_access_key_id`: This variable is used to store the AWS access key ID.\n",
    "- `aws_secret_access_key`: This variable is used to store the AWS secret access key.\n",
    "- `bucket_name`: The name of the S3 bucket from which you want to list the files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "276757c8-f8b2-41c2-8ced-1e0bebefb263",
   "metadata": {
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Get_files_from_S3_bucket.ipynb"
    ]
   },
   "outputs": [],
   "source": [
    "# Mandatory\n",
    "aws_access_key_id = naas.secret.get(\"AWS_ACCESS_KEY_ID\") or \"YOUR_AWS_ACCESS_KEY_ID\"\n",
    "aws_secret_access_key = naas.secret.get(\"AWS_SECRET_ACCESS_KEY\") or \"YOUR_AWS_SECRET_ACCESS_KEY\"\n",
    "bucket_name = \"naas-example\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55fc943a-f791-4a69-919d-9e4e8cd597a4",
   "metadata": {
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Get_files_from_S3_bucket.ipynb"
    ]
   },
   "source": [
    "## Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf014bf3-7ae4-4758-93ee-37c3041db1cf",
   "metadata": {
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Daily_biling_notification_to_slack.ipynb"
    ]
   },
   "source": [
    "### Connect to AWS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a6b9c60-4462-406f-9a03-5354c37039f6",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "s3_client = boto3.client(\n",
    "    \"s3\",\n",
    "    aws_access_key_id=aws_access_key_id,\n",
    "    aws_secret_access_key=aws_secret_access_key\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef77d6ed-4118-4f11-be94-656e98d368bb",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### List objects in the S3 bucket"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15fa4ea7-2f54-4c55-8ae9-8aea0df5a734",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "response = s3_client.list_objects_v2(Bucket=bucket_name)\n",
    "print(\"Results:\", len(response['Contents']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e604b8aa-2234-407c-85b2-3517eb88d99d",
   "metadata": {
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Get_files_from_S3_bucket.ipynb"
    ]
   },
   "source": [
    "## Output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18c8d442-dbff-4900-8048-3ab8a484367e",
   "metadata": {
    "papermill": {},
    "tags": [
     "awesome-notebooks/AWS/AWS_Get_files_from_S3_bucket.ipynb"
    ]
   },
   "source": [
    "### Display pre-signed URL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac8286cf-a7d8-4ba1-be47-7a8f7007f4f9",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Process the response\n",
    "if 'Contents' in response:\n",
    "    for file in response['Contents']:\n",
    "        file_name = file['Key']\n",
    "        print(file_name)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "naas": {
   "notebook_id": "e5cc5c59cca0de48fbaf2a7d65708d416a50968635cadc60e18a669fd52c075a",
   "notebook_path": "AWS/AWS_Get_files_from_S3_bucket.ipynb"
  },
  "papermill": {
   "default_parameters": {},
   "environment_variables": {},
   "parameters": {},
   "version": "2.3.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}