GitHub/GitHub_Download_repository_from_URL.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a61230dc-6327-4a06-80dd-74377a33abdf",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "<img width=\"8%\" alt=\"GitHub.png\" src=\"https://raw.githubusercontent.com/jupyter-naas/awesome-notebooks/master/.github/assets/logos/GitHub.png\" style=\"border-radius: 15%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96ab0035-d52a-4a9a-a69b-81305045ae98",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "# GitHub - Download repository from URL\n",
    "<a href=\"https://bit.ly/3JyWIk6\">Give Feedback</a> | <a href=\"https://github.com/jupyter-naas/awesome-notebooks/issues/new?assignees=&labels=bug&template=bug_report.md&title=GitHub+-+Download+repository+from+URL:+Error+short+description\">Bug report</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea164626-423c-4f0c-a036-0416c357c177",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "**Tags:** #github #download #repository #url #api #zip"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ee9f194-1670-4761-93e8-e41ce3753f76",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "**Author:** [Florent Ravenel](http://linkedin.com/in/florent-ravenel)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7df8099f-1e92-405f-9598-f2f9346e4c08",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "**Last update:** 2023-04-24 (Created: 2023-04-10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55aebec7-d8db-4ad1-8fa4-136e546ce940",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "**Description:** This notebook explains how to download a repository from a URL."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed4c1d04-cdb6-48c2-a4d3-90ffdcea5676",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "**References:**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8e2d836-d8bc-4979-a12a-249d25ae402f",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "## Input"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94d46544-00c4-4b60-a06a-d3b274e30d4c",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "881385e5-ead0-4538-84e7-c2fefe1fc0b1",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "import urllib\n",
    "import os\n",
    "import zipfile"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1887dd61-cdeb-4e63-a2c9-4fbefea935a1",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Setup Variables\n",
    "- `url`: URL of the repository to be downloaded\n",
    "- `branch_master`: Name of the master branch\n",
    "- `output_dir`: Path of the directory as output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f5fcb84e-6abb-4242-a8b0-eb0101ca743e",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Inputs\n",
    "url = \"https://github.com/jupyter-naas/data-product-framework\"\n",
    "branch_master = \"master\"\n",
    "\n",
    "# Outputs\n",
    "output_dir = \".\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "128acdfb-fb2f-4004-aa3b-b75f669ab262",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "## Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47a52092-ac98-4eb8-bdc9-4ffdd2c2dc84",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Download repository"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0449c192-4518-463b-a4ea-3499aafd247a",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "repo_name = url.split(\"/\")[-1].split(\"/\")[0]\n",
    "zip_url = f\"{url}/archive/refs/heads/{branch_master}.zip\"\n",
    "output_path = os.path.join(output_dir, f\"{repo_name}.zip\")\n",
    "urllib.request.urlretrieve(zip_url, output_path)\n",
    "print(\"✅ Repo archive downloaded:\", output_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23c9073b-1676-45aa-a073-d6dd6149cbf5",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Unzip and rename reposistory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "573d1d9b-b3e4-4fe8-baa8-bd8600d2c437",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "with zipfile.ZipFile(output_path, 'r') as zip_ref:\n",
    "    # extract files on root dir\n",
    "    zip_ref.extractall(output_dir)\n",
    "print(\"✅ Folder unzip\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2dc1ca2-fd98-4a22-80d7-3013b4450f97",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-10T13:51:38.287088Z",
     "iopub.status.busy": "2023-04-10T13:51:38.286833Z",
     "iopub.status.idle": "2023-04-10T13:51:38.290774Z",
     "shell.execute_reply": "2023-04-10T13:51:38.289705Z",
     "shell.execute_reply.started": "2023-04-10T13:51:38.287061Z"
    },
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Remove ZIP"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "336f3d41-86a6-44b3-bad0-23e218fa7c15",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "os.remove(output_path)\n",
    "print(\"✅ ZIP removed from root folder\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87a0e3b6-290d-48a2-9b03-bb1e61f2bdf5",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "## Output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ebdabe6d-2c7e-4b79-bbea-69fead4e61df",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "source": [
    "### Display result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "163c414b-de56-488f-91a7-d6184266548a",
   "metadata": {
    "papermill": {},
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(\"✅ Repo downloaded available in dir:\", output_dir)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "naas": {
   "notebook_id": "1002d8195ec68aad7c2c4d45777ae51f859a5f5aaac8f2e7596fbf46203b65ff",
   "notebook_path": "GitHub/GitHub_Download_repository_from_URL.ipynb"
  },
  "papermill": {
   "default_parameters": {},
   "environment_variables": {},
   "parameters": {},
   "version": "2.4.0"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}