source/notebooks/clms_portal.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ee766cee",
   "metadata": {},
   "source": [
    "\n",
    "<div>\n",
    "<img src=\"https://clms-prod.eea.europa.eu/static/media/EU-Cop-logo.4120d6e3.svg\" width=\"200\"/>\n",
    "</div>\n",
    "<div>\n",
    "<img src=\"https://clms-prod.eea.europa.eu/static/media/ccl-icon-land-text.bfaba89b.svg\" width=\"200\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c25c35cd",
   "metadata": {},
   "source": [
    "***\n",
    "The goal of this notebook is to help you kickstart your project and to provide an example on how to download a dataset. \n",
    "You can see all the API documentation here: <br> https://eea.github.io/clms-api-docs/ <br>\n",
    "You can view information regarding the Machine2Machine download here: <br>https://land.copernicus.eu/en/how-to-guides/download-spatial-data/m2m-download\n",
    "***\n",
    "\n",
    "## Sign up and token declaration\n",
    "\n",
    "* Go to https://land.copernicus.eu/en  \n",
    "* Register using EU login\n",
    "* After signing in to the portal navigate to 'My profile' under your username\n",
    "* Create a new token, press Copy to clipboard and paste it in a .json file called 'my_saved_key.json' in the same folder as this notebook\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "fe906156",
   "metadata": {},
   "source": [
    "# Prepare the environment\n",
    "\n",
    "To run this notebook you need to install the following dependencies.\n",
    "\n",
    "```bash\n",
    "pip install pyjwt requests cryptography\n",
    "\n",
    "```\n",
    "\n",
    "After installing them, you are ready to proceed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ffe3ebdd",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import jwt\n",
    "import time\n",
    "import requests\n",
    "\n",
    "base_url = 'https://land.copernicus.eu'"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f9dcd2e4",
   "metadata": {},
   "source": [
    "## Do the authentication steps\n",
    "\n",
    "First load the service key downloaded from the website and create a grant request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f2c116b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load saved key from filesystem\n",
    "service_key = json.load(open('my_saved_key.json', 'rb'))\n",
    "\n",
    "private_key = service_key['private_key'].encode('utf-8')\n",
    "\n",
    "claim_set = {\n",
    "    \"iss\": service_key['client_id'],\n",
    "    \"sub\": service_key['user_id'],\n",
    "    \"aud\": service_key['token_uri'],\n",
    "    \"iat\": int(time.time()),\n",
    "    \"exp\": int(time.time() + (60 * 60)),\n",
    "}\n",
    "grant = jwt.encode(claim_set, private_key, algorithm='RS256')"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "920e1a41",
   "metadata": {},
   "source": [
    "With the grant, you can request the access token to use the API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "7a796920",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "07oMdfuv2ULcbmcHkHeY0x-bNlHNibu-xK1yEeErCoRdwwMCKMS5IGE0w_bXaPA1ZKPKzxxqPQpkSAPSBKLX-Q==\n"
     ]
    }
   ],
   "source": [
    "result = requests.post(\n",
    "        service_key[\"token_uri\"],\n",
    "        headers={\n",
    "            \"Accept\": \"application/json\",\n",
    "            \"Content-Type\": \"application/x-www-form-urlencoded\",\n",
    "        },\n",
    "        data={\n",
    "            \"grant_type\": \"urn:ietf:params:oauth:grant-type:jwt-bearer\",\n",
    "            \"assertion\": grant,\n",
    "        },\n",
    ")\n",
    "\n",
    "access_token_info_json = result.json()\n",
    "access_token = access_token_info_json.get('access_token')\n",
    "print(access_token)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "901c689e",
   "metadata": {},
   "source": [
    "### Find the items that can be downloaded\n",
    "\n",
    "This step makes a search query and prints the title, UID and link of the first 25 items found.\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d7247ac7",
   "metadata": {},
   "source": [
    "## Search prepackaged files that can be downloaded\n",
    "\n",
    "You can use the `@search` endpoint to search for datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35cab1cd",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "url_downloadable_prepackaged = f\"{base_url}/api/@search?portal_type=DataSet&metadata_fields=UID&metadata_fields=downloadable_files\"\n",
    "headers = {'Accept': 'application/json', 'Authorization': f'Bearer {access_token}'}\n",
    "response_downloadable_prepackaged = requests.get(url_downloadable_prepackaged, headers=headers)\n",
    "\n",
    "json_downloadable_prepackaged = response_downloadable_prepackaged.json()\n",
    "\n",
    "for item in json_downloadable_prepackaged.get('items', []):\n",
    "    print('Product information:')\n",
    "    print('\\n')\n",
    "    print(3*' '+'Product title: \"'+ item['title'].replace(':','')+'\"')\n",
    "    print(3*' '+'UID: \"'+item['UID'].replace(' ','')+'\"')\n",
    "    print(3*' '+'Product link: '+ item['@id']+'')\n",
    "    print('\\n')\n",
    "    \n",
    "    for downloadable_file in item.get('downloadable_files', {}).get('items', []):\n",
    "        print('Download option for \"'+item['title'].replace(':','')+'\":')\n",
    "        display(downloadable_file)\n",
    "        print(2*'\\n')\n",
    "\n",
    "    print('\\n')\n",
    "    print(100*'_') "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ac18c2cb",
   "metadata": {},
   "source": [
    "### Download example dataset\n",
    "\n",
    "You are welcome to specify the configuration of your data request by using the `data` variable. \n",
    "If the request is successful, then a task ID will be created under your account and when the data extraction is ready to be downloaded you will receive an email with the download link. \n",
    "\n",
    "\n",
    "Further documentation can be found here: https://eea.github.io/clms-api-docs/download.html#request-the-download"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "be4f3957",
   "metadata": {},
   "outputs": [],
   "source": [
    "url_download_item = f\"{base_url}/api/@datarequest_post\"\n",
    "headers = {'Accept': 'application/json', 'Authorization': f'Bearer {access_token}'}\n",
    "data = {'Datasets': [{'DatasetID': 'a8d945f0edd143a0a5240c28bafa23da', 'DatasetDownloadInformationID': 'f328dbcc-9069-4240-b8bb-6d5df918671a', 'OutputFormat': 'Geotiff', 'OutputGCS': 'EPSG:4326', 'NUTS': 'ITC11'}, {'DatasetID': 'a8d945f0edd143a0a5240c28bafa23da', 'DatasetDownloadInformationID': 'f328dbcc-9069-4240-b8bb-6d5df918671a', 'OutputFormat': 'Geotiff', 'OutputGCS': 'EPSG:4326', 'NUTS': 'ITC1'}]}\n",
    "response_download_item = requests.post(url_download_item, headers=headers, json=data)\n",
    "print('Response:',response_download_item)\n",
    "print('Message with TaskID:\\n'+ response_download_item.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3035bdd3",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "052313b6",
   "metadata": {},
   "source": [
    "### Download example prepackaged file\n",
    "\n",
    "You are welcome to specify the configuration of your data request by using the json variable. \n",
    "If the request is successful, then a task ID will be created under your account and when the data extraction is ready to be downloaded you will receive an email with the download link. <br> <br>\n",
    "Further documentation can be found here: https://eea.github.io/clms-api-docs/download.html#download-prepackaged-files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e32431d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "url_download_prepackaged = f\"{base_url}/api/@datarequest_post\"\n",
    "headers = {'Accept': 'application/json', 'Authorization': f'Bearer {access_token}'}\n",
    "data = {'Datasets': [\n",
    "    {\n",
    "     'DatasetID': 'd30959acacf84e418c658ecaf6673ebf', \n",
    "     'FileID': '7b2c4ca3-749c-4c3d-b4cd-a848e7465175'\n",
    "    }, \n",
    "    {\n",
    "     'DatasetID': 'd30959acacf84e418c658ecaf6673ebf', \n",
    "     'FileID': '345b7d52-fa71-43da-a182-dca7a74a16e2'\n",
    "    }\n",
    "]}\n",
    "print(url_download_prepackaged)\n",
    "response_download_prepackaged = requests.post(url_download_prepackaged, headers=headers, json=data)\n",
    "print('Response:',response_download_prepackaged)\n",
    "print('Message with TaskID:\\n'+response_download_prepackaged.text)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "clms-api-docs",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.1"
  },
  "vscode": {
   "interpreter": {
    "hash": "dbf6530399c2ea83a9ae1fe9e4aed2b52e90d7bb4ce1ba8b44ed3e8af87da4c6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}