notebooks/cmip6-config.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Configure Data Node for CMIP6 publishing and access\n",
    "\n",
    "Publishing\n",
    "----------\n",
    "Edit `/esg/config/esgcet/esg.ini` -- global publishing settings.\n",
    "\n",
    "This file by default must be modified by the node adminstrator, given default permission.\n",
    "\n",
    "Changes under `DEFAULT` group:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "[DEFAULT]\n",
    "# Set up Globus endpoint if publishing to Globus - add Globus entry and replace UUID\n",
    "thredds_file_services =\n",
    "        HTTPServer | /thredds/fileServer/       | HTTPServer    | fileservice\n",
    "        GridFTP    | gsiftp://my.esgf.node:2811/ | GRIDFTP       | fileservice\n",
    "        OpenDAP    | /thredds/dodsC/            | OpenDAPServer | fileservice\n",
    "        Globus     | globus:UUID/ | Globus | fileservice\n",
    "\n",
    "#  Add additional THREDDS root, include path up to but not including CMIP6 CMIP6\n",
    "thredds_dataset_roots =\n",
    "        esg_dataroot | /esg/data\n",
    "        my_cmip6_dataroot | /path/to/cmip6/data\n",
    "\n",
    "# certs directory is commonly missing in old deployments - check and add this\n",
    "hessian_service_certs_location = /etc/grid-security/certificates\n",
    "        \n",
    "#   To see info during publishing check log level\n",
    "log_level = INFO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add to `esg.ini` section`config.cmip6` with the following:   \n",
    "\n",
    "**NOTE:** For `pid_credentials` Nodes in _North America_ should list `pcmdi10.llnl.gov` first\n",
    "\n",
    "_Europe/Asia_: keep `handle-esgf-trusted.dkrz.de` first in server listing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[config:cmip6]\n",
    "pid_credentials =\n",
    "    handle-esgf-trusted.dkrz.de | 5671  | esgf-pid | esgf-publisher | <secret>  | true\n",
    "    pcmdi10.llnl.gov            | 5671  | esgf-pid | esgf-publisher | <secret>            | true\n",
    "pid_exchange_name = esgffed-exchange\n",
    "pid_prefix = 21.14100\n",
    "cmor_table_path = /usr/local/cmip6-cmor-tables/Tables\n",
    "data_specs_version = file  # “file” means the publisher will follow the DR tag specified in each file with “data_specs_version” attribut\n",
    "# FOR test publishing use this one CDF2CIM_CLIENT_WS_HOST = https://test-cdf2cim-api.es-doc.org\n",
    "CDF2CIM_CLIENT_WS_HOST=https://cdf2cim.es-doc.org  # for production \n",
    "CDF2CIM_CLIENT_GITHUB_USER = <your_GitHub_username>\n",
    "CDF2CIM_CLIENT_GITHUB_ACCESS_TOKEN = <your_GitHub_personal_access_token_for_cdf2cim>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set up regular user account for cmor table access.  Recommended not to publish from the root account. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "usermod <linux-username> -G esgfpub -a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Correct thredds catalog permissions "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chgrp -R esgfpub /esg/content/thredds/esgcet\n",
    "chmod g+rwx /esg/content/thredds/esgcet\n",
    "chmod g+rwx /esg/content/thredds/esgcet/catalog.xml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Access\n",
    "------\n",
    "Edit `/esg/config/esgf_policies_local.xml`.  Insert a single rule to make data available  freely to all."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "<policies xmlns=\"http://www.esgf.org/security\">\n",
    "<!--Makes data available to all-->\n",
    "    <policy resource=\".*CMIP6.*\" attribute_type=\"ANY\" attribute_value=\"\" action=\"Read\"/>\n",
    "</policies>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ensure that you have an up-to-date IdP whitelist file installed (for ESGF versions 2.x; future installations using Ansible should not require this step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cd /tmp # or other working directory\n",
    "git clone https://github.com/ESGF/config\n",
    "cp config/esgf-prod/esgf_idp_static.xml /esg/config"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}