demos/grib2io-v2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0e4f03c8-2c84-4e04-b94f-66004d841981",
   "metadata": {},
   "source": [
    "# **grib2io v2.0 Demo**\n",
    "\n",
    "This notebook demos some of the new code design and features of grib2io v2.0.  The sample file used is a GRIB2 files containing GFS model forecasts from the 00Z cycle on a 0.25 deg. global lat/lon grid with a lead time of 6 hours. A similar sample file can be downloaded from [here](http://nomads.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4b0108b0-a65c-4bef-9867-266faf6f4009",
   "metadata": {},
   "outputs": [],
   "source": [
    "import grib2io\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c423b605-89ac-410d-9220-9a1ce3a9036b",
   "metadata": {
    "tags": []
   },
   "source": [
    "## **Opening GRIB2 files**\n",
    "\n",
    "Note that nothing has changed between v1 and v2 with respect to to opening GRIB2 files and the same attributes exist."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b6fbebdb-93af-4e0f-8c7d-33a191cce1a5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mode = rb\n",
      "name = /home/ericengle/Downloads/gfs.t00z.pgrb2.0p25.f006\n",
      "messages = 743\n",
      "current_message = 0\n",
      "size = 544359352\n",
      "closed = False\n",
      "variables = ('4LFTX', 'ABSV', 'ACPCP', 'ALBDO', 'APCP', 'APTMP', 'CAPE', 'CFRZR', 'CICEP', 'CIN', 'CLMR', 'CNWAT', 'CPOFP', 'CPRAT', 'CRAIN', 'CSNOW', 'CWAT', 'CWORK', 'DLWRF', 'DPT', 'DSWRF', 'DZDT', 'FLDCP', 'FRICV', 'GFLUX', 'GRLE', 'GUST', 'HCDC', 'HGT', 'HINDEX', 'HLCY', 'HPBL', 'ICAHT', 'ICEC', 'ICEG', 'ICETK', 'ICETMP', 'ICMR', 'LAND', 'LCDC', 'LFT X', 'LHTFL', 'MCDC', 'MSLET', 'O3MR', 'PEVPR', 'PLPL', 'POT', 'PRATE', 'PRES', 'PRMSL', 'PWAT', 'REFC', 'REFD', 'RH', 'RWMR', 'SFCR', 'SHTFL', 'SNMR', 'SNOD', 'SOILL', 'SOILW', 'SOTYP', 'SPFH', 'SUNSD', 'TCDC', 'TMAX', 'TMIN', 'TMP', 'TOZNE', 'TSOIL', 'U-GWD', 'UFLX', 'UGRD', 'ULWRF', 'USTM', 'USWRF', 'V-GWD', 'VEG', 'VFLX', 'VGRD', 'VIS', 'VRATE', 'VSTM', 'VVEL', 'VWSH', 'WATR', 'WEASD', 'WILT')\n",
      "levels = ('0-0.1 m underground', '0.01 mb', '0.02 mb', '0.04 mb', '0.07 mb', '0.1 mb', '0.1-0.4 m underground', '0.2 mb', '0.33-1 sigma layer', '0.4 mb', '0.4-1 m underground', '0.44-0.72 sigma layer', '0.44-1 sigma layer', '0.7 mb', '0.72-0.94 sigma layer', '0.995 sigma level', '0C isotherm', '1 hybrid level', '1 mb', '1-2 m underground', '10 m above ground', '10 m above mean sea level', '10 mb', '100 m above ground', '100 mb', '1000 m above ground', '1000 mb', '15 mb', '150 mb', '180-0 mb above ground', '1829 m above mean sea level', '2 hybrid level', '2 m above ground', '2 mb', '20 m above ground', '20 mb', '200 mb', '250 mb', '255-0 mb above ground', '2743 m above mean sea level', '3 mb', '30 m above ground', '30 mb', '30-0 mb above ground', '300 mb', '3000-0 m above ground', '350 mb', '3658 m above mean sea level', '40 m above ground', '40 mb', '400 mb', '4000 m above ground', '450 mb', '5 mb', '50 m above ground', '50 mb', '500 mb', '550 mb', '600 mb', '6000-0 m above ground', '650 mb', '7 mb', '70 mb', '700 mb', '750 mb', '80 m above ground', '800 mb', '850 mb', '90-0 mb above ground', '900 mb', '925 mb', '950 mb', '975 mb', 'PV=-2e-06 (Km^2/kg/s) surface', 'PV=2e-06 (Km^2/kg/s) surface', 'boundary layer cloud layer', 'cloud ceiling', 'convective cloud bottom level', 'convective cloud layer', 'convective cloud top level', 'entire atmosphere', 'entire atmosphere (considered as a single layer)', 'high cloud bottom level', 'high cloud layer', 'high cloud top level', 'highest tropospheric freezing level', 'low cloud bottom level', 'low cloud layer', 'low cloud top level', 'max wind', 'mean sea level', 'middle cloud bottom level', 'middle cloud layer', 'middle cloud top level', 'planetary boundary layer', 'surface', 'top of atmosphere', 'tropopause')\n",
      "\n"
     ]
    }
   ],
   "source": [
    "g = grib2io.open(\"gfs.t00z.pgrb2.0p25.f006\")\n",
    "print(g)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38859f42-94c5-4d30-b241-f0666b3716e9",
   "metadata": {},
   "source": [
    "### Here are some key things to know about what grib2io is doing when opening a GRIB2 file.\n",
    "* The messages in the file are indexed automatically.  In other words, GRIB2 sections 0, 1, 2, 3, 4, and 5 are read, unpacked, and used to create Grib2Message objects.\n",
    "* Message, bitmap, and data offsets are recorded.\n",
    "* GRIB2 messages that contain submessages are automatically flattened.\n",
    "* At this point, bitmap and data sections are not read.  These sections make up the majority of packed message, so we defer reading these parts of the message until absolutely necessary."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9b653c9-e619-4f12-a13f-8e30e1192f75",
   "metadata": {
    "tags": []
   },
   "source": [
    "## **Investigation of the file object index**\n",
    "\n",
    "Underneath the hood, the indexing has changed significantly. There no longer is GRIB2-specific metadata from messages, but just information about location\n",
    "and size of messages.  We are now storing the GRIB2 message object in the index in key 'msg'.\n",
    "\n",
    "One might think that storing all of these objects in the index would negatively impact performance, but the new GRIB2 message object is extremely lightweight (more on that later) and acts like a templated container of metadata.  At this point, the data section is not a part of the GRIB2 message object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b393d29a-249a-48e5-9b03-f56a298d3a95",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dict_keys(['offset', 'bitmap_offset', 'data_offset', 'size', 'data_size', 'submessageOffset', 'submessageBeginSection', 'isSubmessage', 'messageNumber', 'msg'])\n"
     ]
    }
   ],
   "source": [
    "print(g._index.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34aae855-f7af-456d-97b1-92cbebce9cb2",
   "metadata": {},
   "source": [
    "## **Reading GRIB2 Messages from file**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "678b5675-ce0b-439e-b9ef-2ce1b4d0ffe9",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Use read(), seek(), tell()\n",
    "\n",
    "The `grib2io.open` class provides custom implementations of read(), seek(), and tell().  While these methods are traditionally used for traversing bytes, these custom implementations operate on units of GRIB2 messages.  Note that when these methods are used, you are not acutally performning any I/O to the GRIB2 file.  You are positioning inside the list contained by `grib2io.open._index['msg']`.  The following example puts the \"file object\" pointer to the 100th message and reads the next 15 messages (from message 100).  In this example, there is no file IO -- just communicating with the Grib2Message objects stored in the index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "73192896-d405-43a3-b012-4a1500076e8f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GRIB2 file poistion:  100\n",
      "Number of messages read:  15\n",
      "100:d=2022-10-28 00:00:00:UGRD:U-Component of Wind (m s-1):1 mb:6:00:00\n",
      "101:d=2022-10-28 00:00:00:VGRD:V-Component of Wind (m s-1):1 mb:6:00:00\n",
      "102:d=2022-10-28 00:00:00:ABSV:Absolute Vorticity (s-1):1 mb:6:00:00\n",
      "103:d=2022-10-28 00:00:00:O3MR:Ozone Mixing Ratio (kg kg-1):1 mb:6:00:00\n",
      "104:d=2022-10-28 00:00:00:HGT:Geopotential Height (gpm):2 mb:6:00:00\n",
      "105:d=2022-10-28 00:00:00:TMP:Temperature (K):2 mb:6:00:00\n",
      "106:d=2022-10-28 00:00:00:RH:Relative Humidity (%):2 mb:6:00:00\n",
      "107:d=2022-10-28 00:00:00:SPFH:Specific Humidity (kg kg-1):2 mb:6:00:00\n",
      "108:d=2022-10-28 00:00:00:VVEL:Vertical Velocity (Pressure) (Pa s-1):2 mb:6:00:00\n",
      "109:d=2022-10-28 00:00:00:DZDT:Vertical Velocity (Geometric) (m s-1):2 mb:6:00:00\n",
      "110:d=2022-10-28 00:00:00:UGRD:U-Component of Wind (m s-1):2 mb:6:00:00\n",
      "111:d=2022-10-28 00:00:00:VGRD:V-Component of Wind (m s-1):2 mb:6:00:00\n",
      "112:d=2022-10-28 00:00:00:ABSV:Absolute Vorticity (s-1):2 mb:6:00:00\n",
      "113:d=2022-10-28 00:00:00:O3MR:Ozone Mixing Ratio (kg kg-1):2 mb:6:00:00\n",
      "114:d=2022-10-28 00:00:00:HGT:Geopotential Height (gpm):3 mb:6:00:00\n"
     ]
    }
   ],
   "source": [
    "g.seek(100)\n",
    "print('GRIB2 file poistion: ',g.tell())\n",
    "msgs = g.read(15)\n",
    "print('Number of messages read: ',len(msgs))\n",
    "for msg in msgs:\n",
    "    print(msg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de09b9a6-8510-4c39-84b5-c9ae254552ca",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Use index, slice, or key notation on the file object\n",
    "\n",
    "**_IMPORTANT: In grib2io v2.0, you no longer have to specify a 2nd index when select 1 message !!_**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "2717065b-fde4-457f-9eb5-fa076dbee3f2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "50:d=2022-10-28 00:00:00:UGRD:U-Component of Wind (m s-1):0.07 mb:6:00:00\n"
     ]
    }
   ],
   "source": [
    "# Index\n",
    "msg = g[50]\n",
    "print(msg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1bab3105-7bba-4206-93fb-79350ef50a6e",
   "metadata": {},
   "source": [
    "**_IMPORTANT:  In grib2io v2.0, a 0th message now exists !!_**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8eb41c02-897d-49e3-abae-6d471d00924b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:d=2022-10-28 00:00:00:PRMSL:Pressure Reduced to MSL (Pa):mean sea level:6:00:00\n"
     ]
    }
   ],
   "source": [
    "# Index\n",
    "msg = g[0]\n",
    "print(msg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "01d8a9fe-da1e-4d55-91c9-95fe9d5433a9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "80:d=2022-10-28 00:00:00:UGRD:U-Component of Wind (m s-1):0.4 mb:6:00:00\n",
      "81:d=2022-10-28 00:00:00:VGRD:V-Component of Wind (m s-1):0.4 mb:6:00:00\n",
      "82:d=2022-10-28 00:00:00:ABSV:Absolute Vorticity (s-1):0.4 mb:6:00:00\n",
      "83:d=2022-10-28 00:00:00:O3MR:Ozone Mixing Ratio (kg kg-1):0.4 mb:6:00:00\n",
      "84:d=2022-10-28 00:00:00:HGT:Geopotential Height (gpm):0.7 mb:6:00:00\n",
      "85:d=2022-10-28 00:00:00:TMP:Temperature (K):0.7 mb:6:00:00\n",
      "86:d=2022-10-28 00:00:00:RH:Relative Humidity (%):0.7 mb:6:00:00\n",
      "87:d=2022-10-28 00:00:00:SPFH:Specific Humidity (kg kg-1):0.7 mb:6:00:00\n",
      "88:d=2022-10-28 00:00:00:VVEL:Vertical Velocity (Pressure) (Pa s-1):0.7 mb:6:00:00\n",
      "89:d=2022-10-28 00:00:00:DZDT:Vertical Velocity (Geometric) (m s-1):0.7 mb:6:00:00\n"
     ]
    }
   ],
   "source": [
    "# Slice\n",
    "msgs = g[80:90]\n",
    "for msg in msgs:\n",
    "    print(msg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a4f4329a-edb4-4d2a-bacb-42434fc17eb6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "623:d=2022-10-28 00:00:00:CAPE:Convective Available Potential Energy (J kg-1):surface:6:00:00\n",
      "709:d=2022-10-28 00:00:00:CAPE:Convective Available Potential Energy (J kg-1):180-0 mb above ground:6:00:00\n",
      "722:d=2022-10-28 00:00:00:CAPE:Convective Available Potential Energy (J kg-1):90-0 mb above ground:6:00:00\n",
      "724:d=2022-10-28 00:00:00:CAPE:Convective Available Potential Energy (J kg-1):255-0 mb above ground:6:00:00\n"
     ]
    }
   ],
   "source": [
    "# Key - where string is the GRIB2 shortName attribute\n",
    "msgs = g['CAPE']\n",
    "for msg in msgs:\n",
    "    print(msg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d84aafc1-8a35-42bd-a471-1a60128713b9",
   "metadata": {},
   "source": [
    "### Selection of GRIB2 messages based on attributes\n",
    "\n",
    "Use **_any_** Grib2Message object attribute name as a keyword for the file object select() method. **NOTE:** The select() method will always return a list."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ad9577e-f66f-4e0f-adce-d73b50267e21",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Example 1: Select temperature message (TMP) that use product definition template number of 8."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "bf207496-3d16-4a6e-b91b-ed9ba0acf0b8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "646:d=2022-10-28 00:00:00:TMP:Temperature (K):low cloud top level:6:00:00\n",
      "\t 6:00:00 0 - Average\n",
      "647:d=2022-10-28 00:00:00:TMP:Temperature (K):middle cloud top level:6:00:00\n",
      "\t 6:00:00 0 - Average\n",
      "648:d=2022-10-28 00:00:00:TMP:Temperature (K):high cloud top level:6:00:00\n",
      "\t 6:00:00 0 - Average\n"
     ]
    }
   ],
   "source": [
    "msgs = g.select(shortName='TMP',productDefinitionTemplateNumber=8)\n",
    "for msg in msgs:\n",
    "    print(msg)\n",
    "    print('\\t',msg.duration,msg.statisticalProcess)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9e51c1c-053e-4694-8eb1-9aa8e373c43a",
   "metadata": {},
   "source": [
    "#### Example 2: Select U-wind messages (UGRD) at geometric heights above ground."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a65c32ac-a5a3-438a-a9a9-ee47cfa15896",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "587:d=2022-10-28 00:00:00:UGRD:U-Component of Wind (m s-1):10 m above ground:6:00:00\n",
      "\t 103 - ['Specified Height Level Above Ground', 'm']\n",
      "674:d=2022-10-28 00:00:00:UGRD:U-Component of Wind (m s-1):20 m above ground:6:00:00\n",
      "\t 103 - ['Specified Height Level Above Ground', 'm']\n",
      "676:d=2022-10-28 00:00:00:UGRD:U-Component of Wind (m s-1):30 m above ground:6:00:00\n",
      "\t 103 - ['Specified Height Level Above Ground', 'm']\n",
      "678:d=2022-10-28 00:00:00:UGRD:U-Component of Wind (m s-1):40 m above ground:6:00:00\n",
      "\t 103 - ['Specified Height Level Above Ground', 'm']\n",
      "680:d=2022-10-28 00:00:00:UGRD:U-Component of Wind (m s-1):50 m above ground:6:00:00\n",
      "\t 103 - ['Specified Height Level Above Ground', 'm']\n",
      "685:d=2022-10-28 00:00:00:UGRD:U-Component of Wind (m s-1):80 m above ground:6:00:00\n",
      "\t 103 - ['Specified Height Level Above Ground', 'm']\n",
      "688:d=2022-10-28 00:00:00:UGRD:U-Component of Wind (m s-1):100 m above ground:6:00:00\n",
      "\t 103 - ['Specified Height Level Above Ground', 'm']\n"
     ]
    }
   ],
   "source": [
    "msgs = g.select(shortName='UGRD',typeOfFirstFixedSurface=103)\n",
    "for msg in msgs:\n",
    "    print(msg)\n",
    "    print('\\t',msg.typeOfFirstFixedSurface)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd928fe2-b256-4745-94d4-ea5f9c0ebdbc",
   "metadata": {},
   "source": [
    "## **Grib2Message Object Metadata Attributes**\n",
    "\n",
    "The Grib2Message object, attributes, and metadata storage have changed **significantly** from v1.0.0 or v2.0.0.  The Grib2Message class inherits from a Grib2Message base class that contains attributes for sections that are not templates.  GRIB2 Sections 3, 4, and 5 are defined by templates and therefore the attributes can be different between templates of the same section.  All template classes are defined as a `dataclasses.dataclass` and each attribute within each template class is a class variable and defined to be a `dataclasses.field` object.  Further, each attribute, defined as a field has a default value that is a unique descriptor class with custom implementations of the `__get__` and `__set__` protocols.\n",
    "\n",
    "Below is an example of all of the metadata associated with a GRIB2 message for a GFS 500mb HGT field.  Each metadata item you see is being created dynamically from the numeric GRIB2 section metadata.  For example, in section 4, there is the attribute `parameterCategory`.  This attribute has a custom descriptor class, `grib2io.templates.ParameterCategory`, whose `__get__` protocol points to the appropriate index location in the section 4 array.  If we wanted to change this attribute value, you can perform `msg.parameterCategory = 5`.  This would trigger the `__set__` protocol for the `grib2io.templates.ParameterCategory` descriptor class which would put that new value into the appropriate index location in the section 4 array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "be3b0d5a-1dee-4d55-a01e-49711aee8245",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Section 0: discipline = 0 - Meteorological Products\n",
       "Section 1: originatingCenter = 7 - US National Weather Service - NCEP (WMC)\n",
       "Section 1: originatingSubCenter = 0 - None\n",
       "Section 1: masterTableInfo = 2 - Version Implemented on 4 November 2003\n",
       "Section 1: localTableInfo = 1 - Number of local table version used.\n",
       "Section 1: significanceOfReferenceTime = 1 - Start of Forecast\n",
       "Section 1: year = 2022\n",
       "Section 1: month = 10\n",
       "Section 1: day = 28\n",
       "Section 1: hour = 0\n",
       "Section 1: minute = 0\n",
       "Section 1: second = 0\n",
       "Section 1: refDate = 2022-10-28 00:00:00\n",
       "Section 1: productionStatus = 0 - Operational Products\n",
       "Section 1: typeOfData = 1 - Forecast Products\n",
       "Section 3: interpretationOfListOfNumbers = 0 - There is no appended list\n",
       "Section 3: gridDefinitionTemplateNumber = 0 - Latitude/Longitude\n",
       "Section 3: shapeOfEarth = 6 - Earth assumed spherical with radius = 6,371,229.0 m\n",
       "Section 3: earthRadius = 6371229.0\n",
       "Section 3: earthMajorAxis = None\n",
       "Section 3: earthMinorAxis = None\n",
       "Section 3: resolutionAndComponentFlags = [0, 0, 1, 1, 0, 0, 0, 0]\n",
       "Section 3: ny = 721\n",
       "Section 3: nx = 1440\n",
       "Section 3: scanModeFlags = [0, 0, 0, 0, 0, 0, 0, 0]\n",
       "Section 3: latitudeFirstGridpoint = 90.0\n",
       "Section 3: longitudeFirstGridpoint = 0.0\n",
       "Section 3: latitudeLastGridpoint = -90.0\n",
       "Section 3: longitudeLastGridpoint = 359.75\n",
       "Section 3: gridlengthXDirection = 0.25\n",
       "Section 3: gridlengthYDirection = -0.25\n",
       "Section 4: productDefinitionTemplateNumber = 0 - Analysis or forecast at a horizontal level or in a horizontal layer at a point in time. (see Template 4.0)\n",
       "Section 4: fullName = Geopotential Height\n",
       "Section 4: units = gpm\n",
       "Section 4: shortName = HGT\n",
       "Section 4: leadTime = 6:00:00\n",
       "Section 4: unitOfFirstFixedSurface = Pa\n",
       "Section 4: valueOfFirstFixedSurface = 50000.0\n",
       "Section 4: unitOfSecondFixedSurface = None\n",
       "Section 4: valueOfSecondFixedSurface = 50000.0\n",
       "Section 4: validDate = 2022-10-28 06:00:00\n",
       "Section 4: duration = None\n",
       "Section 4: level = 500 mb\n",
       "Section 4: parameterCategory = 3\n",
       "Section 4: parameterNumber = 5\n",
       "Section 4: typeOfGeneratingProcess = 2 - Forecast\n",
       "Section 4: generatingProcess = 96 - Global Forecast System Model T1534 - Forecast hours 00-384 T574 - Forecast hours 00-192 T190 - Forecast hours 204-384\n",
       "Section 4: backgroundGeneratingProcessIdentifier = 0\n",
       "Section 4: hoursAfterDataCutoff = 0\n",
       "Section 4: minutesAfterDataCutoff = 0\n",
       "Section 4: unitOfTimeRange = 1 - Hour\n",
       "Section 4: forecastTime = 6\n",
       "Section 4: typeOfFirstFixedSurface = 100 - ['Isobaric Surface', 'Pa']\n",
       "Section 4: scaleFactorOfFirstFixedSurface = 0\n",
       "Section 4: scaledValueOfFirstFixedSurface = 50000\n",
       "Section 4: typeOfSecondFixedSurface = 255 - ['Missing', 'unknown']\n",
       "Section 4: scaleFactorOfSecondFixedSurface = 0\n",
       "Section 4: scaledValueOfSecondFixedSurface = 0\n",
       "Section 5: dataRepresentationTemplateNumber = 3 - Grid Point Data - Complex Packing and Spatial Differencing (see Template 5.3)\n",
       "Section 5: numberOfPackedValues = 1038240\n",
       "Section 5: typeOfValues = 0 - Floating Point\n",
       "Section 5: refValue = 482101.0625\n",
       "Section 5: binScaleFactor = 1\n",
       "Section 5: decScaleFactor = 2\n",
       "Section 5: nBitsPacking = 13\n",
       "Section 5: groupSplittingMethod = 1 - General Group Splitting\n",
       "Section 5: typeOfMissingValueManagement = 0 - No explicit missing values included within the data values\n",
       "Section 5: priMissingValue = None\n",
       "Section 5: secMissingValue = None\n",
       "Section 5: nGroups = 28103\n",
       "Section 5: refGroupWidth = 0\n",
       "Section 5: nBitsGroupWidth = 4\n",
       "Section 5: refGroupLength = 1\n",
       "Section 5: groupLengthIncrement = 1\n",
       "Section 5: lengthOfLastGroup = 41\n",
       "Section 5: nBitsScaledGroupLength = 7\n",
       "Section 5: spatialDifferenceOrder = 2 - Second-Order Spatial Differencing\n",
       "Section 5: nBytesSpatialDifference = 2\n",
       "Section 6: bitMapFlag = 255 - A bit map does not apply to this product."
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "msg = g.select(shortName='HGT',level='500 mb')[0]\n",
    "msg"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c6692e1-c114-451a-82d7-fe7df4e2d43a",
   "metadata": {},
   "source": [
    "### Grib2Message wgrib2-like `__str__`\n",
    "\n",
    "The Grib2Message object provides a wgrib2-like 1-liner string for `__str__` (as shown above).  The Grib2Message object `__repr__` will provide all metadata (see above)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8e58d8ff-6568-42ec-aabd-253df2f4fae9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "348:d=2022-10-28 00:00:00:HGT:Geopotential Height (gpm):500 mb:6:00:00\n"
     ]
    }
   ],
   "source": [
    "print(msg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f5fff1c-fdd4-41ad-8700-560945ac32f5",
   "metadata": {},
   "source": [
    "### Grib2Message Construction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "8a58e36a-7a78-44a2-ba79-d372d069115a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Section 0:  [1196575042          0          0          2     811530]\n",
      "Section 1:  [   7    0    2    1    1 2022   10   28    0    0    0    0    1]\n",
      "Section 2:  b''\n",
      "Section 3:  [        0   1038240         0         0         0         6         0\n",
      "         0         0         0         0         0      1440       721\n",
      "         0        -1  90000000         0        48 -90000000 359750000\n",
      "    250000    250000         0]\n",
      "Section 4:  [    0     0     3     5     2     0    96     0     0     1     6   100\n",
      "     0 50000   255     0     0]\n",
      "Section 5:  [   1038240          3 1223386786          1          2         13\n",
      "          0          1          0 1649987994 4294967295      28103\n",
      "          0          4          1          1         41          7\n",
      "          2          2]\n"
     ]
    }
   ],
   "source": [
    "print('Section 0: ',msg.section0)\n",
    "print('Section 1: ',msg.section1)\n",
    "print('Section 2: ',msg.section2)\n",
    "print('Section 3: ',msg.section3)\n",
    "print('Section 4: ',msg.section4)\n",
    "print('Section 5: ',msg.section5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02a74a99-5ae2-4866-b92d-145c5ab93e93",
   "metadata": {},
   "source": [
    "The Grib2Message class is dynamically created and is unique to GRIB2's grid definition, production definition, and data representation template numbers. These templates contain metadata attributes unique to the definition of the given template.\n",
    "\n",
    "Here we can use the class method resolution order (`__mro__`) to show the various classes that make up the Grib2Message class. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "d5f5fe66-95a6-4c13-9f6c-4d151951688e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'grib2io._grib2io.Grib2Message.__new__.<locals>.Msg'>\n",
      "<class 'grib2io._grib2io._Grib2Message'>\n",
      "<class 'grib2io.templates.GridDefinitionTemplate0'>\n",
      "<class 'grib2io.templates.ProductDefinitionTemplate0'>\n",
      "<class 'grib2io.templates.DataRepresentationTemplate3'>\n",
      "<class 'object'>\n"
     ]
    }
   ],
   "source": [
    "for c in msg.__class__.__mro__:\n",
    "    print(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea5785df-1299-410b-be44-2f7bfda16271",
   "metadata": {},
   "source": [
    "The template classes listed above contain the attributes as class variables that are unique to its template.\n",
    "\n",
    "You will notice that the `ProductDefinitionTemplate0` is blank.  This is because it contains the common attributes used by the rest of the PTDNs, so they are defined in class `_Grib2Message`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "cf6e0fa9-d48a-45d7-ba39-e669128b51e9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['latitudeFirstGridpoint', 'longitudeFirstGridpoint', 'latitudeLastGridpoint', 'longitudeLastGridpoint', 'gridlengthXDirection', 'gridlengthYDirection']\n",
      "['parameterCategory', 'parameterNumber', 'typeOfGeneratingProcess', 'generatingProcess', 'backgroundGeneratingProcessIdentifier', 'hoursAfterDataCutoff', 'minutesAfterDataCutoff', 'unitOfTimeRange', 'forecastTime', 'typeOfFirstFixedSurface', 'scaleFactorOfFirstFixedSurface', 'scaledValueOfFirstFixedSurface', 'typeOfSecondFixedSurface', 'scaleFactorOfSecondFixedSurface', 'scaledValueOfSecondFixedSurface']\n",
      "['refValue', 'binScaleFactor', 'decScaleFactor', 'nBitsPacking', 'groupSplittingMethod', 'typeOfMissingValue', 'priMissingValue', 'secMissingValue', 'nGroups', 'refGroupWidth', 'nBitsGroupWidth', 'refGroupLength', 'groupLengthIncrement', 'lengthOfLastGroup', 'nBitsScaledGroupLength', 'spatialDifferenceOrder', 'nBytesSpatialDifference']\n"
     ]
    }
   ],
   "source": [
    "print(grib2io.templates.GridDefinitionTemplate0._attrs)\n",
    "print(grib2io.templates.ProductDefinitionTemplate0._attrs)\n",
    "print(grib2io.templates.DataRepresentationTemplate3._attrs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29b86ddd-d219-4237-b321-c3586f67240f",
   "metadata": {},
   "source": [
    "Now lets take a look at how the metadata attributes are linked to their origin locations in a given section.  Here we will look at the Grib2Message `refDate` attribute, whose components are stored in Section 1.\n",
    "\n",
    "To reiterate, `msg.refDate` does not hold a value that would be a `datetime.datetime` object.  The `refDate` attribute is defined as a descriptor class, specifically `grib2io.templates.RefDate()`, whose `__get__` protocol returns a `datetime.datetime` object generated from the date components in section 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f30fe213-8b49-4d6c-90d2-daecc0a047d6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[   7    0    2    1    1 2022   10   28    0    0    0    0    1]\n",
      "2022-10-28 00:00:00\n",
      "2022 10 28 0 0 0\n"
     ]
    }
   ],
   "source": [
    "print(msg.section1)\n",
    "print(msg.refDate)\n",
    "print(msg.year,msg.month,msg.day,msg.hour,msg.minute,msg.second)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7a4a793-0db2-41cd-92ac-e099e194de94",
   "metadata": {},
   "source": [
    "To demonstrate that these attributes are dynamically created, lets change the year and minute. We do this by setting the attributes to new values.  You will see that values have changed in section1 and other attributes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "192776fa-7a6f-4f62-8aa6-0532b52695b1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[   7    0    2    1    1 2023   10   28    0   55    0    0    1]\n",
      "2023-10-28 00:55:00\n"
     ]
    }
   ],
   "source": [
    "msg.year = 2023\n",
    "msg.minute = 55\n",
    "print(msg.section1)\n",
    "print(msg.refDate)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "721b0fac-0aba-4ae8-b101-78688b7e9b92",
   "metadata": {},
   "source": [
    "**IMPORTANT:** In grib2io v2.0, all date and time attributes are `datetime.datetime` or `datetime.timedelta` objects.  Let use precipitation as an example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "b9b885c4-78e1-4427-b739-f20412e66488",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "595:d=2022-10-28 00:00:00:APCP:Total Precipitation (kg m-2):surface:6:00:00\n",
      "2022-10-28 00:00:00 <class 'datetime.datetime'>\n",
      "2022-10-28 06:00:00 <class 'datetime.datetime'>\n",
      "6:00:00 <class 'datetime.timedelta'>\n",
      "6:00:00 <class 'datetime.timedelta'>\n"
     ]
    }
   ],
   "source": [
    "msg = g['APCP'][0]\n",
    "print(msg)\n",
    "print(msg.refDate,type(msg.refDate))\n",
    "print(msg.validDate,type(msg.validDate))\n",
    "print(msg.leadTime,type(msg.leadTime))\n",
    "print(msg.duration,type(msg.duration))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61622ab4-a4b1-45e5-9759-b6909f3b6ab5",
   "metadata": {},
   "source": [
    "This allows for datetime arithmetic\n",
    "\n",
    "The Grib2Message object defines the property `pdy` that provides the reference date as a string in `'YYYYMMDD'` format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "9a9b252d-80a3-4ded-b677-042dd0f13eca",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2022-10-28 06:00:00\n",
      "2022-10-28 06:00:00\n",
      "True\n",
      "20221028\n"
     ]
    }
   ],
   "source": [
    "print(msg.refDate + msg.leadTime)\n",
    "print(msg.validDate)\n",
    "print(msg.refDate+msg.leadTime==msg.validDate)\n",
    "print(msg.pdy)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f370a0d6-1b03-4045-9144-9f927fb9dc18",
   "metadata": {},
   "source": [
    "The Grib2Message object provides a method, `attrs_by_section()` to get attribute names and values (if desired) for a given GRIB2 section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "ce2531d3-4249-4375-8f7f-467c0c65c50f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gridDefinitionTemplateNumber 0 - Latitude/Longitude\n",
      "shapeOfEarth 6 - Earth assumed spherical with radius = 6,371,229.0 m\n",
      "earthRadius 6371229.0\n",
      "earthMajorAxis None\n",
      "earthMinorAxis None\n",
      "resolutionAndComponentFlags [0, 0, 1, 1, 0, 0, 0, 0]\n",
      "ny 721\n",
      "nx 1440\n",
      "scanModeFlags [0, 0, 0, 0]\n",
      "latitudeFirstGridpoint 90.0\n",
      "longitudeFirstGridpoint 0.0\n",
      "latitudeLastGridpoint -90.0\n",
      "longitudeLastGridpoint 359.75\n",
      "gridlengthXDirection 0.25\n",
      "gridlengthYDirection -0.25\n"
     ]
    }
   ],
   "source": [
    "for k,v in msg.attrs_by_section(3,values=True).items():\n",
    "    print(k,v)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0f5d5e5-fa2f-4c4c-92ad-04194d14c355",
   "metadata": {},
   "source": [
    "Any GRIB2 metadata attribute that is a code value means that it is associated with a code table. That attribute is likely to be of type `grib2io.templates.Grib2Metadata` which is a class to hold the code value, the definition given its lookup table.\n",
    "\n",
    "Let use the GRIB2 metadata attribute `discipline` as an example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "dd52eded-54ec-4d3a-87e1-5ef71e093c86",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'grib2io.templates.Grib2Metadata'>\n",
      "0 - Meteorological Products\n",
      "0\n",
      "Meteorological Products\n",
      "0.0\n"
     ]
    }
   ],
   "source": [
    "print(type(msg.discipline))\n",
    "print(msg.discipline)\n",
    "print(msg.discipline.value)\n",
    "print(msg.discipline.definition)\n",
    "print(msg.discipline.table)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d16934e-6883-464f-9032-9bad804b98c4",
   "metadata": {},
   "source": [
    "## A Quick Detour to NCEP GRIB2 Tables\n",
    "\n",
    "grib2io contains the [NCEP GRIB2 Tables](https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/) stored as python dictionaries in `grib2io.tables`.  You are able to access them directly,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "6008740b-d574-46ba-acd2-d5c957ca0d17",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'0': 'Meteorological Products', '1': 'Hydrological Products', '2': 'Land Surface Products', '3': 'Satellite Remote Sensing Products', '4': 'Space Weather Products', '5-9': 'Reserved', '10': 'Oceanographic Products', '11-19': 'Reserved', '20': 'Health and Socioeconomic Impacts', '21-191': 'Reserved', '192-254': 'Reserved for Local Use', '255': 'Missing'}\n",
      "{'0': 'Unperturbed High-Resolution Control Forecast', '1': 'Unperturbed Low-Resolution Control Forecast', '2': 'Negatively Perturbed Forecast', '3': 'Positively Perturbed Forecast', '4': 'Multi-Model Forecast', '5-191': 'Reserved', '192-254': 'Reserved for Local Use', '192': 'Perturbed Ensemble Member', '255': 'Missing'}\n"
     ]
    }
   ],
   "source": [
    "print(grib2io.tables.section0.table_0_0)\n",
    "print(grib2io.tables.section4.table_4_6)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dfc6271-09e7-4e5c-895b-7badf3190ff5",
   "metadata": {},
   "source": [
    "However, it is more useful to use the various helper functions in `grib2io.tables` to view the NCEP GRIB2 code tables and lookup values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "fccd2e59-e9a7-4677-82bc-6479a8c6d5b8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0: Meteorological Products\n",
      "1: Hydrological Products\n",
      "2: Land Surface Products\n",
      "3: Satellite Remote Sensing Products\n",
      "4: Space Weather Products\n",
      "5-9: Reserved\n",
      "10: Oceanographic Products\n",
      "11-19: Reserved\n",
      "20: Health and Socioeconomic Impacts\n",
      "21-191: Reserved\n",
      "192-254: Reserved for Local Use\n",
      "255: Missing\n"
     ]
    }
   ],
   "source": [
    "for k,v in grib2io.tables.get_table('0.0').items():\n",
    "    print(f'{k}: {v}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "a1c60f68-47f0-4ab3-886f-4f917a1127df",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Value 2 from table 4.6 means Negatively Perturbed Forecast\n"
     ]
    }
   ],
   "source": [
    "print('Value 2 from table 4.6 means',grib2io.tables.get_value_from_table(2,'4.6'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "9bec5fcb-a18e-40d0-86f8-5df882cc9a6f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Discipline value of 0 with paramater category and number of 1 and 10 defines variable ['Convective Precipitation', 'kg m-2', 'ACPCP']\n"
     ]
    }
   ],
   "source": [
    "discipline, parmcat, parmnum = 0, 1, 10\n",
    "varinfo = grib2io.tables.get_varinfo_from_table(discipline,parmcat,parmnum)\n",
    "print(f'Discipline value of {discipline} with paramater category and number of {parmcat} and {parmnum} defines variable {varinfo}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04b80dc2-6efb-4f1c-9119-c0fe95aecf6f",
   "metadata": {},
   "source": [
    "Some metadata attributes exist for more easier understanding of information from the Grib2Message.\n",
    "\n",
    "The `level` attribute will provide a layer/level string that is **_exactly the same as wgrib2_** and is generated by `grib2io.tables.get_wgrib2_level_string()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "8748e0fd-f517-499b-bc06-9487dc878ec3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "surface\n",
      "kg m-2\n",
      "unknown\n"
     ]
    }
   ],
   "source": [
    "print(msg.level)\n",
    "print(msg.units) # Units of the variable\n",
    "print(msg.unitOfFirstFixedSurface) # Units of the level (or layer)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45e04309-7799-4e4f-99bb-d99112645159",
   "metadata": {},
   "source": [
    "## **Unpacking Data from Grib2Message Objects**\n",
    "\n",
    "Lets go back to the 500 mb HGT example.\n",
    "\n",
    "Data are accessible via the `data` attribute.\n",
    "\n",
    "**IMPORTANT:** If you unfamiliar with Python/NumPy, by default, multi-dimensional array data are ordered in row-major order.  This is how C/C++ order array data in memory.  Fortran orders array data in column-major order. So 2D array will be ordered (ny, nx)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "581e4759-6c3f-4884-9d3d-42fd38c5f1a0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "348:d=2023-10-28 00:55:00:HGT:Geopotential Height (gpm):500 mb:6:00:00\n",
      "[[5097.0503 5097.0503 5097.0503 ... 5097.0503 5097.0503 5097.0503]\n",
      " [5095.2505 5095.2505 5095.2505 ... 5095.2505 5095.2505 5095.2505]\n",
      " [5092.9307 5092.9307 5092.9307 ... 5092.9106 5092.9106 5092.9106]\n",
      " ...\n",
      " [5086.5303 5086.5107 5086.5107 ... 5086.5703 5086.5503 5086.5303]\n",
      " [5086.4106 5086.4106 5086.3906 ... 5086.4307 5086.4307 5086.4106]\n",
      " [5085.7905 5085.7905 5085.7905 ... 5085.7905 5085.7905 5085.7905]]\n",
      "(721, 1440)\n"
     ]
    }
   ],
   "source": [
    "msg = g.select(shortName='HGT',level='500 mb')[0]\n",
    "print(msg)\n",
    "hgtdata = msg.data\n",
    "print(hgtdata)\n",
    "print(hgtdata.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "714b6e8d-e199-4d23-9f25-bd49e5340198",
   "metadata": {},
   "source": [
    "## **Geospatial Information from Grib2Message Objects**\n",
    "\n",
    "GRIB2 Section 3 contains all of the geospatial information about the message."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "299da96b-d92c-4367-a7c0-3566ba4e6004",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[        0   1038240         0         0         0         6         0\n",
      "         0         0         0         0         0      1440       721\n",
      "         0        -1  90000000         0        48 -90000000 359750000\n",
      "    250000    250000         0]\n",
      "gridDefinitionTemplateNumber: 0 - Latitude/Longitude\n",
      "shapeOfEarth: 6 - Earth assumed spherical with radius = 6,371,229.0 m\n",
      "earthRadius: 6371229.0\n",
      "earthMajorAxis: None\n",
      "earthMinorAxis: None\n",
      "resolutionAndComponentFlags: [0, 0, 1, 1, 0, 0, 0, 0]\n",
      "ny: 721\n",
      "nx: 1440\n",
      "scanModeFlags: [0, 0, 0, 0]\n",
      "latitudeFirstGridpoint: 90.0\n",
      "longitudeFirstGridpoint: 0.0\n",
      "latitudeLastGridpoint: -90.0\n",
      "longitudeLastGridpoint: 359.75\n",
      "gridlengthXDirection: 0.25\n",
      "gridlengthYDirection: -0.25\n"
     ]
    }
   ],
   "source": [
    "print(msg.section3)\n",
    "for k,v in msg.attrs_by_section(3,values=True).items():\n",
    "    print(f'{k}: {v}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f6fa893-ee53-48b1-a1ed-f514f3ce6e4f",
   "metadata": {},
   "source": [
    "grib2io also provides the coorindate system in terms of PROJ parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "d3717757-5287-46c0-bf62-6abc9ba65d44",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'a': 6371229.0, 'b': 6371229.0, 'proj': 'longlat'}\n"
     ]
    }
   ],
   "source": [
    "print(msg.projParameters)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc8b828b-caea-48dd-9ce1-8f1ea88316cd",
   "metadata": {},
   "source": [
    "### Generating Latitude and Longitude Value\n",
    "\n",
    "Latitude and Longitude values can be generated via `grid()` or `latlons()` methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "89ffe3ef-0f20-402d-80c6-7074b5506247",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 90.    90.    90.   ...  90.    90.    90.  ]\n",
      " [ 89.75  89.75  89.75 ...  89.75  89.75  89.75]\n",
      " [ 89.5   89.5   89.5  ...  89.5   89.5   89.5 ]\n",
      " ...\n",
      " [-89.5  -89.5  -89.5  ... -89.5  -89.5  -89.5 ]\n",
      " [-89.75 -89.75 -89.75 ... -89.75 -89.75 -89.75]\n",
      " [-90.   -90.   -90.   ... -90.   -90.   -90.  ]]\n",
      "[[0.0000e+00 2.5000e-01 5.0000e-01 ... 3.5925e+02 3.5950e+02 3.5975e+02]\n",
      " [0.0000e+00 2.5000e-01 5.0000e-01 ... 3.5925e+02 3.5950e+02 3.5975e+02]\n",
      " [0.0000e+00 2.5000e-01 5.0000e-01 ... 3.5925e+02 3.5950e+02 3.5975e+02]\n",
      " ...\n",
      " [0.0000e+00 2.5000e-01 5.0000e-01 ... 3.5925e+02 3.5950e+02 3.5975e+02]\n",
      " [0.0000e+00 2.5000e-01 5.0000e-01 ... 3.5925e+02 3.5950e+02 3.5975e+02]\n",
      " [0.0000e+00 2.5000e-01 5.0000e-01 ... 3.5925e+02 3.5950e+02 3.5975e+02]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lats, lons = msg.grid()\n",
    "print(lats)\n",
    "print(lons)\n",
    "msg.grid() == msg.latlons() # latlons() is an alias of grid()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97daa45a-c692-4fb9-846a-52b06cbe5ddf",
   "metadata": {},
   "source": [
    "**NEW** In grib2io v2.0, computed lats and lons are stored so they do not have to regenerated if you call the methods from another message with the same coordinate system.  This is determined by taking the SHA-1 of section 3 and storing that as a key in a dictionary where lats and lons are the values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "bde10f5a-7f40-4fc5-8159-9d49b6a46c1e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "e8e4ad7e93f0f476804d00043204634232ebe9aa\n",
      "{'e8e4ad7e93f0f476804d00043204634232ebe9aa': (array([[ 90.  ,  90.  ,  90.  , ...,  90.  ,  90.  ,  90.  ],\n",
      "       [ 89.75,  89.75,  89.75, ...,  89.75,  89.75,  89.75],\n",
      "       [ 89.5 ,  89.5 ,  89.5 , ...,  89.5 ,  89.5 ,  89.5 ],\n",
      "       ...,\n",
      "       [-89.5 , -89.5 , -89.5 , ..., -89.5 , -89.5 , -89.5 ],\n",
      "       [-89.75, -89.75, -89.75, ..., -89.75, -89.75, -89.75],\n",
      "       [-90.  , -90.  , -90.  , ..., -90.  , -90.  , -90.  ]]), array([[0.0000e+00, 2.5000e-01, 5.0000e-01, ..., 3.5925e+02, 3.5950e+02,\n",
      "        3.5975e+02],\n",
      "       [0.0000e+00, 2.5000e-01, 5.0000e-01, ..., 3.5925e+02, 3.5950e+02,\n",
      "        3.5975e+02],\n",
      "       [0.0000e+00, 2.5000e-01, 5.0000e-01, ..., 3.5925e+02, 3.5950e+02,\n",
      "        3.5975e+02],\n",
      "       ...,\n",
      "       [0.0000e+00, 2.5000e-01, 5.0000e-01, ..., 3.5925e+02, 3.5950e+02,\n",
      "        3.5975e+02],\n",
      "       [0.0000e+00, 2.5000e-01, 5.0000e-01, ..., 3.5925e+02, 3.5950e+02,\n",
      "        3.5975e+02],\n",
      "       [0.0000e+00, 2.5000e-01, 5.0000e-01, ..., 3.5925e+02, 3.5950e+02,\n",
      "        3.5975e+02]]))}\n"
     ]
    }
   ],
   "source": [
    "print(msg._sha1_section3)\n",
    "print(grib2io._grib2io._latlon_datastore)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd40ea3d-265b-44d3-aacd-362e8c36eebf",
   "metadata": {},
   "source": [
    "## **Interpolation**\n",
    "\n",
    "**NEW** In grib2io v2.0 is the ability to perform spatial interpolation from one grid to another by interfacing to the [NCEPLIBS-ip](https://github.com/NOAA-EMC/NCEPLIBS-ip) library.\n",
    "\n",
    "The Grib2Message object already contains the necessary information about the input grid. So all we need to go here is to define the output grid (i.e. the grid that we want to interpolate to).  We do that by specifcy the GRIB2 Grid Definition Template Number and Template values.  Here we will interpolate to the NBM CONUS 2.5km Lambert Conformal grid.\n",
    "\n",
    "Here we place the template number and template into a `Grib2GridDef` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "8fc7d619-13a8-4b5c-b611-9ee503dec88c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "30\n",
      "[1, 0, 6371200, 255, 255, 255, 255, 2345, 1597, 19229000, 233723400, 48, 25000000, 265000000, 2539703, 2539703, 0, 80, 25000000, 25000000, -90000000, 0]\n"
     ]
    }
   ],
   "source": [
    "nbm_co_gdtn = 30\n",
    "nbm_co_gdt = [        1,         0,   6371200,       255,       255,       255,\n",
    "                    255,      2345,      1597,  19229000, 233723400,        48,\n",
    "               25000000, 265000000,   2539703,   2539703,         0,        80,\n",
    "               25000000,  25000000, -90000000,         0]\n",
    "grid_def_out = grib2io.Grib2GridDef(nbm_co_gdtn,nbm_co_gdt)\n",
    "print(grid_def_out.gdtn)\n",
    "print(grid_def_out.gdt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62a36006-352d-41bc-9075-599231bdb734",
   "metadata": {},
   "source": [
    "To interpolate a Grib2Message object, call the `interpolate()` method.  The return from the method is a new Grib2Message object.  All metadata remain the same except for section 3 metadata.\n",
    "\n",
    "**NOTE 1: The interpolation option can be a string or numeric value determined by the NCEPLIBS-ip library.**\n",
    "\n",
    "**NOTE 2: The message number of interpolated messages is -1. This is becasue this message does not originate from a physical file.  Any Grib2Message object created from scatch will have message number of -1 until it is associated with a file object.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "565d5c0e-419a-4efd-9224-165b98ad6c58",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-1:d=2023-10-28 00:55:00:HGT:Geopotential Height (gpm):500 mb:6:00:00\n",
      "[        0   3744965         0       255        30         1         0\n",
      "   6371200       255       255       255       255      2345      1597\n",
      "  19229000 233723400        48  25000000 265000000   2539703   2539703\n",
      "         0        80  25000000  25000000 -90000000         0]\n",
      "gridDefinitionTemplateNumber: 0 - Latitude/Longitude\n",
      "shapeOfEarth: 6 - Earth assumed spherical with radius = 6,371,229.0 m\n",
      "earthRadius: 6371229.0\n",
      "earthMajorAxis: None\n",
      "earthMinorAxis: None\n",
      "resolutionAndComponentFlags: [0, 0, 1, 1, 0, 0, 0, 0]\n",
      "ny: 721\n",
      "nx: 1440\n",
      "scanModeFlags: [0, 0, 0, 0]\n",
      "latitudeFirstGridpoint: 90.0\n",
      "longitudeFirstGridpoint: 0.0\n",
      "latitudeLastGridpoint: -90.0\n",
      "longitudeLastGridpoint: 359.75\n",
      "gridlengthXDirection: 0.25\n",
      "gridlengthYDirection: -0.25\n",
      "[[5885.6235 5885.614  5885.574  ... 5849.2563 5849.1816 5849.145 ]\n",
      " [5885.62   5885.6123 5885.5825 ... 5849.261  5849.1973 5849.1465]\n",
      " [5885.6167 5885.611  5885.592  ... 5849.278  5849.207  5849.1426]\n",
      " ...\n",
      " [5264.842  5264.8643 5264.8887 ... 5419.2915 5421.1416 5422.9863]\n",
      " [5264.0977 5264.1206 5264.1484 ... 5418.9565 5420.8115 5422.6636]\n",
      " [5263.35   5263.378  5263.4053 ... 5418.6245 5420.4863 5422.3438]]\n"
     ]
    }
   ],
   "source": [
    "new = msg.interpolate('bilinear',grid_def_out)\n",
    "print(new)\n",
    "print(new.section3)\n",
    "for k,v in msg.attrs_by_section(3,values=True).items():\n",
    "    print(f'{k}: {v}')\n",
    "print(new.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6f276fe-0656-47e6-aa7f-c58cce6b929c",
   "metadata": {},
   "source": [
    "## **Creating Grib2Message Object from Scratch**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0815538b-8074-4501-93ad-727bbdba424c",
   "metadata": {},
   "source": [
    "The majority of the time, you will most likely have an existing Grib2Message object that you could use a guideline or template in building a new Grib2Message object.  To demo, this we are going to use a bit of information from scratch and from an existing Grib2Message.\n",
    "\n",
    "We will construct a Grib2Message object containing the **10-m Wind Speed from the U- and V-wind components**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "37a48ddd-dbc3-4413-b0c9-a9b96bdb1247",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "587:d=2022-10-28 00:00:00:UGRD:U-Component of Wind (m s-1):10 m above ground:6:00:00\n",
      "588:d=2022-10-28 00:00:00:VGRD:V-Component of Wind (m s-1):10 m above ground:6:00:00\n"
     ]
    }
   ],
   "source": [
    "uwind = g.select(shortName='UGRD',level='10 m above ground')[0]\n",
    "vwind = g.select(shortName='VGRD',level='10 m above ground')[0]\n",
    "print(uwind)\n",
    "print(vwind)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2aa91d30-1f12-4f61-acb2-013c46eeba3a",
   "metadata": {},
   "source": [
    "Create new Grib2Message object.\n",
    "\n",
    "Here we generate the message by only passing Grid Definition Template Number, Product Definition Template Number, and Data Representation Template Number which we can borrow from either of the `uwind` or `vwind` objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "daaa3b41-987c-49eb-aa00-de21bb93a9fd",
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "year 0 is out of range",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[36], line 2\u001b[0m\n\u001b[1;32m      1\u001b[0m wspd \u001b[38;5;241m=\u001b[39m grib2io\u001b[38;5;241m.\u001b[39mGrib2Message(gdtn\u001b[38;5;241m=\u001b[39muwind\u001b[38;5;241m.\u001b[39mgdtn,pdtn\u001b[38;5;241m=\u001b[39muwind\u001b[38;5;241m.\u001b[39mpdtn,drtn\u001b[38;5;241m=\u001b[39muwind\u001b[38;5;241m.\u001b[39mdrtn)\n\u001b[0;32m----> 2\u001b[0m \u001b[38;5;28;43mprint\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mwspd\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/Projects/GitHub-NOAA/grib2io/build/lib.linux-x86_64-cpython-311/grib2io/_grib2io.py:698\u001b[0m, in \u001b[0;36m_Grib2Message.__str__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    697\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__str__\u001b[39m(\u001b[38;5;28mself\u001b[39m):\n\u001b[0;32m--> 698\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m (\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_msgnum\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m:d=\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrefDate\u001b[49m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m:\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mshortName\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m:\u001b[39m\u001b[38;5;124m'\u001b[39m\n\u001b[1;32m    699\u001b[0m             \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mfullName\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m (\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39munits\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m):\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mlevel\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m:\u001b[39m\u001b[38;5;124m'\u001b[39m\n\u001b[1;32m    700\u001b[0m             \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mleadTime\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m'\u001b[39m)\n",
      "File \u001b[0;32m~/Projects/GitHub-NOAA/grib2io/build/lib.linux-x86_64-cpython-311/grib2io/templates.py:188\u001b[0m, in \u001b[0;36mRefDate.__get__\u001b[0;34m(self, obj, objtype)\u001b[0m\n\u001b[1;32m    187\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__get__\u001b[39m(\u001b[38;5;28mself\u001b[39m, obj, objtype\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mNone\u001b[39;00m):\n\u001b[0;32m--> 188\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mdatetime\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mdatetime\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mobj\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msection1\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;241;43m5\u001b[39;49m\u001b[43m:\u001b[49m\u001b[38;5;241;43m11\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m)\u001b[49m\n",
      "\u001b[0;31mValueError\u001b[0m: year 0 is out of range"
     ]
    }
   ],
   "source": [
    "wspd = grib2io.Grib2Message(gdtn=uwind.gdtn,pdtn=uwind.pdtn,drtn=uwind.drtn)\n",
    "print(wspd)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79cf4162-fea6-4ea2-9e5c-369ba73220f7",
   "metadata": {},
   "source": [
    "Notice how printing the new object results in an error, specifically the `refDate`.  Since the GRIB2 message object was created from scratch it does have the appropriate metadata filled in yet.\n",
    "\n",
    "Lets take at the sections 0-5.  Lets get a list of attributes and then populate them.  To perform this correctly, you need to some familiarity with the NCEP GRIB2 tables.\n",
    "\n",
    "### Section 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "42870183-c599-4f0c-96fc-0bcc0091b021",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['discipline']\n",
      "{'discipline': Grib2Metadata(0, table = '0.0')}\n",
      "[1196575042          0          0          2          0]\n"
     ]
    }
   ],
   "source": [
    "print(wspd.attrs_by_section(0))\n",
    "wspd.discipline = 0 # 0 = Meteorological Products\n",
    "print(wspd.attrs_by_section(0, values=True))\n",
    "print(wspd.section0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ceecda0f-5b29-4e76-abf0-92d187a2960e",
   "metadata": {},
   "source": [
    "### Section 1\n",
    "\n",
    "Here we are going to cheat a bit by populating section 1 attributes from the uwind variable.  The following simulates populating the attributes by hand"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "8310ca77-ee1b-4c6f-aa5a-9a8cba4ef88d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "originatingCenter: 7 - US National Weather Service - NCEP (WMC)\n",
      "originatingSubCenter: 0 - None\n",
      "masterTableInfo: 2 - Version Implemented on 4 November 2003\n",
      "localTableInfo: 1 - Number of local table version used.\n",
      "significanceOfReferenceTime: 1 - Start of Forecast\n",
      "year: 2022\n",
      "month: 10\n",
      "day: 28\n",
      "hour: 0\n",
      "minute: 0\n",
      "second: 0\n",
      "refDate: 2022-10-28 00:00:00\n",
      "productionStatus: 0 - Operational Products\n",
      "typeOfData: 1 - Forecast Products\n"
     ]
    }
   ],
   "source": [
    "for attr in wspd.attrs_by_section(1):\n",
    "    try:\n",
    "        cmd = 'wspd.'+attr+' = uwind.'+attr+'.value'\n",
    "        exec(cmd)\n",
    "    except(AttributeError):\n",
    "        cmd = cmd.replace('.value','')\n",
    "        exec(cmd)\n",
    "\n",
    "for k,v in wspd.attrs_by_section(1,values=True).items():\n",
    "    print(f'{k}: {v}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2e7bca5-47f1-4b7d-97f6-84d4e1ffa25a",
   "metadata": {},
   "source": [
    "### Section 3\n",
    "\n",
    "For the rest of the sections, we will simply populate from `uwind` variable and adjust as needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "44a1bbac-279f-4198-b866-32d0921fc434",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gridDefinitionTemplateNumber: 0 - Latitude/Longitude\n",
      "shapeOfEarth: 6 - Earth assumed spherical with radius = 6,371,229.0 m\n",
      "earthRadius: 6371229.0\n",
      "earthMajorAxis: None\n",
      "earthMinorAxis: None\n",
      "resolutionAndComponentFlags: [0, 0, 1, 1, 0, 0, 0, 0]\n",
      "ny: 721\n",
      "nx: 1440\n",
      "scanModeFlags: [0, 0, 0, 0]\n",
      "latitudeFirstGridpoint: 90.0\n",
      "longitudeFirstGridpoint: 0.0\n",
      "latitudeLastGridpoint: -90.0\n",
      "longitudeLastGridpoint: 359.75\n",
      "gridlengthXDirection: 0.25\n",
      "gridlengthYDirection: -0.25\n"
     ]
    }
   ],
   "source": [
    "wspd.section3 = uwind.section3\n",
    "for k,v in wspd.attrs_by_section(3,values=True).items():\n",
    "    print(f'{k}: {v}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "202efcca-4932-4c4f-a438-65ec153960ae",
   "metadata": {},
   "source": [
    "### Section 4\n",
    "\n",
    "Here we need to modify the parameter Category and Number to define the Wind Speed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "cf3876fd-3ab9-4699-964c-fed9d3d48cbd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "productDefinitionTemplateNumber: 0 - Analysis or forecast at a horizontal level or in a horizontal layer at a point in time. (see Template 4.0)\n",
      "fullName: Wind Speed\n",
      "units: m s-1\n",
      "shortName: WIND\n",
      "leadTime: 6:00:00\n",
      "unitOfFirstFixedSurface: m\n",
      "valueOfFirstFixedSurface: 10.0\n",
      "unitOfSecondFixedSurface: None\n",
      "valueOfSecondFixedSurface: 10.0\n",
      "validDate: 2022-10-28 06:00:00\n",
      "duration: None\n",
      "level: 10 m above ground\n",
      "parameterCategory: 2\n",
      "parameterNumber: 1\n",
      "typeOfGeneratingProcess: 2 - Forecast\n",
      "generatingProcess: 96 - Global Forecast System Model  T1534 - Forecast hours 00-384 T574 - Forecast hours 00-192 T190 - Forecast hours 204-384\n",
      "backgroundGeneratingProcessIdentifier: 0\n",
      "hoursAfterDataCutoff: 0\n",
      "minutesAfterDataCutoff: 0\n",
      "unitOfTimeRange: 1 - Hour\n",
      "forecastTime: 6\n",
      "typeOfFirstFixedSurface: 103 - ['Specified Height Level Above Ground', 'm']\n",
      "scaleFactorOfFirstFixedSurface: 0\n",
      "scaledValueOfFirstFixedSurface: 10\n",
      "typeOfSecondFixedSurface: 255 - ['Missing', 'unknown']\n",
      "scaleFactorOfSecondFixedSurface: 0\n",
      "scaledValueOfSecondFixedSurface: 0\n"
     ]
    }
   ],
   "source": [
    "wspd.section4 = uwind.section4\n",
    "wspd.parameterCategory = 2 # Momentum, https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_table4-1.shtml\n",
    "wspd.parameterNumber = 1 # Wind Speed, https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_table4-2-0-2.shtml\n",
    "for k,v in wspd.attrs_by_section(4,values=True).items():\n",
    "    print(f'{k}: {v}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad38a7fc-0ab7-4aec-bb3b-19cb860607e3",
   "metadata": {},
   "source": [
    "### Section 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "2f7492f7-1729-47c7-93d0-113b315eed43",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dataRepresentationTemplateNumber: 3 - Grid Point Data - Complex Packing and Spatial Differencing (see Template 5.3)\n",
      "numberOfPackedValues: 1038240\n",
      "typeOfValues: 0 - Floating Point\n",
      "refValue: -2310.03564453125\n",
      "binScaleFactor: 0\n",
      "decScaleFactor: 2\n",
      "nBitsPacking: 12\n",
      "groupSplittingMethod: 1 - General Group Splitting\n",
      "typeOfMissingValue: 0 - No explicit missing values included within the data values\n",
      "priMissingValue: None\n",
      "secMissingValue: None\n",
      "nGroups: 30222\n",
      "refGroupWidth: 0\n",
      "nBitsGroupWidth: 4\n",
      "refGroupLength: 1\n",
      "groupLengthIncrement: 1\n",
      "lengthOfLastGroup: 37\n",
      "nBitsScaledGroupLength: 7\n",
      "spatialDifferenceOrder: 2 - Second-Order Spatial Differencing\n",
      "nBytesSpatialDifference: 2\n",
      "[   1038240          3 3306184850          0          2         12\n",
      "          0          1          0 1649987994 4294967295      30222\n",
      "          0          4          1          1         37          7\n",
      "          2          2]\n"
     ]
    }
   ],
   "source": [
    "wspd.section5 = uwind.section5\n",
    "for k,v in wspd.attrs_by_section(5,values=True).items():\n",
    "    print(f'{k}: {v}')\n",
    "print(wspd.section5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41624ee3-afd1-4652-9bf5-ef374be9ff8f",
   "metadata": {},
   "source": [
    "**IMPORANT:** So we set section 5 of the `wspd` variable to section 5 of `uwind` variable. This might look harmless, but notice that there is specific information in section 5 related to the packing of the U-wind data.  Having this data present in section 5 for `wspd` could lead to undesirable effects when packing.  The majority of attributes in section 5 will be set by the packing code. So the following will we will sanitze section 5.\n",
    "\n",
    "**NOTE:** The sanitization will likely become an under-the-hood feature of grib2io."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "c6dd22c5-02ec-4102-a989-a6c5c680406b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1038240       3       0       0       3       0       0       1       0\n",
      "       0       0       0       0       0       0       0       0       0\n",
      "       2       0]\n"
     ]
    }
   ],
   "source": [
    "wspd.section5 = np.zeros(wspd.section5.shape,dtype=wspd.section5.dtype)\n",
    "wspd.section5[0] = wspd.nx * wspd.ny\n",
    "wspd.section5[1] = uwind.dataRepresentationTemplateNumber.value\n",
    "wspd.typeOfValues = 0 # Float\n",
    "wspd.binScaleFactor = 0 # Binary Scale Factor\n",
    "wspd.decScaleFactor = 3 # Decimal Scale Factor\n",
    "wspd.groupSplittingMethod = 1 # Group Split\n",
    "wspd.spatialDifferenceOrder = 2 # Second order spatial differencing\n",
    "print(wspd.section5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e496576-5a56-4cd1-a37c-40aa144c17af",
   "metadata": {},
   "source": [
    "We have now defined the packing of the `wspd` variable. We will use the same packing scheme as the `uwind` (Complex Packing with Spatial Differencing)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5955bb3d-c67c-4c88-adf5-2de83df1658d",
   "metadata": {},
   "source": [
    "### Packing the Data and Message\n",
    "\n",
    "First we need to compute the wind speed from the U- and V-Wind components."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "45503451-04a1-4567-8ce6-98c45636c432",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0025712133 26.308275\n"
     ]
    }
   ],
   "source": [
    "wspd.data = np.sqrt(uwind.data**2.0+vwind.data**2.0)\n",
    "print(np.amin(wspd.data),np.amax(wspd.data))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40f2edc5-155b-4907-af8c-2483e8cad378",
   "metadata": {},
   "source": [
    "Next, we can call the `pack()` method to pack the data and create the packed GRIB2 binary message."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "83240fc3-bb52-4467-9b27-6d2e76ba8591",
   "metadata": {},
   "outputs": [],
   "source": [
    "wspd.pack()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e00483eb-c1d7-4521-8d21-ae1ce45160bf",
   "metadata": {},
   "source": [
    "We now have a packed GRIB2 message.  It is stored in the Grib2Message object in the `_msg` attribute. We can take a look at the first 4 and last 4 bytes of the packages message to see that it is a complete messages.  The first 4 bytes should be `'GRIB'` and last 4 `'7777'`.\n",
    "\n",
    "**NOTE:** When we print the message, notice the message number of -1. This is because this Grib2Message object is not associated with a GRIB2 file object (`grib2io.open`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "e1fbe67d-86e3-4839-9341-69daad7ecaaf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'GRIB' b'7777'\n",
      "-1:d=2022-10-28 00:00:00:WIND:Wind Speed (m s-1):10 m above ground:6:00:00\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Section 0: discipline = 0 - Meteorological Products\n",
       "Section 1: originatingCenter = 7 - US National Weather Service - NCEP (WMC)\n",
       "Section 1: originatingSubCenter = 0 - None\n",
       "Section 1: masterTableInfo = 2 - Version Implemented on 4 November 2003\n",
       "Section 1: localTableInfo = 1 - Number of local table version used.\n",
       "Section 1: significanceOfReferenceTime = 1 - Start of Forecast\n",
       "Section 1: year = 2022\n",
       "Section 1: month = 10\n",
       "Section 1: day = 28\n",
       "Section 1: hour = 0\n",
       "Section 1: minute = 0\n",
       "Section 1: second = 0\n",
       "Section 1: refDate = 2022-10-28 00:00:00\n",
       "Section 1: productionStatus = 0 - Operational Products\n",
       "Section 1: typeOfData = 1 - Forecast Products\n",
       "Section 3: gridDefinitionTemplateNumber = 0 - Latitude/Longitude\n",
       "Section 3: shapeOfEarth = 6 - Earth assumed spherical with radius = 6,371,229.0 m\n",
       "Section 3: earthRadius = 6371229.0\n",
       "Section 3: earthMajorAxis = None\n",
       "Section 3: earthMinorAxis = None\n",
       "Section 3: resolutionAndComponentFlags = [0, 0, 1, 1, 0, 0, 0, 0]\n",
       "Section 3: ny = 721\n",
       "Section 3: nx = 1440\n",
       "Section 3: scanModeFlags = [0, 0, 0, 0]\n",
       "Section 3: latitudeFirstGridpoint = 90.0\n",
       "Section 3: longitudeFirstGridpoint = 0.0\n",
       "Section 3: latitudeLastGridpoint = -90.0\n",
       "Section 3: longitudeLastGridpoint = 359.75\n",
       "Section 3: gridlengthXDirection = 0.25\n",
       "Section 3: gridlengthYDirection = -0.25\n",
       "Section 4: productDefinitionTemplateNumber = 0 - Analysis or forecast at a horizontal level or in a horizontal layer at a point in time. (see Template 4.0)\n",
       "Section 4: fullName = Wind Speed\n",
       "Section 4: units = m s-1\n",
       "Section 4: shortName = WIND\n",
       "Section 4: leadTime = 6:00:00\n",
       "Section 4: unitOfFirstFixedSurface = m\n",
       "Section 4: valueOfFirstFixedSurface = 10.0\n",
       "Section 4: unitOfSecondFixedSurface = None\n",
       "Section 4: valueOfSecondFixedSurface = 10.0\n",
       "Section 4: validDate = 2022-10-28 06:00:00\n",
       "Section 4: duration = None\n",
       "Section 4: level = 10 m above ground\n",
       "Section 4: parameterCategory = 2\n",
       "Section 4: parameterNumber = 1\n",
       "Section 4: typeOfGeneratingProcess = 2 - Forecast\n",
       "Section 4: generatingProcess = 96 - Global Forecast System Model  T1534 - Forecast hours 00-384 T574 - Forecast hours 00-192 T190 - Forecast hours 204-384\n",
       "Section 4: backgroundGeneratingProcessIdentifier = 0\n",
       "Section 4: hoursAfterDataCutoff = 0\n",
       "Section 4: minutesAfterDataCutoff = 0\n",
       "Section 4: unitOfTimeRange = 1 - Hour\n",
       "Section 4: forecastTime = 6\n",
       "Section 4: typeOfFirstFixedSurface = 103 - ['Specified Height Level Above Ground', 'm']\n",
       "Section 4: scaleFactorOfFirstFixedSurface = 0\n",
       "Section 4: scaledValueOfFirstFixedSurface = 10\n",
       "Section 4: typeOfSecondFixedSurface = 255 - ['Missing', 'unknown']\n",
       "Section 4: scaleFactorOfSecondFixedSurface = 0\n",
       "Section 4: scaledValueOfSecondFixedSurface = 0\n",
       "Section 5: dataRepresentationTemplateNumber = 3 - Grid Point Data - Complex Packing and Spatial Differencing (see Template 5.3)\n",
       "Section 5: numberOfPackedValues = 1038240\n",
       "Section 5: typeOfValues = 0 - Floating Point\n",
       "Section 5: refValue = 3.0\n",
       "Section 5: binScaleFactor = 0\n",
       "Section 5: decScaleFactor = 3\n",
       "Section 5: nBitsPacking = 15\n",
       "Section 5: groupSplittingMethod = 1 - General Group Splitting\n",
       "Section 5: typeOfMissingValue = 0 - No explicit missing values included within the data values\n",
       "Section 5: priMissingValue = None\n",
       "Section 5: secMissingValue = None\n",
       "Section 5: nGroups = 74433\n",
       "Section 5: refGroupWidth = 0\n",
       "Section 5: nBitsGroupWidth = 5\n",
       "Section 5: refGroupLength = 1\n",
       "Section 5: groupLengthIncrement = 1\n",
       "Section 5: lengthOfLastGroup = 14\n",
       "Section 5: nBitsScaledGroupLength = 5\n",
       "Section 5: spatialDifferenceOrder = 2 - Second-Order Spatial Differencing\n",
       "Section 5: nBytesSpatialDifference = 2\n",
       "Section 6: bitMapFlag = 255 - A bit map does not apply to this product."
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(wspd._msg[0:4],wspd._msg[-4:])\n",
    "print(wspd)\n",
    "wspd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bc9f337-21df-46a7-a17b-64c1a09168cc",
   "metadata": {},
   "source": [
    "## **Writing Grib2Message Objects to a New File**\n",
    "\n",
    "Now that we have created a new Grib2Message object, we can write that message to a new file.\n",
    "\n",
    "Here we create a new GRIB2 file object to write to"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "6d07c85f-58fc-4c50-9f5b-f0675160ae72",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mode = wb\n",
      "name = /home/ericengle/Projects/GitHub-NOAA/grib2io/build/lib.linux-x86_64-cpython-311/wind-speed-demo.grib2\n",
      "messages = 0\n",
      "current_message = 0\n",
      "size = 0\n",
      "closed = False\n",
      "variables = None\n",
      "levels = None\n",
      "\n"
     ]
    }
   ],
   "source": [
    "gout = grib2io.open('wind-speed-demo.grib2',mode='w')\n",
    "print(gout)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d5b1d61-bb72-4f4c-8af0-58290b19c9e1",
   "metadata": {},
   "source": [
    "We can use the `write()` method to write a Grib2Message object to file.  We will use the wind speed message we created above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "ea083c06-8b41-47ed-8efe-5bc31d5e9ac7",
   "metadata": {},
   "outputs": [],
   "source": [
    "gout.write(wspd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "30470732-13f9-430f-b53c-5436016ffb1d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mode = wb\n",
      "name = /home/ericengle/Projects/GitHub-NOAA/grib2io/build/lib.linux-x86_64-cpython-311/wind-speed-demo.grib2\n",
      "messages = 1\n",
      "current_message = 1\n",
      "size = 1464114\n",
      "closed = False\n",
      "variables = None\n",
      "levels = None\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(gout)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee372988-6dea-45c9-abef-0613f8ba5281",
   "metadata": {},
   "source": [
    "**NOTE:** Even though we wrote to the new file, the `variables` and `levels` attributes have not been updated.  This feature will be added in the near future."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "e852ea47-cc7e-4aeb-bb83-e18d4626db3f",
   "metadata": {},
   "outputs": [],
   "source": [
    "gout.close()\n",
    "g.close()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}