V1.1.2 - Griddes parser bug fix

Release Notes: v1.1.2

🎯 Feature Enhancement Release

This release significantly enhances the GriddesParser to support all CDO grid types, including rotated pole projections, Gaussian grids, and unstructured meshes, with intelligent fallback handling for unknown attributes.

---

✨ What's New

Enhanced GriddesParser - Universal Grid Type Support

The Enhancement:
Version 1.1.2 expands GriddesParser from basic lonlat grids to comprehensive support for all 7 major CDO grid types, including advanced projections and irregular meshes.

What's Supported Now:

Grid Type	Description	New in v1.1.2
lonlat	Regular latitude-longitude grids	✅ Enhanced
gaussian	Gaussian grids with regular longitude spacing	✨ NEW
gaussian_reduced	Reduced Gaussian grids with variable longitude points	✨ NEW
generic	Generic grids with minimal metadata	✨ NEW
projection	Rotated pole and CF Conventions projections	✨ NEW
curvilinear	2D coordinate arrays for structured grids	✨ NEW
unstructured	Irregular meshes and point clouds	✨ NEW

---

🚀 Key Features

1. Rotated Pole Projection Support (CF Conventions)

Full support for rotated latitude-longitude grids commonly used in regional climate models (RCMs):

from python_cdo_wrapper import CDO

cdo = CDO()
result = cdo.griddes("rotated_grid_data.nc")
grid = result.primary_grid

# Check if grid is rotated
if grid.is_rotated:
    print(f"Rotated pole projection detected!")
    print(f"North pole longitude: {grid.grid_north_pole_longitude}°")
    print(f"North pole latitude: {grid.grid_north_pole_latitude}°")
    print(f"Grid mapping: {grid.grid_mapping_name}")

# Output:
# Rotated pole projection detected!
# North pole longitude: -123.34°
# North pole latitude: 79.95°
# Grid mapping: rotated_latitude_longitude

New GridInfo Fields:

• grid_mapping - Grid mapping variable name
• grid_mapping_name - CF Conventions projection name
• grid_north_pole_longitude - Rotated pole longitude
• grid_north_pole_latitude - Rotated pole latitude

2. Gaussian Grid Support

Support for both regular and reduced Gaussian grids used in global models:

# Regular Gaussian grid
result = cdo.griddes("gaussian_data.nc")
grid = result.primary_grid

if grid.is_gaussian:
    print(f"Gaussian grid: {grid.gridtype}")
    print(f"Grid size: {grid.gridsize} points")
    print(f"Truncation (np): {grid.np}")
    print(f"Dimensions: {grid.xsize} x {grid.ysize}")

# Reduced Gaussian grid
result = cdo.griddes("reduced_gaussian_data.nc")
grid = result.primary_grid

if grid.gridtype == "gaussian_reduced":
    print(f"Reduced Gaussian grid")
    print(f"Variable longitude points per latitude row")
    if grid.rowlon:
        print(f"Longitude points: {grid.rowlon[:5]}...")  # First 5 rows

New GridInfo Fields:

• np - Gaussian grid truncation parameter
• rowlon - Number of longitude points per latitude row (reduced Gaussian)

3. Unstructured Grid Support

Support for irregular meshes and point clouds:

result = cdo.griddes("unstructured_mesh.nc")
grid = result.primary_grid

if grid.is_unstructured:
    print(f"Unstructured grid detected")
    print(f"Total points: {grid.points}")
    print(f"Vertices per cell: {grid.nvertex}")
    print(f"Grid size: {grid.gridsize}")

New GridInfo Fields:

• points - Number of points in unstructured grid
• nvertex - Number of vertices per cell

4. Intelligent Fallback Mechanism

The Problem:
CDO grids may have custom or unrecognized attributes from specialized models or future CDO versions.

The Solution:
Unknown attributes are automatically stored in raw_attributes dictionary for inspection:

result = cdo.griddes("custom_grid.nc")
grid = result.primary_grid

# Known attributes parsed normally
print(f"Grid type: {grid.gridtype}")
print(f"Size: {grid.xsize} x {grid.ysize}")

# Unknown attributes stored for inspection
if grid.raw_attributes:
    print(f"\nUnknown attributes found:")
    for key, value in grid.raw_attributes.items():
        print(f"  {key}: {value}")

# Output:
# Grid type: lonlat
# Size: 50 x 20
#
# Unknown attributes found:
#   custom_attr1: special_value
#   custom_number: 42
#   custom_float: 3.14159

Benefits:

• ✅ Never fails on unknown attributes
• ✅ Preserves all information for debugging
• ✅ Future-proof for new CDO features
• ✅ Enables inspection of custom grid metadata

5. Comprehensive GridInfo Properties

Eight new convenience properties for grid type checking:

result = cdo.griddes("data.nc")
grid = result.primary_grid

# Grid type classification
print(f"Regular grid: {grid.is_regular}")           # lonlat, generic, projection
print(f"Gaussian grid: {grid.is_gaussian}")         # gaussian, gaussian_reduced
print(f"Structured grid: {grid.is_structured}")     # has regular dimensions
print(f"Unstructured grid: {grid.is_unstructured}") # irregular mesh

# Projection information
print(f"Rotated projection: {grid.is_rotated}")     # rotated pole
print(f"Has projection: {grid.has_projection}")     # any projection info

# Spatial extent (if available)
if grid.lon_range:
    print(f"Longitude range: {grid.lon_range[0]}° to {grid.lon_range[1]}°")
if grid.lat_range:
    print(f"Latitude range: {grid.lat_range[0]}° to {grid.lat_range[1]}°")

---

🎯 Use Cases

Regional Climate Model Analysis

from python_cdo_wrapper import CDO

cdo = CDO()

# Analyze RCM grid with rotated pole projection
result = cdo.griddes("EUR-11_rotated.nc")
grid = result.primary_grid

if grid.is_rotated:
    print(f"European domain with rotated pole")
    print(f"Rotated coordinates: {grid.xname}/{grid.yname}")
    print(f"Resolution: {grid.xinc}° x {grid.yinc}°")
    print(f"Domain size: {grid.xsize} x {grid.ysize}")
    
    # Coordinate transformation information
    print(f"\nProjection details:")
    print(f"  Mapping: {grid.grid_mapping_name}")
    print(f"  Pole: ({grid.grid_north_pole_longitude}°E, "
          f"{grid.grid_north_pole_latitude}°N)")

Global Model Comparison

# Compare different global model grids
models = ["ERA5_gaussian.nc", "CMIP6_lonlat.nc", "IFS_reduced.nc"]

for model_file in models:
    result = cdo.griddes(model_file)
    grid = result.primary_grid
    
    print(f"\n{model_file}:")
    print(f"  Type: {grid.gridtype}")
    print(f"  Size: {grid.gridsize} points")
    
    if grid.is_gaussian:
        print(f"  Gaussian truncation: N{grid.np}")
    elif grid.is_regular:
        print(f"  Resolution: {grid.xinc}° x {grid.yinc}°")

Unstructured Mesh Inspection

# Analyze unstructured mesh from ocean model
result = cdo.griddes("ocean_mesh.nc")
grid = result.primary_grid

if grid.is_unstructured:
    print(f"Ocean mesh analysis:")
    print(f"  Total mesh points: {grid.points:,}")
    print(f"  Cell structure: {grid.nvertex}-vertex cells")
    print(f"  Grid efficiency: {grid.gridsize / grid.points:.2%}")

---

🧪 Testing

Comprehensive Test Coverage

Added 13 new test cases covering all grid types and edge cases:

Grid Type Tests:

1. ✅ Regular lonlat grids
2. ✅ Rotated pole projection grids
3. ✅ Gaussian grids (regular)
4. ✅ Reduced Gaussian grids
5. ✅ Generic grids
6. ✅ Curvilinear grids
7. ✅ Unstructured grids

Feature Tests:
8. ✅ Projection parameter parsing
9. ✅ GridInfo convenience properties (lonlat)
10. ✅ GridInfo convenience properties (rotated)
11. ✅ Unknown attribute fallback
12. ✅ Type-safe attribute parsing
13. ✅ Error handling for malformed output

Test Files:

• tests/test_parsers/test_grid.py - Unit tests with sample CDO outputs
• Sample outputs for 7 different grid types included as fixtures

All tests pass on:

• ✅ Ubuntu (Python 3.9, 3.10, 3.11, 3.12)
• ✅ macOS (Python 3.9, 3.10, 3.11, 3.12)

---

💥 Breaking Changes

None - This is a backward-compatible enhancement. All existing code continues to work as before.

New Features:

• Existing GridInfo fields remain unchanged
• New optional fields with default None values
• New convenience properties are additive
• raw_attributes field is optional

---

🔧 Technical Details

Type-Safe Attribute Parsing

New _parse_grid_attribute() method handles automatic type detection:

# Integer fields
gridsize, xsize, ysize, np, points, nvertex → int

# Float fields  
xfirst, xinc, yfirst, yinc, scanningMode,
grid_north_pole_longitude, grid_north_pole_latitude → float

# Float list fields (space-separated)
xvals, yvals, levels → list[float]

# Integer list fields (space-separated)
rowlon → list[int]

# String fields (with quote stripping)
gridtype, xname, xlongname, xunits, etc. → str

Robust Parsing:

• ✅ Automatic type conversion based on field name
• ✅ Graceful handling of parsing failures (fallback to raw string)
• ✅ Preservation of all data (unknown → raw_attributes)
• ✅ No data loss on parse errors

Known vs Unknown Attributes

The parser distinguishes between:

Known Attributes (mapped to GridInfo fields):

• Standard grid attributes: gridtype, gridsize, xsize, ysize, etc.
• Coordinate attributes: xfirst, xinc, xvals, etc.
• Projection attributes: grid_mapping, grid_north_pole_*, etc.
• Special attributes: np, rowlon, points, nvertex, etc.

Unknown Attributes (stored in raw_attributes):

• Custom model-specific attributes
• Future CDO attributes not yet supported
• Experimental or non-standard fields

This ensures forward compatibility with future CDO versions.

---

📦 Installation & Upgrade

Upgrading from v1.1.1 or Earlier

# Upgrade to the latest version
pip install --upgrade python-cdo-wrapper

# Or with shapefile support
pip install --upgrade python-cdo-wrapper[shapefiles]

New Installation

# Core package
pip install python-cdo-wrapper

# With shapefile support
pip install python-cdo-wrapper[shapefiles]

Prerequisites

• Python >= 3.9
• CDO >= 1.9.8 (recommended: >= 2.0.0)
• Optional: geopandas >= 0.10.0, shapely >= 2.0.0 (for shapefile masking)

🔍 Migration Guide

For Users with Custom Grid Types

If you have custom or non-standard grids, this release makes them easier to work with:

Before v1.1.2:

# Unknown attributes were silently ignored or caused errors
result = cdo.griddes("custom_grid.nc")
grid = result.primary_grid
# custom_projection_param - LOST! ❌

v1.1.2 and Later:

# Unknown attributes preserved in raw_attributes
result = cdo.griddes("custom_grid.nc")
grid = result.primary_grid

# Access custom attributes
if grid.raw_attributes:
    custom_param = grid.raw_attributes.get("custom_projection_param")
    print(f"Custom parameter: {custom_param}")  # ✅ Preserved!

For Users with Rotated Grids

Before v1.1.2:

# Projection info was not parsed
result = cdo.griddes("EUR-11_rotated.nc")
grid = result.primary_grid
# grid_north_pole_longitude - Not available ❌

v1.1.2 and Later:

# Full projection support
result = cdo.griddes("EUR-11_rotated.nc")
grid = result.primary_grid

if grid.is_rotated:
    print(f"Pole: ({grid.grid_north_pole_longitude}, "
          f"{grid.grid_north_pole_latitude})")  # ✅ Available!

---

🙏 Acknowledgments

Special thanks to:

• The CDO team for comprehensive grid type support
• CF Conventions community for standardized projection metadata
• Regional climate modeling community for rotated pole projection use cases

---

🐛 Known Issues & Limitations

Current Limitations

1. Curvilinear Coordinates: Full coordinate arrays (xvals, yvals) for curvilinear grids are typically stored in the NetCDF file itself, not in griddes output. The parser captures size and metadata, but full 2D arrays should be read from the file using xarray.

2. Reduced Gaussian Rowlon: Some reduced Gaussian grids may not include the rowlon array in griddes output. In these cases, grid.rowlon will be None.

Future Improvements

• Add support for more projection types (Lambert Conformal, Mercator, etc.)
• Add GridSpec factory methods for common Gaussian grids
• Expand documentation with more regional model examples
• Add validation for projection parameter combinations

For complete details, see CHANGELOG.md.

---

📞 Support & Feedback

Found a bug? Please open an issue on GitHub:
https://github.com/NarenKarthikBM/python-cdo-wrapper/issues

Have questions? Check out:

• Documentation: README.md
• Examples: examples/
• Migration Guide: MIGRATION_GUIDE.md

---

🔗 Resources

• GitHub Repository: https://github.com/NarenKarthikBM/python-cdo-wrapper
• PyPI Package: https://pypi.org/project/python-cdo-wrapper/
• CDO Documentation: https://code.mpimet.mpg.de/projects/cdo/
• CF Conventions (Projections): http://cfconventions.org/cf-conventions/cf-conventions.html#grid-mappings-and-projections

---

Thank you for using python-cdo-wrapper! 🎉

---

Note: This release adds comprehensive grid type support while maintaining full backward compatibility. All existing code will continue to work as before.