Add CMIP7 Support: Global Attributes

[siligam]

Summary

This PR adds comprehensive CMIP7 support to pycmor, including global attributes generation, YAML configuration validation, and extensive user documentation. All changes maintain backward compatibility with CMIP6.

Changes

🎯 Core Functionality

Global Attributes (src/pycmor/std_lib/global_attributes.py)

• Added CMIP7GlobalAttributes class extending GlobalAttributes
• Fixed get_frequency(): Falls back to rule_dict when drv is None
• Updated get_Conventions(): Returns "CF-1.10 CMIP-7.0" for CMIP7
• Fixed variant label indices: Convert to strings for netCDF compliance
• Added methods: get_product(), get_data_specs_version(), get_sub_experiment_id()

YAML Validation (src/pycmor/core/validate.py)

• Added CMIP7 fields to general schema:
  • CMIP7_DReq_metadata (optional, for Data Request metadata file)
  • Made CMIP_Tables_Dir optional (not needed for CMIP7)
• Added CMIP7 fields to rules schema:
  • compound_name (CMIP7 compound names like atmos.tas.tavg-h2m-hxy-u.mon.GLB)
  • realm, frequency, table_id (can come from compound_name)
  • grid, nominal_resolution (recommended metadata)
  • institution_id (required for CMIP7)
• Fixed typo: instition_id → institution_id (kept old one for backward compatibility)
• Made flexible: cmor_variable optional when compound_name provided

📚 Documentation

New Documentation

• doc/cmip7_configuration.rst (573 lines)

  • Complete YAML configuration guide for CMIP7
  • Minimum requirements and optional fields explained
  • Multiple complete examples (atmospheric, ocean, unstructured grids)
  • CMIP6 to CMIP7 migration guide
  • Troubleshooting section
  • Validation and running instructions

• examples/cmip7-example.yaml (98 lines)

  • Working CMIP7 configuration examples
  • Examples with and without compound names
  • Atmospheric and ocean variables
  • Unstructured grid configuration

Updated Documentation

• doc/quickstart.rst: Added note directing CMIP7 users to new guide
• doc/index.rst: Added cmip7_configuration to table of contents
• README.rst: Updated to mention CMIP6 and CMIP7 support

✅ Testing

Unit Tests (tests/unit/test_cmip7_global_attributes.py)

• 11 tests covering CMIP7 global attributes:
  • Global attributes structure and presence
  • CMIP7-specific attributes (mip_era, Conventions, product)
  • Format validation (creation_date, tracking_id, variant_label)
  • Type validation (all attributes are strings)
  • Source type derivation from CVs
  • License text correctness
  • Further info URL generation

Integration Tests (tests/integration/test_cmip7_yaml_validation.py)

• 10 tests covering YAML validation:
  • Minimal CMIP7 configuration
  • Full CMIP7 configuration with optional fields
  • CMIP7 without compound names
  • CMIP6 backward compatibility
  • Field validation (compound_name, institution_id, variant_label)
  • YAML file parsing

All 21 tests passing ✅

Key Features

CMIP7 Compound Names

# Instead of:
cmor_variable: tas
frequency: mon
realm: atmos
table_id: Amon

# Use:
compound_name: atmos.tas.tavg-h2m-hxy-u.mon.GLB

Minimum YAML Configuration

general:
  cmor_version: "CMIP7"
  CV_Dir: "/path/to/CMIP7-CVs"
  CMIP7_DReq_metadata: "/path/to/dreq_metadata.json"

rules:
  - name: tas
    compound_name: atmos.tas.tavg-h2m-hxy-u.mon.GLB
    model_variable: temp2
    inputs:
      - path: /path/to/data
        pattern: "*.nc"
    
    # 5 Required identifiers
    source_id: AWI-CM-1-1-HR
    institution_id: AWI              # ← NEW in CMIP7!
    experiment_id: historical
    variant_label: r1i1p1f1
    grid_label: gn
    
    output_directory: /path/to/output

CMIP6 vs CMIP7 Comparison

Field	CMIP6	CMIP7
Variable	cmor_variable: tas	compound_name: atmos.tas...
Metadata source	CMOR Tables	Data Request API
Frequency	From table	From compound_name or explicit
Realm	model_component: atmos	realm: atmos (or from compound_name)
Institution	Optional	institution_id required
Variant label	r1i1p1 (3 parts)	r1i1p1f1 (4 parts)
Tables directory	Required	Optional

Backward Compatibility

✅ All CMIP6 configurations continue to work

• Existing CMIP6 YAML files validate and run without changes
• CMIP6 tests still passing
• No breaking changes to existing functionality

Migration Path

Users can migrate from CMIP6 to CMIP7 by:

1. Update general section:

   cmor_version: "CMIP7"
   CV_Dir: "/path/to/CMIP7-CVs"
   CMIP7_DReq_metadata: "/path/to/dreq_metadata.json"

2. Update each rule:

   • Add compound_name (recommended) or keep cmor_variable + frequency + realm + table_id
   • Add institution_id (required)
   • Add grid and nominal_resolution (recommended)

See doc/cmip7_configuration.rst for complete migration guide.

Testing

# Run unit tests
pytest tests/unit/test_cmip7_global_attributes.py -v

# Run integration tests
pytest tests/integration/test_cmip7_yaml_validation.py -v

# Validate YAML configuration
pycmor validate config examples/cmip7-example.yaml

Documentation

• Configuration Guide: doc/cmip7_configuration.rst
• Data Request API: doc/cmip7_interface.rst (existing)
• Controlled Vocabularies: doc/cmip7_controlled_vocabularies.rst (existing)
• Example YAML: examples/cmip7-example.yaml

Checklist

• Code follows project style guidelines
• Tests added and passing (21 tests)
• Documentation updated
• Backward compatibility maintained
• CMIP6 tests still passing
• Example configurations provided
• Migration guide included

Additional Notes

This PR focuses on the YAML configuration workflow, which is how most users interact with pycmor. The implementation:

• Validates CMIP7-specific fields
• Generates all 29 required CMIP7 global attributes
• Provides comprehensive documentation for users
• Maintains full backward compatibility with CMIP6

Future work could include:

• Integration with CMIP7 Data Request API for runtime variable queries
• DRS (Data Reference Syntax) updates for CMIP7
• Additional CMIP7-specific pipelines or processing steps

[mandresm]

Closes #210 #198

[pgierz]

Closing, all features are in prep-release