Skip to content

v1.1.1 - BinaryOPQuery execution bug fix

Latest
Latest

Choose a tag to compare

View all tags
@NarenKarthikBM NarenKarthikBM released this 17 Dec 11:08
V1.1.1
6d79728
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Release Notes: v1.1.1

🐛 Bug Fix Release

This is a critical bug fix release that corrects the command generation for binary operations in CDO queries.

🔧 What's Fixed

Binary Operation Command Generation

The Issue:
Version 1.1.0 incorrectly used bracket notation [ ... ] for binary operations, which is not valid CDO syntax for binary operators. This caused "too many inputs" errors when using nested binary operations like ifthen inside sub.

The Fix:
Binary operators (add, sub, mul, div, min, max) now use CDO's operator chaining syntax instead of brackets.

Command Generation - Before vs After:

Scenario v1.1.0 (Incorrect) v1.1.1 (Correct)
Left has operators cdo -sub [ -yearmean file1 ] file2 ❌ cdo -sub -yearmean file1 file2 ✅
Both have operators cdo -sub [ -yearmean file1 ] [ -timmean file2 ] ❌ cdo -sub -yearmean file1 -timmean file2 ✅
Nested binary ops Failed with "too many inputs" ❌ cdo -sub -ifthen mask.nc data.nc clim.nc ✅

Key Points:

  • ✅ Binary operators use operator chaining (no brackets)
  • ✅ Operators are applied to their respective input files left-to-right
  • ✅ Supports nested binary operations (e.g., ifthen inside sub)
  • ✅ All operations execute in a single CDO command (no temporary files)
  • 📝 Bracket notation is only for variadic operators (merge, cat) - not binary operators

💥 Breaking Changes

None - This is a bug fix release that makes the package work correctly. If you were using binary operations in v1.1.0, they may not have been working properly.

🎯 What's Working Now

Example 1: Simple Anomaly Calculation

from python_cdo_wrapper import CDO, F

cdo = CDO()

# Calculate anomalies - works correctly now!
anomaly = cdo.query("data.nc").sub(F("climatology.nc")).compute()
# Generates: cdo -sub data.nc climatology.nc

Example 2: Operations on Both Sides

# With operators on both operands - single command!
result = (
    cdo.query("data.nc")
    .select_var("tas")
    .year_mean()
    .sub(F("climatology.nc").time_mean())
    .compute()
)
# Generates: cdo -sub -yearmean -selname,tas data.nc -timmean climatology.nc

Example 3: Nested Binary Operations (The Bug Case)

# The original issue: ifthen inside sub
masked_data = cdo.query("data.nc").ifthen(F("mask.nc"))
anomaly = masked_data.sub(F("climatology.nc")).compute()
# Generates: cdo -sub -ifthen mask.nc data.nc climatology.nc
# This now works correctly! ✅

Example 4: Complex Multi-Step Operations

# Standardized anomaly: (data - mean) / std
std_anomaly = (
    cdo.query("data.nc")
    .sub(F("climatology.nc"))
    .div(F("std_dev.nc"))
    .compute()
)
# Generates: cdo -div -sub data.nc climatology.nc std_dev.nc

🧪 Testing

New Integration Tests

Added 5 comprehensive integration tests that execute actual CDO commands with real NetCDF files:

  1. ✅ Binary operations with operators on left operand
  2. ✅ Binary operations with operators on both sides
  3. ✅ Nested binary operations (ifthen inside sub) - the original bug case
  4. ✅ Anomaly calculations (data - time_mean)
  5. ✅ Chained binary operations (sub then div)

All tests pass with flying colors on:

  • ✅ Ubuntu (Python 3.9, 3.10, 3.11, 3.12)
  • ✅ macOS (Python 3.9, 3.10, 3.11, 3.12)

📦 Installation & Upgrade

Upgrading from v1.1.0

# Upgrade to the bug fix release
pip install --upgrade python-cdo-wrapper

# Or with shapefile support
pip install --upgrade python-cdo-wrapper[shapefiles]

New Installation

# Core package
pip install python-cdo-wrapper

# With shapefile support
pip install python-cdo-wrapper[shapefiles]

Prerequisites

  • Python >= 3.9
  • CDO >= 1.9.8 (recommended: >= 2.0.0)
  • Optional: geopandas >= 0.10.0, shapely >= 2.0.0 (for shapefile masking)

🔗 Technical Details

Why Operator Chaining?

CDO processes operators right-to-left and applies them to their respective input files. Binary operators (which take exactly 2 inputs) use operator chaining syntax, not bracket notation.

From the CDO Tutorial:

"Operators can be chained directly to their input files without brackets for binary operations."
CDO Tutorial: Combining Operators

Example Flow:

cdo -sub -yearmean file1.nc -timmean file2.nc output.nc

CDO evaluation:

  1. Apply -yearmean to file1.nc → result1
  2. Apply -timmean to file2.nc → result2
  3. Apply -sub with (result1, result2) → output

📝 Documentation Updates

All documentation has been updated to reflect the correct syntax:

  • ✅ README.md - Updated binary operation examples
  • ✅ CHANGELOG.md - Documented the fix with examples
  • ✅ MIGRATION_GUIDE.md - Updated v1.0.0 examples
  • ✅ .github/copilot-instructions.md - Updated all references
  • ✅ examples/v1.0.0-demo.ipynb - Updated example outputs

🙏 Acknowledgments

Special thanks to:

  • The CDO team for excellent documentation on operator chaining

🐛 Known Issues & Limitations

None in this release!

This bug fix resolves all known command generation issues with binary operations.

Future Improvements

  • Consider adding more optimization hints for large-scale operations
  • Expand documentation with more climate science use cases
  • Add performance benchmarks for operator chaining

📊 Full Changelog

[v1.1.1] - 2025-12-17

Fixed

  • BinaryOpQuery command generation: Corrected to use CDO's operator chaining syntax (removed incorrect bracket notation)
    • Binary operators (sub, add, mul, div, etc.) now use standard operator chaining
    • ❌ Old (incorrect): cdo -sub [ -yearmean file1 ] [ file2 ]
    • ✅ New (correct): cdo -sub -yearmean file1 -timmean file2
    • Operators are applied directly to their respective input files from left to right
    • Supports nested binary operations (e.g., ifthen inside sub)
    • All operations execute in single CDO command
    • Bracket notation is NOT used for binary operators - only for variadic operators (merge, cat)

Testing

  • Added 5 integration tests for binary operations with actual CDO execution
  • All tests pass on Ubuntu and macOS with Python 3.9-3.12

Documentation

  • Updated all documentation to reflect correct operator chaining syntax
  • Added technical notes explaining CDO's evaluation model
  • Updated examples in README, MIGRATION_GUIDE, and notebooks

For complete details, see CHANGELOG.md.

📞 Support & Feedback

Found a bug? Please open an issue on GitHub:
https://github.com/NarenKarthikBM/python-cdo-wrapper/issues

Have questions? Check out:

🔗 Resources

Thank you for using python-cdo-wrapper! 🎉

Note: This release fixes a critical bug in v1.1.0. All users on v1.1.0 are encouraged to upgrade immediately.

Assets 2
Loading