v0.17.0

Earth2Studio general release 0.17.0

🚨 Breaking Changes 🚨

• NNJA and GDAS data sources had breaking behavior and constructor API changes to help with consistency between them.

📕 Changelog

🌎 Added

• Added GHCN hourly data source (GHCNHourly), superseding the deprecated ISD source
• Added EarthMover ERA5 0.25 degree reanalysis data source
• Added EarthMover IFS 0.1 degree data source and forecast source hosted by BrightBand
• Added async_workers and retries parameters to GFS / GFS_FX, HRRR / HRRR_FX,
  GEFS_FX / GEFS_FX_721x1440, CFS_FX / CFS_FX_Flux and NCAR_ERA5 data sources
• Added shared obstore byte-range helpers (obstore_store_from_url,
  obstore_read_range, obstore_fetch_to_cache) in earth2studio.data.utils
• Added dynamical.org analysis and forecast data sources, reading anonymous Icechunk
  repositories: DynamicalAIFS, DynamicalAIFS_ENS, DynamicalGFS, DynamicalGEFS,
  DynamicalHRRR, DynamicalMRMS, DynamicalGFS_FX, DynamicalGEFS_FX,
  DynamicalHRRR_FX, DynamicalICON_EU_FX, DynamicalIFS_ENS,
  DynamicalIFS_ENS_FX,
  DynamicalAIFS_FX and DynamicalAIFSENS_FX.
• Added Aurora v1.5 deterministic and ensemble model wrapper (Aurora1p5, Aurora1p5Ensemble)
• Added StormCast CONUS prognostic model (StormCastCONUS)
• Added DataReplay for replaying DataSource and ForecastSource data through the
  prognostic iterator interface.
• Added NNJA satellite observation data frame source (NNJAObsSat)
• Added StormScope NSRDB solar irradiance (GHI) estimation model (StormScopeDxNSRDB)
• Added COSMO-REA downscaling diagnostic model (CorrDiffCosmoEra5) for diffusion downscaling
  of ERA5 to COSMO-REA6 (6 km) and COSMO-REA2 (2.2 km) over Europe.

⚙️ Changed

• Renamed GHCNLexicon to GHCNDailyLexicon for consistency with the new hourly lexicon
• Updated MeteosatFCI reader and lexicon to include all channels.
• Updated StormScope model package to use improved higher resolution checkpoints. Model
  now defaults to using 3 km and 10 minute spatiotemporal resolution, and includes
  predictions for GOES GLM Lightning density.
• Migrated GFS / GFS_FX data sources from s3fs to obstore for index and byte-range
  GRIB fetches; downloads now use bounded concurrency with retry on transient errors
• Migrated the remaining GRIB byte-range data sources (HRRR / HRRR_FX, GEFS_FX /
  GEFS_FX_721x1440, CFS_FX / CFS_FX_Flux, NCAR_ERA5) from s3fs/gcsfs to obstore with
  bounded concurrency and retry on transient errors
• Migrated HTTP-backed GRIB sources (GFS ncep, HRRR nomads, CFS nomads) to
  obstore HTTPStore against the NOMADS HTTPS endpoint (GFS ncep previously used
  FTP); no fsspec fallbacks remain in the GRIB byte-range sources
• Refactored UFS observation sources (UFSObsConv, UFSObsSat) onto the shared
  obstore byte-range helpers
• Zarr-reading data sources (ARCO, WB2ERA5 and other WeatherBench 2 sources, and
  the rx prescriptive sources) now read via obstore-backed zarr stores instead of
  fsspec
• UFS observation sources (UFSObsConv, UFSObsSat) now tolerate missing diag files
  by warning and skipping instead of erroring
• Updated the OPERA data source to represent undetect values as -99.0, while
  retaining NaN for no-data values.
• NNJA Obs data source now accepts any time / tolerance rather than 6-hour strides
• Renamed NomadsGDASObsConv max_workers parameter to async_workers for
  consistency with other observation data sources

🔨 Fixed

• Fixed PrecipitationAFNOv2 and WindgustAFNO passing latitude and longitude in
  swapped order to cos_zenith_angle, which produced an incorrect solar-zenith-angle
  input channel.
• Fixed incorrect interpolation values at the lower grid edge when an output
  coordinate matched the first input coordinate.
• Corrected the PrecipitationAFNOv2 docstring: the model predicts precipitation
  accumulated over the following six hours [t, t+6h], not the prior six hours.
• Fixed DerivedRH mixed-phase saturation blend clipping the liquid-water fraction
  ratio to 1.2 instead of 1.0 before squaring, which let the effective weight reach
  1.44 and inflated relative humidity above freezing.
• Changed ISD schema source type to string since the field is alphanumeric. Enforced
  float32 dtypes for lat, lon, elev, and observation.
• Fixed NNJA observation sources blocking the shared fsspec IO loop with
  CPU-bound PrepBUFR decode work, which stalled concurrent fetches from other
  data sources.
• Fixed ACE2 distributed inference failures caused by nondeterministic variable ordering
  across MPI ranks.
• Improved ACE2ERA5 inference performance by caching yearly forcing values
• Fixed ACE2ERA5 forcing data retrieval for static variables requested across
  multiple times.
• Added CF-convention scale/offset when retrieving JPSS data.

📦 Dependencies

• Removed multi-storage-client from the data optional dependency group,
  succeeded by obstore
• Added icechunk>=2.0.0 to the data optional dependency group (Python ≥3.12 only)

Special thank you to our external contributors:

• Ryan Abernathey
• Amanda Dufek
• Marshall
• Ian Palacín

v0.16.0

Earth2Studio general release 0.16.0

🚨 Breaking Changes 🚨

• No notable breaks this release 🎉

📕 Changelog

🌎 Added

• Added AIFS 2.0 prognostic model (AIFS2) with wave and 10 hPa pressure level support
• Added AIFS 2.0 ensemble prognostic model (AIFS2ENS) with stochastic noise injection
• Added U-CAST prognostic model (UCast) with 1.5-degree global ERA5
  forecasting support
• Added wave variables to IFS data source for AIFS2 support
• Added NCEP CFSv2 operational forecast data sources for the pressure-level
  pgbf and surface-flux flxf products (CFS_FX, CFS_FX_Flux), backed by
  either NOMADS (real-time) or the AWS Open Data mirror (archive)
• Added NCEP CFSv2 6-hourly 9-month reforecast data sources
  (CFS_Reforecast_FX, CFS_Reforecast_FX_Flux), covering 1981-12-12
  through 2011-03-27 with cycles every 5 days, served from the NCEI HTTPS
  archive
• Added IBTrACS tropical cyclone track DataFrame source (IBTrACS)
• Added checkpoint/session utilities and restart support for deterministic,
  diagnostic, and ensemble inference workflows
• Added EUMETNET OPERA European weather radar composite DataSource for DBZH
  reflectivity, rain rate, and 1-hour accumulation (OPERA)
• Added support for cumulative variables in ARCO data source
• Added DLESyM-v0-ISCCP-ERA5 climate model

⚙️ Changed

• UFS GSI observation sources (UFSObsConv, UFSObsSat) now fetch from S3 via native
  obstore instead of s3fs to avoid the Python-GIL bottleneck that caps fsspec's
• Renamed AIFS runoff and snowfall variables to ro06 and sf06 and added six-hour
  accumulated IFS/AIFS data aliases.
• Automatic test skipping for missing optional dependencies via
  pytest_ignore_collect hook in test/conftest.py

🔨 Fixed

• Fixed ARCO data source ARCO_TIME_STOP fallback to 2025-12-31,
  reflecting the most recent available data in the bucket
• Fixed ZarrBackend chunk metadata reload to skip coordinate arrays when reopening Zarr stores.

📦 Dependencies

• Added obstore>=0.8 to core dependency list
• Removed aifs and aifsens from the [all] extra due to dependency conflicts with
  aifs2 (incompatible anemoi-models versions).

Special thank you to our external contributors:

• Minh Vu
• vaibhavi
• Travis A. O'Brien

v0.15.0

Earth2Studio general release 0.15.0

🚨 Breaking Changes 🚨

• No notable breaks this release 🎉

📕 Changelog

🌎 Added

• Added Himawari-8/9 AHI ISatSS L2 Full Disk satellite data source (HimawariAHI)
• Added GHCN-Daily global station observation data frame source (GHCNDaily)
• Added NNJA conventional (in-situ + GPS RO) observation data source (NNJAObsConv)
• Added GOES Geostationary Lightning Mapper L2 LCFA event data source (GOESGLM)
• Added real-time GDAS conventional observation data source (NomadsGDASObsConv)
• Added quality field to E2STUDIO_SCHEMA for observation QC markers
• Added Climate in a Bottle tropical cyclone guidance and odds ration example
• Added Orbit-2 precipitation downscaling model

⚙️ Changed

• UFS Satellite Obs source is now the only one that provides the UFS specific channel_index and general sensor_index fields, all other now provide sensor_index only which can be used to consistently map from UFS to L1 products
• Added wavenumber (cm⁻¹) field to all satellite schemas
• Changed channel_index to sensor_index in E2STUDIO_SCHEMA
• Disabled Atlas example from documentation build due to slow performance

🔨 Fixed

• Fixed potential uint16 underflow in UFS channel index expansion
• Fixed S3 upload bug in server utilities
• Fixed pres obs from UFS to be Pascal units
• Fixed chunk downloading race condition in file system cache for Zarr data sources
• Fixed 180° longitude misalignment in ECMWF open-data sources for some GRIB files with non-standard lon origin

📦 Dependencies

• Removed nested_asyncio for Python 3.14 compatibility and updated async data sources
• Bumping minimum ecmwf-opendata version to 0.3.29 to resolve IFS data request errors

Special thank you to our external contributors:

• Parth Katlana
• Isaac Lyngaas
• Viktor Cikojević

v0.14.0

Earth2Studio general release 0.14.0

🚨 Breaking Changes 🚨

• Following PyTorch 2.11, Earth2Studio dependencies now point to CUDA 13.X installs by default. Later CUDA versions are still usable, however special install procedures might be required.

📕 Changelog

🌎 Added

• Added GenCast 1 degree Mini model
• Added CAMS Global atmospheric composition forecast data source (CAMS_FX)
• Added MetOp AMSU-A Level 1B brightness temperature data source (MetOpAMSUA)
• Added MetOp AVHRR Level 1B radiance/brightness temperature data source (MetOpAVHRR)
• Added JPSS ATMS Level 1 BUFR brightness-temperature data source (JPSS_ATMS)
• Added JPSS CrIS FSR Level 1 spectral radiance data source (JPSS_CRIS)
• Added MetOp IASI Level 1C infrared brightness temperature data source (MetOpIASI)
• Added NClimGrid daily CONUS gridded climate data source (NClimGridDaily)
• Added MTG-I FCI Level-1C Full Disk satellite radiance data source (MeteosatFCI)

🔨 Fixed

• Fixed source code links in documentation

📦 Dependencies

• Added eumdac>=3.1.0 to data optional dependency group for EUMETSAT Data Store access
• Moved package dependencies defaults to CUDA 13.0 following torch
• Moving suggested Python version to 3.13
• Pointing physicsnemo to git main source for torch 2.11 compatibility
• Updated ACE2 dependency hash with setuptools fix
• Updated models using PhysicsNeMo to compliance with nvidia-physicsnemo>=2.0

Special thank you to our external contributors:

• Jerry Lin
• Björn Bethge
• Iype Eldho

v0.13.0

Earth2Studio general release 0.13.0

🚨 Breaking Changes 🚨

• Renamed tolerance parameter in ISD data source to time_tolerance
• Changed GraphCastOperational and GraphCastSmall latitude input / output to be [90,-90]
• Removed device from cbottle SR load_model API

📕 Changelog

🌎 Added

• Added NOAA UFS observation dataframe sources for satellite and conventional data
• Added Earth2Studio base schema for dataframe sources
• Added Planetary Computer data source for ECMWF IFS analysis data
• Added accumulated variable support to NCAR ERA5 data source
• Added routine to tile tensors to higher-dimensional tensors
• Added routine to concatenate tensors along specified coordinate dimension
• Added Planetary Computer data source for GOES cloud and moisture imagery
• Added ability to have seperate data and model cache locations via env variables
• Added random dataframe source
• Added base reflectivity to MRMS data source
• Added fetch_dataframe utility function
• Added data assimilation model class
• Added equirectangular interpolation data assimilation model
• Added StormCast SDA model
• Added beta serve utils with inference server and client implementations
• Added HealPix data assimilation (HealDA) model
• Added energy_score metric for multivariate ensemble forecast verification

⚙️ Changed

• Changed available date for ARCO data source to be dynamically updated
• Changed ISD data frame return to master schema
• handshake_coords is now accepting list of dimensions while remaining backwards-compatible
• Updated CBottle infill to mixture of model checkpoints
• Updated GraphCastOperational and GraphCastSmall latitude input / output to be [90,-90]
• Updated GraphCast models to support multiple time inputs, multiple times will be looped not batched
• Renamed tolerance parameter in ISD data source to time_tolerance

❌ Removed

• Removed device from cbottle SR load_model API

🔨 Fixed

• Fixed timezone bug in CBottleVideo that converted UTC time to local system time
• Bug in cbottle datasource resulting identical samples for multiple samples
• Bug in StormCast loading out-of-date model package, introduced in 5518edecbabee371c824b34f0f2ec269a4d6094f
• Bug in spherical perturbations which did not use lmax from the SHT transform

📦 Dependencies

• Added pyarrow explicitly to core dependencies
• Updated CBottle repo to NVlabs location
• Updated Makani dependency hash to later version with pyproject toml fix
• Updated ACE2 dependency hash with setuptools fix
• Updated models using PhysicsNeMo to compliance with nvidia-physicsnemo>=2.0

Special thank you to our external contributors:

• Jerry Lin
• abhaygoudannavar

v0.12.1

Earth2Studio patch release 0.12.1

🚨 Breaking Changes 🚨

• No notable breaks this release 🎉

📕 Changelog

⚙️ Changed

• Moved (most) cBottle checkpoints to huggingface repo
• Added config.json download for HF models to track downloads

📦 Dependencies

• Added onnxscript to onnx models to add support for newer torch versions
• Moved lock file / CI system to torch 2.9.1

v0.12.0

Earth2Studio general release 0.12.0

🚨 Breaking Changes 🚨

• No notable breaks this release 🎉

📕 Changelog

🌎 Added

• Added CorrDiff CMIP6->ERA5 diagnostic model
• Added derived TCWV diagnostic model
• Added verbose option for inference workflows in run.py to toggle off tqdm
• Added Atlas prognostic model
• Added StormScope GOES and MRMS models

⚙️ Changed

• Removed AIFS-ENS invariants from model input / outputs and moved into model wrapper
• CDS and NCAR lexica have been updated with variables for AIFS-ENS
• Switch grib reads in HRRR data sources from xarray to pygrib to fix memory leak
• Updated AIFS model to 1.1 checkpoint version
• Updated GFS lexicon to include composite reflectivity
• Moved most data sources from cfgrib to pygrib

🔨 Fixed

• Made tp in HRRR and GFS units m for package consistency
• Fixed batched inference support for AIFS and AIFS ENS

📦 Dependencies

• Added pygrib to core dependency group
• Moved cfgrib to optional dependency group
• Resolved conflict between AIFS and AIFS ENS groups
• Removed old numcodecs limit

Special thank you to our external contributors:

• Yuki H. Takano

v0.11.0

Earth2Studio general release 0.11.0

🚨 Breaking Changes 🚨

• Removed IMERG data source
• ECMWF data sources for IFS and AIFS have been updated to separate classes for getting analysis data and forecast data

📕 Changelog

🌎 Added

• Added general PlanetaryComputerData source for pulling Planetary Computer STAC assets
• Added ECMWF AIFSENS ensemble prognostic model wrapper
• Added MRMS data source
• Added IFS and IFS-ENS initial state datasources
• Added local datasource example with Microsoft Aurora model
• Added NOAA's Integrated Surface Database (ISD) data frame source

⚙️ Changed

• Removed tp06 field from Graphcast operational model
• Removed static fields from Graphcast model input / outputs
• Moved StormCast and DLESyM checkpoints to Huggingface
• Change previous IFS, IFS_ENS, AIFS and AIFS_ENS sources to IFS_FX, IFS_ENS_FX, AIFS_FX
  and AIFS_ENS_FX to reflect that they are forecast sources
• Support multiple x, coords pairs in batch function decorator

❌ Removed

• Removed support for returning multiple samples at once from IFS ENS data sources
• IMERG datasource

🔨 Fixed

• Vertical wind variables in GFS / GEFS to be Pa s-1

📦 Dependencies

• Removed Aurora-fork option which had a temporary patch for package conflict that is
  resolved in the source aurora package
• Capped PhysicsNeMo to below 2.0

Special thank you to our external contributors:

• Rodrigo Almeida
• David Ahijevych

v0.10.0

Earth2Studio general release 0.10.0

🚨 Breaking Changes 🚨

• Dropped Python 3.10 support

📕 Changelog

🌎 Added

• Added Mic PlanetaryComputerData source for pulling Planetary Computer STAC assets
• Added ECMWF AIFSENS ensemble prognostic model wrapper
• Added MRMS data source
• Added IFS and IFS-ENS initial state datasources
• Local datasource example
• Added TimeWindow datasource wrapper for fetching data at multiple time offsets
• Added NOAA's Integrated Surface Database (ISD) data frame source

⚙️ Changed

• Updated CBottle data source to mixture of experts models, added additional parameters
  to load_model to align with other cBottle models.
• Fixed duplicate geo-potential at surface ids in AIFS, IFS data source and orography
  source, orography is denoted by lower case z
• Updated package caching default to None, which will default to true for remote
  packages and false for local packages
• Changed IFS to be a forecast data source.
• InferenceOutputSource can now accept an Xarray Dataset directly as an argument
• InferenceOutputSource returns data consistently in ("time", "lead_time", "variable")
  order
• Added support for ERA5 model levels and additional variables in ARCO data source
• Changed the HRRR_X and HRRR_Y coordinates of the HRRR data source to match the native
  LCC coordinates
• Updated CorrDiffTaiwan model wrapper to use latest PhysicsNeMo APIs.

⚠️ Deprecated

• Added deprecated warning for IMERG data source. This will be removed in the next
  release.

🔨 Fixed

• Fixed typo: InferenceOuputSource renamed to InferenceOutputSource
• StormCast ensures that conditioning variables are in the correct order
• NetCDFBackend unit change to ensure timedeltas are correctly decoded by xarray
• GFS data source for early 2021 dates
• Updated corrdiff wrapper for newer physicsnemo performance improvements

📦 Dependencies

• Dropped support for Python 3.10
• Bumped CBottle (and Earth2Grid) versions
• Capped JAX version, due to numpy 2.0 requirement conflicting with NV PyTorch containers
• Temp limit globus-sdk for intake-esgf
• Added multi-storage client into data dependency group

Special thank you to our external contributors:

• Joshua-Elms
• Rodrigo Almeida

v0.9.0

Earth2Studio general release 0.9.0

🚨 Breaking Changes 🚨

• This is the last release with Python 3.10 support! Please upgrade Python version to at least 3.11 for main branch and future releases.

📕 Changelog

🌎 Added

• Async Zarr IO backend with non-blocking write calls
• Different compression codec support in the ZarrBackend with zarr_codecs parameter
• IO performance example
• Unified CorrDiff Wrapper
• Added UV script dependencies to all examples
• New metrics: Brier score, fractions skill score, log spectral distance, mean absolute error.
• Option to compute error of ensemble mean in rmse and mae.
• Added FourCastNet 3 model

⚙️ Changed

• Zarr IO Backend now uncompressed by default
• Allow HCBV perturbation to handle constant outputs (like land sea mask, or
  geopotential at surface)
• test/models/dx/test_corrdiff.py is now test/models/dx/test_corrdiff_taiwan.py
• Updated APIs for optional dependency managment utils with improved error messages
• Allow Zarr backends to user datetime and timedelta arrays for Zarr 3.0

🔨 Fixed

• Incorrect datetime utc timezone calculation in SFNO wrapper was fixed.
• DLWP output coords lead_time array to have proper shape
• Fixed data sources using GCFS throwing error at end of script from aiohttp session
  clean up
• Fixed HRRR_FX valid lead time check for date times not on 6 hour interval
• Removed time limits for WB2 climatology data source

📦 Dependencies

• Adding rich to core dependencies
• Updated torch-harmonics to 0.8.0
• Updated Makani to 0.2.1

Special thank you to our external contributors:

• Rodrigo Almeida