FINAM v1.1

FINAM is an open-source component-based model coupling framework for environmental models.
It aims at enabling bi-directional online couplings of models for different compartments like geo-, hydro-, pedo- and biosphere.

The framework is built in Python, with well-defined interfaces for data exchange.
This approach allows for coupling of models irrespective of their internal structure, architecture or programming language.

Resources

• FINAM homepage
• FINAM documentation
• FINAM source code
• FINAM GitLab group, containing further related projects

Changelog

Features

• Adds Masking, UnMasking and Clip adapters for explicit mask handling and grid clipping (!294)
• Adds ToCRS and ToUnstructured adapters for CRS conversion and conversion to unstructured grids (!294)
• Adds equal_crs helper to the data submodule and automatic axis attributes from CRS information (!294)
• Adds unit handling to Callback and Scale adapters (!294)
• Adds mask support for WeightedSum component (!296)

Bugfixes

• Fixes mask propagation through regridding adapters and through input/output info exchange (!293, !295)
• Fixes GridToValue and ValueToGrid metadata handling for masks (!294)
• Fixes CRS transformations in regridding adapters to keep axis order stable (!294)
• Fixes regridding error types and wrong error messages from regrid adapters (!294, !300)
• Fixes compatibility with NumPy's changed reshape parameter name (!297)
• Fixes transformation between compatible grids (!303, !305)
• Fixes generation order of CRS-derived axis attributes (!301)

Documentation

• Adds blog posts for the FINAM v1.0 release and model-code changes for coupling with FINAM (!288, !292)
• Fixes minor spelling, example and link issues in documentation (!294, !298)

FINAM v1.0.1

A minor bugfix release.

FINAM is an open-source component-based model coupling framework for environmental models.
It aims at enabling bi-directional online couplings of models for different compartments like geo-, hydro-, pedo- and biosphere.

The framework is built in Python, with well-defined interfaces for data exchange.
This approach allows for coupling of models irrespective of their internal structure, architecture or programming language.

Resources

• FINAM homepage
• FINAM documentation
• FINAM source code
• FINAM GitLab group, containing further related projects

Changelog

Bug fixes

• make GridToValue and ValueToGrid compatible with masked data (!291)

FINAM v1.0

The first stable release of FINAM: Version 1.0 🎉

FINAM is an open-source component-based model coupling framework for environmental models.
It aims at enabling bi-directional online couplings of models for different compartments like geo-, hydro-, pedo- and biosphere.

The framework is built in Python, with well-defined interfaces for data exchange.
This approach allows for coupling of models irrespective of their internal structure, architecture or programming language.

Resources

• FINAM homepage
• FINAM documentation
• FINAM source code
• FINAM GitLab group, containing further related projects

Changelog

Breaking changes

• submodule modules renamed to components for consistency (!289)
• argument modules renamed to components in Composition for consistency (!289)
• Components now implement _next_time instead of the property next_time for consistency (!283)
• All fields of Composition are now private (!273)
• Input.source is private, Input.get_source() becomes property Input.source, Input.set_source becomes a getter (!273)
• Output.targets is private, Output.get_targets() becomes property Output.targets (!273)
• Composition metadata was restructured to hold components and adapters in separate sub-dictionaries (!274)
• Time components implement method _next_time instead of property next_time (!283)
• Info now has properties for grid, time and mask (!286)
• all init-args of Info are now optional (!286)
• Info.accepts has changed signature: renamed ignore_none to incoming_donwstream (!286)
• Info.accepts now only checks: grid, mask and units (other meta data can differ) (!286)
• Grid.to_/from_canonical now allows additional dimensions (!286)
• data_shape now a property of GridBase (!286)
  • NoGrid can be initialized with dim or data_shape now
  • NoGrid.data_shape can have -1 entries for variable size dimensions
  • if only dim given to NoGrid, all entries in data_shape will be -1

Features

• Components and adapters automatically provide default metadata that can be extended by implementations (!274, !276)
• Grid class now have attributes providing connectivity information for the contained cells (!275)
  • cells_connectivity: connectivity array as used by ESMF and VTK
  • cells_definition: cell definition as used by PyVista and legacy VTK
  • cells_offset: location of the start of each cell in cells_connectivity
• added convenience functions and constants to grid_tools (!275)
  • get_cells_matrix: convert cells_connectivity or cells_definition back to the default cells matrix used in the Grid class (can be used to convert VTK-grids into FINAM-grids)
  • INV_VTK_TYPE_MAP: inverse mapping to VTK_TYPE_MAP - FINAM cell type to VTK cell type
  • VTK_CELL_DIM: parametric dimension for each VTK cell type
• Grid class now reusable when having different data locations and better grid type casting (!278)
  • added copy method to grids with optional argument deep (False by default) to create a copy of a grid
  • added setter for data_location in order to set a new data location (e.g. after copying a grid)
  • added class attribute valid_locations in order to check the set data location (esri-grid only supports cells, unstructured-points only support points)
  • added missing casting methods to convert esri to uniform and uniform to rectilinear (when you want to use point data on an esri-grid, you can cast it to uniform first)
  • added axes_attributes also to unstructured grids
• Grid method compatible_with now has a check_location argument to optionally check data location (!280)
• added Mask enum with two options: (!286)
  • Mask.FLEX for flexible masking
  • Mask.NONE to explicitly use plain numpy arrays
• added mask attribute and init-arg to Info : can be a Mask value or a valid mask for numpy.ma.MaskedArray (!286)
• data.tools.prepare now applies masks to data if set in Info object (!286)
• ARegridding now has a out_mask arg (!286)
• RegridNearest and RegridLinear now support explicitly masked data (input doesn't have Mask.FLEX) (!286)
• adapters now have an in_info property (!286)

Bug fixes

• cells for structured grids in 3D are now created correctly (no negative Volume in VTK/ESMF) (!286)
• cf_units.py was broken for pint>=0.24 (!282)

Documentation

• Minor fixes in documentation examples and links (!272)
• Adds a book section on composition, component and adapter metadata (!274)

FINAM v0.5.1

FINAM is an open-source component-based model coupling framework for environmental models.
It aims at enabling bi-directional online couplings of models for different compartments like geo-, hydro-, pedo- and biosphere.

The framework is built in Python, with well-defined interfaces for data exchange.
This approach allows for coupling of models irrespective of their internal structure, architecture or programming language.

Changelog

Bug fixes

• Fix unquantified masked arrays loosing mask in fm.data.prepare() (#115, !270)

FINAM v0.5.0

FINAM is an open-source component-based model coupling framework for environmental models.
It aims at enabling bi-directional online couplings of models for different compartments like geo-, hydro-, pedo- and biosphere.

The framework is built in Python, with well-defined interfaces for data exchange.
This approach allows for coupling of models irrespective of their internal structure, architecture or programming language.

Resources

• FINAM homepage
• FINAM documentation
• FINAM source code
• FINAM GitLab group, containing further related projects

Changelog

Features

• Components and adapters can provide a dictionary of meta data (!259)
• Class Composition hat a property metadata that collects and returns the meta data from all components and adapters (!259)
• Automatic conversion between compatible grids (!255)
• Adds methods to_canonical, from_canonical and get_transform_to to grid classes (!255)
• Adds support for masked grids using numpy.ma.MaskedArray (!258, !260)
• Adds convenience functions for dealing with masked arrays in data.tools (!260):
  is_masked_array, has_masked_values, filled, to_compressed, from_compressed, check_data_covers_domain

Documentation

• Adds a book chapter on wrapping existing models for FINAM (!256)
• Adds a book section on masked data (!262)

Bug fixes

• No more logging of expected FinamNoDataError in inputs during the connect phase (!257)

Other

• FINAM is now available on Conda via conda-forge

FINAM v0.4.0

FINAM is an open-source component-based model coupling framework for environmental models.
It aims at enabling bi-directional online couplings of models for different compartments like geo-, hydro-, pedo- and biosphere.

The framework is built in Python, with well-defined interfaces for data exchange.
This approach allows for coupling of models irrespective of their internal structure, architecture or programming language.

Resources

• FINAM homepage
• FINAM documentation
• FINAM source code
• FINAM GitLab group, containing further related projects

Changelog

New scheduling algorithm

• FINAM uses a new scheduling algorithm that allows components to use future data instead of only past/current (!157, !159)
• New adapters to resolve circular coupling through the use of delayed data (!187)
• It is now possible to set up static couplings that run only once and have no explicit time or stepping (!166)
• FINAM can handle different starting times of components by pushing initial data twice (!206):
  Once for the common starting time, and once for the actual component time
• Components are no longer required to push all outputs on every step (!208)

Data and metadata rework

• Outputs check compatibility between metadata of inputs if there is more than one target input (!104)
• Add data tools function compatible_units to check for convertibility (!105)
• Components can exchange their starting time through the Info object (!111)
• Info exchange is automated by the ConnectHelper by specifying transfer rules at initialization (!154)
• Info now requires time in constructor (can be None) (!111)
• Scheduler checks for dead links that don't work in terms of push/pull combination (!112)
• IInput, IOutput and IAdapter have new internally used properties needs_push and needs_pull (!112)
• to_xarray now checks the data shape if the data is not flat (!130)
• Outputs can be flagged static for data that is only used during initialization, or that is constant (!166)
• Inputs can be flagged static for constant data (!171)
• Outputs accept and convert compatible units, not only exactly equal units (!215)
• Outputs check that subsequent data pushes don't share memory (!217)
• Exchanged xarray data has no time coordinate anymore, only a dimension without values (for performance and usability) (!223)
• Remove the xarray wrapping completely. Use numpy arrays in pint Quantity (!235)
• Outputs and adapters can have a memory_limit and write data to disk if the limit is exceeded (!238)

Components

• Add modules.WeightedSum for aggregation of multiple inputs (!105)
• Add modules.SimplexNoise for generating spatio-temporal noise (!131)
• Add modules.TimeTrigger to forward data from pull-based to push-based components (!131)
• Add modules.ScheduleLogger to visualize scheduling/module updates through ASCII charts (!160)
• Add modules.DebugPushConsumer as a push-based variant of the debug consumer (!165)
• Add modules.UserControl that lets users control FINAM runs from the terminal (!184)
• modules.DebugConsumer and modules.DebugPushConsumer can use optional callbacks for better debugging (!176)
• Components can be renamed using the method with_name() (!243)

Adapters

• Add adapters.Histogram to extract a histogram from grid values (!182)
• Add adapters.DelayFixed, adapters.DelayToPull and adapters.DelayToPush to resolve circular coupling through the use of delayed data (!187)
• Add adapters.StepTime for step-wise interpolation (!194)
• Restructuring of time integration adapters (!194)
  • adapters.IntegrateTime renamed to adapters.AvgOverTime
  • Add adapters.SumOverTime for sum/Area under Curve integration
• Adapters have a method finalize() for cleanup (!226).
• Adapters can be renamed using the method with_name() (!243)

Other

• Remove module core, subpackages now under finam (!106)
• Rename IOutput.source_changed() to source_updated (!107)
• Rename LogError to ErrorLogger (!107)
• Rename abstract SDK classes: (!107)
  • AAdapter is now Adapter
  • AComponent is now Component
  • ATimeComponent is now TimeComponent
• Changed arguments for create_connector() (!111)
  • Removed required_out_infos
  • Renamed required_in_data to pull_data
  • Added arguments to specify info exchange rules
• All error types are in module errors now, and re-exported at top level (!116)
• Overwriting _validate() and _finalize() in components is now mandatory (!156)
• Input and output slots can be accessed from components directly, e.g. comp["A"] instead of comp.inputs["A"] (!147)
• Inputs and outputs can be marked as static for constant data without time information (!166, !171)
• New helper function tools.inspect() to inspect components, adapters and I/O slots (!197)
• Publish on PyPI, starting with the next release (!198, !200, !201)
• Added benchmarks for the most important FINAM functions
  (see the benchmarks README)
• Added profiling for full runs to the CI (!221)
• Optimization of data tool functions, with approx. 20-fold speedup of basic push+pull
  (!222, !223, !224, !228, !229, !237).
• Add two more log levels: TRACE (most verbose) and PROFILE (between DEBUG and INFO) (!240)