whats_new.rst

What's New

v3.13.3 (2024/02/02)

New Features

Breaking changes

Deprecations

Bug fixes

Documentation

• Improve documentation for :pynetCDF4 dependency. (@enekomartinmartinez)

Performance

Internal Changes

• Support for Python 3.12. (@enekomartinmartinez)
• Support for :pynumpy >= 1.24. (@enekomartinmartinez)
• Correct some warnings management in the tests. (@enekomartinmartinez)
• Fix :pynumpy requirements to >= 1.23 to follow NEP29. (@enekomartinmartinez)
• Custom error messages when :pynetCDF4 is missing (435). (@enekomartinmartinez)

v3.13.2 (2024/01/09)

New Features

Breaking changes

Deprecations

Bug fixes

• Add support for XMILE's DELAY function through DelayFixed (433). (@enekomartinmartinez)

Documentation

Performance

Internal Changes

v3.13.1 (2024/01/02)

New Features

Breaking changes

Deprecations

Bug fixes

• Fix bug with :pypysd.py_backend.output.ModelOutput initializing all the handlers, and thus, importing netCDF4 when unnecessary (431). (@enekomartinmartinez)

Documentation

Performance

Internal Changes

v3.13.0 (2023/12/25)

New Features

• Include new method :pypysd.py_backend.model.Model.copy which allows copying a model (131). (@enekomartinmartinez)
• :pypysd.py_backend.model.Model.select_submodel now takes an optional argument inplace when set to :pyFalse it will return a modified copy of the model instead of modifying the original model (131). (@enekomartinmartinez)
• :pypysd.py_backend.model.Model.export will now save also time component information if changed (e.g. final time, time step...). (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

• Set the pointer of :pypysd.py_backend.statefuls.DelayFixed to 0 during initialization (427). (@enekomartinmartinez)
• :pypysd.py_backend.model.Model.export now works with Macros. (@enekomartinmartinez)

Documentation

• Improve documentation of methods in :pypysd.py_backend.model.Model and :pypysd.py_backend.model.Macro includying cross-references and rewrite the one from :pypysd.py_backend.model.Macro.set_components. (@enekomartinmartinez)
• Include documentation about the new method :pypysd.py_backend.model.Model.copy and update documentation from :pypysd.py_backend.model.Model.select_submodel. (@enekomartinmartinez)

Performance

• Improved performace of :pypysd.py_backend.output.DataFrameHandler by creating the dataframe at the end of the run (374 and 330). (@easyas314159 and @enekomartinmartinez)

Internal Changes

• Move old :pypysd.py_backend.model.Macro.set_components to :pypysd.py_backend.model.Macro._set_components, and create new method with the same name without the new argument. (@enekomartinmartinez)
• Move old :pypysd.py_backend.model.Macro.set_stateful to :pypysd.py_backend.model.Macro._set_stateful. (@enekomartinmartinez)
• Make integration tests filter only specific warnings. (@enekomartinmartinez)
• Include warnings in :pypysd.py_backend.model.Macro.set_components when changing the behaviour of the component (58). (@enekomartinmartinez)

v3.12.0 (2023/10/02)

New Features

Breaking changes

• Remove support for Python 3.7 and 3.8. (@enekomartinmartinez)
• Update the cellrange retrievel to be compatible with :pyopenpyxl >= 3.1. It breaks the code for lower versions. (@enekomartinmartinez)

Deprecations

Bug fixes

• Update benchmarking tools to avoid :pyFutureWarning for positional keys from :pypandas. (@enekomartinmartinez)

Documentation

• Update requirements. (@enekomartinmartinez)

Performance

Internal Changes

• Update libraries used for building documentation and .readthedocs.yml. (@enekomartinmartinez)
• Force to use :pyxarray >= 2023.09. (@enekomartinmartinez)
• Use pandas[excel] instead of xlrd dependency in the requirements. (@gdrosos)

v3.11.0 (2023/08/01)

New Features

• Add the possibility to run a model one or several steps at a time, updating model variables in the process. (@rogersamso)

Breaking changes

Deprecations

Bug fixes

Documentation

• Add the Running models one (or more) step(s) at a time section in advanced_usage. (@rogersamso)

Performance

Internal Changes

• Modify signature of the :pypysd.py_backend.output.ModelOutput class. It now only accepts the path of the results file (@rogersamso)
• Add the :pypysd.py_backend.output.ModelOutput.collect method to the :pypysd.py_backend.output.ModelOutput class. (@rogersamso)
• Add the :pypysd.py_backend.model.Model.set_stepper and :pypysd.py_backend.model.Model.step methods to the :pypysd.py_backend.model.Model class. (@rogersamso)
• Add several internal methods to the :pypysd.py_backend.model.Model class, to avoid code repetition. (@rogersamso)
• Add :pyFutureWarning for Python 3.7 support deprecation. (@enekomartinmartinez)

v3.10.0 (2023/04/28)

New Features

• Parse TABBED ARRAYS Vensim function. (@rogersamso)
• Add support for Vensim's POWER function. (@rogersamso)
• Add possibility to pass data_files in netCDF format. (@rogersamso)
• Add support for XMILE's non-negative flows and stocks. (@enekomartinmartinez)
• Add support for XMILE's MIN and MAX functions with one argument. (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

• Set the final_subscripts to an empty dictionary for ELMCOUNT function in :pypysd.builders.python_expressions_builder.CallBuilder.build_function_call. (@rogersamso)
• Define comp_subtype of Unchangeable tabbed arrays as Unchangeable. This is done in :pypysd.builders.python.python_expressions_builder.ArrayBuilder.build. (@rogersamso)

Documentation

• Add information about slack channel https://slofile.com/slack/sdtoolsandmet-slj3251. (@enekomartinmartinez)
• Update XMILE stocks section. (@enekomartinmartinez)

Performance

Internal Changes

• Add a weekly scheduled run to all CI workflows, which run each Monday at 06:00 UTC. (@EwoutH)
• Fix CI pipeline for Python 3.11 and remove Python 3.10 pipeline in favour of 3.11. (@kinow)
• Add non_negative argument in :pypysd.translators.structures.abstract_expressions.IntegStructure. (@enekomartinmartinez)

v3.9.1 (2023/03/11)

New Features

• Add :pynumpy.py as translation for the call to the function PI(). (@lionel42)

Breaking changes

Deprecations

Bug fixes

• Set :pynumpy <1.24 to avoid errors with least squares equation in :pypysd.py_backend.allocation.allocate_available. (@enekomartinmartinez)
• Keep the attributes of a component when using :pypysd.py_backend.model.Macro.set_components to avoid losing coords or arguments information. (@enekomartinmartinez)
• Set :pyopenpyxl <3.1 to avoid errors due to non-backwards compatible changes. (@enekomartinmartinez)
• Include time dependency in random functions to avoid them using constant cache. (@enekomartinmartinez)

Documentation

Performance

Internal Changes

• Run test for Python 3.11 with ubuntu-latest (hdf5-headers need to be installed using apt manager). (@enekomartinmartinez)

v3.9.0 (2022/12/15)

New Features

• Parses and ignores reality check functions during translation of Vensim models. (@rogersamso)

Breaking changes

Deprecations

Bug fixes

• Fix issue with the classification of variables in modules and submodules (388). When a model had a view with 3 sublevels (e.g. energy-transformation.losses) but another view was defined with only two of them (e.g. energy-transformation), the variables in the second view were placed in the main model file. Now, if this happens, the variables in the second view will be placed in a main.py file (i.e. energy/transformation/main.py). (@rogersamso)
• Fix bug on the CLI when passing a hyphen as first value to the --subview-sep argument (388). (@rogersamso)
• Fix bug on the CLI when parsing initial conditions (395). (@rogersamso)

Documentation

• The Splitting Vensim views in different files section in command_line_usage has been updated to include an example of the usage of the --subview-sep CLI argument. (@rogersamso)

Performance

Internal Changes

• The :py_merge_nested_dicts method from the :pypysd.translators.vensim.vensim_file.VensimFile class has been made a static method, as it does not need to access any attribute of the instance, and it does facilitate unit testing. (@rogersamso)
• The pysd/translators/vensim/parsing_grammars/element_object.peg grammar has been modified to be able to parse reality check elements. (@rogersamso)
• :pypysd.translators.vensim.vensim_element.Constraint and :pypysd.translators.vensim.vensim_element.TestInputs classes have been added, which inherit from the also newly created :pypysd.translators.vensim.vensim_element.GenericComponent, which include the :pyparse and :pyget_abstract_component methods. (@rogersamso and @enekomartinmartinez)
• The :pypysd.translators.structures.abstract_model.AbstractSection class now has two extra attributes (:pyconstraints and :pyinput_tests), which hold the :pypysd.translators.structures.abstract_model.AbstractConstraint and :pypysd.translators.structures.abstract_model.AbstractTestInputs objects. (@rogersamso)

v3.8.0 (2022/11/03)

New Features

• Adds ncfile.py module with helper functions to export a subset or all of the data_vars in netCDF files generated with PySD to :pypandas.DataFrame, csv or tab files. (@rogersamso)
• Adds possibility to initialize and export a subset or all external objects to netCDF, and then initialize the external objects from the file. (@rogersamso)

Breaking changes

Deprecations

• Deprecate :pypysd.py_backend.model.Model._get_dependencies replacing it with :pypysd.py_backend.model.Model.get_dependencies. (@enekomartinmartinez)

Bug fixes

• Include new class :pypysd.py_backend.utils.Dependencies to return by :pypysd.py_backend.model.Model.get_dependencies (379). (@lionel42)

Documentation

• Updates the getting_started page with instructions on how to use the new helper functions for netCDF files. (@rogersamso)
• Updates the advanced_usage page with instructions on how to export externals to netCDF and initialize a model from it. (@rogersamso)
• Update citation information to include the new paper published in JOSS. (@enekomartinmartinez)

Performance

• Initializing external objects from netCDF file is much faster than reading from spreadsheet files.(@rogersamso)

Internal Changes

• Adds the :pypysd.py_backend.model.Macro.serialize_externals and :pypysd.py_backend.model.Macro.initialize_external_data methods, and a few other private methods.(@rogersamso)
• Adds the :pypysd.py_backend.utils.UniqueDims class for renaming model dimensions with unique names.(@rogersamso)
• Force :pypysd.py_backend.external.External objects to always have the full element dimensions, missing dimensions are filled with numpy.nan. (@enekomartinmartinez)
• Add dependabot configuration for GitHub Actions updates. (@EwoutH)
• Include new error messages for initialization of :pypysd.py_backend.lookups.HardcodedLookups (376). (@enekomartinmartinez)
• Include new warning message when a translated variable has several types or subtypes. (@enekomartinmartinez)
• Set CI test to run in parallel in 2 cores. (@enekomartinmartinez)

v3.7.1 (2022/09/19)

New Features

Breaking changes

Deprecations

Bug fixes

• Fix bugs with :pypandas.DataFrame 1.5.0 (366). (@enekomartinmartinez)

Documentation

Performance

Internal Changes

v3.7.0 (2022/09/19)

New Features

• Simulation results can now be stored as netCDF4 files. (@rogersamso)
• The CLI also accepts netCDF4 file paths after the -o argument. (@rogersamso)

Breaking changes

Deprecations

Bug fixes

• Fix bug when a WITH LOOKUPS argument has subscripts. (@enekomartinmartinez)
• Fix bug of exporting csv files with multiple subscripts variables. (@rogersamso)
• Fix bug of missing dimensions in variables defined with not all the subscripts of a range (364). (@enekomartinmartinez)
• Fix bug when running a model with variable final time or time step and progressbar (361). (@enekomartinmartinez)

Documentation

• Add Storing simulation results on a file section in the getting_started page. (@rogersamso)
• Include cookbook information in the getting_started page. (@enekomartinmartinez)
• Include an introduction of main historical changes in the about page. (@enekomartinmartinez)

Performance

• Exporting outputs as netCDF4 is much faster than exporting a pandas DataFrame, especially for large models. (@rogersamso)

Internal Changes

• Make PySD work with :pyparsimonius 0.10.0. (@enekomartinmartinez)
• Add netCDF4 dependency for tests. (@rogersamso)
• Improve warning message when replacing a stock with a parameter. (@enekomartinmartinez)
• Include more pytest parametrizations in some test and make them translate the models in temporary directories. (@enekomartinmartinez)
• Include lychee-action in the GHA workflow to check the links. (@enekomartinmartinez)
• Update License. (@enekomartinmartinez)
• Include Maintained? Yes and Contributions welcome badges. (@enekomartinmartinez)
• Update links to the new repository location. (@enekomartinmartinez)
• Reduce relative precision from 1e-10 to 1e-5 to compute the saving times and final time. (@enekomartinmartinez)
• Add convergence tests for euler integration method. (@enekomartinmartinez)
• Include build docs check in the GHA workflow to avoid warnings with sphinx. (@enekomartinmartinez)

v3.6.1 (2022/09/05)

New Features

Breaking changes

Deprecations

Bug fixes

Documentation

Performance

Internal Changes

• Set :pyparsimonius requirement to 0.9.0 to avoid a breaking-change in the newest version. Pending to update PySD to run it with :pyparsimonious 0.10.0. (@enekomartinmartinez)

v3.6.0 (2022/08/31)

New Features

• Include warning messages when a variable is defined in more than one view, when a control variable appears in a view or when a variable doesn't appear in any view as a workbench variable (357). (@enekomartinmartinez)
• Force variables in a module to be saved alphabetically for being able to compare differences between versions (only for the models that are split by views). (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

• Classify control variables in the main file always (357). (@enekomartinmartinez)

Documentation

Performance

Internal Changes

• Include :pypysd.translators.structures.abstract_model.AbstractControlElement child of :pypysd.translators.structures.abstract_model.AbstractElement to differentiate the control variables. (@enekomartinmartinez)

v3.5.2 (2022/08/15)

New Features

Breaking changes

Deprecations

Bug fixes

• Make sketch's font_size optional. (@enekomartinmartinez)

Documentation

• Correct typos.

Performance

Internal Changes

v3.5.1 (2022/08/11)

New Features

Breaking changes

Deprecations

Bug fixes

• Fix bug generated when :EXCEPT: keyword is used with subscript subranges (352). (@enekomartinmartinez)
• Fix bug of precision error for :pypysd.py_backend.allocation.allocate_by_priority (353). (@enekomartinmartinez)
• Fix bug of constant cache assignment. (@enekomartinmartinez)

Documentation

Performance

• Improve the performance of reading :pypysd.py_backend.external.External data with cellrange names by loading the data in memory with :pypandas. As recommended by :pyopenpyxl developers, this is a possible way of improving performance to avoid parsing all rows up each time for getting the data (issue 1867 in openpyxl). (@enekomartinmartinez)

Internal Changes

v3.5.0 (2022/07/25)

New Features

• Add support for subscripted arguments in :pypysd.py_backend.functions.ramp and :pypysd.py_backend.functions.step (344). (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

• Fix bug related to the order of elements in 1D GET expressions (343). (@enekomartinmartinez)
• Fix bug in request 0 values in allocate by priority (345). (@enekomartinmartinez)
• Fix a numerical error in starting time of step and ramp. (@enekomartinmartinez)

Documentation

• Include new PySD logo. (@enekomartinmartinez)

Performance

Internal Changes

• Ignore 'distutils Version classes are deprecated. Use packaging.version instead' error in tests as it is an internal error of xarray. (@enekomartinmartinez)
• Add a warning message when a subscript range is duplicated in a variable reference. (@enekomartinmartinez)

v3.4.0 (2022/06/29)

New Features

• Add support for Vensim's ALLOCATE AVAILABLE (:pypysd.py_backend.allocation.allocate_available) function (339). Integer allocation cases have not been implemented neither the fixed quantity and constant elasticity curve priority functions. (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

Documentation

• Improve the documentation of the :pypysd.py_backend.allocation module. (@enekomartinmartinez)

Performance

Internal Changes

• Add a class to manage priority profiles so it can be also used by the many-to-many allocation. (@enekomartinmartinez)

v3.3.0 (2022/06/22)

New Features

• Add support for Vensim's ALLOCATE BY PRIORITY (:pypysd.py_backend.allocation.allocate_by_priority) function (263). (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

• Fix bug of using subranges to define a bigger range (335). (@enekomartinmartinez)

Documentation

Performance

Internal Changes

• Improve error messages for pysd.py_backend.External objects. (@enekomartinmartinez)

v3.2.0 (2022/06/10)

New Features

• Add support for Vensim's GET TIME VALUE (:pypysd.py_backend.functions.get_time_value) function (332). Not all cases have been implemented. (@enekomartinmartinez)
• Add support for Vensim's VECTOR SELECT (:pypysd.py_backend.functions.vector_select) function (266). (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

Documentation

Performance

Internal Changes

v3.1.0 (2022/06/02)

New Features

• Add support for Vensim's VECTOR SORT ORDER (:pypysd.py_backend.functions.vector_sort_order) function (326). (@enekomartinmartinez)
• Add support for Vensim's VECTOR RANK (:pypysd.py_backend.functions.vector_rank) function (326). (@enekomartinmartinez)
• Add support for Vensim's VECTOR REORDER (:pypysd.py_backend.functions.vector_reorder) function (326). (@enekomartinmartinez)

Breaking changes

Deprecations

Bug fixes

Documentation

• Add the section /development/adding_functions with examples for developers. (@enekomartinmartinez)

Performance

Internal Changes

• Include a template for PR.

v3.0.1 (2022/05/26)

New Features

Breaking changes

Deprecations

Bug fixes

• Simplify subscripts dictionaries for :pypysd.py_backend.data.TabData objects. (@enekomartinmartinez)

Documentation

• Improve tests/README.md.
• Minor improvements in the documentation.

Performance

Internal Changes

• Add Python 3.10 to CI pipeline and include it in the supported versions list. (@enekomartinmartinez)
• Correct LICENSE file extension in the setup.py. (@enekomartinmartinez)
• Move from importlib's :pyload_module to :pyexec_module. (@enekomartinmartinez)
• Remove warnings related to :pyset usage. (@enekomartinmartinez)
• Move all the missing test to :pypytest. (@enekomartinmartinez)
• Remove warning messages from test and make test fail if there is any warning. (@enekomartinmartinez)

v3.0.0 (2022/05/23)

New Features

• The new Abstract Model Representation <structure/structure_index> translation and building workflow will allow to add new output languages in the future. (@enekomartinmartinez)
• Added new properties to the :pypysd.py_backend.model.Macro to make more accessible some information: :py.namespace, :py.subscripts, :py.dependencies, :py.modules, :py.doc. (@enekomartinmartinez)

• Cleaner Python models: (@enekomartinmartinez)

  • :py_namespace and :py_dependencies dictionaries have been removed from the file.
  • Variables original names, dependencies metadata now are given through :pypysd.py_backend.components.Component.add decorator, instead of having them in the docstring.
  • Merging of variable equations is now done using the coordinates to a pre-allocated array, instead of using the magic function :pypysd.py_backend.utils.xrmerge().
  • Arranging and subseting arrays are now done inplace instead of using the magic function :pypysd.py_backend.utils.rearrange().

Breaking changes

• Set the argument :pyflatten_output from :py.run to :pyTrue by default. Previously it was set to :pyFalse by default. (@enekomartinmartinez)
• Move the docstring of the model to a property, :py.doc. Thus, it is not callable anymore. (@enekomartinmartinez)
• Allow the function :pypysd.py_backend.functions.pulse to also perform the operations performed by :pypysd.py_backend.functions.pulse_train() and :pypysd.py_backend.functions.pulse_magnitude(). (@enekomartinmartinez)
• Change first argument of :pypysd.py_backend.functions.active_initial, now it is the stage of the model and not the time. (@enekomartinmartinez)
• Simplify the function :pypysd.py_backend.utils.rearrange() orienting it to perform simple rearrange cases for user interaction. (@enekomartinmartinez)
• Move :pypysd.py_backend.statefuls.Model and :pypysd.py_backend.statefuls.Macro to :pypysd.py_backend.model.Model and :pypysd.py_backend.model.Macro, respectively. (@enekomartinmartinez)
• Manage all kinds of lookups with the :pypysd.py_backend.lookups.Lookups class. (@enekomartinmartinez)
• Include a second optional argument to lookups functions to set the final coordinates when a subscripted variable is passed as an argument. (@enekomartinmartinez)

Deprecations

• Remove :pypysd.py_backend.utils.xrmerge(), :pypysd.py_backend.functions.pulse_train(), :pypysd.py_backend.functions.pulse_magnitude(), :pypysd.py_backend.functions.lookup(), :pypysd.py_backend.functions.lookup_discrete(), :pypysd.py_backend.functions.lookup_extrapolation(), :pypysd.py_backend.functions.logical_and(), :pypysd.py_backend.functions.logical_or(), :pypysd.py_backend.functions.bounded_normal(), :pypysd.py_backend.functions.log(). (@enekomartinmartinez)
• Remove old translation and building files (:pypysd.translation). (@enekomartinmartinez)

Bug fixes

• Generate the documentation of the model when loading it to avoid losing information when replacing a variable value (310, 312). (@enekomartinmartinez)
• Make random functions return arrays of the same shape as the variable, to avoid repeating values over a dimension (309, 312). (@enekomartinmartinez)
• Fix bug when Vensim's :MACRO: definition is not at the top of the model file (306, 312). (@enekomartinmartinez)
• Make builder identify the subscripts using a main range and subrange to allow using subscripts as numeric values as Vensim does (296, 301, 312). (@enekomartinmartinez)
• Fix bug of missmatching of functions and lookups names (116, 312). (@enekomartinmartinez)
• Parse Xmile models case insensitively and ignoring the new lines characters (203, 253, 312). (@enekomartinmartinez)
• Add support for Vensim's :EXCEPT: keyword (168, 253, 312). (@enekomartinmartinez)
• Add support for Xmile's FORCST and SAFEDIV functions (154, 312). (@enekomartinmartinez)
• Add subscripts support for Xmile (289, 312). (@enekomartinmartinez)
• Fix numeric error bug when using :pyreturn_timestamps and time step with non-integer values. (@enekomartinmartinez)

Documentation

• Review the whole documentation, refract it, and describe the new features. (@enekomartinmartinez)

Performance

• The variables defined in several equations are now assigned to a pre-allocated array instead of using :pypysd.py_backend.utils.xrmerge(). (@enekomartinmartinez)
• The arranging and subseting of arrays is now done inplace instead of using the magic function :pypysd.py_backend.utils.rearrange(). (@enekomartinmartinez)
• The grammars for Parsimonious are only compiled once per translation. (@enekomartinmartinez)

Internal Changes

• The translation and the building of models has been totally modified to use the Abstract Model Representation <structure/structure_index>. (@enekomartinmartinez)