20-01-2022
The MultiForest Optimization Notebook was developed for the project "MultiForest - Management for multifunctionality in European forests in the era of bioeconomy" (https://www.jyu.fi/BERG/berg-projects-1/forest-values). The framework provides a set of rules that can be combined to create a unique multi-objective optimization problem. Particularly, it can be used to optimize forest management for forest ecosystem service and biodiversity objectives, while seeking an efficient management solution for individual forest entities (forest stands). The optimization problem can be created by adjusting the settings in a Jupyter notebook and a graphical user interphase (GUI).
The optimization tool was developed under the lead of the company Silo AI together with project partners from: University of Jyväskylä, Technical University of Munich, Norwegian Institute of Bioeconomy Research, and Swedish University of Agricultural Sciences.
- Anaconda Python (https://www.anaconda.com/products/individual), install for User only (does not require administrative permissions)
- Python 3 version (Python 2 should also work though, but not tested)
- Run conda prompt
- Ortools and Pandas packages ("conda install pip"&"pip install ortools, pandas")
- Go to the directory containing the repository
- Run "jupyter notebook"
- From the files tab open the corresponding .ipynb file
- A browser window should open and you are ready to go
- Quick introductions to Jupyter Notebook can be found in e.g., Youtube (see e.g., https://www.youtube.com/watch?v=jZ952vChhuI)
- Create Anaconda environment "conda env create -f mainEnvironment.yml" from the directory containing the repository
- Activate the environtment using "conda activate multiForest"
- Everything should work by running "jupyter notebook"
import multiFunctionalOptimization as MFO
Imports the package from the Python file.
mfo = MFO.MultiFunctionalOptimization()
Creates an instance of the class.
mfo.readData(filename, sampleRatio=0.1 )
Reads the data from filename.
It takes a random sample of stands to reduce computation time (if commented out "#", whole data is taken).
Input data consits of forest stand/inventory plot data simulated under different management regimes with a forest growth model. The data can be grouped into three types: 1) Indexing the data: stand ID, year, regime; 2) Indicators for assessing forest ecosystem services (they can differ between countries); 3) Additional info like climate change scenario, represented area by NFI plot, region/province, or NUTS2 level.
The first year gets only the regime “initial_state” and can be considered as starting point for all regimes (same value for all regimes). This gets important, if indicator performances to the current situation have to be evaluated (e.g. no decline in biodiversity is allowed).
mfo.calculateTotalValuesFromRelativeValues(columnTypes=columnTypes)
columnTypes is a dictionary, which has entries where the column name is the key and then we have a tupe with python data type and column type. Possible column types are:
- "Relative to Area" = indicator value relates 1 hectar -> scaled to represented area of NFI plot
- "Relative to volume" = indicator relates to standing V (e.g. %-share of deciduous trees) -> scaled to the represented volume of the plot
- "Absolute Value" = takes the inticator value as it is
If columTypes is an empty dictionary, then the class tries to make all the column data types to float if possible. For column types "Relative to Area" and "Relative to volume" the class makes a new column, which name "Total_" combined with the original column type.
mfo.addRegimeClassifications(regimeClassNames = regimeClassNames,regimeClassregimes=regimeClassregimes)
(OPTIONAL) Classify the regimes into category and create a new column, e.g. indicating if regime is "CCF_3, CCF_4, BAUwGTR" (TRUE/FLASE)
(OPTIONAL) Calculate additional columns from data. As an example, mfo.data["new_column_name"] = mfo.data["column1"].values*mfo.data["column2].values creates a new column which is the itemwise product of two old columns.
mfo.finalizeData(initialRegime="initial_state")
Prepares the data for optimization and makes columns with name "Relative_", which means related to the intial values. For example, a value 1.1 in a stand in certain year would mean that the value has grown 10% from the beginning to the certain year.
InitialValues = { "key":value } is an optional dictionary, which gives initial values of objectives when they cannot be calculated from the input data (as defined in the previous step). The keys are the keys of objectives and the values are the values of objectives.
mfo.defineObjectives(objectives,initialValues = initialValues)
Defines objectives following the objective format defined in section Objective format and call
Define objectives using the format defined in section Constraint format and call mfo.calculateObjectiveRanges(debug=True). Evaluates the ranges of objectives by optimizing individually the mfo.defineConstraints(constraints)
mfo.showGui()
Shows the graphical user interface. Uses the sliders to define the preferences as described in section Defining preferences using sliders
Forest growth simulations under different management scenarios on stand/NFI plot level represent the input data. Different climate change scenarios are handled in seperate input data and optimized individually. Data can come from different ecosystem models and needs to be grouped in columns that:
- Indexing the data: stand ID, year, regime (mandatory columns)
- Indicators representing ecosystem services (country or model specific)
- Additional information (optional)
IMPORTANT: the first year of the data (starting year) is only once represented for each stand/plot and is indicated by the regime "initial_state". It represents the starting point for all simulated regimes - same initial stand situation for all regimes. All later years can be represented multiple times and are indicated by the corresponding regime (BAU; CCF; SA). This gets important, if indicator performances to the current situation have to be evaluated (e.g. no decline in biodiversity is allowed).
The objective function can be defined based on up to eight standardised attributes, arranged in the below order.
(1) "Unique_key" : [ (2) "Long human readable name", (3) "column name", (4) "max/min objective", (5) "year wise aggregation", (6) "stand wise aggregation" (, (7) target year OR (8) string of periodic targets) ]
(1) "Unique_key": short identifier of the objective problem, includes usually the indicator name.
(2) "Long human readable name": easy and understandable text that appears in the GUI, format of text is free.
(3) "column name": definition of the indicator column that is optimized for.
(4) "max/min objective": define if the objective is "max"imised or "min"imised.
(5) "year wise aggregation":
- optimize the "min"imum value over all time periods,
- optimize the "average" over all time periods,
- optimize the "firstYear" value,
- optimize the "sum" over all years,
- optimize a "targetYearWithSlope", aim to achive a target value in a certain year and keep it above this value for all years afterwards, consider a linear increase from the initial situation,
- optimize a "targetYear", same as above but without linear increase,
- optimize the "lastYear" over all time periods,
- optimize for "periodicTargets", each simulation period gets a certain target value (except initial year),
- optimize that the yearly increase is minimum "minYearlyIncrease",
- optimize that the yearly increase is maximum "maxYearlyIncrease",
- optimize that the periodic decrease is maximum "maxDecreaseDuringNPeriods",
- optimize that the periodic increase is mimimum "minIncreaseDuringNPeriods".
(6) "stand wise aggregation":
- take the "sum" of all stand/plot values, or of a smaller set "subsetSum"
- take the "areaWeightedAverage" of all stand/plot values,
- take the "areaWeightedSum" of all stand/plot values.
Additional comment: the combination of "Total_"indicator values & "sum" results in the same outcome as the combination of basic indicator value and "areaWeightedSum"; e.g. "Total_HarvestedV" & "sum" = "HarvestedV" & "areaWeightedSum"; both calculate the total harvest over the whole landscape.
The following two attributes are optional.
(7) target year: define the year that is optimized. Can be any year except the initial year. Is used for the objectives "targetYear" & "targetYearWithSlope".
(8) periodic targets: define one target value for each simulation step (except initial year). Is for example used for the demand values of GLOBIOM.
Most of the options are what they say they are, but some may require additional description:
- minYearlyIncrease: This is mathematically
, where f is the standwise aggregated objective. Note that minYearlyIncrease can be used only with maximized objectives.
- maxYearlyIncrease: This is mathematically
, where f is the standwise aggregated objective. Note that maxYearlyIncrease can be used only with minimized objectives.
- maxDecreaseDuringNPeriods: This is mathematically
, where f is the standwise aggregated objective. Note that maxDecreaseDuringNPeriods can be used only with minimized objectives.
- minIncreaseDuringNPeriods: This is mathematically
, where f is the standwise aggregated objective. Note that minIncreaseDuringNPeriods can be used only with maximized objectives.
If the initial values are not available in the input data, e.g. for indicators like increment or harvested volume (see Finnish case), the values can be defined separately. Reason, those values are usually calculated after the first simulation step, why those are not available for the first year.
Values can be defined by InitialValues = { "key":value }, where "key" is the indicator column and value the missing initial value for the first year.
The constaints can be defined based on the attributes in the below order.
(1) "Shortname": [(2) "constraint type", (3) "human readable name", (4) (regimes), (5) "column in data"]
(1) "Shortname": short identifier for the constraint
(2) "constraint type":
- Allowed regimes: Only certain regimes are allowed on certain stands. There needs to be a column in the dataset, which has value 1 if the constraint applies to the stand and 0 if it does not. Requires also a list of allowed regimes for the stands, for which the column has value 1. Other regimes are not allowed on those stands (e.g. only CCF on Peatland).
- Species reduction: Defined for not allowing species amounts to reduce, but can be applied to any reduction. Requires the column name that we are following, the number of periods that we calculate the reduction for and a number between 0 and 1 which says how much the amount can relatively decrease. Assumes that the value is aggregated over the stands using sum, that is the constraint applies to the sum over all stands. Will not allow the amount to decrease more than the given relative amount during given number of periods.
- less than: The amount from one column needs to be less than a value from another column. Analogous to comparing to objectives and requiring one to be less than the other on each stand. Standwise aggregation for each column must be given and uses similar keywords to objectives.
(3) "human readable name": easy and understandable text that appears in the GUI, format of text is free.
(4) "(regimes)": specify which regimes are considered, e.g. CCFregimes = [regime for regime in mfo.regimes if "CCF" in regime] + ["SA"]
(5) "column in data": specify which columns is delegating the constraint, e.g. column indicating (0, or 1) if soil is peat
Note that constraints can also established by defining an objective and then using the epsilon constraint in the GUI to set its value.
The GUI offers three ways of adjustments.
Epsilon constraint values: These are hard targets that need to be first fulfiled. If the optimization does not match the epsilon contraints, the problem is not solved.
Reference point: These are so called "soft targets". If a reference point cannot be reached, the slider will be reduced until the next possible value. If higher values are possible than the defined reference point, the sliders will increase to the highest possible value.
Enabled constraints: By ticking the box those additional constraints can be considered.
ATTENTION: remember to push the button "Change constraints". The enabled constraints are first considered after the button has been pushed (the same is the case if they need to be changed).
TIP: if enabled constraints need to be considered, start with those before adjusting and optimizing for epsilon constraints and reference points!
Currently, three kinds of data are exported and saved as a CSV.
objective ranges: mfo.objectiveRanges describes the minimum and maximum possible value for individually optimized objectives, the possible range of outcomes. Further, the objective ranges are saved as .json file)
objective value: By pushing the button "Print solution" the optimal solution for each objective is exported that can be reached under the multiple objectives.
solution & solution_alldata: The first data set contains the ideal regime (or share of regimes) for each stand under the multiple objectives. The second data set (solution_alldata) additionally contains the indicator values and their timely development for each stand under the optimal regime.
The project MultiForest was supported under the umbrella of ERA-NET Cofund "ForestValue" by Academy of Finland, Business Finland, Federal Ministry of Agriculture, Forestry, Environment & Water Management (Austria), Agency for Renewable Resources (Germany), Research Council of Norway, Vinnova/Formas/SWEA (Sweden). ForestValue has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement N° 773324.
