Calibration and Validation Using Calibrar

General structure of calibrar

The general structure of the calibrar workflow is shown in @fig-calibrar-diag.

The main function of the package is calibrate(), which performs minimisation of the objective function. The two compulsory arguments (i.e. without default settings) of calibrate() are the starting search points (par) argument and the function to minimise (fn).

The user can give a vector of values as initial parameters if he has already an idea of the values of the parameters. In that case, the initial exploration will be done following a normal distribution around the starting points given.

The function to minimise fn results from createObjectiveFunction() that combines the observed and simulated data, according to calibration information coming from a csv file read by getCalibrationInfo(), as discussed below.

Calibration info

Calibration information must be provided as a CSV file that contains information about the data that will be used in the calibration process (see Table \ref{tbl-calibration-table}). This file will be read by the getCalibrationInfo() function of the calibrar package.

library(kableExtra)

data = read.csv('_static/osmose-peru-calibration_settings.csv', header=TRUE, nrows=10)
kable(data, booktabs = T) |>
kable_styling(latex_options = c("striped", "scale_down"), position="center") |>
column_spec(1, width = "10em") |>
column_spec(2, width = "3em") |>
column_spec(3, width = "3em") |>
row_spec(0, bold=TRUE)

The table columns are:

• variable is the list of the variables that will be fit by the model with observations.
• type is the objective function that will be used in the fitting. It can either be a native function of the calibrar package or a user defined function.
• calibrate indicates whether the data is used in the calibration.
• weights provides the relative weights used to combine the partial objective values obtained for each variable.
• useData indicates whether data are read from the disk. If useData=FALSE, the observed value is set to NULL and the likelihood function is expected to use simulated data only. The latter option can be particularly useful to set penalties in the model outputs or parameters, where no observed data are needed.

The user can provide a user-defined objective function instead of the ones provided in calibrar. These functions must be defined as follows:

user_rms <- function(obs, sim) {
  value = sqrt(sum((obs - sim)^2) / length(obs))
}

Warning

If useData = TRUE, the name of the CSV file must be consistent with the name of the variable. For instance, if the variable is PagellusErythrinus.thr, the data file must be PagellusErythrinus.thr.csv

Warning

The column names (i.e. the header) are important and must be as shown in @tbl-calibration-table

Observed data

The getObservedData() function gets back the observed data from csv files and transforms them into a list format. The data must be provided as CSV files. Biomass and yield data must be provided as shown in @tbl-peru-biomass and @tbl-peru-yield. They must have a year and a month column, and one column for each species of the Osmose configuration considered. Therefore, there is one file for all the species.

data = read.csv('_static/peru_biomass-index_month.csv', header=TRUE, nrows=10)
kable(data, booktabs = T) |>
kable_styling(latex_options = c("striped", "scale_down"), position="center")

data = read.csv('_static/peru_yield_month.csv', header=TRUE, nrows=10)
kable(data, booktabs = T) |>
kable_styling(latex_options = c("striped", "scale_down"), position="center")

Catch at length data must be formatted as shown on Table \ref{tbl-peru-catatlen}. In this case, there will be one file per species and the columns correspond to the size-classes considered.

data = read.csv('_static/peru_catchatlength-anchoveta_month.csv', header=TRUE, nrows=10)
data = data[, 1:10]
kable(data, booktabs = T) |>
kable_styling(latex_options = c("striped", "scale_down"), position="center")

Note

The CSV data separator must be ,

Simulated data

Simulated data are returned as a list format by run_model(), the second main function of the package that must be defined by the user. It must be defined as follows:

run_model = function(param, ...) {
  ...   
}

The param argument is the list of the calibrated parameters, which is passed by the` calibrate function.

The run_model function must achieve the following three tasks:

• write the set of parameters provided by calibrate() in a format the Osmose model is able to read
• launch the Osmose simulation from a command line with this set of parameters (by calling the run_osmose function of the Osmose package)
• get back the simulation outputs and return them after a transformation into a list format.