Optim param fitting doc

docs/src/inverse_problems/optimization_ode_param_fitting.md

@@ -1,11 +1,11 @@
 # [Parameter Fitting for ODEs using SciML/Optimization.jl and DiffEqParamEstim.jl](@id optimization_parameter_fitting)
 Fitting parameters to data involves solving an optimisation problem (that is, finding the parameter set that optimally fits your model to your data, typically by minimising a cost function). The SciML ecosystem's primary package for solving optimisation problems is [Optimization.jl](https://github.com/SciML/Optimization.jl). It provides access to a variety of solvers via a single common interface by wrapping a large number of optimisation libraries that have been implemented in Julia.
 
-This tutorial demonstrates both how to create parameter fitting cost functions using the [DiffEqParamEstim.jl](https://github.com/SciML/DiffEqParamEstim.jl) package, and how to use Optimization.jl to minimise these. Optimization.jl can also be used in other contexts, such as finding parameter sets that maximise the magnitude of some system behaviour. More details on how to use these packages can be found in their [respective](https://docs.sciml.ai/Optimization/stable/) [documentations](https://docs.sciml.ai/DiffEqParamEstim/stable/).
+This tutorial demonstrates both how to create parameter fitting cost functions using the [DiffEqParamEstim.jl](https://github.com/SciML/DiffEqParamEstim.jl) package, and how to use Optimization.jl to minimise these. Optimization.jl can also be used in other contexts, such as [finding parameter sets that maximise the magnitude of some system behaviour](@ref ref). More details on how to use these packages can be found in their [respective](https://docs.sciml.ai/Optimization/stable/) [documentations](https://docs.sciml.ai/DiffEqParamEstim/stable/).
 
 ## [Basic example](@id optimization_parameter_fitting_basics)
 
-Let us consider a simple catalysis network, where an enzyme ($E$) turns a substrate ($S$) into a product ($P$):
+Let us consider a [Michaelis-Menten enzyme kinetics model](@ref ref), where an enzyme ($E$) converts a substrate ($S$) into a product ($P$):
 ```@example diffeq_param_estim_1 
 using Catalyst
 rn = @reaction_network begin
@@ -18,17 +18,17 @@ From some known initial condition, and a true parameter set (which we later want
 ```@example diffeq_param_estim_1 
 # Define initial conditions and parameters.
 u0 = [:S => 1.0, :E => 1.0, :SE => 0.0, :P => 0.0]
-p_true = [:kB => 1.0, :kD => 0.1, :kP => 0.5]
+ps_true = [:kB => 1.0, :kD => 0.1, :kP => 0.5]
 
 # Generate synthetic data.
 using OrdinaryDiffEq
-oprob_true = ODEProblem(rn, u0, (0.0, 10.0), p_true)
-true_sol = solve(oprob_true, Tsit5())
-data_sol = solve(oprob_true, Tsit5(); saveat=1.0)
+oprob_true = ODEProblem(rn, u0, (0.0, 10.0), ps_true)
+true_sol = solve(oprob_true)
+data_sol = solve(oprob_true; saveat=1.0)
 data_ts = data_sol.t[2:end]
 data_vals = (0.8 .+ 0.4*rand(10)) .* data_sol[:P][2:end]
 
-# Plots the true solutions and the (synthetic data) measurements.
+# Plots the true solutions and the (synthetic) data measurements.
 using Plots
 plot(true_sol; idxs=:P, label="True solution", lw=8)
 plot!(data_ts, data_vals; label="Measurements", seriestype=:scatter, ms=6, color=:blue)
@@ -37,22 +37,22 @@ plot!(data_ts, data_vals; label="Measurements", seriestype=:scatter, ms=6, color
 Next, we will use DiffEqParamEstim to build a loss function to measure how well our model's solutions fit the data.
 ```@example diffeq_param_estim_1 
 using DiffEqParamEstim, Optimization
-p_dummy = [:kB => 0.0, :kD => 0.0, :kP => 0.0]
-oprob = ODEProblem(rn, u0, (0.0, 10.0), p_dummy)
+ps_dummy = [:kB => 0.0, :kD => 0.0, :kP => 0.0]
+oprob = ODEProblem(rn, u0, (0.0, 10.0), ps_dummy)
 loss_function = build_loss_objective(oprob, Tsit5(), L2Loss(data_ts, data_vals), Optimization.AutoForwardDiff(); 
-    maxiters=10000, verbose=false, save_idxs=4)
+    maxiters = 10000, verbose = false, save_idxs = 4)
 nothing # hide
 ```
 To `build_loss_objective` we provide the following arguments:
 - `oprob`: The `ODEProblem` with which we simulate our model (using some dummy parameter values, since we do not know these).
-- `Tsit5()`: The numeric integrator we wish to simulate our model with.
+- `Tsit5()`: The [numeric integrator](@ref ref) we wish to simulate our model with.
 - `L2Loss(data_ts, data_vals)`: Defines the loss function. While [other alternatives](https://docs.sciml.ai/DiffEqParamEstim/stable/getting_started/#Alternative-Cost-Functions-for-Increased-Robustness) are available, `L2Loss` is the simplest one (measuring the sum of squared distances between model simulations and data measurements). Its first argument is the time points at which the data is collected, and the second is the data's values.
 - `Optimization.AutoForwardDiff()`: Our choice of [automatic differentiation](https://en.wikipedia.org/wiki/Automatic_differentiation) framework.
 
 Furthermore, we can pass any number of additional optional arguments, these are then passed to the internal `solve()` function (which is used to solve our ODE). Here we provide the following additional arguments:
-- `maxiters=10000`: If the ODE integrator takes a very large number of steps, that can be a sign of a very poor fit (or stiffness in the ODEs, but that is not a concern for our current example). Reducing the `maxiters` threshold reduces the time we waste on evaluating such models. 
-- `verbose=false`: The simulation of models with highly unsuitable parameter sets typically generate various warnings (such as premature simulation termination due to reaching `maxiters` timesteps). To avoid an overflow of such (here unnecessary) warnings, as we evaluate a large number of parameter sets, we turn warnings off.
-- `save_idxs=4`: The measured species ($P$) is the 4th species in our species vector (`species(rn)`). Since we only assume data is available for $P(t)$ there is no reason to save any other species.
+- `maxiters = 10000`: If the ODE integrator takes a very large number of steps, that can be a sign of a very poor fit (or stiffness in the ODEs, but that is not a concern for our current example). Reducing the `maxiters` threshold reduces the time we waste on evaluating such models. 
+- `verbose = false`: The simulation of models with highly unsuitable parameter sets typically generate various warnings (such as premature simulation termination due to reaching `maxiters` time steps). To avoid an overflow of such (here unnecessary) warnings, as we evaluate a large number of parameter sets, we turn warnings off.
+- `save_idxs = 4`: The measured species ($P$) is the 4th species in our species vector (`species(rn)`). Since data is available for $P(t)$, we will only save the value of this species.
 
 Now we can create an `OptimizationProblem` using our `loss_function` and some initial guess of parameter values from which the optimiser will start:
 ```@example diffeq_param_estim_1 
@@ -63,7 +63,7 @@ nothing # hide
 !!! note
     `OptimizationProblem` cannot currently accept parameter values in the form of a map (e.g. `[:kB => 1.0, :kD => 1.0, :kP => 1.0]`). These must be provided as individual values (using the same order as the parameters occur in in the `parameters(rs)` vector). Similarly, `build_loss_objective`'s `save_idxs` uses the species' indexes, rather than the species directly. These inconsistencies should be remedied in future DiffEqParamEstim releases.
 
-Finally, we can optimise `optprob` to find the parameter set that best fits our data. Optimization.jl only provides a few optimisation methods natively. However, for each supported optimisation package, it provides a corresponding wrapper-package to import that optimisation package for use with Optimization.jl. E.g., if we wish to use [Optim.jl](https://github.com/JuliaNLSolvers/Optim.jl)'s [Nelder-Mead](https://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method) method, we must install and import the OptimizationOptimJL package. A summary of all, by Optimization.jl, supported optimisation packages can be found [here](https://docs.sciml.ai/Optimization/stable/#Overview-of-the-Optimizers). Here, we import the Optim.jl package and uses it to minimise our cost function (thus finding a parameter set that fits the data):
+Finally, we can optimise `optprob` to find the parameter set that best fits our data. Optimization.jl only provides a few optimisation methods natively. However, for each supported optimisation package, it provides a corresponding wrapper-package to import that optimisation package for use with Optimization.jl. E.g., if we wish to use [Optim.jl](https://github.com/JuliaNLSolvers/Optim.jl)'s [Nelder-Mead](https://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method) method, we must install and import the OptimizationOptimJL package. A summary of all, by Optimization.jl supported, optimisation packages can be found [here](https://docs.sciml.ai/Optimization/stable/#Overview-of-the-Optimizers). Here, we import the Optim.jl package and uses it to minimise our cost function (thus finding a parameter set that fits the data):
 ```@example diffeq_param_estim_1 
 using OptimizationOptimJL
 optsol = solve(optprob, Optim.NelderMead())
@@ -87,7 +87,7 @@ sol = solve(optprob, NLopt.LD_LBFGS())
 nothing # hide
 ```
 
-## Optimisation problems with data for multiple species
+## [Optimisation problems with data for multiple species](@id optimization_parameter_fitting_multiple_species)
 Imagine that, in our previous example, we had measurements of the concentration of both *S* and *P*:
 ```@example diffeq_param_estim_1 
 data_vals_S = (0.8 .+ 0.4*rand(10)) .* data_sol[:S][2:end]
@@ -98,32 +98,32 @@ plot!(data_ts, data_vals_S; label="Measured S", seriestype=:scatter, ms=6, color
 plot!(data_ts, data_vals_P; label="Measured P", seriestype=:scatter, ms=6, color=:red)
 ```
 
-In this case we would have to use the `L2Loss(data_ts, hcat(data_vals_S, data_vals_P))` and `save_idxs=[1,4]` arguments in `loss_function`:
+In this case we would have to use the `L2Loss(data_ts, hcat(data_vals_S, data_vals_P))` and `save_idxs=[1, 4]` arguments in `loss_function`:
 ```@example diffeq_param_estim_1 
-loss_function_S_P = build_loss_objective(oprob, Tsit5(), L2Loss(data_ts, Array(hcat(data_vals_S, data_vals_P)')), Optimization.AutoForwardDiff(); maxiters=10000, verbose=false, save_idxs=[1,4])
+loss_function_S_P = build_loss_objective(oprob, Tsit5(), L2Loss(data_ts, Array(hcat(data_vals_S, data_vals_P)')), Optimization.AutoForwardDiff(); maxiters=10000, verbose=false, save_idxs=[1, 4])
 nothing # hide
 ```
 Here, `Array(hcat(data_vals_S, data_vals_P)')` is required to put the data in the right form (in this case, a 2x10 matrix).
 
 We can now fit our model to data and plot the results:
 ```@example diffeq_param_estim_1 
-optprob_S_P = OptimizationProblem(loss_function_S_P, [1.0,1.0, 1.0])
+optprob_S_P = OptimizationProblem(loss_function_S_P, [1.0, 1.0, 1.0])
 optsol_S_P = solve(optprob_S_P, Optim.NelderMead())
-oprob_fitted_S_P = remake(oprob; p=optsol_S_P.u)
-fitted_sol_S_P = solve(oprob_fitted_S_P, Tsit5())
-plot!(fitted_sol_S_P; idxs=[:S, :P], label="Fitted solution", linestyle=:dash, lw=6, color=[:lightblue :pink])
+oprob_fitted_S_P = remake(oprob; p = optsol_S_P.u)
+fitted_sol_S_P = solve(oprob_fitted_S_P)
+plot!(fitted_sol_S_P; idxs=[:S, :P], label="Fitted solution", linestyle = :dash, lw = 6, color = [:lightblue :pink])
 ```
 
-## Setting parameter constraints and boundaries
-Sometimes, it is desired to set boundaries on parameter values. Indeed, this can speed up the optimisation process (by preventing searching through unfeasible parts of parameter space), and can also be a requirement for some optimisation methods. This can be done by passing the `lb` (lower bounds) and `up` (upper bounds) arguments to `OptimizationProblem`. These are vectors (of the same length as the number of parameters), with each argument corresponding to the boundary value of the parameter with the same index (as used in the `parameters(rn)` vector). If we wish to constrain each parameter to the interval $(0.1, 10.0)$ this can be done through:
+## [Setting parameter constraints and boundaries](@id optimization_parameter_fitting_constraints)
+Sometimes, it is desirable to set boundaries on parameter values. Indeed, this can speed up the optimisation process (by preventing searching through unfeasible parts of parameter space), and can also be a requirement for some optimisation methods. This can be done by passing the `lb` (lower bounds) and `up` (upper bounds) arguments to `OptimizationProblem`. These are vectors (of the same length as the number of parameters), with each argument corresponding to the boundary value of the parameter with the same index (as used in the `parameters(rn)` vector). If we wish to constrain each parameter to the interval $(0.1, 10.0)$ this can be done through:
 ```@example diffeq_param_estim_1 
-optprob = OptimizationProblem(loss_function, [1.0, 1.0, 1.0]; lb = [0.1, 0.1, 0.1], ub = [10.0, 10.0, 10.0])
+optprob = OptimizationProblem(loss_function, [1.0, 1.0, 1.0]; lb = [1e-1, 1e-1, 1e-1], ub = [1e1, 1e1, 1e1])
 nothing # hide
 ```
 
 In addition to boundaries, Optimization.jl also supports setting [linear and non-linear constraints](https://docs.sciml.ai/Optimization/stable/tutorials/constraints/#constraints) on its output solution for some optimizers.
 
-## Parameter fitting with known parameters
+## [Parameter fitting with known parameters](@id optimization_parameter_fitting_known_parameters)
 If we from previous knowledge know that $kD = 0.1$, and only want to fit the values of $kB$ and $kP$, this can be achieved through `build_loss_objective`'s `prob_generator` argument. First, we create a function (`fixed_p_prob_generator`) that modifies our `ODEProblem` to incorporate this knowledge:
 ```@example diffeq_param_estim_1 
 fixed_p_prob_generator(prob, p) = remake(prob; p = vcat(p[1], 0.1, p[2]))
@@ -142,8 +142,8 @@ optsol_fixed_kD = solve(optprob_fixed_kD, Optim.NelderMead())
 nothing # hide
 ```
 
-## [Fitting parameters on the logarithmic scale](@id optimization_parameter_fitting_logarithmic_scale)
-Often it can be advantageous to fit parameters on a [logarithmic, rather than linear, scale](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008646). The best way to proceed is to simply replace each parameter in the model definition by its logarithmic version:
+## [Fitting parameters on the logarithmic scale](@id optimization_parameter_fitting_log_scale)
+Often it can be advantageous to fit parameters on a [logarithmic scale (rather than on a linear scale)](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008646). The best way to do this is to simply replace each parameter in the model definition by its logarithmic version:
 ```@example diffeq_param_estim_2
 using Catalyst
 rn = @reaction_network begin
@@ -152,23 +152,39 @@ rn = @reaction_network begin
     10^kP, SE --> P + E
 end
 ```
-And then proceeding, by keeping in mind that parameter values are logarithmic. Here, setting
+And then going forward, by keeping in mind that parameter values are logarithmic. Here, setting
 ```@example diffeq_param_estim_2
 p_true = [:kB => 0.0, :kD => -1.0, :kP => 10^(0.5)]
 nothing # hide
 ```
 corresponds to the same true parameter values as used previously (`[:kB => 1.0, :kD => 0.1, :kP => 0.5]`).
 
-## Parameter fitting to multiple experiments
+## [Parameter fitting to multiple experiments](@id optimization_parameter_fitting_multiple_experiments)
 Say that we had measured our model for several different initial conditions, and would like to fit our model to all these measurements simultaneously. This can be done by first creating a [corresponding `EnsembleProblem`](@ref advanced_simulations_ensemble_problems). How to then create loss functions for these are described in more detail [here](https://docs.sciml.ai/DiffEqParamEstim/stable/tutorials/ensemble/).
 
-## Optimisation solver options
+## [Optimisation solver options](@id optimization_parameter_fitting_solver_options)
 Optimization.jl supports various [optimisation solver options](https://docs.sciml.ai/Optimization/stable/API/solve/) that can be supplied to the `solve` command. For example, to set a maximum number of seconds (after which the optimisation process is terminated), you can use the `maxtime` argument:
 ```@example diffeq_param_estim_1 
-optsol_fixed_kD = solve(optprob, Optim.NelderMead(); maxtime=100)
+optsol_fixed_kD = solve(optprob, Optim.NelderMead(); maxtime = 100)
 nothing # hide
 ```
 
+---
+## [Citation](@id structural_identifiability_citation)
+If you use this functionality in your research, please cite the following paper to support the authors of the Optimization.jl package:
+```
+@software{vaibhav_kumar_dixit_2023_7738525,
+	author = {Vaibhav Kumar Dixit and Christopher Rackauckas},
+	month = mar,
+	publisher = {Zenodo},
+	title = {Optimization.jl: A Unified Optimization Package},
+	version = {v3.12.1},
+	doi = {10.5281/zenodo.7738525},
+  	url = {https://doi.org/10.5281/zenodo.7738525},
+	year = 2023
+}
+```
+
 ---
 ## References
 [^1]: [Alejandro F. Villaverde, Dilan Pathirana, Fabian Fröhlich, Jan Hasenauer, Julio R. Banga, *A protocol for dynamic model calibration*, Briefings in Bioinformatics (2023).](https://academic.oup.com/bib/article/23/1/bbab387/6383562?login=false)

test/reactionsystem_core/events.jl

@@ -243,10 +243,6 @@ let
     sol_dsl = solve(ODEProblem(rn_dsl, u0, tspan, ps), Tsit5())
     sol_prog = solve(ODEProblem(rn_prog, u0, tspan, ps), Tsit5())
     @test sol_dsl == sol_prog
-
-    sol_dsl = solve(SDEProblem(rn_dsl, u0, tspan, ps), ImplicitEM(); seed = 1234)
-    sol_prog = solve(SDEProblem(rn_prog, u0, tspan, ps), ImplicitEM(); seed = 1234)
-    @test sol_dsl == sol_prog
 end
 
 # Checks that misformatted events yields errors in the DSL.
@@ -313,6 +309,65 @@ end
 
 ### Additional Correctness Tests ###
 
+# Tests that events are properly triggered for SDEs.
+# Tests for continuous events, and all three types of discrete events.
+let
+    # Creates model with all types of events. The `e` parameters track whether events are triggered.
+    rn = @reaction_network begin
+        @parameters e1=0 e2=0 e3=0 e4=0
+        @continuous_events begin
+            [X ~ 1000.0] => [e1 ~ 1]
+        end
+        @discrete_events begin
+            [1.0] => [e2 ~ 1]
+            1.0 => [e3 ~ 1]
+            (Y > 1000.0) & (e4==0) => [e4 ~ 1]
+        end
+        (p,d), 0 <--> X
+        (p,d), 0 <--> Y
+    end
+
+    # Simulates the model for conditions where it *definitely* will cross `X = 1000.0`
+    u0 = [:X => 999.9, :Y => 999.9]
+    ps = [:p => 10.0, :d => 0.001]
+    sprob = SDEProblem(rn, u0, (0.0, 2.0), ps)
+    sol = solve(sprob, ImplicitEM(); seed)
+
+    # Checks that all `e` parameters have been updated properly.
+    @test sol.ps[:e1] == 1
+    @test sol.ps[:e2] == 1
+    @test sol.ps[:e3] == 1
+    @test sol.ps[:e4] == 1
+end
+
+# Tests that events are properly triggered for Jump simulations.
+# Tests for all three types of discrete events.
+let
+    # Creates model with all types of events. The `e` parameters track whether events are triggered.
+    rn = @reaction_network begin
+        @parameters e1=0 e2=0 e3=0
+        @discrete_events begin
+            [1.0] => [e1 ~ 1]
+            # 1.0 => [e2 ~ 1]
+            (X > 1000.0) & (e3==0) => [e3 ~ 1]
+        end
+        (p,d), 0 <--> X
+    end
+
+    # Simulates the model for conditions where it *definitely* will cross `X = 1000.0`
+    u0 = [:X => 999]
+    ps = [:p => 10.0, :d => 0.001]
+    dprob = DiscreteProblem(rn, u0, (0.0, 2.0), ps)
+    jprob = JumpProblem(rn, dprob, Direct(); rng)
+    sol = solve(jprob, SSAStepper(); seed)
+
+    # Checks that all `e` parameters have been updated properly.
+    # Note that periodic discrete events are currently broken for jump processes.
+    @test sol.ps[:e1] == 1
+    @test_broken sol.ps[:e2] == 1
+    @test sol.ps[:e3] == 1
+end
+
 # Compares simulations using MTK type events with those generated through callbacks.
 # Jump simulations must be handles differently (since these only accepts discrete callbacks).
 # Checks for all types of discrete callbacks, and for continuous callbacks.