Benchmarks for model X parameter set combinations

[Vaibhav-Chopra-GT]

Description

Added benchmarks for SPM, SPMe, and DFN with all the parameter sets (except Sulzer2019 and Xu2019).

Some concerns -

• I could not solve the DFN model with Mohtat2020 parameter sets for the benchmarks (commented code). I tried varying dt_max and used different solvers but nothing worked. Here is a reproducible example for the same-

import numpy as np
import pybamm
import matplotlib.pyplot as plt

def prepare_model_Mohtat2020(model):
    geometry = model.default_geometry

    param = pybamm.ParameterValues("Mohtat2020")
    param.process_model(model)
    param.process_geometry(geometry)

    var_pts = {"x_n": 20, "x_s": 20, "x_p": 20, "r_n": 30, "r_p": 30, "y": 10, "z": 10}
    mesh = pybamm.Mesh(geometry, model.default_submesh_types, var_pts)

    disc = pybamm.Discretisation(mesh, model.default_spatial_methods)
    disc.process_model(model)

solver = pybamm.CasadiSolver()
model = pybamm.lithium_ion.DFN()
c_rate = 1
tmax = 4000 / c_rate
nb_points = 500
t_eval = np.linspace(0, tmax, nb_points)
prepare_model_Mohtat2020(model)
solver.solve(model, t_eval=t_eval)
solution = solver.solve(model, t_eval=t_eval)
voltage = solution['Terminal voltage [V]']
c_s_n_surf = solution['Negative particle surface concentration']
c_s_p_surf = solution['Positive particle surface concentration']
t = solution["Time [s]"].entries
x = solution["x [m]"].entries[:, 0]
f, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(13,4))

ax1.plot(t, voltage(t))
ax1.set_xlabel(r'$Time [s]$')
ax1.set_ylabel('Terminal voltage [V]')

ax2.plot(t, c_s_n_surf(t=t, x=x[0]))  
ax2.set_xlabel(r'$Time [s]$')
ax2.set_ylabel('Negative particle surface concentration')

ax3.plot(t, c_s_p_surf(t=t, x=x[-1])) 
ax3.set_xlabel(r'$Time [s]$')
ax3.set_ylabel('Positive particle surface concentration')

plt.tight_layout()
plt.show()

The error-

The linesearch algorithm failed with too small a step.
At t = 3.7305e-005 and h = 3.3406e-019, the corrector convergence failed repeatedly or with |h| = hmin.
At t = 3.73046e-005 and h = 3.12482e-019, the corrector convergence failed repeatedly or with |h| = hmin.
At t = 3.73044e-005 and h = 3.16429e-020, the corrector convergence failed repeatedly or with |h| = hmin.
At t = 3.7305e-005 and h = 3.3406e-019, the corrector convergence failed repeatedly or with |h| = hmin.
Traceback (most recent call last):
  File "d:\LearningProjects\python\second.py", line 25, in <module>
    solver.solve(model, t_eval=t_eval)
  File "C:\Users\Vaibhav Chopra\AppData\Local\Programs\Python\Python39\lib\site-packages\pybamm\solvers\base_solver.py", line 1018, in solve
    new_solution = self._integrate(
  File "C:\Users\Vaibhav Chopra\AppData\Local\Programs\Python\Python39\lib\site-packages\pybamm\solvers\casadi_solver.py", line 263, in _integrate
    raise pybamm.SolverError(
pybamm.expression_tree.exceptions.SolverError: Maximum number of decreased steps occurred at t=0.0. Try solving the model up to this time only or reducing dt_max (currently, dt_max=5.010622778599846).
PS D:\LearningProjects> python -u "d:\LearningProjects\python\second.py"
The linesearch algorithm failed with too small a step.
At t = 3.7305e-005 and h = 3.3406e-019, the corrector convergence failed repeatedly or with |h| = hmin.
At t = 3.73046e-005 and h = 3.12482e-019, the corrector convergence failed repeatedly or with |h| = hmin.
At t = 3.73044e-005 and h = 3.16429e-020, the corrector convergence failed repeatedly or with |h| = hmin.
At t = 3.7305e-005 and h = 3.3406e-019, the corrector convergence failed repeatedly or with |h| = hmin.
Traceback (most recent call last):
  File "d:\LearningProjects\python\second.py", line 25, in <module>
    solver.solve(model, t_eval=t_eval)
  File "C:\Users\Vaibhav Chopra\AppData\Local\Programs\Python\Python39\lib\site-packages\pybamm\solvers\base_solver.py", line 1018, in solve
    new_solution = self._integrate(
  File "C:\Users\Vaibhav Chopra\AppData\Local\Programs\Python\Python39\lib\site-packages\pybamm\solvers\casadi_solver.py", line 263, in _integrate
    raise pybamm.SolverError(
pybamm.expression_tree.exceptions.SolverError: Maximum number of decreased steps occurred at t=0.0. Try solving the model up to this time only or reducing dt_max (currently, dt_max=5.010622778599846).

• Should I group these benchmarks in some way. For example- multiple files with each file containing benchmarks of a particular parameter or model.
• As discussed with @brosaplanella on slack I have used ScikitsDaeSolver for some benchmarks, will this create a problem or is it safe to use different solvers for similar benchmarks?

Type of change

Please add a line in the relevant section of CHANGELOG.md to document the change (include PR #) - note reverse order of PR #s. If necessary, also add to the list of breaking changes.

• New feature (non-breaking change which adds functionality)
• Optimization (back-end change that speeds up the code)
• Bug fix (non-breaking change which fixes an issue)

Key checklist:

• No style issues: $ flake8
• All tests pass: $ python run-tests.py --unit
• The documentation builds: $ cd docs and then $ make clean; make html

You can run all three at once, using $ python run-tests.py --quick.

Further checks:

• Code is commented, particularly in hard-to-understand areas
• Tests added that prove fix is effective or that feature works

[codecov]

Codecov Report

Merging #2086 (deefe3d) into develop (2c74a4f) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #2086   +/-   ##
========================================
  Coverage    99.38%   99.38%           
========================================
  Files          348      348           
  Lines        19255    19255           
========================================
  Hits         19136    19136           
  Misses         119      119           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c74a4f...deefe3d. Read the comment docs.

[brosaplanella]

Hi Vaibhav! It looks pretty good, but there are some changes I suggested (mostly to make use of parameters in benchmarks so you can simplify your code a lot).

I couldn't attend last meeting, so if any of my comments contradicts anything you agreed previously do let me know (and apologies in advanced).

[valentinsulzer]

I agree with Ferran's comments about reducing code duplication. You could also use a subclass to do this, e.g.

class BaseTiming:
    def time_setup(self):
        self.param.process_model(self.model)
        compute_discretisation(self.model, self.param).process_model(self.model)
class TimeBuildSPMNCA_Kim2011(BaseTiming):
    def __init__(self):
        self.param = pybamm.ParameterValues("NCA_Kim2011")
        self.model = pybamm.lithium_ion.SPM()

For the Mohtat2020 model, it isn't working because the initial concentrations are for a completely discharged cell. Just skip that one for now and we can come back to it

[Vaibhav-Chopra-GT]

Thank you for the review @tinosulzer, @brosaplanella! I have parameterized the benchmarks and I am still looking into the solvers. Could you please review it again and give your feedback. Also, keeping the voltage same for every parameter did not throw any error, should I keep it the same or change it to match every parameter's lower voltage.

[brosaplanella]

Nice, it looks much cleaner now!

[brosaplanella]

Thank you for the review @tinosulzer, @brosaplanella! I have parameterized the benchmarks and I am still looking into the solvers. Could you please review it again and give your feedback. Also, keeping the voltage same for every parameter did not throw any error, should I keep it the same or change it to match every parameter's lower voltage.

For the voltage, it would be better to test down to the right voltage specified in the parameters so we ensure we have a full discharge. You can just get the value for Lower voltage cut-off [V] and format the string of the experiment.

[priyanshuone6]

Could you make a global variable with a list of these parameter sets outside the classes and use that in all the functions to reduce code duplication

[valentinsulzer]

Thanks, looks good!

[brosaplanella]

@priyanshuone6 do you have any further comments on the PR or shall I merge it?

[priyanshuone6]

Let’s merge this