ENH: integrate.solve_ivp: add verbose as optional argument to print time step info

[ma-sadeghi]

Reference issue

Fixes #11799

What does this implement/fix?

This PR adds verbose as an optional argument to solve_ivp method. The motivation, as described in the issue, was that for large-enough problems, solve_ivp takes a significant amount of time to complete and it feels like it's stuck.

Additional information

This is a sample output of calling solve_ivp with verbose=True:

1:  t = 2.01808e-05,  dt = 2.01808e-05
2:  t = 2.21989e-04,  dt = 2.01808e-04
3:  t = 1.12904e-03,  dt = 9.07053e-04
4:  t = 2.37318e-03,  dt = 1.24414e-03
5:  t = 4.15499e-03,  dt = 1.78181e-03
6:  t = 6.78473e-03,  dt = 2.62974e-03
7:  t = 1.00000e-02,  dt = 3.21527e-03

[miladsade96]

@ma-sadeghi Please use fix #11799 , fixes #11799 or closes #11799 to link related issue to this pull request in order to auto-close the issue after merging this PR.

[ma-sadeghi]

@everlookneversee Do you know why the build is failing?

[miladsade96]

@ma-sadeghi As you can see two of azure pipelines are cancelled after 60 minutes. I have no idea about that because i have no permission to azure pipeline details and re-running that. this issue happens sometimes in the other pull requests. @rgommers @ilayn Do you have any idea?!

[rgommers]

It's just Azure having a bad day, please just ignore the timeout.

[rgommers]

Thanks @ma-sadeghi.

Many older solvers have this kind of feature, but I'm not 100% sure if we want to introduce it for new code. So I'll wait for a second opinion here. @nmayorov any thoughts?

[nmayorov]

I think it will be useful -- being able to see that integration is at least progressing. I personally like running optimization routine with verbosity.

I sort of introduced a format which is used in least_squares, solve_bvp and I believe here as well. So maybe it's better to use a similar table-like format to reinforce some consistency within scipy. Here how it looks from least_squares:

   Iteration     Total nfev        Cost      Cost reduction    Step norm     Optimality   
       0              1         2.0389e+04                                    1.60e+06    
       1              2         2.8935e+02      2.01e+04       5.77e-02       1.83e+04    
       2              3         2.8884e+02      5.08e-01       6.11e-05       1.84e-01    
       3              4         2.8884e+02      6.27e-11       1.04e-09       1.35e-08    

I think the reasonable set of columns are Step number, Time, Step size (positive as solver.step_size), and probably Event (True/False, whether an event took place during the last step, printed only if events are present).

Also there could be a simple termination report, like "Successful reached the end in n steporFailed` (formulate exact phases).

And there are 3 levels of verbosity:

• 0 - silent
• 1 - termination report
• 2 - progress by iteration + termination report

These are only suggestions to consider.

[mdhaber]

There seemed to be some support here so I implemented @nmayorov's suggestions using formatting from solve_bvp. This should probably use more modern f-strings, and the events column should probably print True or False rather than an integer.

Alternatively, we could add a callback is suggested in the original issue.

[mdhaber]

What do you think, @tupui? Add verbose or close this in favor of a callback interface?

[tupui]

What do you think, @tupui? Add verbose or close this in favor of a callback interface?

I would say that one does not prevent the other if verbose is already done here, hence I am 0. Personally I am never using verbose flags and prefer callbacks with full introspection support (not a half backed solution as with optimize), which you can use to print anything you want.

[mdhaber]

@Kai-Striega I saw your post to the list, so I thought I'd ask you to weigh in here. If you'd be willing to contribute a callback interface to solve_ivp, I think we could close this, gh-14176, and gh-14771. I don't think it would take very much domain knowledge (just enough to figure out what variables are important to pass into the callback). Most of it is interface (what does the object passed into the callback look like, and when is callback called) and showing how the callback can be used to display information (like in this PR) and how it can be used to count events and terminate the solver (e.g. depending on what the callback function returns).