Omp

[jguterl]

• Fix sfsetnk issue with first call to sf

• Fix v2cde issue in boundary.m

• Add option to keep bandwith adjustment when isphion=1 (variable bbb.ExtendedJacPhi)

• Add out of bound checks in the routine csrbnd. That should allow to extend the jacobian bandwidth for phi equations with the banded preconditionner.

• Add EvalDumpJac for jacobian bandwith analaysis (see UEDGEToolBox)

• Modification exmain.c for proper interruption of parallel evaluation with openmp: adding "stop" to abort exmain by waiting for the end of the jacobian/pandf1 evaluation before interrupting the process. Critical to avoid threads to be hanging after interruption (abort/exit cannot be used with openmp). Added a trap in nksol to interrupt nksol when exmain_aborted=True (requires adding dependency of svr on bbb in makefile and chnaing order in which pakcages are loaded in uedge.py). (Can we interrupt each thread wth a siglongjmp and sigaction?. Here is an example with pthreads:https://stackoverflow.com/questions/8138168/signal-handling-in-openmp-parallel-program. Apparently this is not possible. Need a condition somewhere in the threaded loop.)

• Semantic modification for fortran code parsing purpose:

  • Modif of aphrates.m to allow parsing of assigned variables in pandf1. This is a
  • Remove a non-standard linebreak in odesetup.m at line 7329 to allow parsing of the file.
  • Took out SROTMhG subroutine from comutil.m into a separate file to avoid parsing it(to messy to be parsed...)
  • Fix number of arguments in convsr_aux: convsr_aux(ix,iy,yl) -> convsr_aux(ix,iy)

• Add roadblockers in pandf to prevent non-validated usage of pandf with openmp

• Add GridFileName to read gridfile from a file different than gridue and add reading capabilities of simagxs and sibdrys (backward compatibility with old gridue files is maintained).

• add githash into the version.py so githash can be printed out...

• Parallelization of Pandf1:

  • Added Interface for pandf1 call in rhsnk entry
  • Added option for range of indexes in convert_vo and convert_aux for call to pandf1 with xc=-1 and yc>0

Guide for OMP JAc debug debug:

1. Check that Jocabian bandwith are consistent between jac_calc and localjacbuilder routines.
2. Check that newly added variables assigned in the scope of pandf are in the list of variables which are privatethreaded by Forthon.
3. Check that no newly added variables are assigned first out of the scope of pandf (then it needs to be copied first in each thread).
4. Check that newly added variables are properly initialized. It means that they should be initialized within pandf for the range of indexes used by pandf. Example: a qqty is not intialized for neutrals but later on used assuming it is zero in a loop over all species. Wth openmp, behavior of unitialized variables are undetermined in non-master threads. Such error can be checked using the debug flags FARGSDEBUG in setup.py.

---

Suggestion for improvment:

• Get rid of sptopdp and replace it with an interface, which is fortran compliant and is actually present in Fortran for that purpose.
• Create a new package (e.g. "phy") to completely separate "physics" from "numerics". "phy" should contain everyting in the scope of pandf1 (convert,potcur,boundary,pandf,...). This is pure copy-past and git can track moving files so it reamins fully compataible with previous versions.
  Because python requires call the parent object of method/variables, a decision needs to be made quickly...
• Versioning this new physics package with a sub git-repo. That will completely decouple the solver from the physics (let saysomeone want to work on improving the solver while someone else work on improving the physics package. That will guarantee backward compability. Good example is the openmp implementation independent of the physics in pandf1.)
• Replace distutils by setuptools in setup.py (compliance with python3 guidelines)
• Replace timing routine with routine compataible with openmp (see tick tock in parallel.F90)
• Put uedge code into a submodule and avoid mixing up py scripts and uedge.py
• Use setupgit to automatically set the version
• Add criterion for log interpolation in non-orthogonal cells (time consuming in convert and pandf!)
• Slice pandf in suroutines (not into too many small subroutines because subroutine have non-negligible time call (because on the stack)