Add new "WRF" mesoscale forcing capabilities (#652)

* Initial commit of WRF forcing

* Hacking away WRF forcing

* more hacking

* Some compilation related fixes

* Some more compilation related fixes

* And more compilation related fixes

* fix some more compilation issues

* initial commit temperature forcing

* WRF file class

* missed header file

* Move WRF file read to a separate class

* Missing header file

* cleaning up

* hacking on some errors

* moving constructor to cpp file

* more clean up header

* Comment some errors

* Accomodate comments from Shreyas

* some bug fixes

* some more bug fixes

* Fix errors with unique_ptr

* Fix compilation errors

* missing headers

* Compiled code ready for testing momentum forcing

* Debug Print statements

* WRF momentum forcing working; rebase on top of latest main

* Temperature forcing from WRF

* update cmakelists

* Update ABL to call temperature forcing

* Compiled WRF temperature forcing

* Update field init to initialize from profile and wall function surface t flux

* some compilation related bug fixes

* Output statistics of K_sgs

* put a check on tke field existence

* clang format some of the stats modifications

* Indirect momentum forcing

* k_sgs output statistics

* Bug fix

* Indirect forcing for both momentum and temperature

* Bug fix in temp forcing

* bug fix in polynomial fit

* read gain coeff from input file

* Multi-level WRF forcing

* Output wrf forcing in the stats file

* Bug fix - add forward declaration of WRF classes

* Bug fix - fixed the missing pointers to wrf mom and temp classes

* Height varying Bodyforce implementation

* remove dt from source term; instead supply bodyforce with dt

* Add bodyforce from WRF from precursor to temperature equation

* Fixed a typo in CmakeLists

* body force magnitude not needed for height vary

* WRF tendency forcing

* missing gain coeff read and missing call to tendency forcing in theta equation

* Create ABLWrfForcing superclass

* Perform parm parse in ABLWrfForcing constructor

* Put `m_wrf_ht` back in associated ABLWrfForcing*.H

* Add comments and debugging output

* Add dev readme

* Move hard-coded scaleFact to one place

* Add input weighting vector for ABLWrfForcing, with defaults

* Rename input weighting member vars

* Add forcing_transition; implement updateWeights()

* Move indirectForcingInit to mean-height update; fix reinitialization flag

* Implement weighting in Z^T W Z; save direct error profiles in indirect case

* Add additional inputs for different forcing_transition modes

* Add option `normalize_by_zmax` for consistency with SOWFA

* Minor cleanup

* Fix bug with when to call indirectForcingInit()

* Update conditions for calling indirectForcingInit()

* Add function override weighting profile based on the chosen forcing_transition

Still need to set m_transition_height at this point

* Implement indirectToConstant forcing transition

based on SOWFA6/src/ABLForcing/drivingForce/drivingForce.C

* Rearrange parmparse queries in ABLWrfForcing constructor

* Implement forcing_transition = "indirectToDirect"

* Add screen debug print control

Also, changed forcing_scheme to be a required input

* Switch to int debug flag

* has_var always returns 0 status code if the varid pointer is NULL

This may affect other functions in the netcdf interface as well!

* WIP: Implement transition_height read from netcdf

Lots of unexpected behaviors when attempting to use new netcdf file with
additional `transition_height` variable. Something appears to get corrutped
in memory. When running with full profile assimilation (transition_height
is unused) with the original input netcdf file (results should be identical to
before this commit), the forcing is corrupted, resulting in CFL blowing up
(violating the CFL constraint from step 2) and the debug flag somehow gets
turned off (the error profile is not printed to screen).

When running with partial profile assimilation (transition_height should be
read from the netcdf input), the solver performs no iterations after step 1--
is the forcing somehow 0?

* Restore backwards compatibility and expected behavior

Comment out changes to ABLWRFfile for now, add "FIXME" tag

* Implement workaround for d8db484

* Interpolate the instantaneous transition height; add datfile.is_open check

* Update README.profileAssim.md

* Fix merge

* Fixes to allow specifying surface temperature flux history w/o WRF profiles

Notes:
* `Temperature.source_terms = ABLWrfForcingTemp` needs to be set
* `ABLWrfForcing[Mom|Temp].forcing_scheme` does not need to be specified
* `nheight` and `arraySize` can be 0 in the forcing input file

* Allow WRF input file to not have any height information (if forcing_scheme=="")

* Fix formatting

* Add NETCDF preprocessor directive

* Cleanup BodyForce operator

* Remove unused inline function closest_index

* Improve style and readability

* More formatting fixes

* Define var for use in kernel function

* More clang-tidy fixes

* Rename ABLWrf.* source files

* Rename ABLWrfForcing class

* Create ABLMesoscaleInput.* from ABLMesoscaleForcing.* (previously part of ABLWrf.*)

* Generalize ABLWRFfile --> ABLMesoscaleInput

* Generalize member variable and function names ("wrf" --> "meso")

* Make consistent the names of register_meso_*_forcing() and associated pointers

* Minor cleanup

* Add ABL.mesoscale_forcing ncfile input; ABL.WRFforcing is deprecated

* Rename source classes ABLWrfForcing* --> ABLMesoForcing*

* Formatting

* Formatting

* Fix for memory leak

Discovered between 8746f4c..f45e5c7; cause unclear

* Fix to work with #667

* Fix merge, compatibility issues; do some cleanup

* Fix formatting

* formatting

* cppcheck and clang-tidy fixes

* Explicitly cast to int to avoid clang-tidy bugprone-narrowing-conversions warning

* gah formatting

* Additional clang-tidy cleanup

* More cppcheck

* Fix ABLMesoscaleInput instantiation

* Formatting

* Fixes for C++ "best practice"

* stupid formatting

* stupid formatting

* CodeQL clean up, remove transition_height workaround

Workaround was put in place in commit ec5eb95,
see that commit message for details. Affects indirect profile assimilation
with the following options set:
- forcing_transition = indirectToConstant or indirectToDirect
- constant_transition_height is NOT set

* formatting

* formatting

* clang-tidy fixes

* Silly formatting

* make codeql happy

---------

Co-authored-by: ShashankNREL <shashank.yellapantula@nrel.gov>

amr-wind/equation_systems/icns/source_terms/ABLMesoForcingMom.H

@@ -0,0 +1,83 @@
+#ifndef ABLMESOFORCINGMOM
+#define ABLMESOFORCINGMOM
+
+#include <AMReX_REAL.H>
+#include <AMReX_Vector.H>
+#include <memory>
+#include "amr-wind/core/FieldRepo.H"
+#include "amr-wind/equation_systems/icns/MomentumSource.H"
+#include "amr-wind/core/SimTime.H"
+#include "amr-wind/utilities/FieldPlaneAveragingFine.H"
+#include "amr-wind/utilities/ncutils/nc_interface.H"
+#include "amr-wind/wind_energy/ABLMesoscaleForcing.H"
+#include "amr-wind/wind_energy/ABLMesoscaleInput.H"
+
+namespace amr_wind::pde::icns {
+
+/** Forcing term for meso-micro scale coupled ABL simulations
+ *  \ingroup icns_src we_abl
+ *
+ *  \sa ABL
+ */
+
+class ABLMesoForcingMom
+    : public ABLMesoscaleForcing
+    , public MomentumSource::Register<ABLMesoForcingMom>
+{
+public:
+    static std::string identifier() { return "ABLMesoForcingMom"; }
+
+    explicit ABLMesoForcingMom(const CFDSim& sim);
+
+    ~ABLMesoForcingMom() override;
+
+    void operator()(
+        const int lev,
+        const amrex::MFIter& mfi,
+        const amrex::Box& bx,
+        const FieldState fstate,
+        const amrex::Array4<amrex::Real>& src_term) const override;
+
+    void mean_velocity_init(const ABLMesoscaleInput& ncfile);
+
+    void mean_velocity_init(
+        const VelPlaneAveragingFine& vavg, const ABLMesoscaleInput& ncfile);
+
+    void
+    mean_velocity_heights(std::unique_ptr<ABLMesoscaleInput> const& ncfile);
+
+    void mean_velocity_heights(
+        const VelPlaneAveragingFine& vavg,
+        std::unique_ptr<ABLMesoscaleInput> const& ncfile);
+
+    amrex::Vector<amrex::Real>& mom_u_error() { return m_err_U; }
+    amrex::Vector<amrex::Real>& mom_v_error() { return m_err_V; }
+
+private:
+    // these are inputs
+    amrex::Gpu::DeviceVector<amrex::Real> m_meso_ht;
+    amrex::Gpu::DeviceVector<amrex::Real> m_meso_u_vals;
+    amrex::Gpu::DeviceVector<amrex::Real> m_meso_v_vals;
+    amrex::Gpu::DeviceVector<amrex::Real> m_meso_avg_error;
+
+    // these are the instantaneous planar averages
+    amrex::Gpu::DeviceVector<amrex::Real> m_velAvg_ht;
+    amrex::Gpu::DeviceVector<amrex::Real> m_uAvg_vals;
+    amrex::Gpu::DeviceVector<amrex::Real> m_vAvg_vals;
+
+    // these specify the source term
+    amrex::Gpu::DeviceVector<amrex::Real> m_error_meso_avg_U;
+    amrex::Gpu::DeviceVector<amrex::Real> m_error_meso_avg_V;
+
+    amrex::Vector<amrex::Real> m_err_U;
+    amrex::Vector<amrex::Real> m_err_V;
+
+    // these should be m_ind_polyOrder+1 (currently, cubic polynomials are
+    // assumed)
+    amrex::Array<amrex::Real, 4> m_poly_coeff_U;
+    amrex::Array<amrex::Real, 4> m_poly_coeff_V;
+};
+
+} // namespace amr_wind::pde::icns
+
+#endif // ABLMESOFORCINGMOM_H