Skip to content

Conversation

@danieljvickers
Copy link
Member

@danieljvickers danieljvickers commented Sep 29, 2025

User description

Description

This PR adds a 1st-order moving immersed boundary method, which will need to be iterated upon. This implimented will require some additional work via adding rotations, higher-order time stepping, and performance optimizations. But it puts in the framework for moving immersed boundaries that can be built upon. I would like to move forward with merging, and then going back to optimize the code, since the current functionality completely captures the scope of a new feature. The code still performs optimally when the new feature is disabled.

This PR also refactors the patch frame work to separate the notions of ICPP and IB patches, which allows IB patches to be moved into a common directory.

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

Scope

  • This PR comprises a set of related changes with a common goal

If you cannot check the above box, please split your PR into multiple PRs that each have a common goal.

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.
Provide instructions so we can reproduce.
Please also list any relevant details for your test configuration

  • I verified this version by running multiple tests of immersed boundaries moving through a stationary and moving fluid, and creating vortexes. A quiver plot of a circle traveling through a stationary fluid on a 50x50 grid can be viewed at: https://youtu.be/7App5aeE4rI. I will soon add a new example case that contains this case file run. View below:

Test Configuration:

import json
import math

Mu = 1.84e-05
gam_a = 1.4

# Configuring case dictionary
print(
    json.dumps(
        {
            # Logistics
            "run_time_info": "T",
            # Computational Domain Parameters
            # For these computations, the cylinder is placed at the (0,0,0)
            # domain origin.
            # axial direction
            "x_domain%beg": 0.0e00,
            "x_domain%end": 6.0e-03,
            # r direction
            "y_domain%beg": 0.0e00,
            "y_domain%end": 6.0e-03,
            "cyl_coord": "F",
            "m": 50,
            "n": 50,
            "p": 0,
            "dt": 6.0e-5,
            "t_step_start": 0,
            "t_step_stop": 1000,
            "t_step_save": 100,
            # Simulation Algorithm Parameters
            # Only one patches are necessary, the air tube
            "num_patches": 1,
            # Use the 5 equation model
            "model_eqns": 2,
            "alt_soundspeed": "F",
            # One fluids: air
            "num_fluids": 1,
            # time step
            "mpp_lim": "F",
            # Correct errors when computing speed of sound
            "mixture_err": "T",
            # Use TVD RK3 for time marching
            "time_stepper": 3,
            # Use WENO5
            "weno_order": 5,
            "weno_eps": 1.0e-16,
            "weno_Re_flux": "T",
            "weno_avg": "T",
            "avg_state": 2,
            "mapped_weno": "T",
            "null_weights": "F",
            "mp_weno": "T",
            "riemann_solver": 2,
            "wave_speeds": 1,
            # We use ghost-cell
            "bc_x%beg": -3,
            "bc_x%end": -3,
            "bc_y%beg": -3,
            "bc_y%end": -3,
            # Set IB to True and add 1 patch
            "ib": "T",
            "num_ibs": 1,
            "viscous": "T",
            # Formatted Database Files Structure Parameters
            "format": 1,
            "precision": 2,
            "prim_vars_wrt": "T",
            "E_wrt": "T",
            "parallel_io": "T",
            # Patch: Constant Tube filled with air
            # Specify the cylindrical air tube grid geometry
            "patch_icpp(1)%geometry": 3,
            "patch_icpp(1)%x_centroid": 3.0e-03,
            # Uniform medium density, centroid is at the center of the domain
            "patch_icpp(1)%y_centroid": 3.0e-03,
            "patch_icpp(1)%length_x": 6.0e-03,
            "patch_icpp(1)%length_y": 6.0e-03,
            # Specify the patch primitive variables
            "patch_icpp(1)%vel(1)": 0.00e00,
            "patch_icpp(1)%vel(2)": 0.0e00,
            "patch_icpp(1)%pres": 1.0e00,
            "patch_icpp(1)%alpha_rho(1)": 1.0e00,
            "patch_icpp(1)%alpha(1)": 1.0e00,
            # Patch: Cylinder Immersed Boundary
            "patch_ib(1)%geometry": 2,
            "patch_ib(1)%x_centroid": 4.5e-03,
            "patch_ib(1)%y_centroid": 3.0e-03,
            "patch_ib(1)%radius": 0.2e-03,
            "patch_ib(1)%slip": "F",
            "patch_ib(1)%moving_ibm": 1,
            "patch_ib(1)%vel(1)": -0.05e00,
            "patch_ib(1)%vel(2)": 0.0,
            "patch_ib(1)%vel(3)": 0.0,
            # Fluids Physical Parameters
            "fluid_pp(1)%gamma": 1.0e00 / (gam_a - 1.0e00),  # 2.50(Not 1.40)
            "fluid_pp(1)%pi_inf": 0,
            "fluid_pp(1)%Re(1)": 2500000,
        }
    )
)
  • What computers and compilers did you use to test this:

I tested this locally on GNU compilers with and without MPI enabled. I also tested on Wingtip with NVHPC compilers with and without MPI and on GPUs.

Checklist

  • I have added comments for the new code
  • I added Doxygen docstrings to the new code
  • I have made corresponding changes to the documentation (docs/)
  • I have added regression tests to the test suite so that people can verify in the future that the feature is behaving as expected
  • I have added example cases in examples/ that demonstrate my new feature performing as expected.
    They run to completion and demonstrate "interesting physics"
  • I ran ./mfc.sh format before committing my code
  • New and existing tests pass locally with my changes, including with GPU capability enabled (both NVIDIA hardware with NVHPC compilers and AMD hardware with CRAY compilers) and disabled
  • This PR does not introduce any repeated code (it follows the DRY principle)
  • I cannot think of a way to condense this code and reduce any introduced additional line count

If your code changes any code source files (anything in src/simulation)

To make sure the code is performing as expected on GPU devices, I have:

  • Checked that the code compiles using NVHPC compilers
  • Checked that the code compiles using CRAY compilers
  • Ran the code on either V100, A100, or H100 GPUs and ensured the new feature performed as expected (the GPU results match the CPU results)
  • Ran the code on MI200+ GPUs and ensure the new features performed as expected (the GPU results match the CPU results)
  • Enclosed the new feature via nvtx ranges so that they can be identified in profiles
  • Ran a Nsight Systems profile using ./mfc.sh run XXXX --gpu -t simulation --nsys, and have attached the output file (.nsys-rep) and plain text results to this PR
  • Ran a Rocprof Systems profile using ./mfc.sh run XXXX --gpu -t simulation --rsys --hip-trace, and have attached the output file and plain text results to this PR.

PR Type

Enhancement


Description

• Implements a first-order moving immersed boundary method with framework for future optimizations
• Refactors patch system to separate ICPP and IB patch functionality into distinct modules
• Adds new m_ib_patches module with geometric shape functions for circles, rectangles, spheres, cuboids, cylinders, airfoils, and STL models
• Introduces moving boundary support with position propagation using Euler's method and velocity handling
• Adds MPI support for broadcasting moving immersed boundary parameters
• Updates toolchain to support cross-target compatibility and new moving boundary parameters
• Includes CMake configuration updates to exclude specific modules from post-process target


Diagram Walkthrough

flowchart LR
  A["Original m_patches module"] --> B["m_icpp_patches module"]
  A --> C["m_ib_patches module"]
  C --> D["Moving IB implementation"]
  D --> E["Position propagation"]
  D --> F["Velocity handling"]
  G["MPI support"] --> D
  H["Toolchain updates"] --> D
Loading

File Walkthrough

Relevant files
Enhancement
12 files
m_icpp_patches.fpp
Refactor patches module to separate ICPP and IB functionality

src/pre_process/m_icpp_patches.fpp

• Renamed module from m_patches to m_icpp_patches and refactored to
separate ICPP and IB patch functionality
• Removed IB patch processing
logic and parameters from all geometry subroutines
• Added s_icpp_
prefix to all patch geometry subroutines for clarity
• Removed
optional ib_flag parameters and IB-specific conditional logic
throughout

+188/-761
m_ibm.fpp
Add moving immersed boundary method implementation             

src/simulation/m_ibm.fpp

• Added moving immersed boundary support with s_propagate_mib and
s_update_mib subroutines
• Implemented Euler's method for boundary
position updates and velocity handling
• Added
moving_immersed_boundary_flag to track if any boundaries are moving

Enhanced ghost point velocity assignment to handle moving boundaries

+98/-4   
m_mpi_common.fpp
Add integer reduction and fix MPI compilation structure   

src/common/m_mpi_common.fpp

• Added s_mpi_allreduce_integer_sum subroutine for integer reduction
operations
• Fixed conditional compilation structure for MPI
initialization
• Consolidated QBMM variable handling across
pre-process and simulation targets

+33/-22 
m_initial_condition.fpp
Update initial condition to use separated patch modules   

src/pre_process/m_initial_condition.fpp

• Updated to use separate m_ib_patches and m_icpp_patches modules

Modified patch application logic to call s_apply_ib_patches and
s_apply_icpp_patches separately

+5/-4     
m_mpi_proxy.fpp
Add MPI support for moving immersed boundary parameters   

src/simulation/m_mpi_proxy.fpp

• Added MPI broadcast support for new moving IB parameters moving_ibm
and vel

+2/-1     
m_global_parameters.fpp
Initialize moving immersed boundary parameters                     

src/pre_process/m_global_parameters.fpp

• Added initialization of moving immersed boundary parameters
moving_ibm and vel with default values

+6/-0     
m_derived_types.fpp
Add moving immersed boundary fields to patch parameters   

src/common/m_derived_types.fpp

• Added moving_ibm integer flag and vel velocity array to
ib_patch_parameters type

+5/-0     
m_checker.fpp
Add placeholder for moving IBM validation                               

src/pre_process/m_checker.fpp

• Added placeholder s_check_moving_IBM subroutine for future
validation

+5/-1     
m_start_up.fpp
Integrate moving immersed boundary updates in startup       

src/simulation/m_start_up.fpp

• Added conditional call to s_update_mib when moving immersed
boundaries are present

+4/-0     
m_start_up.fpp
Update startup module imports for separated patches           

src/pre_process/m_start_up.fpp

• Updated module imports to use separated m_ib_patches and
m_icpp_patches

+3/-1     
case.py
Enhance case FPP generation for cross-target compatibility

toolchain/mfc/case.py

• Modified FPP generation to include pre-processing includes in
simulation target
• Added support for @:analytical function access
across targets

+8/-6     
case_dicts.py
Add moving immersed boundary parameters to case dictionaries

toolchain/mfc/run/case_dicts.py

• Added moving_ibm parameter and vel velocity components to IB patch
parameter dictionaries

+10/-2   
Formatting
2 files
case.fpp
Minor formatting change in case include                                   

src/common/include/case.fpp

• Added empty line in analytical macro definition

+1/-0     
run.py
Minor formatting adjustment                                                           

toolchain/mfc/run/run.py

• Minor formatting change with extra whitespace

+1/-1     
Configuration changes
2 files
CMakeLists.txt
Update CMake configuration for separated modules                 

CMakeLists.txt

• Added exclusion of m_compute_levelset.fpp and m_ib_patches.fpp from
post_process target
• Added stdc++ library linking for SILO support

+8/-1     
settings.json
Disable fortls language server in VSCode settings               

.vscode/settings.json

• Disabled fortls language server by setting fortran.fortls.disabled
to true

+1/-1     
New feature
1 files
m_ib_patches.fpp
New immersed boundary patches module with geometric shapes

src/common/m_ib_patches.fpp

• Creates a new module m_ib_patches dedicated to immersed boundary
(IB) patch handling
• Implements geometric shape functions for IB
patches including circles, rectangles, spheres, cuboids, cylinders,
airfoils, and STL models
• Adds coordinate conversion utilities for
cylindrical to cartesian and spherical coordinates
• Provides levelset
computation capabilities for various geometric shapes used in immersed
boundary methods

+1037/-0
Additional files
6 files
1dHardcodedIC.fpp [link]   
2dHardcodedIC.fpp [link]   
3dHardcodedIC.fpp [link]   
ExtrusionHardcodedIC.fpp [link]   
m_compute_levelset.fpp [link]   
m_model.fpp [link]   

…ndaries with moving imersed boundary variables
…r what is applied each loop. Wrote a seaparate function and working on passing everything in
…ib patch definitions. This is a precommit before I start separating the two files
@qodo-merge-pro
Copy link
Contributor

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Possible Issue

New moving-IB integration allocates ghost point arrays using globally reduced maxima but still declares them with sizes based on local num_gps/num_inner_gps in interfaces and loops. Verify consistency between allocation size, assumed-shape interfaces, and loops to avoid out-of-bounds or uninitialized elements on ranks with fewer ghost points.

$:GPU_UPDATE(host='[ib_markers%sf]')

! find the number of ghost points and set them to be the maximum total across ranks
call s_find_num_ghost_points(num_gps, num_inner_gps)
call s_mpi_allreduce_integer_sum(num_gps, max_num_gps)
call s_mpi_allreduce_integer_sum(num_inner_gps, max_num_inner_gps)

$:GPU_UPDATE(device='[num_gps, num_inner_gps]')
@:ALLOCATE(ghost_points(1:int(max_num_gps * 1.2)))
@:ALLOCATE(inner_points(1:int(max_num_inner_gps * 1.2)))

$:GPU_ENTER_DATA(copyin='[ghost_points,inner_points]')
Incomplete Feature

s_update_mib recomputes ib markers and ghost points each step but does not free/realloc ghost point arrays when counts increase beyond the 1.2 safety factor, nor does it update patch_id_fp anywhere after initial setup. Confirm time-stepping call sites and resizing logic to prevent overflow and stale patch IDs.

!> Resets the current indexes of immersed boundaries and replaces them after updating
!> the position of each moving immersed boundary
impure subroutine s_update_mib(num_ibs, levelset, levelset_norm)

    integer, intent(in) :: num_ibs
    type(levelset_field), intent(inout) :: levelset
    type(levelset_norm_field), intent(inout) :: levelset_norm

    integer :: i

    ! Clears the existing immersed boundary indices
    ib_markers%sf = 0

    do i = 1, num_ibs
        if (patch_ib(i)%moving_ibm /= 0) then
            call s_propagate_mib(i)  ! TODO :: THIS IS DONE TERRIBLY WITH EULER METHOD
        end if
    end do

    ! recompute the new ib_patch locations and broadcast them.
    call s_apply_ib_patches(ib_markers%sf(0:m, 0:n, 0:p), levelset, levelset_norm)
    call s_populate_ib_buffers() ! transmits the new IB markers via MPI

    ! recalculate the ghost point locations and coefficients
    call s_find_num_ghost_points(num_gps, num_inner_gps)
    $:GPU_UPDATE(device='[num_gps, num_inner_gps]')

    call s_find_ghost_points(ghost_points, inner_points)
    $:GPU_UPDATE(device='[ghost_points, inner_points]')

    call s_compute_image_points(ghost_points, levelset, levelset_norm)
    $:GPU_UPDATE(device='[ghost_points]')

    call s_compute_interpolation_coeffs(ghost_points)
    $:GPU_UPDATE(device='[ghost_points]')

end subroutine s_update_mib
API Refactor Risk

The IC/IB patch logic was split; icpp routines were renamed (s_* -> s_icpp_*), and IB logic removed from this module. Ensure all call sites were updated (e.g., m_initial_condition now calls s_apply_icpp_patches and s_apply_ib_patches) and that any remaining references to old names are removed to prevent link-time or runtime errors.

    use m_mpi_common

    use m_ib_patches

    implicit none

    private; public :: s_apply_icpp_patches

    real(wp) :: x_centroid, y_centroid, z_centroid
    real(wp) :: length_x, length_y, length_z

    integer :: smooth_patch_id
    real(wp) :: smooth_coeff !<
    !! These variables are analogous in both meaning and use to the similarly
    !! named components in the ic_patch_parameters type (see m_derived_types.f90
    !! for additional details). They are employed as a means to more concisely
    !! perform the actions necessary to lay out a particular patch on the grid.

    real(wp) :: eta !<
    !! In the case that smoothing of patch boundaries is enabled and the boundary
    !! between two adjacent patches is to be smeared out, this variable's purpose
    !! is to act as a pseudo volume fraction to indicate the contribution of each
    !! patch toward the composition of a cell's fluid state.

    real(wp) :: cart_x, cart_y, cart_z
    real(wp) :: sph_phi !<
    !! Variables to be used to hold cell locations in Cartesian coordinates if
    !! 3D simulation is using cylindrical coordinates

    type(bounds_info) :: x_boundary, y_boundary, z_boundary  !<
    !! These variables combine the centroid and length parameters associated with
    !! a particular patch to yield the locations of the patch boundaries in the
    !! x-, y- and z-coordinate directions. They are used as a means to concisely
    !! perform the actions necessary to lay out a particular patch on the grid.

    character(len=5) :: istr ! string to store int to string result for error checking

contains

    impure subroutine s_apply_icpp_patches(patch_id_fp, q_prim_vf)

        type(scalar_field), dimension(1:sys_size), intent(inout) :: q_prim_vf
        integer, dimension(0:m, 0:m, 0:m), intent(inout) :: patch_id_fp

        integer :: i

        !  3D Patch Geometries
        if (p > 0) then

            do i = 1, num_patches

                if (proc_rank == 0) then
                    print *, 'Processing patch', i
                end if

                !> ICPP Patches
                !> @{
                ! Spherical patch
                if (patch_icpp(i)%geometry == 8) then
                    call s_icpp_sphere(i, patch_id_fp, q_prim_vf)
                    ! Cuboidal patch
                elseif (patch_icpp(i)%geometry == 9) then
                    call s_icpp_cuboid(i, patch_id_fp, q_prim_vf)
                    ! Cylindrical patch
                elseif (patch_icpp(i)%geometry == 10) then
                    call s_icpp_cylinder(i, patch_id_fp, q_prim_vf)
                    ! Swept plane patch
                elseif (patch_icpp(i)%geometry == 11) then
                    call s_icpp_sweep_plane(i, patch_id_fp, q_prim_vf)
                    ! Ellipsoidal patch
                elseif (patch_icpp(i)%geometry == 12) then
                    call s_icpp_ellipsoid(i, patch_id_fp, q_prim_vf)
                    ! Spherical harmonic patch
                elseif (patch_icpp(i)%geometry == 14) then
                    call s_icpp_spherical_harmonic(i, patch_id_fp, q_prim_vf)
                    ! 3D Modified circular patch
                elseif (patch_icpp(i)%geometry == 19) then
                    call s_icpp_3dvarcircle(i, patch_id_fp, q_prim_vf)
                    ! 3D STL patch
                elseif (patch_icpp(i)%geometry == 21) then
                    call s_icpp_model(i, patch_id_fp, q_prim_vf)
                end if
            end do
            !> @}

            ! 2D Patch Geometries
        elseif (n > 0) then

            do i = 1, num_patches

                if (proc_rank == 0) then
                    print *, 'Processing patch', i
                end if

                !> ICPP Patches
                !> @{
                ! Circular patch
                if (patch_icpp(i)%geometry == 2) then
                    call s_icpp_circle(i, patch_id_fp, q_prim_vf)
                    ! Rectangular patch
                elseif (patch_icpp(i)%geometry == 3) then
                    call s_icpp_rectangle(i, patch_id_fp, q_prim_vf)
                    ! Swept line patch
                elseif (patch_icpp(i)%geometry == 4) then
                    call s_icpp_sweep_line(i, patch_id_fp, q_prim_vf)
                    ! Elliptical patch
                elseif (patch_icpp(i)%geometry == 5) then
                    call s_icpp_ellipse(i, patch_id_fp, q_prim_vf)
                    ! Unimplemented patch (formerly isentropic vortex)
                elseif (patch_icpp(i)%geometry == 6) then
                    call s_mpi_abort('This used to be the isentropic vortex patch, '// &
                                     'which no longer exists. See Examples. Exiting.')
                    ! Spherical Harmonic Patch
                elseif (patch_icpp(i)%geometry == 14) then
                    call s_icpp_spherical_harmonic(i, patch_id_fp, q_prim_vf)
                    ! Spiral patch
                elseif (patch_icpp(i)%geometry == 17) then
                    call s_icpp_spiral(i, patch_id_fp, q_prim_vf)
                    ! Modified circular patch
                elseif (patch_icpp(i)%geometry == 18) then
                    call s_icpp_varcircle(i, patch_id_fp, q_prim_vf)
                    ! TaylorGreen vortex patch
                elseif (patch_icpp(i)%geometry == 20) then
                    call s_icpp_2D_TaylorGreen_vortex(i, patch_id_fp, q_prim_vf)
                    ! STL patch
                elseif (patch_icpp(i)%geometry == 21) then
                    call s_icpp_model(i, patch_id_fp, q_prim_vf)
                end if
                !> @}
            end do

            ! 1D Patch Geometries
        else

            do i = 1, num_patches

                if (proc_rank == 0) then
                    print *, 'Processing patch', i
                end if

                ! Line segment patch
                if (patch_icpp(i)%geometry == 1) then
                    call s_icpp_line_segment(i, patch_id_fp, q_prim_vf)
                    ! 1d analytical
                elseif (patch_icpp(i)%geometry == 16) then
                    call s_icpp_1d_bubble_pulse(i, patch_id_fp, q_prim_vf)
                end if
            end do

        end if

    end subroutine s_apply_icpp_patches

    !>          The line segment patch is a 1D geometry that may be used,
    !!              for example, in creating a Riemann problem. The geometry
    !!              of the patch is well-defined when its centroid and length
    !!              in the x-coordinate direction are provided. Note that the
    !!              line segment patch DOES NOT allow for the smearing of its
    !!              boundaries.
    !! @param patch_id patch identifier
    !! @param patch_id_fp Array to track patch ids
    !! @param q_prim_vf Array of primitive variables
    subroutine s_icpp_line_segment(patch_id, patch_id_fp, q_prim_vf)

        integer, intent(in) :: patch_id
        integer, dimension(0:m, 0:n, 0:p), intent(inout) :: patch_id_fp
        type(scalar_field), dimension(1:sys_size), intent(inout) :: q_prim_vf

        ! Generic loop iterators
        integer :: i, j, k

        ! Placeholders for the cell boundary values
        real(wp) :: pi_inf, gamma, lit_gamma
        @:HardcodedDimensionsExtrusion()
        @:Hardcoded1DVariables()

        pi_inf = fluid_pp(1)%pi_inf
        gamma = fluid_pp(1)%gamma
        lit_gamma = (1._wp + gamma)/gamma
        j = 0
        k = 0

        ! Transferring the line segment's centroid and length information
        x_centroid = patch_icpp(patch_id)%x_centroid
        length_x = patch_icpp(patch_id)%length_x

        ! Computing the beginning and end x-coordinates of the line segment
        ! based on its centroid and length
        x_boundary%beg = x_centroid - 0.5_wp*length_x
        x_boundary%end = x_centroid + 0.5_wp*length_x

        ! Since the line segment patch does not allow for its boundaries to
        ! be smoothed out, the pseudo volume fraction is set to 1 to ensure
        ! that only the current patch contributes to the fluid state in the
        ! cells that this patch covers.
        eta = 1._wp

        ! Checking whether the line segment covers a particular cell in the
        ! domain and verifying whether the current patch has the permission
        ! to write to that cell. If both queries check out, the primitive
        ! variables of the current patch are assigned to this cell.
        do i = 0, m
            if (x_boundary%beg <= x_cc(i) .and. &
                x_boundary%end >= x_cc(i) .and. &
                patch_icpp(patch_id)%alter_patch(patch_id_fp(i, 0, 0))) then

                call s_assign_patch_primitive_variables(patch_id, i, 0, 0, &
                                                        eta, q_prim_vf, patch_id_fp)

                @:analytical()

                ! check if this should load a hardcoded patch
                if (patch_icpp(patch_id)%hcid /= dflt_int) then
                    @:Hardcoded1D()
                end if

                ! Updating the patch identities bookkeeping variable
                if (1._wp - eta < sgm_eps) patch_id_fp(i, 0, 0) = patch_id

            end if
        end do
        @:HardcodedDellacation()

    end subroutine s_icpp_line_segment

    !>  The spiral patch is a 2D geometry that may be used, The geometry
        !!              of the patch is well-defined when its centroid and radius
        !!              are provided. Note that the circular patch DOES allow for
        !!              the smoothing of its boundary.
        !! @param patch_id patch identifier
        !! @param patch_id_fp Array to track patch ids
        !! @param q_prim_vf Array of primitive variables
    impure subroutine s_icpp_spiral(patch_id, patch_id_fp, q_prim_vf)

        integer, intent(in) :: patch_id
        integer, dimension(0:m, 0:n, 0:p), intent(inout) :: patch_id_fp
        type(scalar_field), dimension(1:sys_size), intent(inout) :: q_prim_vf

        integer :: i, j, k !< Generic loop iterators
        real(wp) :: th, thickness, nturns, mya
        real(wp) :: spiral_x_min, spiral_x_max, spiral_y_min, spiral_y_max
        @:HardcodedDimensionsExtrusion()
        @:Hardcoded2DVariables()

        ! Transferring the circular patch's radius, centroid, smearing patch
        ! identity and smearing coefficient information
        x_centroid = patch_icpp(patch_id)%x_centroid
        y_centroid = patch_icpp(patch_id)%y_centroid
        mya = patch_icpp(patch_id)%radius
        thickness = patch_icpp(patch_id)%length_x
        nturns = patch_icpp(patch_id)%length_y

        !
        logic_grid = 0
        do k = 0, int(m*91*nturns)
            th = k/real(int(m*91._wp*nturns))*nturns*2._wp*pi

            spiral_x_min = minval((/f_r(th, 0.0_wp, mya)*cos(th), &
                                    f_r(th, thickness, mya)*cos(th)/))
            spiral_y_min = minval((/f_r(th, 0.0_wp, mya)*sin(th), &
                                    f_r(th, thickness, mya)*sin(th)/))

            spiral_x_max = maxval((/f_r(th, 0.0_wp, mya)*cos(th), &
                                    f_r(th, thickness, mya)*cos(th)/))
            spiral_y_max = maxval((/f_r(th, 0.0_wp, mya)*sin(th), &
                                    f_r(th, thickness, mya)*sin(th)/))

            do j = 0, n; do i = 0, m; 
                    if ((x_cc(i) > spiral_x_min) .and. (x_cc(i) < spiral_x_max) .and. &
                        (y_cc(j) > spiral_y_min) .and. (y_cc(j) < spiral_y_max)) then
                        logic_grid(i, j, 0) = 1
                    end if
                end do; end do
        end do

        do j = 0, n
            do i = 0, m
                if ((logic_grid(i, j, 0) == 1)) then
                    call s_assign_patch_primitive_variables(patch_id, i, j, 0, &
                                                            eta, q_prim_vf, patch_id_fp)

                    @:analytical()
                    if (patch_icpp(patch_id)%hcid /= dflt_int) then
                        @:Hardcoded2D()
                    end if

                    ! Updating the patch identities bookkeeping variable
                    if (1._wp - eta < sgm_eps) patch_id_fp(i, j, 0) = patch_id
                end if
            end do
        end do
        @:HardcodedDellacation()

    end subroutine s_icpp_spiral

    !> The circular patch is a 2D geometry that may be used, for
        !!              example, in creating a bubble or a droplet. The geometry
        !!              of the patch is well-defined when its centroid and radius
        !!              are provided. Note that the circular patch DOES allow for
        !!              the smoothing of its boundary.
        !! @param patch_id is the patch identifier
        !! @param patch_id_fp Array to track patch ids
        !! @param q_prim_vf Array of primitive variables
    subroutine s_icpp_circle(patch_id, patch_id_fp, q_prim_vf)

        integer, intent(in) :: patch_id
        integer, dimension(0:m, 0:n, 0:p), intent(inout) :: patch_id_fp
        type(scalar_field), dimension(1:sys_size), intent(inout) :: q_prim_vf

        real(wp) :: radius

        integer :: i, j, k !< Generic loop iterators
        @:HardcodedDimensionsExtrusion()
        @:Hardcoded2DVariables()

        ! Transferring the circular patch's radius, centroid, smearing patch
        ! identity and smearing coefficient information

        x_centroid = patch_icpp(patch_id)%x_centroid
        y_centroid = patch_icpp(patch_id)%y_centroid
        radius = patch_icpp(patch_id)%radius
        smooth_patch_id = patch_icpp(patch_id)%smooth_patch_id
        smooth_coeff = patch_icpp(patch_id)%smooth_coeff

        ! Initializing the pseudo volume fraction value to 1. The value will
        ! be modified as the patch is laid out on the grid, but only in the
        ! case that smoothing of the circular patch's boundary is enabled.
        eta = 1._wp

        ! Checking whether the circle covers a particular cell in the domain
        ! and verifying whether the current patch has permission to write to
        ! that cell. If both queries check out, the primitive variables of
        ! the current patch are assigned to this cell.

        do j = 0, n
            do i = 0, m

                if (patch_icpp(patch_id)%smoothen) then

                    eta = tanh(smooth_coeff/min(dx, dy)* &
                               (sqrt((x_cc(i) - x_centroid)**2 &
                                     + (y_cc(j) - y_centroid)**2) &
                                - radius))*(-0.5_wp) + 0.5_wp

                end if

                if (((x_cc(i) - x_centroid)**2 &
                     + (y_cc(j) - y_centroid)**2 <= radius**2 &
                     .and. &
                     patch_icpp(patch_id)%alter_patch(patch_id_fp(i, j, 0))) &
                    .or. &
                    patch_id_fp(i, j, 0) == smooth_patch_id) &
                    then

                    call s_assign_patch_primitive_variables(patch_id, i, j, 0, &
                                                            eta, q_prim_vf, patch_id_fp)

                    @:analytical()
                    if (patch_icpp(patch_id)%hcid /= dflt_int) then
                        @:Hardcoded2D()
                    end if

                end if
            end do
        end do
        @:HardcodedDellacation()

    end subroutine s_icpp_circle

    !>  The varcircle patch is a 2D geometry that may be used

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High-level Suggestion

Refactor the code to eliminate duplication in geometry calculation logic between m_icpp_patches.fpp and m_ib_patches.fpp. This can be achieved by creating shared geometry subroutines that handle the common calculations. [High-level, importance: 8]

Solution Walkthrough:

Before:

! In m_icpp_patches.fpp
subroutine s_icpp_sphere(patch_id, patch_id_fp, q_prim_vf)
  ...
  do k = 0, p; do j = 0, n; do i = 0, m
    is_inside = ((x_cc(i) - x_centroid)**2 + ... <= radius**2)
    if (is_inside and alter_patch(...)) then
      call s_assign_patch_primitive_variables(...)
    endif
  end do; end do; end do
end subroutine

! In m_ib_patches.fpp
subroutine s_ib_sphere(patch_id, ib_markers_sf)
  ...
  do k = 0, p; do j = 0, n; do i = 0, m
    is_inside = ((x_cc(i) - x_centroid)**2 + ... <= radius**2)
    if (is_inside) then
      ib_markers_sf(i, j, k) = patch_id
    endif
  end do; end do; end do
end subroutine

After:

! In a new shared module e.g., m_patch_geometry.fpp
subroutine s_apply_sphere_geometry(patch_id, patch_type, patch_id_fp, q_prim_vf)
  ! patch_type is 'icpp' or 'ib'
  ...
  if (patch_type == 'icpp') then
    x_centroid = patch_icpp(patch_id)%x_centroid
    ...
  else if (patch_type == 'ib') then
    x_centroid = patch_ib(patch_id)%x_centroid
    ...
  endif

  do k = 0, p; do j = 0, n; do i = 0, m
    is_inside = ((x_cc(i) - x_centroid)**2 + ... <= radius**2)
    if (is_inside) then
      if (patch_type == 'icpp' and alter_patch(...)) then
        call s_assign_patch_primitive_variables(...)
      else if (patch_type == 'ib') then
        patch_id_fp(i, j, k) = patch_id
      endif
    endif
  end do; end do; end do
end subroutine

Comment on lines +376 to +385
if (((x_cc(i) - x_centroid)**2 &
+ (y_cc(j) - y_centroid)**2 <= radius**2 &
.and. &
patch_icpp(patch_id)%alter_patch(patch_id_fp(i, j, 0))) &
.or. &
patch_id_fp(i, j, 0) == smooth_patch_id) &
then

patch_id_fp(i, j, 0) = patch_id
else
if (((x_cc(i) - x_centroid)**2 &
+ (y_cc(j) - y_centroid)**2 <= radius**2 &
.and. &
patch_icpp(patch_id)%alter_patch(patch_id_fp(i, j, 0))) &
.or. &
(.not. present(ib_flag) .and. patch_id_fp(i, j, 0) == smooth_patch_id)) &
then

call s_assign_patch_primitive_variables(patch_id, i, j, 0, &
eta, q_prim_vf, patch_id_fp)

@:analytical()
if (patch_icpp(patch_id)%hcid /= dflt_int) then
@:Hardcoded2D()
end if

end if
end if
end do
end do
@:HardcodedDellacation()

end subroutine s_circle

!! @param patch_id is the patch identifier
!! @param patch_id_fp Array to track patch ids
!! @param q_prim_vf Array of primitive variables
!! @param ib True if this patch is an immersed boundary
subroutine s_airfoil(patch_id, patch_id_fp, q_prim_vf, ib_flag)

integer, intent(in) :: patch_id
integer, dimension(0:m, 0:n, 0:p), intent(inout) :: patch_id_fp
type(scalar_field), dimension(1:sys_size), intent(inout) :: q_prim_vf
logical, optional, intent(in) :: ib_flag

real(wp) :: x0, y0, f, x_act, y_act, ca_in, pa, ma, ta, theta
real(wp) :: xa, yt, xu, yu, xl, yl, xc, yc, dycdxc, sin_c, cos_c
integer :: i, j, k
integer :: Np1, Np2

if (.not. present(ib_flag)) return
x0 = patch_ib(patch_id)%x_centroid
y0 = patch_ib(patch_id)%y_centroid
ca_in = patch_ib(patch_id)%c
pa = patch_ib(patch_id)%p
ma = patch_ib(patch_id)%m
ta = patch_ib(patch_id)%t
theta = pi*patch_ib(patch_id)%theta/180._wp

Np1 = int((pa*ca_in/dx)*20)
Np2 = int(((ca_in - pa*ca_in)/dx)*20)
Np = Np1 + Np2 + 1

allocate (airfoil_grid_u(1:Np))
allocate (airfoil_grid_l(1:Np))

airfoil_grid_u(1)%x = x0
airfoil_grid_u(1)%y = y0

airfoil_grid_l(1)%x = x0
airfoil_grid_l(1)%y = y0

eta = 1._wp

do i = 1, Np1 + Np2 - 1
if (i <= Np1) then
xc = x0 + i*(pa*ca_in/Np1)
xa = (xc - x0)/ca_in
yc = (ma/pa**2)*(2*pa*xa - xa**2)
dycdxc = (2*ma/pa**2)*(pa - xa)
else
xc = x0 + pa*ca_in + (i - Np1)*((ca_in - pa*ca_in)/Np2)
xa = (xc - x0)/ca_in
yc = (ma/(1 - pa)**2)*(1 - 2*pa + 2*pa*xa - xa**2)
dycdxc = (2*ma/(1 - pa)**2)*(pa - xa)
end if

yt = (5._wp*ta)*(0.2969_wp*xa**0.5_wp - 0.126_wp*xa - 0.3516_wp*xa**2._wp + 0.2843_wp*xa**3 - 0.1015_wp*xa**4)
sin_c = dycdxc/(1 + dycdxc**2)**0.5_wp
cos_c = 1/(1 + dycdxc**2)**0.5_wp

xu = xa - yt*sin_c
yu = yc + yt*cos_c

xl = xa + yt*sin_c
yl = yc - yt*cos_c

xu = xu*ca_in + x0
yu = yu*ca_in + y0

xl = xl*ca_in + x0
yl = yl*ca_in + y0

airfoil_grid_u(i + 1)%x = xu
airfoil_grid_u(i + 1)%y = yu

airfoil_grid_l(i + 1)%x = xl
airfoil_grid_l(i + 1)%y = yl

end do

airfoil_grid_u(Np)%x = x0 + ca_in
airfoil_grid_u(Np)%y = y0

airfoil_grid_l(Np)%x = x0 + ca_in
airfoil_grid_l(Np)%y = y0

do j = 0, n
do i = 0, m

if (.not. f_is_default(patch_ib(patch_id)%theta)) then
x_act = (x_cc(i) - x0)*cos(theta) - (y_cc(j) - y0)*sin(theta) + x0
y_act = (x_cc(i) - x0)*sin(theta) + (y_cc(j) - y0)*cos(theta) + y0
else
x_act = x_cc(i)
y_act = y_cc(j)
end if
call s_assign_patch_primitive_variables(patch_id, i, j, 0, &
eta, q_prim_vf, patch_id_fp)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Initialize the eta variable to 1._wp before its conditional assignment to prevent the use of an uninitialized variable. [possible issue, importance: 8]

Suggested change
if (((x_cc(i) - x_centroid)**2 &
+ (y_cc(j) - y_centroid)**2 <= radius**2 &
.and. &
patch_icpp(patch_id)%alter_patch(patch_id_fp(i, j, 0))) &
.or. &
patch_id_fp(i, j, 0) == smooth_patch_id) &
then
patch_id_fp(i, j, 0) = patch_id
else
if (((x_cc(i) - x_centroid)**2 &
+ (y_cc(j) - y_centroid)**2 <= radius**2 &
.and. &
patch_icpp(patch_id)%alter_patch(patch_id_fp(i, j, 0))) &
.or. &
(.not. present(ib_flag) .and. patch_id_fp(i, j, 0) == smooth_patch_id)) &
then
call s_assign_patch_primitive_variables(patch_id, i, j, 0, &
eta, q_prim_vf, patch_id_fp)
@:analytical()
if (patch_icpp(patch_id)%hcid /= dflt_int) then
@:Hardcoded2D()
end if
end if
end if
end do
end do
@:HardcodedDellacation()
end subroutine s_circle
!! @param patch_id is the patch identifier
!! @param patch_id_fp Array to track patch ids
!! @param q_prim_vf Array of primitive variables
!! @param ib True if this patch is an immersed boundary
subroutine s_airfoil(patch_id, patch_id_fp, q_prim_vf, ib_flag)
integer, intent(in) :: patch_id
integer, dimension(0:m, 0:n, 0:p), intent(inout) :: patch_id_fp
type(scalar_field), dimension(1:sys_size), intent(inout) :: q_prim_vf
logical, optional, intent(in) :: ib_flag
real(wp) :: x0, y0, f, x_act, y_act, ca_in, pa, ma, ta, theta
real(wp) :: xa, yt, xu, yu, xl, yl, xc, yc, dycdxc, sin_c, cos_c
integer :: i, j, k
integer :: Np1, Np2
if (.not. present(ib_flag)) return
x0 = patch_ib(patch_id)%x_centroid
y0 = patch_ib(patch_id)%y_centroid
ca_in = patch_ib(patch_id)%c
pa = patch_ib(patch_id)%p
ma = patch_ib(patch_id)%m
ta = patch_ib(patch_id)%t
theta = pi*patch_ib(patch_id)%theta/180._wp
Np1 = int((pa*ca_in/dx)*20)
Np2 = int(((ca_in - pa*ca_in)/dx)*20)
Np = Np1 + Np2 + 1
allocate (airfoil_grid_u(1:Np))
allocate (airfoil_grid_l(1:Np))
airfoil_grid_u(1)%x = x0
airfoil_grid_u(1)%y = y0
airfoil_grid_l(1)%x = x0
airfoil_grid_l(1)%y = y0
eta = 1._wp
do i = 1, Np1 + Np2 - 1
if (i <= Np1) then
xc = x0 + i*(pa*ca_in/Np1)
xa = (xc - x0)/ca_in
yc = (ma/pa**2)*(2*pa*xa - xa**2)
dycdxc = (2*ma/pa**2)*(pa - xa)
else
xc = x0 + pa*ca_in + (i - Np1)*((ca_in - pa*ca_in)/Np2)
xa = (xc - x0)/ca_in
yc = (ma/(1 - pa)**2)*(1 - 2*pa + 2*pa*xa - xa**2)
dycdxc = (2*ma/(1 - pa)**2)*(pa - xa)
end if
yt = (5._wp*ta)*(0.2969_wp*xa**0.5_wp - 0.126_wp*xa - 0.3516_wp*xa**2._wp + 0.2843_wp*xa**3 - 0.1015_wp*xa**4)
sin_c = dycdxc/(1 + dycdxc**2)**0.5_wp
cos_c = 1/(1 + dycdxc**2)**0.5_wp
xu = xa - yt*sin_c
yu = yc + yt*cos_c
xl = xa + yt*sin_c
yl = yc - yt*cos_c
xu = xu*ca_in + x0
yu = yu*ca_in + y0
xl = xl*ca_in + x0
yl = yl*ca_in + y0
airfoil_grid_u(i + 1)%x = xu
airfoil_grid_u(i + 1)%y = yu
airfoil_grid_l(i + 1)%x = xl
airfoil_grid_l(i + 1)%y = yl
end do
airfoil_grid_u(Np)%x = x0 + ca_in
airfoil_grid_u(Np)%y = y0
airfoil_grid_l(Np)%x = x0 + ca_in
airfoil_grid_l(Np)%y = y0
do j = 0, n
do i = 0, m
if (.not. f_is_default(patch_ib(patch_id)%theta)) then
x_act = (x_cc(i) - x0)*cos(theta) - (y_cc(j) - y0)*sin(theta) + x0
y_act = (x_cc(i) - x0)*sin(theta) + (y_cc(j) - y0)*cos(theta) + y0
else
x_act = x_cc(i)
y_act = y_cc(j)
end if
call s_assign_patch_primitive_variables(patch_id, i, j, 0, &
eta, q_prim_vf, patch_id_fp)
eta = 1._wp
if (patch_icpp(patch_id)%smoothen) then
eta = tanh(smooth_coeff/min(dx, dy)* &
(sqrt((x_cc(i) - x_centroid)**2 &
+ (y_cc(j) - y_centroid)**2) &
- radius))*(-0.5_wp) + 0.5_wp
end if
if (((x_cc(i) - x_centroid)**2 &
+ (y_cc(j) - y_centroid)**2 <= radius**2 &
.and. &
patch_icpp(patch_id)%alter_patch(patch_id_fp(i, j, 0))) &
.or. &
patch_id_fp(i, j, 0) == smooth_patch_id) &
then
call s_assign_patch_primitive_variables(patch_id, i, j, 0, &
eta, q_prim_vf, patch_id_fp)

Comment on lines 1659 to 1671
if (patch_icpp(patch_id)%smoothen) then
if (eta > patch_icpp(patch_id)%model_threshold) then
eta = 1._wp
end if
else
if (patch_icpp(patch_id)%smoothen) then
if (eta > patch_icpp(patch_id)%model_threshold) then
eta = 1._wp
end if
if (eta > patch_icpp(patch_id)%model_threshold) then
eta = 1._wp
else
if (eta > patch_icpp(patch_id)%model_threshold) then
eta = 1._wp
else
eta = 0._wp
end if
eta = 0._wp
end if
call s_assign_patch_primitive_variables(patch_id, i, j, k, &
eta, q_prim_vf, patch_id_fp)

! Note: Should probably use *eta* to compute primitive variables
! if defining them analytically.
@:analytical()
end if
call s_assign_patch_primitive_variables(patch_id, i, j, k, &
eta, q_prim_vf, patch_id_fp)
Copy link
Contributor

@qodo-merge-pro qodo-merge-pro bot Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Correct the logic for eta calculation in s_icpp_model. When not smoothing, eta should be either 0._wp or 1._wp; otherwise, it should retain its fractional value. [possible issue, importance: 7]

Suggested change
if (patch_icpp(patch_id)%smoothen) then
if (eta > patch_icpp(patch_id)%model_threshold) then
eta = 1._wp
end if
else
if (patch_icpp(patch_id)%smoothen) then
if (eta > patch_icpp(patch_id)%model_threshold) then
eta = 1._wp
end if
if (eta > patch_icpp(patch_id)%model_threshold) then
eta = 1._wp
else
if (eta > patch_icpp(patch_id)%model_threshold) then
eta = 1._wp
else
eta = 0._wp
end if
eta = 0._wp
end if
call s_assign_patch_primitive_variables(patch_id, i, j, k, &
eta, q_prim_vf, patch_id_fp)
! Note: Should probably use *eta* to compute primitive variables
! if defining them analytically.
@:analytical()
end if
call s_assign_patch_primitive_variables(patch_id, i, j, k, &
eta, q_prim_vf, patch_id_fp)
if (.not. patch_icpp(patch_id)%smoothen) then
if (eta > patch_icpp(patch_id)%model_threshold) then
eta = 1._wp
else
eta = 0._wp
end if
end if
call s_assign_patch_primitive_variables(patch_id, i, j, k, &
eta, q_prim_vf, patch_id_fp)

Comment on lines 280 to 282
do while (airfoil_grid_u(k)%x < x_act)
k = k + 1
end do
Copy link
Contributor

@qodo-merge-pro qodo-merge-pro bot Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Add a bounds check to the do while loop that iterates with k to prevent a potential out-of-bounds memory access. [possible issue, importance: 8]

Suggested change
do while (airfoil_grid_u(k)%x < x_act)
k = k + 1
end do
do while (k < Np .and. airfoil_grid_u(k)%x < x_act)
k = k + 1
end do

Comment on lines +895 to +899
if (proc_rank == 0 .and. mod(cell_num, ncells/100) == 0) then
write (*, "(A, I3, A)", advance="no") &
char(13)//" * Generating grid: ", &
nint(100*real(cell_num)/ncells), "%"
end if
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Add a check to ensure ncells is at least 100 before calculating the modulus to prevent a potential division-by-zero error. [possible issue, importance: 7]

Suggested change
if (proc_rank == 0 .and. mod(cell_num, ncells/100) == 0) then
write (*, "(A, I3, A)", advance="no") &
char(13)//" * Generating grid: ", &
nint(100*real(cell_num)/ncells), "%"
end if
if (proc_rank == 0 .and. ncells >= 100 .and. mod(cell_num, ncells/100) == 0) then
write (*, "(A, I3, A)", advance="no") &
char(13)//" * Generating grid: ", &
nint(100*real(cell_num)/ncells), "%"
end if

@codecov
Copy link

codecov bot commented Sep 29, 2025

Codecov Report

❌ Patch coverage is 40.86022% with 330 lines in your changes missing coverage. Please review.
✅ Project coverage is 41.75%. Comparing base (eb152c5) to head (35fa4ad).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/common/m_ib_patches.fpp 42.45% 198 Missing and 27 partials ⚠️
src/pre_process/m_icpp_patches.fpp 39.23% 68 Missing and 11 partials ⚠️
src/simulation/m_ibm.fpp 4.00% 24 Missing ⚠️
src/simulation/m_start_up.fpp 0.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1006      +/-   ##
==========================================
- Coverage   41.94%   41.75%   -0.19%     
==========================================
  Files          69       70       +1     
  Lines       19904    20126     +222     
  Branches     2496     2504       +8     
==========================================
+ Hits         8348     8403      +55     
- Misses      10007    10180     +173     
+ Partials     1549     1543       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Collaborator

@wilfonba wilfonba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

benchmark

@danieljvickers
Copy link
Member Author

This last commit that I made only changed comments and the spell checker to make sure that all of the spelling passes. No changes were made to src. Since the spell check passes, we are safe to merge, I believe.

@sbryngelson
Copy link
Member

i don't think any test has gone through frontier ci yet?

@danieljvickers
Copy link
Member Author

After you'd extended the time, all tests passed but spelling.

@danieljvickers
Copy link
Member Author

Found that test run here: https://github.com/MFlowCode/MFC/actions/runs/18237509358

@sbryngelson sbryngelson merged commit c86fdd9 into MFlowCode:master Oct 5, 2025
33 checks passed
@danieljvickers danieljvickers deleted the add-moving-imersed-boundaries branch October 17, 2025 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants