Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Coupled overset runs are broken #936

Closed
ashesh2512 opened this issue Feb 9, 2022 · 2 comments
Closed

Coupled overset runs are broken #936

ashesh2512 opened this issue Feb 9, 2022 · 2 comments
Labels
bug Something isn't working

Comments

@ashesh2512
Copy link
Contributor

ashesh2512 commented Feb 9, 2022 •
edited
Loading

Detailed description of the bug

Recently a few of us have noticed coupled overset runs failing in nalu. Not exactly sure where the issue lies. We have a regression test that does use coupled overset runs - overset cylinder. However - 1) The location of the hole does not change in that regression test which is the scenario where others have run into issues 2) The ghosting doesn't change significantly which is the case with turbine runs. Additionally, for a stationary mesh, coupled runs still appear to do fine.

Details to reproduce bug/build error

SHA ID of the Nalu-Wind commit you are using -
This bug is present as of the latest commit. Not quite sure when it was introduced, but this capability was thoroughly tested and worked as of the following commits -
Nalu-Wind GIT Commit SHA: 50ef780
Trilinos Version: 12.17-gadef041e9e

Details of your simulation (input file, mesh description, etc.) -
nrel5mw.txt

Stack trace of the runtime error output -
nrel5mw.log
@ashesh2512 ashesh2512 added the bug Something isn't working label Feb 9, 2022
@ddement
Copy link
Contributor

ddement commented Aug 1, 2023

@ashesh2512 Do you know where we can get access to the nrel5mw_overset.exo mesh used in this case? Could you perhaps put it in the proj-shared folder on Frontier's Orion filesystem? Also, what recipe did you use for running this case (number of nodes, GPUs, machine used, etc)?

ddement added a commit that referenced this issue Sep 19, 2023
@ddement
Fixes errors in ghosting update for some overset mesh problems.
4c5ae63 
Instead of trying to modify the ghosting for overset problems with
relative motion, we now just rebuild the ghosting from scratch.
This appears to fix errors such as those reported in #936
@ddement ddement mentioned this issue Sep 19, 2023
Merged
psakievich added a commit that referenced this issue Dec 23, 2023
@ddement @psakievich
Fixes errors in ghosting update for some overset mesh problems. (#1214)
70e576b 
Instead of trying to modify the ghosting for overset problems with
relative motion, we now just rebuild the ghosting from scratch.
This appears to fix errors such as those reported in #936

Co-authored-by: dcdemen <dcdemen@sandia.gov>
Co-authored-by: psakievich <psakiev@sandia.gov>
psakievich added a commit that referenced this issue Apr 30, 2024
@wjhorne @psakievich
@neilmatula @gantech @saitoasukakawaii @rcknaus @djglaze @ddement @jrood-nrel @itopcuoglu @mbkuhn
Rebase Multiphase_Dev and add VOF overset dpdx improvements (#1239)
003056d 
* Add Timers for FSI (#1221)

* Add Timers for FSI

* Style

* Add FSI section to Input File docs. (#1220)

* Add FSI section to Input File docs.

* Updating the FSI docs to incorporate PR comments.

* Relax projection for FSI mapping to account for curvature (#1223)

* Relax projection for FSI mapping to account for curvature

* Style

* Format again

* Add option to dump mesh on failed jacobian check (#1226)

* Add option to dump mesh on failed jacobian check

* Style

* Cleaner, parallel consistent impl

* Correctly set current_coordinates for restart (#1227)

* Actually set current_coordinates for restart

* Remove redundant call updating displacements

* Revert "Remove redundant call updating displacements"

This reverts commit dca015c.

* Fix restart issue for FSI simulations (#1228)

---------

Co-authored-by: Ganesh Vijayakumar <ganesh.vijayakumar@nrel.gov>

* Update FieldRegistry.h (#1229)

missing std::.

* FSI: Set ramping defaults to false (#1231)

* FSI: Set ramping defaults to false

Since we are moving to split meshes as the current strategy setting
ramping defaults to `False`.

I have been investigating why the temporal ramping is causing
simulations to fail since the bug fixes regarding hub motion, and I
can't find any issues with it at the moment. So it seems best to keep
that turned off as well.

* Update real defaults

* add a string-function temperature IC/dirichlet bc (#1198)

* add a string-function temperature IC/dirichlet bc

* undo stk deprecation fixes

---------

Co-authored-by: psakievich <psakiev@sandia.gov>

* Converted to the new STK simple_fields workflow (#1233)

STK is migrating to a new strategy for registering and managing
Fields, where sizing information is purely specified at run-time
instead of the previous technique of specifying it in a confusing
blend of both compile-time and run-time information.  The compile-time
specification was just a suggestion, as it could be overridden
(possibly inconsistently) at run-time to support variable-length
Fields.  This made it unclear what the true size of a Field was
and where it should be specified.

As an example, registering a vector field on the entire mesh
previously looked like this:

  using VectorField = stk::mesh::Field<double, stk::mesh::Cartesian3d>;
  VectorField & field = meta.declare_field<VectorField>(stk::topology::NODE_RANK, "velocity");
  stk::mesh::put_field_on_mesh(field, meta.universal_part(), 3, nullptr);

and now, it looks like this:

  using VectorField = stk::mesh::Field<double>;
  VectorField & field = meta.declare_field<double>(stk::topology::NODE_RANK, "velocity");
  stk::mesh::put_field_on_mesh(field, meta.universal_part(), 3, nullptr);

  stk::io::set_field_output_type(field, stk::io::FieldOutputType::VECTOR_3D); // Optional

The only template parameter for a Field is now the datatype parameter.
Sizing information now exclusively comes from put_field_on_mesh() calls.
The optional set_field_output_type() function call registers with the
IO sub-system how a multi-component Field should be subscripted in
Exodus files.  If this call is left off, you will get the default
[_1, _2, _3] subscripting.  With the above call, you will instead get
[_x, _y, _z] subscripting.

The MetaData::use_simple_fields() flag is set everywhere possible in
the code to prevent accidental regressions before the old behavior
is formally deprecated and removed.  This will yield a run-time error
if the old-style extra template parameters are used anywhere.  These
calls to use_simple_fields() can be removed in the future once the
STK Mesh back-end has removed support for the old behavior.

This wasn't a completely straightforward conversion due to nalu-wind
making heavy use of various algorithm selections based on the
templated Field type.  The ScalarFieldType, VectorFieldType,
TensorFieldType, and GenericFieldType types are now all identical,
so different techniques had to be used to switch behaviors.

* Fixes errors in ghosting update for some overset mesh problems. (#1214)

Instead of trying to modify the ghosting for overset problems with
relative motion, we now just rebuild the ghosting from scratch.
This appears to fix errors such as those reported in #936

Co-authored-by: dcdemen <dcdemen@sandia.gov>
Co-authored-by: psakievich <psakiev@sandia.gov>

* Revert "Converted to the new STK simple_fields workflow (#1233)" (#1234)

This reverts commit 8e8f4d5.

* Fixed the iblank inconsistency at shared nodes. The iblank field is returned to Tioga after the stk::mesh::copy_owned_to_shared operation, prior to field interpolation (#1238)

* Multiphase milestone (#1222)

* Divide through by density to get velocity form

* Fix VOF velocity flux to finalize

* Fix missing pressure gradient density norm and sharpen interfaces more

* Add diffusion term to VOF

* center droplet and provide velocity

* first version of sloshing tank

* sloshing tank case, can change parameters

* Mass-momentum consistency with initial vel scale and diffusion values

* Remove unused forced mass flux from pressure equation

* clean-up and safe settings for VOF advection

* Density face definition for minimized buoyancy noise

* rho_ref == initial density

* Change initial density to a specified function and introduce wall bcs for VOF

* sloshing tank pressure profile

* user function for generic flat water level in z

* Cleaning

* Balanced Buoyancy Forcing

* Complete milestone implementation

* Full implementation with fixed testing

* Fix allocation of fields

* Formatting

* Documentation of VOF advection scheme

---------

Co-authored-by: whorne <whorne@sandia.gov>
Co-authored-by: Michael Kuhn <mkuhn@nrel.gov>

* Rebase and pressure gradient improvements for VOF

* Fixed interface width work

* Allow slip of VOF at wall bcs

---------

Co-authored-by: psakievich <psakiev@sandia.gov>
Co-authored-by: neilmatula <72527988+neilmatula@users.noreply.github.com>
Co-authored-by: Ganesh Vijayakumar <ganesh.vijayakumar@nrel.gov>
Co-authored-by: 四月是你的谎言 <31875095+saitoasukakawaii@users.noreply.github.com>
Co-authored-by: rcknaus <rcknaus@sandia.gov>
Co-authored-by: djglaze <44038705+djglaze@users.noreply.github.com>
Co-authored-by: ddement <ddement@gatech.edu>
Co-authored-by: dcdemen <dcdemen@sandia.gov>
Co-authored-by: Jon Rood <jon.rood@nrel.gov>
Co-authored-by: itopcuoglu <114435459+itopcuoglu@users.noreply.github.com>
Co-authored-by: whorne <whorne@sandia.gov>
Co-authored-by: Michael Kuhn <mkuhn@nrel.gov>
@mbkuhn
Copy link
Contributor

mbkuhn commented Aug 5, 2024

Due to the age of this issue, and that a merged PR claims to fix it, I'll consider this issue closed.

@mbkuhn mbkuhn closed this as completed Aug 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ddement @mbkuhn @ashesh2512