Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multispecies cosmology #104

Closed

Conversation

WillHicks96
Copy link
Contributor

@WillHicks96 WillHicks96 commented May 20, 2021

This contains changes necessary for using Grackle with cosmology problems (working out of @jobordner's enzo-prolong branch). The changes are as follows:

  1. Small bug fix in enzo_SolveHydroEquations.cpp, where colindex was initialized as NULL and never updated, which caused segfaults when doing color field advection.
  2. Adding color fields to refresh -- this is part of PR #49
  3. Initialize chemistry fields in enzo_EnzoInitialCosmology.cpp
  4. Explicitly passing cosmology units into Grackle because the first calls to initialize_grackle_chemistry_data() and setup_grackle_units() happen before enzo_units are set to comoving

Mesh
  - added Box class for computing loop indices
  - replaced loop_limits in FieldFace with Box

Cello
  - fix: changed std::numeric_limits<T>::digits to digits10

Charm
  - removed obsolete sum_long_double_[4-8]_type reductions; switched
    to n_type

Data
  - added rank parameter to FieldFace constructor for Box class,
    and since cello:rank() not available in test_FieldFace.cpp
  - removed unused FluxData methods

Enzo
  - implementd EnzoProlong to call ENZO interpolate() but with local data
    (extends coarse region by duplicating edge values)
  - changed EnzoSolverDiagonal to use temporary instead of "diagonal_D"
Cello
  - bug fix: use digits10 not digits in cello::digits_max()
Charm
  - add missing array indexing [in] to static
    EnzoBlock::NumberOfBaryonFields
Refresh
  - removed "new" from refresh-related identifiers
  - removed obsolete debugging code
Build
  - added linux_clang config
Refresh
  - Adding support for EnzoProlong's interpolation region overlapping
    multiple extra blocks
Problem
  - Added virtual Prolong::padding()
  - Removed Refresh::id_solver_
Mesh
  - Cleaning Box API
  - Added support for third "extra" Block to Box
  - Converted Box attributes to arrays
Refresh
  - Implemented most of refresh_extra_field_faces_
Mesh
  - separated Box::compute_block_start from Box::compute_region()
  - Added support for fourth "array" Block type to Box
  - set gr3_[] default to g3_
  - added Box::restrict_limits() to confine loop limits to Block
Charm
  - Added optional MsgRefresh::oname_[block|type]_ for debugging

Refresh

  - Added TRACE_COMM debugging
  - Implemented and debugged refresh_load_extra_face_ (still needs
    field value copies)
  - Implemented refresh_extra_send_()

Mesh
  - added Block::padded_face_array_[] to store padded arrays
  - added Block::padded_face_array()
  - added Block::padded_face_array_[de]allocate()
  - added Block::name(Index) to get name of neighboring blocks

Data
  - added DataMsg::padded_face_
  - added DataMsg::padded_face_field_list_
  - added DataMsg::set_padded_face()
  - added padded_face_field_list_

Enzo
  - added missing array copy for 3D problems

Tools
  - changed obsolete xrange to range in plot_mesh.py
Testing

  - removed "dump" from Output in PPML tests which was failing in
    random queueing

Debugging

  - added temporary cello::sum() and cello::copy() used in
    enzo_prolong debugging

  - added cello::hex_string() to create pseudo-random hex string for
    MsgRefresh::tag_ and DataMsg::tag_ for debugging messages

  - updated debugging code in MsgRefresh
  - updated debugging code in control_refresh
  - updated debugging code in DataMsg

EnzoProlong

  - added Block::refresh_extra_apply_ to call EnzoProlong only after
    all expected face data have been received

  - added refresh_type parameter to refresh_load_extra_face_

Mesh

  - Updated Box to not overwrite receive block with extra since needed
    for EnzoProlong array receive
  - Added Box_type to Box class for
  - Added Box::get_region_size()
  - Added rank,level,face,child to Box constructor instead of separate
    set_face(), set_child(), etc.

Method

  - fixed timestepping in EnzoMethodGravity to match ENZO (had grouping error)
  - removed const from virtual double Method::timestep() required by change
    to ENzoMethodGravity::timestep()

Data
  - added ENABLE_PADDING to FieldFace
  - Moved padded_face_array_ from Data to FieldData::padded_array_
  - added padded_array_dimensions_[]
  - added padded_array_fields_[]
  - removed FieldFace::box_ since doesn't need to be stored
  - cleaned padded array code in
    FieldFace::[array|face]_to_[array|face]()
  - added code to FieldFace to copy send-recv face zones to padded
    array in FieldFace::[array|face]_to_[array|face]()
  - fixed typo c3[3] = {3} in FieldFace::face_to_face()

Messages
  - updated padded_array attributes in DataMsg

ENZO

  - added int() around sign() calls in some fortran code (calcdiss)
Build
  - removed obsolete ncsa-bw check in build.sh that points to user
    home directory

Testing
  - added test_Box (empty)
  - updated test_Field for coarse fields

Debugging
  - removed control_refresh.cpp TRACE_COMM debugging
  - updated debugging in DataMsg
  - cleaned FieldFace debugging
  - added temporary ENABLE_EXTRA to control_refresh.cpp
  - added MsgRefresh::set_block_[name|type]

Mesh
  - cleaning Box API
Prolong
  - converting face padded_arrays to single coarse field per block
    (incomplete)
Data
  - added FieldData::coarse_values()
  - added FieldData::[de]allocate_coarse()
  - added FieldData::coarse_dimensions()
Documentation

  - formatting

Debugging

  - updated MsgRefresh.cpp, control_refresh.cpp
  - cleaned DataMsg

EnzoProlong

  - renamed refresh_extra_*() as refresh_coarse_*()
  - updated call to refresh_load_coarse_face_()
  - updated refresh_load_coarse_face_() parameters
  - added main S->r send for padded prolong

Mesh

  - refactored Box API: split get_limits() into get_start_stop() and
    get_start_size()
  - refactored Box to clarify role of get_start_*() parameters
    (first defines block to intersect region, second to define
     index coordinate system)
  - added Box::coarse_size_[] and coarse_ghost_[] initialized by
    set_coarse()
  - bugfix in FieldData::coarse_dimensions() for odd ghost depths

Charm

  - renamed and updated DataMsg::set_padded_face() as
    set_coarse_array()
  - updated DataMsg coarse_array parameters

Prolong

  - renamed Prolong::padding() as Prolong::coarse_padding()
  - removed padding code from FieldFace since handled by
    FieldData::coarse_array_
  - removed unused Prolong::monotonic_ and ::positive_
  - changed Prolong::apply() to void return
  - added virtual Prolong::array_sizes_valid (true default)

Fortran

  - removed obsolete [DEBUG_ERROR_WARNING]_MESSAGE defines
Cello

  - cleaning macros for sizing, saving, loading message data

  - updated random number generator in cello::hex_string() to not
    be the same on different processors

Refresh

  - removed redundant Box method calls
  - fixed interpolation factor: add not average
  - finished implementing Block::refresh_coarse_apply_()

Data

  - renamed DataMsg::field_[array|data}_ as field_[array|data]_u_ to indicate union
  - replaced manual size/load/save with cello.hpp macros
  - fixed data size check ASSERT call position
  - updated print() to include tag_
  - added Box BlockType::none for not intersecting region

EnzoProlong

  - finished implementing

Tools

  - updated valgrind to include additional issues in generated valgrind.org file
Problem

  - added MethodRefresh to manually add refresh operations between other Methods

Refresh

  - added debug to control_refresh.cpp
  - fixed logic for calling refresh_load_[field|coarse]_face_ and corresponding counters
    (note count expected receives not number sent)
  - fixed BlockType argument to box_sr.get_start_stop for (iam3,iap3) indices
  - fixed BlockType arguments to box_[er|se].get_start_stop() for (ifms3,ifps3) indices
  - fixed rr scaling to be num_children() only when levels actually differ
  - fixed loop limits in DataMsg::update()
  - fixed (re)setting pad = 0 in else of if (refresh_fine) in FieldFace face_to_array()
    array_to_face()
  - simplified calls to prolong->apply in FieldFace::[array|field]_to_field()
  - adding Refresh::set_min_face_rank() and ::set_ghost_depth(), ::sync() (not called yet)
  - added return_bypass to return_enum for Blocks that have nothing to do in a Solver

Mesh

  - split BlockType::coarse into BlockType::coarse_receive and
    BlockType::coarse_extra

Config

  - added MethodRefresh parameters to Config
    - method_refresh_field_list;
    - method_refresh_particle_list;
    - method_refresh_ghost_depth;
    - method_refresh_min_face_rank;
    - method_refresh_all_fields;
    - method_refresh_all_particles;
  - made Config::index_schedule public for consistency

Enzo

  - adding debugging to EnzoSolverBiCgStab
  - incorporating ENZO Fortran code error handling into EnzoMethodPpm to
    exit immediately on errors
  - adding ENZO errors
  - added missing ierror parameter to flux_[hll|twoshock] argument
    lists (was in call)
  - added returns after errors in [xyz]euler_sweeps to exit with error earlier
    and not try to "compute" with nan's
Testing

  - fixed B_COPY -> B_copy in test_cosmo.incl
  - implemented test_Box.cpp (no checks, currently for comparing
    with expected values by hand)
  - updated test_FieldFace for change to FieldFace::set_ghost() arguments
    (int instead of bool)
  - add src/Enzo/test_interpolate.F for testing setup for calling
    interpolate.F (old but hadn't added to git)

Mesh

  - changed Block::create_face() arguments from logical lg3 to values g3
  - added lpad argument to Box::get_start_[stop|size]() whether to
    include padding (need false for fine blocks true for coarse)
  - removed temporary set_padding(0) in refresh_coarse_apply_
    (not needed with lpad argument)
  - moved box set_padding() inside set_box_()
  - fixed coarse loop indices in array_to_face()
  - update FieldFace::ghost_ from bool to int for size
  - fixed logic for computing pad (0 if not refresh_fine)
  - changed Box to add padding when get_start_[stop|size] called not
    when computing region (since not always used)
  - added Box::apply_padding_()

Method

  - renamed i_f as index_field in MethodDebug
  - updated EnzoSolverDd::pack_field_() for including ghosts only when
    refresh_type == refresh_fine
  - added EXIT_ON_ERROR to SolveHydroEquations (EnzoMethodPpm)
  - removed obsolete setting ierror to constants in [yz]euler_sweep.F

Refresh

  - BUG FIX: fixed use of i_f index instead of index_field in
    refresh_coarse_apply_
  - fixed FieldData::coarse dimensions() for non-centered fields
    (cx was used instead of cy)
Input

  - removed prolong parameters for testing
  - added input/schedule_time_0.01.incl
  - removed old/unused fields from test_cosmo.incl

Testing

  - removed mesh-balanced test since redundant with adapt_L5
  - run adapt_L5 regression test to 0.05 instead of 0.1 to speed up regression
  - changed gravity solver tolerance from 0.01 to 0.1 to speed up regression
  - updated test/index.php for above changes
  - cleaning: removed commented-out tests

Prolong

  - readded TRACE_PROLONG output to Prolong objects for verifying type

Parameters

  - removed obsolete interpolation_method parameter
Debug

  - consolidating debug macros in cello.hpp
  - removing debug code from control_refresh.cpp

Refresh

  - incorporate refresh->field_list_dst() as well as field_list_src()
    for padded prolong refresh
  - duplicated Box object in refresh_coarse_apply_() that was
    used for two separate index computations
  - moved FieldFace::accumulate_() into Refresh::accumulate()
  - Added Refresh::box_accumulate_adjust()

Output

  - added use_min_max parameter to OutputImage()
  - fixed OutputImage pup()--added missing attributes
  - changed log() in OutputImage to log(fabs())

Prolong

  - consolidated duplicated code in ProlongLinear::apply_() for
    accumulate = 0 and 1
  - added support for accumulate in EnzoProlong

Method

  - removed non-accumulated fields from ir_post refresh in EnzoMethodGravity
    (accelerations and density)
  - added DEBUG_COPY_DENSITIES to EnzoMethodGravity
  - removed errant debug code from EnzoMethodPmDeposit

Testing

  - removed obsolete collapse tests test_method_gravity_cg-[18].unit
  - Cleaning Sync, preparing to add error checking

Initial

  - Moved initialize_[prolong|restrict] before initialize_[method|solver]
    since initialize_[prolong|restrict] create "default" prolong/restrict
    which must be first

Prolong

  - moved coarse_padding() from Prolong to Refresh, so can set to 0
    if refresh accumulate_ is true
  - Made Prolong::coarse_padding() protected but friend to
    Refresh::coarse_padding()

Problem

  - changed Problem::[prolong|restrict]_ to lists [prolong|restrict]_list_
  - changed all NULL's to nullptr's in problem_Problem.?pp
  - changed initialize_[prolong|restrict]_ to create default prolong/restrict
    check that it's creating the first prolong/restrict objects in
    the lists
  - changed Problem::[prolong|restrict]() methods to accept optional argument
    i, with default 0 to return default first in list
  - added "use_linear" argument to EnzoProlong constructor

Refresh

  - adding prolong and restrict to Refresh

Testing

  - cleaning test_Sync

Parameters

  - added Prolong : enzo : use_linear parameter for debugging
    (EnzoProlong::apply() calls ProlongLinear::apply())

Solvers

  - Changed restrict/prolong arguments to EnzoSolver[Dd|Mg0] to
    index_prolong/index_restrict
  - adding debugging to Sync to test for over-counting (INCOMPLETE)
  - moved [mul|div]_by_density() from FieldFace class to cello:: namespace

Debug

  - removed tag_ attribute and other debugging code from MsgRefresh
  - and DataMsg addressed some minor compiler warning messages (-Wall)
  - add checks when copying fields for debugging in EnzoMethodGravity
  - and EnzoMethodPpm

Rferesh

  - Added index_[prolong|refresh] to Refresh
  - Removed Prolong / Restrict from FieldFace (use from Refresh instead)

Parameters

  - Changed default Field:prolong from "linear" to "enzo"

Solver

  - Added prolong/restrict arguments to Solvers
  - changed EnzoSolverBiCgStab::res_tol_ from long double to double
    (suspect this was causing compiler issues with optimization)
Control

  - bug fix: Moved deallocating fluxes from MethodFluxCorrect.cpp to
    control_compute.cpp to avoid memory leak if MethodFluxCorrect isn't
    called

Debug

  - Updating MethodDebug for added parameters (incomplete)
  - Added parameters to "debug" method: "print", "coarse", "ghost"
    (incomplete)
  - added debug code for copying fields in EnzoSolverBiCgStab

Parameters

  - Added method_refresh_prolong

Method

  - EnzoMethodGravity updates
    - Added index_prolong parameter to EnzoMethodGravity
    - Re-added accelerations to ir_post refresh in EnzoMethodGravity
    - fixed typo: naming ir_exit_ refresh

Solver

  - EnzoSolverBiCgStab can produce poor solutions at mesh level jumps
    if restart != 1: resetting to 1 with WARNING message to bypass
    until this is addressed

Cleaning

  - Removed obsolete EnzoMethodCheckGravity
  - Removed obsolete files in tools directory
  - Added tools/index.org file to document remaining available tools
Mesh

  - Added BlockTrace class to aid traversing distributed
    array-of-octrees partition (range of elements in root-block array)
  - Added [pr]_method_output_<foo>() entry methods for MethodOutput

Parameters

  - Added initial_hdf5_* parameters for InitialHdf5
  - Renamed method_refresh_<foo> to method_<foo>
  - Added method_output_blocking parameter to define scope of
    domain for each output file

Testing

  - Added test_BlockTrace unit tests
  - Removed unused code from test/index.php

Method / Output

  - Added MethodOutput "method" for writing HDF5 output

Cello

  - fixed LOAD_ARRAY_TYPE()

Charm

  - Added MsgOutput Charm++ message for use by MethodOutput
  - cleaned MsgRefresh: removed debug code

Initial

  - Added InitialHdf5 stub

Cleaning

  - addressed some -Wall compiler warnings
Build

  - removed obsolete "new_output" from SConstruct

Output

  - Removed unused index parameter from Io class hierarchy
  - removed unused Io::data_count_
  - Copied expand_name() and directory() from Output to Cello so
    can use in MethodOutput
  - removed NEW_OUTPUT from control_output.cpp

Mesh

  - Added index_home to BlockTrace (not necessarily the same
    as index_root)
  - Added data_size, [save|load]_data to Index

Cello

  - Added string, object, and object_ptr to size/load/save type macros

Charm

  - Updating MsgOutput for block name, file pointer, and tag (for
    debugging)
  - Converting manual size/load/save in MsgOutput::[un]pack() to
    use cello.hpp macros

Parameters

  - Added method_output_[file|path]_name parameters
  - Fixed method_field_list to be list of string not integer

Method

  - Implementing MethodOutput (sequencing works, file created and closed,
    need to add HDF5 write calls)
  - Added MethodOutput ScalarData for file counter (needed per-block
    since may have multiple Blocks writing files on same pe
  - Updated MethodRefresh field and particle lists to be
    string not int

Simulation

  - Removed Simulation sync objects for obsolete "new_output"
Input

  - Removed flux_correct from test_double_mach to avoid crashing

Charm

  - Added IoBlock, Block lower/upper to MsgOutput

Problem

  - Added Factory argument to Problem::initialize_method()
    (required for MethodOutput)
  - Made argument orderings in Problem::create_[*]() more consistent

Output

  - Added data_size() and [load|save]_data() to Io class hierarchy
    for including in MsgOutput
  - Made Io class hierarchy pup-able
  - Updated Io classes to store metadata and not access via
    Block / Hierarchy classes (since IoBlock may be accessed
    on different Block than where created in MethodOutput)
  - Finished Block and field data output in MethodOutput
    (particle data still not implemented) (untested at scale)

Mesh

  - Added Index::operator[]
  - Added virtual Block::factory() for use by MethodOutput()

Tools

  - Made plot_mesh.py more forgiving, supporting any text file
    containing "words" that look like Block names B0:0_1:00
Method

  - removed debugging code from MethodOutput
  - added support to output particle data
Output

  - bug fix: check for any fields or particles in MethodOutput
    lists before setting in DataMsg
  - bug fix: added missing HDF5 file close to MethodOutput
  - added "writing data file" monitor output to MethodOutput
  - Cleaning and optimizing OutputImage (incomplete)
  - Added trace_memory debugging to OutputImage

Testing

  - added new-output to test-adapt-L5-P1 regression test
  - fixed typo in test/index.php

Parameters

  - removed image_block_size parameter
  - replaced image_block_size with image_size in input files
  - changed image_size default from [0,0] to [512,512]

Documentation

  - marked image_block_size parameter as depreciated

Data

  - added FieldFace::ghost() (for debugging MethodImage)

Tools

  - Tweaked valgrind-org to check src not source for source files

Cleaning

  - Removed unused file control_Control.hpp
  - Spell-checked comment in control_stopping.cpp
Checkpoint

  - Adding MethodCheckpoint (incomplete)
  - Added Simulation_r_write_checkpoint_[output|method]
  - Added Main::p_[output|method]_checkpoint

Data

  - bug fix: removede delete particle_data_ in update
    (deleted in destructor if needed)

Parameters

  - renamed parameters method_output_[file|path]_name as
    method_[file|path]_name

Initial

  - moved InitialHdf5 to EnzoInitialHdf5
  - removed obsolete EnzoMethodHydro
  - fixed array initializers in EnzoInitialMusic

Config

  - Split EnzoConfig::read() into subfunctions
@bwoshea
Copy link
Contributor

bwoshea commented May 30, 2021

@WillHicks96 there's something funny with this PR - there are 289 changed files, which doesn't align with the nominal content of the PR. Looking at the commits, this seems to include a lot of @jobordner 's WIP PR (#97 ), and probably some other stuff. Would you mind issuing a cleaner PR?

@mabruzzo
Copy link
Contributor

@WillHicks96 @bwoshea I'm pretty sure all of those file changes from PR #97 will persist until that PR is merged into main. So it's probably not worth re-creating this PR until after that.

Charm
  - Added MsgInitial message type for EnzoInitialHdf5
  - Added Block::r_initial_new_[next|continue]()
  - Added EnzoBlock::p_initial_hdf5_recv() barrier to start
    of EnzoInitialHdf5 to prevent sending data to non-existing
    Blocks

Input
  - Reverted to "music" instead of "hdf5" for initial_music.incl
  - Updated test_cosmo.incl for "hdf5" input

Cleaning
  - Moved TAG_LEN message tag length declaration to cello.hpp
  - Removed unused functions in cello namespace
  - Moved common code into apply_initial_()
  - Removed unused Simulation::r_initialize_hierarchy()

Mesh
  - Added Block::initial()
  - Updated Block::apply_initial_ to support new initialization
  - bug fix: delay incrementing Block::index_initial_ until after
    Initial::enforce_block() is called

Initial

  - Finished implementing EnzoInitialHdf5 for unigrid IC's
  - Added initial_new_* methods for updated initialization supporting
    communication (currently unigrid only)
  - Added calls to Block::initial_done() required for new initialization

Parameters

  - Added Initial:new parameter to enable "new" initialization
    supporting communication within an initialization step (currently
    unigrid only).  Default is false
  - Added Initial:hdf5:max_level maximum level for HDF5 initialization
    (untested over 0 (default))
  - Added Initial:hdf5:format for "enzo", "music", or "inits"
    (only "music" supported)
  - Added Initial:hdf5:blocking analagous to MethodOutput for EnzoInitialHdf5
  - Added Initial:hdf5:max_level
  - Removed initial_hdf5 parameters from Config (initial_hdf5 parameters
    are in EnzoConfig)

Disk
  - Added File::file_read_scalar for non-array meta-data
@bwoshea
Copy link
Contributor

bwoshea commented Jun 3, 2021

@mabruzzo thanks for pointing that out! We probably want to hold off on merging this PR until #97 is merged in that case, but I think it's reasonable to review it in the interim.

@bwoshea
Copy link
Contributor

bwoshea commented Jun 3, 2021

@WillHicks96 Given that there are so many files modified (with the inclusion of files from PR #97 ), could you comment on the files that you modified for this specific PR?

@WillHicks96
Copy link
Contributor Author

@bwoshea @mabruzzo Oh yes, I can definitely see how this is confusing--I added these changes on top of @jobordner's branch that includes the commits from PR #97, so the hundreds of file changes associated with that PR should go away once it goes through.

The changes I made are limited only to the following 4 files: enzo_EnzoMethodGrackle.cpp, enzo_EnzoInitialCosmology.cpp, enzo_SolveHydroEquations.cpp, and enzo_EnzoMethodPpm.cpp

@mabruzzo
Copy link
Contributor

mabruzzo commented Jun 5, 2021

Just a heads up: Because PR #97 branched off from the main branch so long ago, this doesn't seem to include some improvements that I previously introduced to EnzoMethodGrackle.

It might take a little work to resolve these differences. I'm happy to help, if you run into any problems.

James Bordner and others added 12 commits June 5, 2021 12:16
Build

   - Removed checking for VERSION file in Charm++ for compatability
     with Charm++ v7.0.0

Charm

  - Updated FieldMsg to include child_index()

Solver

  - Added buffering and synchronization for "self" for restrict in DD
    (already in prolong)
Solver

  - Added buffering and synchronization for "self" for restrict in Mg0
  - Global change NULL -> nullptr in EnzoSolverMg0
Charm

- Changed MsgInitial::data_delete_ default from false to true
    to address memory leak

Parameters

  - Added missing Grackle chemistry_data attributes to operator|(PUP)
    in enzo_EnzoConfig.hpp (values are already in enzo-project/main)

Grackle

  - Removed trailing _ from parameter and local variables: reserved by
    convention for private member attributes

  - Changed "*c = _set_default_chemistry_parameters()" call to
    "set_default_chemistry_parameters(c)" to avoid unnecessary copy
Input

  - Added velocities to "conserved" field group (needed for momentum)
  - Updated Testing : time_final for checkpoint_ppm tests for updates to
    "conserved" field group

Charm

  - Added MsgOutput::operator=() to complete "the big three" (all-or-none of
    destructor, assignment operator, and copy constructor); moved common code
    into MsgOutput::copy_()

Stopping

  - Added printing of counts of remaining messages for MsgInitial and
    MsgOutput
  - Set DataMsg::face_fluxes_delete_[] to true in load_data() instead of copying
    since locally created
  - Delete face_fluxes_list_ elements explicitly instead of clearing list

Data

  - Replace fluxes_ std::vector with delete_fluxes_ and raw fluxes_
    array instead for pointing into single-array allocation by FluxData
  - updated FaceFluxes and FluxData (partial) data methods
    (data_size(), save_data(), load_data()) with cello macros
  - added FaceFluxes::print() for debugging
  - add missing initializers to FaceFluxes constructor
  - completed FaceFluxes "big three"
  - added FluxData single_array parameter to constructor
  - FluxData allocates a single array for all FaceFluxes objects if single_array
    parameter is true
  - Updated FaceFluxes::get_size() to return total size and allow optional arguments
  - Added FaceFluxes::copy_() to avoid common code in big three

Output

  - Bug fix: image_ghost=true parameter to OutputImage modified
    root_size parameter: added const and use copy for input Parameters
    root_size and root_blocks

Enzo Compute

  - added single_flux_array parameter to EnzoMethodPpm, SolveHydroEquations
  - updated SolveHydroEquations for single_flux_array

Parameters

  - added method_flux_correct_single_array (Method : flux_correct : single_array)
    parameter
Parameters

  - Removed check on memory usage in test_cosmo.incl: was failing with
    false positive in SMP mode

Data

  - Added mutex node lock for FieldFace for field scaling by density
    (fixes sporadic race condition errors in SMP mode)

Memory

  - Replaced free() and malloc() with std::free() and std::malloc() in
    new/delete override

Initial

  - Added mutex node lock for InitialValue (fixes threading errors
    in SMP mode related to evaluating parameter expressions )

Cello

  - Added bounds checking to MaskPng class (failed in SMP mode)

Method

  - Removed debug code from pgas2d_dual.F
Input

  - adding group_list = ["has_mass"] for merging with enzo-project/main

Charm
  - Converting public attributes to private in MsgOutput

Adapt

  - Updated adapt to not delay delete (bug workaround for old Charm++)
    which fixes SMP bug on Frontera (adapt-L5 crashes at first coarsen
    in cycle 291)

  - removed Block : delete_ attribute

Checkpoint

  - Added optional argument to CkStartCheckpoint for number of
    simultaneous writers (1)

Method

  - Only add acceleration Field in EnzoMethodGravity if rank
    sufficiently large (bug fix for enzo-prolong + new-output merge)

Solver

  - reordered i_msg_* attributes to address -Wall out-of-order
    initialization warnings

Testing

  - Fixed test/index.php prolong_linear bug
jobordner and others added 2 commits August 18, 2021 10:07
Build

  - added "new_adapt" variable to define NEW_ADAPT
  - added default for CELLO_VAR in config files

Adapt

  - Added quiescence detection (QD) after doneInserting() call
  - Added p_adapt_update() entry methods (Main and Block) for
    additional QD
  - Changed Block::adapt_ to local variable

Checkpoint

  - Added single-thread to CkStartCheckpoint (should be generalized
    with parameter)

Cleaning

  - Removed unused r_adapt_*() entry methods

Monitor

  - Added NEW_ADAPT to Monitor::header()

Testing

  - Fixed Balance tests to include parallel_arg
@peeples
Copy link
Contributor

peeples commented Jan 28, 2022

Hi Will, now that PR #97 has been merged, what is the status of this PR?

@WillHicks96 WillHicks96 closed this Feb 8, 2022
@WillHicks96 WillHicks96 deleted the multispecies_cosmology branch April 12, 2023 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants