Preparing v0.6 Release #200
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Increment YGM version number in CMakeLists.txt * changed messages to use instead of hard-coded value. * Cleaned up cmake build, and moved the library target description to one place. Co-authored-by: Trevor Steil <trevor.steil1@gmail.com>
* Initial ygm::io::multi_output functionality * Removes unnecessary closing of ofstreams in destructor * Checks prefix path provided to ygm::io::multi_output does not exist as a regular file, and changes filename variable name to subpath * Adds tests for ygm::io::multi_output * Adds ygm::io::daily_output and simple tests * Adds test to check correct files are created by ygm::io::multi_output * Fixing clean-up of test files written * Forces filename_prefix given to ygm::io::multi_output constructor to be a directory
Adds Boost 1.78 to CI.
* Initial ygm::array implementation * Adds safety checks of array sizes * Adds barrier after resizing completes * Changes name of async_put to async_set * Adds array::async_unary_op_update_value and helpers for commonly used binary and unary operators * Adds functions for getting YGM pointers to arrays and array sizes
* Adding maptrix apis and impls. * Adding async_visit API+impl. * Some more details. * Maptrix API changes. * Adding new maptrix design. * Adding SpMV - first take. * Adding Structure * Porting to develop. * Adding SpMV as a standalone function. * Adding pagerank in examples and other restructuring changes. * Maptrix Impl. * Adding for_all over row_id and col_id. * Moving to experimental directory * Delete assoc_vector, replaced with ygm map. * Delete assoc_vector_impl, replaced with ygm map_impl. * Delete maptrix.hpp, in experimental mode. * Delete adj_impl.hpp, in experimental. * Delete csc_impl.hpp, in experimental. * Delete csr_impl.hpp, in experimental. * Delete maptrix_impl.hpp, in experimental. * Delete spmv.hpp, in experimental. * Moved as a part of alg_spmv.hpp. * Moved as a part of alg_spmv.hpp. * Add nicer examples or, tests. * Brain storming spmv row. * Moved within containers. * Moved within containers. * Adding new changes. * Adding timing details. * . * Changing API to insert_if_missing_else_visit. * Fix. * Fix. * Adding norm check. * Added OpPlus OpTimes. * Adding OpTimes. * Adding fixes from pull-request. * Adding fixes. * Delete spmv_row, not supporting for now. * Adding references. * Fixing references. * Changing line parsing. * Changing to 32bit. * Cleaning up webgraph example to use as general SpMV example * Removing extra headers * Switching to webgraph_spmv.cpp as new alg_spmv.cpp * Removing maptrix_visit * Cleaning up alg_pagerank.cpp * Removing unused 2D hasher * Removes unneeded header and changes a function name in adj_impl.hpp * Removing unused code in SpMV * Removing extra headers in column_view_impl.hpp * Removes unused headers in maptrix_impl and passes default value to row_view and column_view * Removes unused headers in maptrix_impl and row_view_impl * clang-format on maptrix.hpp Co-authored-by: Ancy Sarah Sarah Tom <tom7@catalyst160.llnl.gov> Co-authored-by: Trevor Steil <trevor.steil1@gmail.com>
* Manually buffers ygm::io::multi_output writing * Adds buffering to ygm::io::daily_output * Updates ygm::io output tests to use buffering
* initial pass on implementing the Layout class. ygm::comm::layout() returns a const reference. * Removed comments and cleaned up ygm::Layout internals. * Reordered ygm::comm::Layout member for consistency. * Added tests for ygm::Layout::is_strided() and ygm::Layout::is_local(), and added functions to get const refs to all local and strided ranks. * Refactored ygm::Layout -> ygm::detail::layout.
* Adds missing function to get a YGM pointer to a counting set * Adds comm() function to counting_set * Fixes typo in counting_set.comm() * Adds test for counting_set's YGM pointer * Adds topk to counting set * Adds barrier to beginning of map's topk * Adds example of counting_set topk
…version and mpi types. Also caching boost to avoid downloading/untaring with every job. (LLNL#86)
…eauture/** and hotfix/** branches trigger jobs. (LLNL#88)
* Adds additional opportunities to flush buffers while processing received messages
LLNL#89) * Tweaked contributing guidelines to reflect CI changes and fixed some typos. * Update CONTRIBUTING.md Co-authored-by: Trevor Steil <trevor.steil1@gmail.com>
…curring on a newline. (LLNL#93)
Co-authored-by: Sudharshan Srinivasan <ssriniv@catalyst159.llnl.gov>
* Switches ygm::container::array to block partitioning * Gives last rank fewer elements during resize to avoid issues in for_all * Removes ability to resize ygm::container::array. No longer sensible with block partitioning * Properly sets local block size of last rank when block is full-sized
* gather data to single std::vector added * cleaned up collective operation, fixed test case, added to all support * Update bag_impl.hpp m_res -> p_res removes unnecessary `mailbox` from `gatherer` lambda passes `outer_data` to `gatherer` by reference uses m_comm to build ygm_ptr (preferred method) * Update bag_impl.hpp Makes vector arguments to lambdas in gather_to_vector const Co-authored-by: Dozier <dozier8@llnl.gov> Co-authored-by: Trevor Steil <trevor.steil1@gmail.com>
* Adds ygm_tag to comm::impl * Removes local_receive function
* Adds initial ygm::detail::interrupt_mask implementation * Adds test for ygm::detail::mask_interrupt * Adds detail::interrupt_mask to map_impl to prevent iterator invalidation during map visits * Adds missing include statement for interrupt_mask in map_impl * Fixes namespacing for interrupt_mask use in map_impl * Gets m_comm from pmap in map_impl::async_visit_group
Significant overhaul removing listener thread and MPI_THREAD_MULTIPLE. * Removal of the listener thread. Removes requirement for MPI_THREAD_MULTIPLE. Improves performance substantially for the triangle counting benchmark. * NR & NLNR routing are supported. * Added the ability to capture primitive values in the async lambda to avoid Cereal overhead. This improves performance a bit, but mostly it makes user code easier to read/debug. Caution: there are no safety rails to prevent capturing a local pointer (but does prevent local references). * Added a full environment setting system, and it now controls the major settings in the runtime. * comm::welcome() added a welcome banner that prints out the current configuration and MPI settings. * API minor change removed the optional parameter to comm::comm(…, int buffer_capacity) that controlled the buffer size. Now it must be controlled by the environment. * Added new mechanism to indicate remote dispatch functions via static initialization.
…o that multi-rank print operations are less likely to get garbled. (LLNL#107)
* Adds Github action to run GCC-8 tests on Ubuntu 20.04 as it is unavailable on Ubuntu 22.04 * Updating version of checkout action to get rid of javascript warning in Github Actions * Updating version of cache action to get rid of javascript warning in Github Actions
Added ygm_container_type tags to containers. Additionally added compile time functions to check container type.
* Adds test of std::vector of YGM sets * Fixes size check in test of vector of YGM sets * Adds move constructor to YGM set
* Removes comm::impl * Properly passes lambdas to pack_lambda operations and removes mention of comm::impl from interrupt_mask * Removes comm copy constructor * Changes comm in arrow_parquet_parser to a reference * Removes comm_impl.hpp includes from comments * Removes bag_impl
* Added comm::progress() and comm::wait_until(Fn). * Added comm::process_incoming()
* Adding consume_all to set. * Made adapter more clear.
* Adds missing filesystem include to multi_output.hpp * Replaces csv_parser.hpp include in ndjson_parser.hpp with line_parser.hpp * Removes unnecessary ygm::io::detail namespace
* fix type of set get_ygm_ptr * add a test for the ygm set pointer * also fix the set pointer for multiset --------- Co-authored-by: Grace Jingying Li <li85@llnl.gov>
* Adding recent version of Arrow to CMakeLists.txt * make -j * Adds CMake flag to require a version of Arrow to be found, specifically for use in CI to make it obvious when Apache Arrow is not being found
* Check calling context in comm.local_progress() * Fix spelling error and increase test trip count
* Change parquet parser wrt schema Specifically, schema() returns physical type rather than logical type. * Add paquet -> json converter * Update Parquet JSON reader example
Change the cmake file to support Apache Arrow v14.0
* Parallel parquet reader v2 * Add comments
* Add tests for parquet reader - Add test for parquet2json convertor - Add additional parallel parquet reading test * Fix wrong indent * Brush up on parque reader test
* Read values using std::optional in the ParquetToJson converter * Support Arrow v15
* Adds print statements to see if obvious bugs exist in disjoint_set::all_compress() * Removes race condition in disjoint_set::all_compress() by preparing all parent queries before sending any * Removing print statements from disjoint_set * Adds check of parent ranks before compressing disjoint sets * Adds check of parent ranks before compressing disjoint sets * Adds check of parent ranks before compressing disjoint sets * Avoid querying parent during disjoint_set::all_compress() when item is a root * Fixes missing reference for local_rep_query inside of update_rep_functor of disjoint_set::all_compress() * More detailed debugging output in all_compress() * Fixing logic for when a parent request comes for an item that had to query its own parent in current all_compress() round * Fixing logic for when a parent request comes for an item that had to query its own parent in current all_compress() round * Cleaning up all_compress() queries as they return * Cleaning up all_compress() queries as they return * Adding additional debug printing to disjoint_set::all_compress() * Avoids potential race condition where all_compress() is using whether parent of item being queried has returned in place of directly tracking whether item has found its root * Cleaning up disjoint_set::all_compress() code * Removing unused code
Co-authored-by: Keita Iwabuchi <iwabuchi1@lln.gov>
* Adds missing async_visit function to disjoint_set * Fixes stopping criteria on while loop in disjoint_set::all_compress()
LFletch1
added a commit
to LFletch1/ygm
that referenced
this pull request
Aug 21, 2024
Preparing v0.6 Release (LLNL#200)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.