Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify Windows CMake setup + Stand-alone examples #720

Merged
merged 11 commits into from
May 7, 2021
Merged

Conversation

upsj
Copy link
Member

@upsj upsj commented Mar 11, 2021

I attempted to simplify our current Windows CMake setup a bit:

  • Require OpenMP >= 3.0 to avoid OpenMP being enabled erroneously on MSVC
  • Add /bigobj flags by default to avoid users having to specify them
  • Always link dynamically against the C runtime library (CRT); If users really want static linking, they just have to specify their own CMAKE_CXX_<CONFIG>_FLAGS and disable tests. We could also try to identify based on the compiler flags whether we need this, but GTest decided to make this a CMake option instead of relying on the compiler flags, so this would need additional code 🤷‍♂️
  • Install DLLs into the binary instead of library path. This is what vcpkg expects by default, and Craig Scott recommends in his book, so I guess this is the right way to go
  • Create all of Ginkgo's libraries in lib/ by default, which allows us to avoid having to specify the PATH in tests, instead using the WORKING_DIRECTORY parameter of add_test to allow the libraries to be found.

Finally, I turned all examples into stand-alone projects which conditionally call find_package(Ginkgo) if we are not in the build tree.

All of these changes are implemented in individual commits, so it should be easy to remove one of them if we consider them unnecessary or harmful.

@upsj upsj added reg:build This is related to the build system. 1:ST:ready-for-review This PR is ready for review plat:windows This is related to the Windows Operating system labels Mar 11, 2021
@upsj upsj self-assigned this Mar 11, 2021
@ginkgo-bot ginkgo-bot added mod:cuda This is related to the CUDA module. mod:openmp This is related to the OpenMP module. reg:ci-cd This is related to the continuous integration system. reg:example This is related to the examples. labels Mar 11, 2021
@upsj upsj removed mod:cuda This is related to the CUDA module. mod:openmp This is related to the OpenMP module. labels Mar 11, 2021
@yhmtsai
Copy link
Member

yhmtsai commented Mar 11, 2021

Is any reason to delete the current workaround?
I do not like we leave the users to set CMAKE_CXX_... or CMAKE_CUDA_...
(You also need to modify CMAKE_CUDA_.. when linking crt.)
CMAKE_CXX_ has some default setting, so it is hard to set it correctly with only change crt in the beginning via command.
And we still need tests when we build static ginkgo.

I prefer keeping CMAKE_CXX_FLAGS empty such that user can add anything without covering original settings.
(users still need to set the bigobj by command or env in current ginkgo though)

@upsj
Copy link
Member Author

upsj commented Mar 11, 2021

/MD, /MT
Yes, the CRT linking issues are probably the most contentious change. Just so we are talking about the same thing:
Both dynamic library + dynamic CRT and static library + dynamic CRT work fine and can build the tests without any modifications.
GTest is the only library that sets the CRT-related flags itself instead of using what is provided in CMAKE_CXX_<CONFIG>_FLAGS. With 2-3 lines of code, we could also make it work for GTest.

In general, I think we should give users the freedom to choose which CRT to use, which means not changing the settings ourselves based on BUILD_SHARED_LIBRARIES, at the same time, it is a property that must be the same across the entire library + application. We could add something like GINKGO_SHARED_CRT, but that again breaks transitivity of compiler flags, which is what caused the GTest issues in the first place. Basically most of this PR is removing parts from the CMake configuration which I believe have more canonical or common ways to do this.

/bigobj
The ideal setting for users would be to just call mkdir build && cd build && cmake ... That currently doesn't work because of the missing /bigobj flag. CMAKE_CXX_FLAG is only modified internally and the change is not visible in the CMakeCache.txt file. Can you explain what you mean by covering original settings?
Finally, I think ginkgo_cuda also works without the /bigobj flag, but I can add them of course.

@sonarcloud
Copy link

sonarcloud bot commented Mar 11, 2021

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@codecov
Copy link

codecov bot commented Mar 11, 2021

Codecov Report

Merging #720 (59c8e23) into develop (6448ba5) will increase coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop     #720   +/-   ##
========================================
  Coverage    94.18%   94.18%           
========================================
  Files          399      399           
  Lines        31032    31032           
========================================
+ Hits         29228    29229    +1     
+ Misses        1804     1803    -1     
Impacted Files Coverage Δ
core/base/extended_float.hpp 92.23% <0.00%> (+0.97%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6448ba5...59c8e23. Read the comment docs.

@yhmtsai
Copy link
Member

yhmtsai commented Mar 11, 2021

thanks, I never know static library works with dynamic crt on windows.
I also did some changes on cuda runtime choice according to the shared/static.
Could you also check it? I am wondering there is another issue.

\bigobj:
I mean cmake -DCMAKE_CXX_FLAGS="" .. will lose the big-obj from ginkgo's cmake.

@upsj
Copy link
Member Author

upsj commented Mar 11, 2021

I didn't see any issues with CUDA static/dynamic linking so far, but I will check.

On /bigobj: CMake variables are scoped, so everything in CMakeCache.txt is used as the input for the CMake scripts, but all changes within the scripts are only temporary during execution, so /bigobj will always be added, even if it is already present in CMAKE_CXX_FLAGS.

@upsj
Copy link
Member Author

upsj commented Mar 11, 2021

The build itself seems to work, even though the CUDA tests segfault during execution. This is the same as a few months ago when I tried to build on Windows first, so I think this is not related to the build configuration. As soon as I manage to fix my CUDA installation, I'll try to identify the issue.

add_test(NAME ${REL_BINARY_DIR}/${test_name} COMMAND ${TEST_TARGET_NAME})
add_test(NAME ${REL_BINARY_DIR}/${test_name}
COMMAND ${TEST_TARGET_NAME}
WORKING_DIRECTORY "$<TARGET_FILE_DIR:ginkgo>")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because do not require put the library path into PATH, will user need to run the test from the lib location?

Copy link
Member Author

@upsj upsj Mar 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either that, or use ctest -C Release|Debug since that uses the correct WORKING_DIRECTORY specified in add_test. As far as I can tell, that issue is only relevant to Windows, though. On Linux, the RPATH points to lib anyways, so the tests can be run from any location.

examples/adaptiveprecision-blockjacobi/CMakeLists.txt Outdated Show resolved Hide resolved
examples/custom-matrix-format/CMakeLists.txt Outdated Show resolved Hide resolved
CMakeLists.txt Show resolved Hide resolved
cmake/build_helpers.cmake Show resolved Hide resolved
Comment on lines +14 to -9
target_link_libraries(custom-stopping-criterion Ginkgo::ginkgo
Threads::Threads)
target_include_directories(${target_name}
PRIVATE ${PROJECT_SOURCE_DIR})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not need the include directory anymore because terry already add the information to Ginkgo::ginkgo target.
Is it correct or anything I miss?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about that. I removed the include since we should not use any code from core in the examples.

examples/ginkgo-ranges/CMakeLists.txt Outdated Show resolved Hide resolved
@upsj
Copy link
Member Author

upsj commented Apr 13, 2021

@ginkgo-project/reviewers This PR is now up to date and ready to review again. I think it's easiest if you go through the individual commits, since there are multiple changes mixed together.

Copy link
Member

@pratikvn pratikvn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just a question whether this change of library paths affects external libraries.

cmake/build_helpers.cmake Show resolved Hide resolved

# We only need to find Ginkgo if we build this example stand-alone
if (NOT GINKGO_BUILD_EXAMPLES)
find_package(Ginkgo 1.4.0 REQUIRED)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a need to require a specific version ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means that we find Ginkgo with version number at least 1.4.0 and the same major version (i.e. no 2.0), so we can be sure that the interfaces we use are available.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I mean do we need 1.4.0 ? 1.3.0 should also work ? Because, technically these are standalone and can be used with an already existing, possibly older version of Ginkgo ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good point, in general I agree. However, with the addition of DpcppExecutor to most examples, they are no longer backwards-compatible with 1.3.0, so we need to modify it in a few places at least.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For ease of maintainance and version bumps, can you create a shared .cmake file somewhere to define a variable like ${GINKGO_VERSION_REQUIRED}` or whatever the name could be? Just so that it's in one place.

You would also need to set that file as installed or it can be to the directory when building the example.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea behind these changes is to make the CMake files as self-sufficient as possible. So I would like to avoid such external dependencies. Another point is that this way, you only need to increment the version requirement if you actually modify the example to use newer parts of Ginkgo, but whether that is useful would be another question entirely.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My fear is that this will easily become hard to manage and some aspects might be missed which make examples actually incompatible but we did not realize that. For that reason, I think I would pretty much always bump the version requirement if it is there.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. So would you be okay with keeping the values in sync manually, but leaving them in here explicitly?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's still somewhat error prone particularly as the number of examples increase, but if you think there is no easy way to integrate a macro in that workflow, fine then.

@upsj upsj added this to the Ginkgo 1.4.0 milestone May 5, 2021
Copy link
Member

@tcojean tcojean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Some small comments.

CMakeLists.txt Outdated Show resolved Hide resolved
INSTALL.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved

# We only need to find Ginkgo if we build this example stand-alone
if (NOT GINKGO_BUILD_EXAMPLES)
find_package(Ginkgo 1.4.0 REQUIRED)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For ease of maintainance and version bumps, can you create a shared .cmake file somewhere to define a variable like ${GINKGO_VERSION_REQUIRED}` or whatever the name could be? Just so that it's in one place.

You would also need to set that file as installed or it can be to the directory when building the example.

add_executable(${target_name} ${target_name}.cpp)
target_link_libraries(${target_name} Ginkgo::ginkgo)
target_include_directories(${target_name} PRIVATE ${PROJECT_SOURCE_DIR})
cmake_minimum_required(VERSION 3.9)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ginkgo itself requires 3.13 now. I guess there is no need, but you could upgrade this as well.

examples/build-setup.sh Outdated Show resolved Hide resolved
examples/custom-matrix-format/CMakeLists.txt Show resolved Hide resolved
examples/papi-logging/CMakeLists.txt Show resolved Hide resolved
examples/minimal-cuda-solver/CMakeLists.txt Show resolved Hide resolved
@upsj upsj added 1:ST:ready-to-merge This PR is ready to merge. and removed 1:ST:ready-for-review This PR is ready for review labels May 7, 2021
Copy link
Member

@yhmtsai yhmtsai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I still have some concern build ginkgo with static runtime library.
I do not have the usage case about requirement of static runtime library on Windows.
It simplifies CMakeFiles a lot, so I am fine with the current one.

@sonarcloud
Copy link

sonarcloud bot commented May 7, 2021

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@upsj upsj merged commit d669c87 into develop May 7, 2021
@upsj upsj deleted the cmake_cleanup branch May 7, 2021 19:49
tcojean added a commit that referenced this pull request Aug 20, 2021
Ginkgo release 1.4.0

The Ginkgo team is proud to announce the new Ginkgo minor release 1.4.0. This
release brings most of the Ginkgo functionality to the Intel DPC++ ecosystem
which enables Intel-GPU and CPU execution. The only Ginkgo features which have
not been ported yet are some preconditioners.

Ginkgo's mixed-precision support is greatly enhanced thanks to:
1. The new Accessor concept, which allows writing kernels featuring on-the-fly
memory compression, among other features. The accessor can be used as
header-only, see the [accessor BLAS benchmarks repository](https://github.com/ginkgo-project/accessor-BLAS/tree/develop) as a usage example.
2. All LinOps now transparently support mixed-precision execution. By default,
this is done through a temporary copy which may have a performance impact but
already allows mixed-precision research.

Native mixed-precision ELL kernels are implemented which do not see this cost.
The accessor is also leveraged in a new CB-GMRES solver which allows for
performance improvements by compressing the Krylov basis vectors. Many other
features have been added to Ginkgo, such as reordering support, a new IDR
solver, Incomplete Cholesky preconditioner, matrix assembly support (only CPU
for now), machine topology information, and more!

Supported systems and requirements:
+ For all platforms, cmake 3.13+
+ C++14 compliant compiler
+ Linux and MacOS
  + gcc: 5.3+, 6.3+, 7.3+, all versions after 8.1+
  + clang: 3.9+
  + Intel compiler: 2018+
  + Apple LLVM: 8.0+
  + CUDA module: CUDA 9.0+
  + HIP module: ROCm 3.5+
  + DPC++ module: Intel OneAPI 2021.3. Set the CXX compiler to `dpcpp`.
+ Windows
  + MinGW and Cygwin: gcc 5.3+, 6.3+, 7.3+, all versions after 8.1+
  + Microsoft Visual Studio: VS 2019
  + CUDA module: CUDA 9.0+, Microsoft Visual Studio
  + OpenMP module: MinGW or Cygwin.


Algorithm and important feature additions:
+ Add a new DPC++ Executor for SYCL execution and other base utilities
  [#648](#648), [#661](#661), [#757](#757), [#832](#832)
+ Port matrix formats, solvers and related kernels to DPC++. For some kernels,
  also make use of a shared kernel implementation for all executors (except
  Reference). [#710](#710), [#799](#799), [#779](#779), [#733](#733), [#844](#844), [#843](#843), [#789](#789), [#845](#845), [#849](#849), [#855](#855), [#856](#856)
+ Add accessors which allow multi-precision kernels, among other things.
  [#643](#643), [#708](#708)
+ Add support for mixed precision operations through apply in all LinOps. [#677](#677)
+ Add incomplete Cholesky factorizations and preconditioners as well as some
  improvements to ILU. [#672](#672), [#837](#837), [#846](#846)
+ Add an AMGX implementation and kernels on all devices but DPC++.
  [#528](#528), [#695](#695), [#860](#860)
+ Add a new mixed-precision capability solver, Compressed Basis GMRES
  (CB-GMRES). [#693](#693), [#763](#763)
+ Add the IDR(s) solver. [#620](#620)
+ Add a new fixed-size block CSR matrix format (for the Reference executor).
  [#671](#671), [#730](#730)
+ Add native mixed-precision support to the ELL format. [#717](#717), [#780](#780)
+ Add Reverse Cuthill-McKee reordering [#500](#500), [#649](#649)
+ Add matrix assembly support on CPUs. [#644](#644)
+ Extends ISAI from triangular to general and spd matrices. [#690](#690)

Other additions:
+ Add the possibility to apply real matrices to complex vectors.
  [#655](#655), [#658](#658)
+ Add functions to compute the absolute of a matrix format. [#636](#636)
+ Add symmetric permutation and improve existing permutations.
  [#684](#684), [#657](#657), [#663](#663)
+ Add a MachineTopology class with HWLOC support [#554](#554), [#697](#697)
+ Add an implicit residual norm criterion. [#702](#702), [#818](#818), [#850](#850)
+ Row-major accessor is generalized to more than 2 dimensions and a new
  "block column-major" accessor has been added. [#707](#707)
+ Add an heat equation example. [#698](#698), [#706](#706)
+ Add ccache support in CMake and CI. [#725](#725), [#739](#739)
+ Allow tuning and benchmarking variables non intrusively. [#692](#692)
+ Add triangular solver benchmark [#664](#664)
+ Add benchmarks for BLAS operations [#772](#772), [#829](#829)
+ Add support for different precisions and consistent index types in benchmarks.
  [#675](#675), [#828](#828)
+ Add a Github bot system to facilitate development and PR management.
  [#667](#667), [#674](#674), [#689](#689), [#853](#853)
+ Add Intel (DPC++) CI support and enable CI on HPC systems. [#736](#736), [#751](#751), [#781](#781)
+ Add ssh debugging for Github Actions CI. [#749](#749)
+ Add pipeline segmentation for better CI speed. [#737](#737)


Changes:
+ Add a Scalar Jacobi specialization and kernels. [#808](#808), [#834](#834), [#854](#854)
+ Add implicit residual log for solvers and benchmarks. [#714](#714)
+ Change handling of the conjugate in the dense dot product. [#755](#755)
+ Improved Dense stride handling. [#774](#774)
+ Multiple improvements to the OpenMP kernels performance, including COO,
an exclusive prefix sum, and more. [#703](#703), [#765](#765), [#740](#740)
+ Allow specialization of submatrix and other dense creation functions in solvers. [#718](#718)
+ Improved Identity constructor and treatment of rectangular matrices. [#646](#646)
+ Allow CUDA/HIP executors to select allocation mode. [#758](#758)
+ Check if executors share the same memory. [#670](#670)
+ Improve test install and smoke testing support. [#721](#721)
+ Update the JOSS paper citation and add publications in the documentation.
  [#629](#629), [#724](#724)
+ Improve the version output. [#806](#806)
+ Add some utilities for dim and span. [#821](#821)
+ Improved solver and preconditioner benchmarks. [#660](#660)
+ Improve benchmark timing and output. [#669](#669), [#791](#791), [#801](#801), [#812](#812)


Fixes:
+ Sorting fix for the Jacobi preconditioner. [#659](#659)
+ Also log the first residual norm in CGS [#735](#735)
+ Fix BiCG and HIP CSR to work with complex matrices. [#651](#651)
+ Fix Coo SpMV on strided vectors. [#807](#807)
+ Fix segfault of extract_diagonal, add short-and-fat test. [#769](#769)
+ Fix device_reset issue by moving counter/mutex to device. [#810](#810)
+ Fix `EnableLogging` superclass. [#841](#841)
+ Support ROCm 4.1.x and breaking HIP_PLATFORM changes. [#726](#726)
+ Decreased test size for a few device tests. [#742](#742)
+ Fix multiple issues with our CMake HIP and RPATH setup.
  [#712](#712), [#745](#745), [#709](#709)
+ Cleanup our CMake installation step. [#713](#713)
+ Various simplification and fixes to the Windows CMake setup. [#720](#720), [#785](#785)
+ Simplify third-party integration. [#786](#786)
+ Improve Ginkgo device arch flags management. [#696](#696)
+ Other fixes and improvements to the CMake setup.
  [#685](#685), [#792](#792), [#705](#705), [#836](#836)
+ Clarification of dense norm documentation [#784](#784)
+ Various development tools fixes and improvements [#738](#738), [#830](#830), [#840](#840)
+ Make multiple operators/constructors explicit. [#650](#650), [#761](#761)
+ Fix some issues, memory leaks and warnings found by MSVC.
  [#666](#666), [#731](#731)
+ Improved solver memory estimates and consistent iteration counts [#691](#691)
+ Various logger improvements and fixes [#728](#728), [#743](#743), [#754](#754)
+ Fix for ForwardIterator requirements in iterator_factory. [#665](#665)
+ Various benchmark fixes. [#647](#647), [#673](#673), [#722](#722)
+ Various CI fixes and improvements. [#642](#642), [#641](#641), [#795](#795), [#783](#783), [#793](#793), [#852](#852)


Related PR: #857
tcojean added a commit that referenced this pull request Aug 23, 2021
Release 1.4.0 to master

The Ginkgo team is proud to announce the new Ginkgo minor release 1.4.0. This
release brings most of the Ginkgo functionality to the Intel DPC++ ecosystem
which enables Intel-GPU and CPU execution. The only Ginkgo features which have
not been ported yet are some preconditioners.

Ginkgo's mixed-precision support is greatly enhanced thanks to:
1. The new Accessor concept, which allows writing kernels featuring on-the-fly
memory compression, among other features. The accessor can be used as
header-only, see the [accessor BLAS benchmarks repository](https://github.com/ginkgo-project/accessor-BLAS/tree/develop) as a usage example.
2. All LinOps now transparently support mixed-precision execution. By default,
this is done through a temporary copy which may have a performance impact but
already allows mixed-precision research.

Native mixed-precision ELL kernels are implemented which do not see this cost.
The accessor is also leveraged in a new CB-GMRES solver which allows for
performance improvements by compressing the Krylov basis vectors. Many other
features have been added to Ginkgo, such as reordering support, a new IDR
solver, Incomplete Cholesky preconditioner, matrix assembly support (only CPU
for now), machine topology information, and more!

Supported systems and requirements:
+ For all platforms, cmake 3.13+
+ C++14 compliant compiler
+ Linux and MacOS
  + gcc: 5.3+, 6.3+, 7.3+, all versions after 8.1+
  + clang: 3.9+
  + Intel compiler: 2018+
  + Apple LLVM: 8.0+
  + CUDA module: CUDA 9.0+
  + HIP module: ROCm 3.5+
  + DPC++ module: Intel OneAPI 2021.3. Set the CXX compiler to `dpcpp`.
+ Windows
  + MinGW and Cygwin: gcc 5.3+, 6.3+, 7.3+, all versions after 8.1+
  + Microsoft Visual Studio: VS 2019
  + CUDA module: CUDA 9.0+, Microsoft Visual Studio
  + OpenMP module: MinGW or Cygwin.


Algorithm and important feature additions:
+ Add a new DPC++ Executor for SYCL execution and other base utilities
  [#648](#648), [#661](#661), [#757](#757), [#832](#832)
+ Port matrix formats, solvers and related kernels to DPC++. For some kernels,
  also make use of a shared kernel implementation for all executors (except
  Reference). [#710](#710), [#799](#799), [#779](#779), [#733](#733), [#844](#844), [#843](#843), [#789](#789), [#845](#845), [#849](#849), [#855](#855), [#856](#856)
+ Add accessors which allow multi-precision kernels, among other things.
  [#643](#643), [#708](#708)
+ Add support for mixed precision operations through apply in all LinOps. [#677](#677)
+ Add incomplete Cholesky factorizations and preconditioners as well as some
  improvements to ILU. [#672](#672), [#837](#837), [#846](#846)
+ Add an AMGX implementation and kernels on all devices but DPC++.
  [#528](#528), [#695](#695), [#860](#860)
+ Add a new mixed-precision capability solver, Compressed Basis GMRES
  (CB-GMRES). [#693](#693), [#763](#763)
+ Add the IDR(s) solver. [#620](#620)
+ Add a new fixed-size block CSR matrix format (for the Reference executor).
  [#671](#671), [#730](#730)
+ Add native mixed-precision support to the ELL format. [#717](#717), [#780](#780)
+ Add Reverse Cuthill-McKee reordering [#500](#500), [#649](#649)
+ Add matrix assembly support on CPUs. [#644](#644)
+ Extends ISAI from triangular to general and spd matrices. [#690](#690)

Other additions:
+ Add the possibility to apply real matrices to complex vectors.
  [#655](#655), [#658](#658)
+ Add functions to compute the absolute of a matrix format. [#636](#636)
+ Add symmetric permutation and improve existing permutations.
  [#684](#684), [#657](#657), [#663](#663)
+ Add a MachineTopology class with HWLOC support [#554](#554), [#697](#697)
+ Add an implicit residual norm criterion. [#702](#702), [#818](#818), [#850](#850)
+ Row-major accessor is generalized to more than 2 dimensions and a new
  "block column-major" accessor has been added. [#707](#707)
+ Add an heat equation example. [#698](#698), [#706](#706)
+ Add ccache support in CMake and CI. [#725](#725), [#739](#739)
+ Allow tuning and benchmarking variables non intrusively. [#692](#692)
+ Add triangular solver benchmark [#664](#664)
+ Add benchmarks for BLAS operations [#772](#772), [#829](#829)
+ Add support for different precisions and consistent index types in benchmarks.
  [#675](#675), [#828](#828)
+ Add a Github bot system to facilitate development and PR management.
  [#667](#667), [#674](#674), [#689](#689), [#853](#853)
+ Add Intel (DPC++) CI support and enable CI on HPC systems. [#736](#736), [#751](#751), [#781](#781)
+ Add ssh debugging for Github Actions CI. [#749](#749)
+ Add pipeline segmentation for better CI speed. [#737](#737)


Changes:
+ Add a Scalar Jacobi specialization and kernels. [#808](#808), [#834](#834), [#854](#854)
+ Add implicit residual log for solvers and benchmarks. [#714](#714)
+ Change handling of the conjugate in the dense dot product. [#755](#755)
+ Improved Dense stride handling. [#774](#774)
+ Multiple improvements to the OpenMP kernels performance, including COO,
an exclusive prefix sum, and more. [#703](#703), [#765](#765), [#740](#740)
+ Allow specialization of submatrix and other dense creation functions in solvers. [#718](#718)
+ Improved Identity constructor and treatment of rectangular matrices. [#646](#646)
+ Allow CUDA/HIP executors to select allocation mode. [#758](#758)
+ Check if executors share the same memory. [#670](#670)
+ Improve test install and smoke testing support. [#721](#721)
+ Update the JOSS paper citation and add publications in the documentation.
  [#629](#629), [#724](#724)
+ Improve the version output. [#806](#806)
+ Add some utilities for dim and span. [#821](#821)
+ Improved solver and preconditioner benchmarks. [#660](#660)
+ Improve benchmark timing and output. [#669](#669), [#791](#791), [#801](#801), [#812](#812)


Fixes:
+ Sorting fix for the Jacobi preconditioner. [#659](#659)
+ Also log the first residual norm in CGS [#735](#735)
+ Fix BiCG and HIP CSR to work with complex matrices. [#651](#651)
+ Fix Coo SpMV on strided vectors. [#807](#807)
+ Fix segfault of extract_diagonal, add short-and-fat test. [#769](#769)
+ Fix device_reset issue by moving counter/mutex to device. [#810](#810)
+ Fix `EnableLogging` superclass. [#841](#841)
+ Support ROCm 4.1.x and breaking HIP_PLATFORM changes. [#726](#726)
+ Decreased test size for a few device tests. [#742](#742)
+ Fix multiple issues with our CMake HIP and RPATH setup.
  [#712](#712), [#745](#745), [#709](#709)
+ Cleanup our CMake installation step. [#713](#713)
+ Various simplification and fixes to the Windows CMake setup. [#720](#720), [#785](#785)
+ Simplify third-party integration. [#786](#786)
+ Improve Ginkgo device arch flags management. [#696](#696)
+ Other fixes and improvements to the CMake setup.
  [#685](#685), [#792](#792), [#705](#705), [#836](#836)
+ Clarification of dense norm documentation [#784](#784)
+ Various development tools fixes and improvements [#738](#738), [#830](#830), [#840](#840)
+ Make multiple operators/constructors explicit. [#650](#650), [#761](#761)
+ Fix some issues, memory leaks and warnings found by MSVC.
  [#666](#666), [#731](#731)
+ Improved solver memory estimates and consistent iteration counts [#691](#691)
+ Various logger improvements and fixes [#728](#728), [#743](#743), [#754](#754)
+ Fix for ForwardIterator requirements in iterator_factory. [#665](#665)
+ Various benchmark fixes. [#647](#647), [#673](#673), [#722](#722)
+ Various CI fixes and improvements. [#642](#642), [#641](#641), [#795](#795), [#783](#783), [#793](#793), [#852](#852)

Related PR: #866
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1:ST:ready-to-merge This PR is ready to merge. plat:windows This is related to the Windows Operating system reg:build This is related to the build system. reg:ci-cd This is related to the continuous integration system. reg:example This is related to the examples.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants