Skip to content

Adding basic support of benchmarks into CMake#4501

Merged
martin-frbg merged 1 commit intoOpenMathLib:developfrom
dmikushin:add-benchmarks-to-cmake
Feb 27, 2024
Merged

Adding basic support of benchmarks into CMake#4501
martin-frbg merged 1 commit intoOpenMathLib:developfrom
dmikushin:add-benchmarks-to-cmake

Conversation

@dmikushin
Copy link
Copy Markdown
Contributor

@dmikushin dmikushin commented Feb 10, 2024

Adding basic support of benchmarks into CMake for single, double, single complex and double complex cases. Each benchmarking target has a suffix to identify the data type, for example ./benchmark_gemm3m_COMPLEX_DOUBLE is a gemm3m.c source compiled with COMPLEX and DOUBLE macros defined.

This PR is done under assumption that the benchmarks folder is not yet wired into CMake compilation.

This PR is not trying to re-create the benchmarks/Makefile behavior. Instead, the purpose of this PR is to compile as many tests as possible from the existing material.

…gle complex and double complex cases. Each benchmarking target has a suffix to identify the data type, for example ./benchmark_gemm3m_COMPLEX_DOUBLE is a gemm3m.c source compiled with COMPLEX and DOUBLE macros defined
@martin-frbg
Copy link
Copy Markdown
Collaborator

right, the benchmarks are currently not available in the cmake build, but the benchmark suite (which mostly dates back to GotoBLAS) is due for an overhaul anyway.

@dmikushin
Copy link
Copy Markdown
Contributor Author

dmikushin commented Feb 10, 2024

We may still wish to enable it in its current state to provide a lot more code coverage. Otherwise, many functions are not tested at all. For example, cgemm3m/zgemm3m are completely untested, unless we accept these tests for now as a compromise. Of course, someone may work on bringing them to a much better state later.

I'm also using this PR to demonstrate the defect in #4500

@martin-frbg
Copy link
Copy Markdown
Collaborator

The benchmarks make no claim to test for accuracy. The GEMM3M functions are supported on a handful of architectures only, and tests for them are included in test/ctest, though it turns out they are not getting built by default currently.

@dmikushin
Copy link
Copy Markdown
Contributor Author

I understand, still having more coverage is better than nothing. I mean, "Perfect is the enemy of good." (c) Voltaire

@martin-frbg martin-frbg added this to the 0.3.27 milestone Feb 27, 2024
@martin-frbg
Copy link
Copy Markdown
Collaborator

will follow this with a PR to make OpenMP optional (only required in the smallscaling benchmark) and probably make building the benchmarks optional as a whole (the gmake build does not include them by default either)

@martin-frbg martin-frbg merged commit 96e4218 into OpenMathLib:develop Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants