Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions .github/scripts/run_benchmarks.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@
# needed. The drivers always exit non-zero (the Jamfile declares them run-fail),
# so success is judged on the data set actually being written.
#
# --sources names extra translation units to build alongside the driver, which is
# how the platforms with no packaged Abseil get one: point --cxxflags at a source
# checkout and --sources at absl/numeric/int128.cc, the only file of it that
# absl::uint128 and absl::int128 need.
#
# .github/scripts/run_benchmarks.sh --compiler g++-14 --out /tmp/results
# .github/scripts/run_benchmarks.sh --msvc --elements 5000000 --out C:/results

Expand All @@ -20,6 +25,7 @@ set -e
COMPILER=""
EXTRA_CXXFLAGS=""
EXTRA_LDFLAGS=""
EXTRA_SOURCES=""
ELEMENTS=20000000
REPETITIONS=5
OUT=""
Expand All @@ -30,6 +36,7 @@ while [ $# -gt 0 ]; do
--compiler) COMPILER="$2"; shift 2 ;;
--cxxflags) EXTRA_CXXFLAGS="$2"; shift 2 ;;
--ldflags) EXTRA_LDFLAGS="$2"; shift 2 ;;
--sources) EXTRA_SOURCES="$2"; shift 2 ;;
--elements) ELEMENTS="$2"; shift 2 ;;
--repetitions) REPETITIONS="$2"; shift 2 ;;
--out) OUT="$2"; shift 2 ;;
Expand Down Expand Up @@ -67,12 +74,14 @@ for sign in u128 i128; do
# the widest vectors need.
executable="./benchmark_$sign.exe"
# shellcheck disable=SC2086
cl -nologo -std:c++20 -O2 -EHsc -DNDEBUG $INCLUDES "$source_file" \
-Fe:"benchmark_$sign.exe" -link -LARGEADDRESSAWARE
cl -nologo -std:c++20 -O2 -EHsc -DNDEBUG $EXTRA_CXXFLAGS $INCLUDES \
"$source_file" $EXTRA_SOURCES \
-Fe:"benchmark_$sign.exe" -link -LARGEADDRESSAWARE $EXTRA_LDFLAGS
else
executable="./benchmark_$sign"
# shellcheck disable=SC2086
$COMPILER -std=c++20 -O2 -DNDEBUG $EXTRA_CXXFLAGS $INCLUDES "$source_file" \
$COMPILER -std=c++20 -O2 -DNDEBUG $EXTRA_CXXFLAGS $INCLUDES \
"$source_file" $EXTRA_SOURCES \
-o "benchmark_$sign" $EXTRA_LDFLAGS
fi

Expand Down
17 changes: 17 additions & 0 deletions .github/workflows/benchmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@
# 128-bit vectors of the default length do not fit in a 32-bit address space. The
# element count lands in the data set and is printed in the documentation, so the
# tables stay honest about what was measured.
#
# Linux and macOS take Abseil from a package. Windows has none, so those jobs
# check out the pinned Abseil release and build the single translation unit that
# absl::uint128 and absl::int128 need, with the same flags as the rest of the
# benchmark. Building all of Abseil to link one object would cost more CI time
# than the benchmarks themselves, and compiling it here is what keeps the
# comparison fair: both sides of every row come out of one -O2 build.

#
# Every pull request gets a full set of runs: the step summary of each job
Expand Down Expand Up @@ -65,6 +72,9 @@ env:
ELEMENTS: ${{ inputs.elements || '20000000' }}
EMULATED_ELEMENTS: ${{ inputs.emulated_elements || '2000000' }}
REPETITIONS: ${{ inputs.repetitions || '5' }}
# The Abseil release the Windows jobs build against, latest LTS at the time of
# writing. Linux and macOS take whatever their package manager ships.
ABSEIL_TAG: '20260526.0'

jobs:
linux:
Expand Down Expand Up @@ -189,6 +199,11 @@ jobs:
elements: ''
steps:
- uses: actions/checkout@v6
# Cloned next to the Boost tree that the following step creates, so that the
# build reaches both of them by relative path and nothing has to be
# translated between git-bash and cl.
- name: Fetch Abseil
run: git clone --depth 1 --branch "$ABSEIL_TAG" https://github.com/abseil/abseil-cpp.git ../abseil-cpp
- name: Setup Boost
run: .github/scripts/setup_benchmarks.sh
- uses: TheMrMilchmann/setup-msvc-dev@v4
Expand All @@ -201,6 +216,8 @@ jobs:
: "${COUNT:=$ELEMENTS}"
"$GITHUB_WORKSPACE/.github/scripts/run_benchmarks.sh" \
--msvc \
--cxxflags '-I../abseil-cpp' \
--sources '../abseil-cpp/absl/numeric/int128.cc' \
--elements "$COUNT" \
--repetitions "$REPETITIONS" \
--out "$GITHUB_WORKSPACE/bench-results"
Expand Down
450 changes: 257 additions & 193 deletions doc/modules/ROOT/data/benchmarks-windows-arm64/i128.json

Large diffs are not rendered by default.

463 changes: 261 additions & 202 deletions doc/modules/ROOT/data/benchmarks-windows-arm64/u128.json

Large diffs are not rendered by default.

450 changes: 257 additions & 193 deletions doc/modules/ROOT/data/benchmarks-windows-x64/i128.json

Large diffs are not rendered by default.

463 changes: 261 additions & 202 deletions doc/modules/ROOT/data/benchmarks-windows-x64/u128.json

Large diffs are not rendered by default.

450 changes: 257 additions & 193 deletions doc/modules/ROOT/data/benchmarks-windows-x86/i128.json

Large diffs are not rendered by default.

463 changes: 261 additions & 202 deletions doc/modules/ROOT/data/benchmarks-windows-x86/u128.json

Large diffs are not rendered by default.

Binary file modified doc/modules/ROOT/images/i128_graphs/windows/ARM64_benchmarks.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified doc/modules/ROOT/images/i128_graphs/windows/x64_benchmarks.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified doc/modules/ROOT/images/i128_graphs/windows/x86_benchmarks.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified doc/modules/ROOT/images/u128_graphs/windows/ARM64_benchmarks.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified doc/modules/ROOT/images/u128_graphs/windows/x64_benchmarks.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified doc/modules/ROOT/images/u128_graphs/windows/x86_benchmarks.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
50 changes: 26 additions & 24 deletions doc/modules/ROOT/pages/i128_benchmarks.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ On most platforms we use the builtin `\__int128` as the reference benchmark.
When this is unavailable (such as on 32-bit architectures) we use `boost::multiprecision::int128` (abbreviated as `boost::mp::int128`) as it is widely used, and known to be portable.
On MSVC platforms we use as reference `std::_Signed128` from the header `<__msvc_int128.hpp>` since this is bundled with their compiler.
Lastly, when available, the benchmarks are also run with `absl::int128` which is the 128-bit signed integer from the https://abseil.io[Abseil Libraries].
Linux and macOS take Abseil from a package; Windows has none, so it is built from source at a pinned release with the same compiler and optimization flags as the rest of the table.
Abseil only uses a native 128-bit type when the compiler has one, which MSVC does not, so its portable path is what the Windows rows measure.

NOTE: Each platform reports a single run, several of them from shared CI runners, so treat small differences as noise.

Expand Down Expand Up @@ -166,16 +168,16 @@ image::i128_graphs/linux/ARM32_relative_performance.png[ARM32 Relative Performan

Measured with MSVC 14.51 (pass:[C++]20): 5 passes over 20,000,000 element pairs (100,000,000 operations per cell).

[cols="1,>1,>1,>1"]
[cols="1,>1,>1,>1,>1"]
|===
| Operation | `std::_Signed128` | `int128` | `boost::mp::int128`
| Operation | `std::_Signed128` | `int128` | `boost::mp::int128` | `absl::int128`

| Comparisons | 1,361,518 | 1,271,189 | 4,315,263
| Addition | 115,135 | 107,633 | 2,089,612
| Subtraction | 108,885 | 105,990 | 2,265,849
| Multiplication | 196,081 | 113,761 | 3,024,735
| Division | 1,031,262 | 1,241,974 | 4,191,052
| Modulo | 1,080,810 | 1,327,820 | 4,139,305
| Comparisons | 1,313,422 | 1,301,134 | 4,219,069 | 1,310,527
| Addition | 104,308 | 105,315 | 2,122,367 | 92,929
| Subtraction | 101,850 | 100,240 | 2,328,494 | 93,068
| Multiplication | 167,872 | 112,184 | 3,004,661 | 114,658
| Division | 938,908 | 1,233,042 | 4,238,571 | 17,581,284
| Modulo | 1,101,512 | 1,330,229 | 4,187,828 | 17,626,117
|===

image::i128_graphs/windows/x64_relative_performance.png[x64 Relative Performance, width=100%]
Expand All @@ -184,16 +186,16 @@ image::i128_graphs/windows/x64_relative_performance.png[x64 Relative Performance

Measured with MSVC 14.44 (pass:[C++]20): 5 passes over 20,000,000 element pairs (100,000,000 operations per cell).

[cols="1,>1,>1,>1"]
[cols="1,>1,>1,>1,>1"]
|===
| Operation | `std::_Signed128` | `int128` | `boost::mp::int128`
| Operation | `std::_Signed128` | `int128` | `boost::mp::int128` | `absl::int128`

| Comparisons | 1,191,566 | 1,125,261 | 2,871,780
| Addition | 86,160 | 84,055 | 1,598,093
| Subtraction | 88,263 | 87,934 | 1,769,736
| Multiplication | 505,934 | 81,566 | 2,395,038
| Division | 1,506,403 | 1,480,602 | 3,622,476
| Modulo | 1,722,778 | 1,447,734 | 3,555,076
| Comparisons | 1,163,962 | 1,124,237 | 2,892,406 | 1,176,305
| Addition | 88,279 | 83,389 | 1,604,518 | 86,567
| Subtraction | 88,880 | 86,568 | 1,766,980 | 88,994
| Multiplication | 532,086 | 81,119 | 2,389,516 | 129,947
| Division | 1,507,951 | 1,480,214 | 3,623,405 | 10,773,260
| Modulo | 1,720,557 | 1,443,956 | 3,531,333 | 10,797,356
|===

image::i128_graphs/windows/ARM64_relative_performance.png[ARM64 Relative Performance, width=100%]
Expand All @@ -202,16 +204,16 @@ image::i128_graphs/windows/ARM64_relative_performance.png[ARM64 Relative Perform

Measured with MSVC 14.51 (pass:[C++]20): 5 passes over 10,000,000 element pairs (50,000,000 operations per cell).

[cols="1,>1,>1,>1"]
[cols="1,>1,>1,>1,>1"]
|===
| Operation | `std::_Signed128` | `int128` | `boost::mp::int128`
| Operation | `std::_Signed128` | `int128` | `boost::mp::int128` | `absl::int128`

| Comparisons | 2,084,732 | 1,947,041 | 2,898,710
| Addition | 131,344 | 128,063 | 1,325,852
| Subtraction | 447,258 | 128,145 | 1,454,691
| Multiplication | 877,318 | 149,394 | 2,018,981
| Division | 3,479,471 | 3,113,092 | 3,948,298
| Modulo | 2,683,535 | 3,157,295 | 3,620,409
| Comparisons | 1,642,521 | 1,501,096 | 2,486,760 | 1,517,123
| Addition | 73,033 | 58,313 | 1,218,478 | 55,079
| Subtraction | 517,398 | 66,889 | 1,314,906 | 57,791
| Multiplication | 970,404 | 170,877 | 2,302,831 | 3,865,486
| Division | 2,840,529 | 2,663,730 | 3,362,691 | 9,883,544
| Modulo | 2,390,312 | 2,752,450 | 3,255,144 | 9,862,440
|===

image::i128_graphs/windows/x86_relative_performance.png[x86 Relative Performance, width=100%]
Expand Down
50 changes: 26 additions & 24 deletions doc/modules/ROOT/pages/u128_benchmarks.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ On most platforms we use the builtin `unsigned \__int128` as the reference bench
When this is unavailable (such as on 32-bit architectures) we use `boost::multiprecision::uint128` (abbreviated as `boost::mp::uint128`) as it is widely used, and known to be portable.
On MSVC platforms we use as reference `std::_Unsigned128` from the header `<__msvc_int128.hpp>` since this is bundled with their compiler.
Lastly, when available, the benchmarks are also run with `absl::uint128` which is the 128-bit unsigned integer from the https://abseil.io[Abseil Libraries].
Linux and macOS take Abseil from a package; Windows has none, so it is built from source at a pinned release with the same compiler and optimization flags as the rest of the table.
Abseil only uses a native 128-bit type when the compiler has one, which MSVC does not, so its portable path is what the Windows rows measure.

NOTE: Each platform reports a single run, several of them from shared CI runners, so treat small differences as noise.

Expand Down Expand Up @@ -166,16 +168,16 @@ image::u128_graphs/linux/ARM32_relative_performance.png[ARM32 Relative Performan

Measured with MSVC 14.51 (pass:[C++]20): 5 passes over 20,000,000 element pairs (100,000,000 operations per cell).

[cols="1,>1,>1,>1"]
[cols="1,>1,>1,>1,>1"]
|===
| Operation | `std::_Unsigned128` | `uint128` | `boost::mp::uint128`
| Operation | `std::_Unsigned128` | `uint128` | `boost::mp::uint128` | `absl::uint128`

| Comparisons | 1,313,048 | 1,251,724 | 1,970,110
| Addition | 106,229 | 98,075 | 2,153,287
| Subtraction | 115,275 | 97,252 | 2,318,242
| Multiplication | 115,091 | 112,082 | 2,428,936
| Division | 1,127,057 | 1,086,078 | 3,340,168
| Modulo | 1,249,176 | 1,144,617 | 2,721,519
| Comparisons | 1,351,853 | 1,210,126 | 1,953,037 | 1,199,946
| Addition | 102,157 | 89,607 | 2,241,970 | 91,530
| Subtraction | 112,192 | 89,322 | 2,346,973 | 90,050
| Multiplication | 116,473 | 109,967 | 2,262,252 | 110,565
| Division | 1,135,328 | 1,077,802 | 3,443,408 | 20,871,271
| Modulo | 1,271,929 | 1,165,839 | 2,816,863 | 20,945,166
|===

image::u128_graphs/windows/x64_relative_performance.png[x64 Relative Performance, width=100%]
Expand All @@ -184,16 +186,16 @@ image::u128_graphs/windows/x64_relative_performance.png[x64 Relative Performance

Measured with MSVC 14.44 (pass:[C++]20): 5 passes over 20,000,000 element pairs (100,000,000 operations per cell).

[cols="1,>1,>1,>1"]
[cols="1,>1,>1,>1,>1"]
|===
| Operation | `std::_Unsigned128` | `uint128` | `boost::mp::uint128`
| Operation | `std::_Unsigned128` | `uint128` | `boost::mp::uint128` | `absl::uint128`

| Comparisons | 1,262,216 | 753,322 | 1,757,122
| Addition | 86,038 | 84,791 | 1,535,904
| Subtraction | 86,521 | 87,885 | 1,712,929
| Multiplication | 458,107 | 82,947 | 1,780,541
| Division | 1,334,323 | 920,483 | 2,843,970
| Modulo | 1,491,426 | 915,935 | 2,420,312
| Comparisons | 1,198,039 | 749,538 | 1,780,083 | 1,146,973
| Addition | 88,759 | 85,378 | 1,534,492 | 86,222
| Subtraction | 86,292 | 84,435 | 1,707,180 | 92,099
| Multiplication | 457,638 | 83,669 | 1,785,505 | 127,263
| Division | 1,337,104 | 921,115 | 2,824,979 | 12,558,253
| Modulo | 1,488,718 | 918,404 | 2,433,789 | 12,522,017
|===

image::u128_graphs/windows/ARM64_relative_performance.png[ARM64 Relative Performance, width=100%]
Expand All @@ -202,16 +204,16 @@ image::u128_graphs/windows/ARM64_relative_performance.png[ARM64 Relative Perform

Measured with MSVC 14.51 (pass:[C++]20): 5 passes over 10,000,000 element pairs (50,000,000 operations per cell).

[cols="1,>1,>1,>1"]
[cols="1,>1,>1,>1,>1"]
|===
| Operation | `std::_Unsigned128` | `uint128` | `boost::mp::uint128`
| Operation | `std::_Unsigned128` | `uint128` | `boost::mp::uint128` | `absl::uint128`

| Comparisons | 2,519,936 | 2,006,459 | 1,516,713
| Addition | 132,021 | 129,303 | 1,184,928
| Subtraction | 505,196 | 127,036 | 1,526,177
| Multiplication | 765,417 | 960,622 | 1,459,692
| Division | 2,510,903 | 2,217,091 | 2,816,842
| Modulo | 1,985,290 | 2,271,454 | 2,458,916
| Comparisons | 1,978,828 | 1,761,531 | 1,039,171 | 1,828,790
| Addition | 71,811 | 56,033 | 1,029,278 | 60,322
| Subtraction | 574,255 | 61,036 | 1,351,338 | 59,280
| Multiplication | 938,730 | 1,121,940 | 1,741,827 | 1,668,286
| Division | 2,111,143 | 1,944,458 | 2,592,836 | 11,246,769
| Modulo | 1,858,981 | 1,985,261 | 2,215,040 | 11,258,901
|===

image::u128_graphs/windows/x86_relative_performance.png[x86 Relative Performance, width=100%]
Expand Down
4 changes: 2 additions & 2 deletions test/benchmark_i128.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@
#define BOOST_INT128_HAS_MSVC_INTERNAL_I128
#endif

// Abseil requires at least C++17 (at time of writing)
#if __has_include(<absl/numeric/int128.h>) && defined(__cplusplus) && __cplusplus >= 201703L
#if __has_include(<absl/numeric/int128.h>) && \
(__cplusplus >= 201703L || (defined(_MSVC_LANG) && _MSVC_LANG >= 201703L))
# include <absl/numeric/int128.h>
# ifndef __i386__
# define BOOST_INT128_BENCHMARK_ABSL
Expand Down
4 changes: 2 additions & 2 deletions test/benchmark_u128.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@
#define BOOST_INT128_HAS_MSVC_INTERNAL_I128
#endif

// Abseil requires at least C++17 (at time of writing)
#if __has_include(<absl/numeric/int128.h>) && defined(__cplusplus) && __cplusplus >= 201703L
#if __has_include(<absl/numeric/int128.h>) && \
(__cplusplus >= 201703L || (defined(_MSVC_LANG) && _MSVC_LANG >= 201703L))
# include <absl/numeric/int128.h>
# ifndef __i386__
# define BOOST_INT128_BENCHMARK_ABSL
Expand Down
Loading