From 7bd3f562e45d48cab883f83a09420ec81a3bd278 Mon Sep 17 00:00:00 2001 From: Vlad Frolov Date: Sun, 20 May 2018 17:53:28 +0300 Subject: [PATCH] Clarified the meaning of the "naive" and moved some solutions to the main scoreboard according to the defined meaning --- README.md | 183 +++++++++++------- ...{main_fast.adb => main_naive_pointers.adb} | 8 +- ...{tree_fast.adb => tree_naive_pointers.adb} | 4 +- ...{tree_fast.ads => tree_naive_pointers.ads} | 4 +- c++/Makefile | 26 +++ c++/README.md | 2 +- c++/main-naive-java-like.cpp | 99 ++++++++++ c++/{main-raw.cpp => main-naive-raw.cpp} | 0 ...ared_ptr.cpp => main-naive-shared_ptr.cpp} | 0 c++/main-tuned-raw-with-pool.cpp | 167 ++++++++++++++++ c++/main-tuned-raw.cpp | 160 +++++++++++++++ ...ique_ptr.cpp => main-tuned-unique_ptr.cpp} | 0 d/README.md | 16 +- d/{main_pointers.d => main_naive_pointers.d} | 0 d/{main_nort.d => main_tuned_no_rt.d} | 0 ..._cheating.pas => main_cheating_noheap.pas} | 0 ...w_pointers.pas => main_naive_pointers.pas} | 0 17 files changed, 586 insertions(+), 83 deletions(-) rename ada/{main_fast.adb => main_naive_pointers.adb} (81%) rename ada/{tree_fast.adb => tree_naive_pointers.adb} (97%) rename ada/{tree_fast.ads => tree_naive_pointers.ads} (94%) create mode 100644 c++/Makefile create mode 100644 c++/main-naive-java-like.cpp rename c++/{main-raw.cpp => main-naive-raw.cpp} (100%) rename c++/{main-shared_ptr.cpp => main-naive-shared_ptr.cpp} (100%) create mode 100644 c++/main-tuned-raw-with-pool.cpp create mode 100644 c++/main-tuned-raw.cpp rename c++/{main-unique_ptr.cpp => main-tuned-unique_ptr.cpp} (100%) rename d/{main_pointers.d => main_naive_pointers.d} (100%) rename d/{main_nort.d => main_tuned_no_rt.d} (100%) rename pascal/{main_noheap_cheating.pas => main_cheating_noheap.pas} (100%) rename pascal/{main_raw_pointers.pas => main_naive_pointers.pas} (100%) diff --git a/README.md b/README.md index 016b0eb..298a009 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,35 @@ separately, so it has a few shortcuts, and thus it might be not a completely fair comparison (I will try to implement "fair" C++ solution and also "C++"-like Rust solution to see if the performance can be on par). + +## Metrics + +We define the "naive" implementations as those which a developer with enough +experience in a given language would implement as a baseline "good enough" +solution where correctness is more important than performance. + +However, experienced developers in system programming languages (e.g. C, C++, D) +tend to work comfortably with raw pointers, and that makes the comparison of the +solutions only by speed and memory consumption unfair. High-level abstractions +tend to introduce some performance hit in exchange for safety and +expressiveness. Thus, we added other metrics: "Expressiveness" (1 - pure magic, +10 - easy to get started and express your intent) and "Maintenance Complexity" +(1 - easy to maintain, 5 - ugly yet safe, 6-10 - hard to keep it right, i.e. +risky). The ease of maintenance is estimated for a big project using the given +language and the given approach. + +Thus, here are the metrics: + +* Expressiveness, scores from 1 to 10 - higher value is better (keep in mind + that this is a subjective metric based on the author's experience!) +* Maintenance Complexity, scores from 1 to 10 - smaller value is better (keep in + mind that this is a subjective metric based on the author's experience!) +* Real Time, seconds - smaller value is better +* Slowdown Time (relative speed compared to the best tuned solution) - smaller value is better +* Memory, megabytes - smaller value is better +* Binary Size, megabytes - smaller value is better + + ## Measurements To measure time we used `time` util on Mac OS and Windows (msys2 environment), @@ -36,6 +65,7 @@ and given the limitations of cgroup subsystem (it counts caches and loaded shared objects unless they are already cached or loaded by other processes), we take the lowest memory footprint among all the executions. + ## Results Originally, this benchmark had a goal to implement the same "natural" and @@ -52,69 +82,94 @@ All tables are sorted in an alphabetical order. #### Linux (Arch Linux, x64, Intel Core i7-4710HQ CPU) -| Language | Real Time, seconds | Slowdown Time | Memory, MB | Binary Size, MB | Compiler Version | -| -------------------------------- | ------------------ | ------------- | ---------- | ------------------------------- | --------------------------------- | -| *Best tuned solution* | 0.178 | x1 | 0.38 | | | -| **C++ `shared_ptr` ("ref-counted")** | **0.38** | x2.1 | **0.5** | 0.015 + libstdc++ | Clang 6.0.0 / GCC 8.1.0 | -| C# | 0.70\* | x3.9 | 11 | N/A | .NET Core 2.0 | -| **Go** | **0.38** | x2.1 | 5.8 | 1.2 (static) | Go 1.10.2 | -| Haskell | 0.87 | x4.9 | 3.4 | 3.8 | GHC 8.2.2 | -| JavaScript | 1.12 | x6.3 | 52 | N/A | Node.js 10.1.0 | -| Java (no-limit / `-Xm*50M`) | 0.50 / 0.50 | x2.8 | 142 / 29 | N/A | OpenJDK 1.8.0 | -| Kotlin/JVM (no-limit / `-Xm*50M`) | 0.53 / 0.51 | x2.9 | 144 / 30 | N/A | Kotlinc 1.2.40 + OpenJDK 1.8.0 | -| Kotlin/Native | 5.88 | x33 | 1.2 | 0.239 | Kotlinc-native 0.7 | -| Nim | 1.00 | x5.6 | **0.5** | 0.051 | Nim 0.18 / GCC 8.1.0 | -| OCaml | 0.69 | x3.9 | 3.8 | N/A | OCaml 4.06 | -| PHP | 4.44 | x24.9 | 5.8 | N/A | PHP 7.2.5 | -| Python (CPython) | 12.25 | x68.8 | 5 | N/A | CPython 3.6 | -| Python (PyPy) | 3.20 | x18 | 48.5 | N/A | PyPy 6.0.0 | -| **Rust "idiomatic"** | **0.37** | x2.1 | **0.5** | 0.427 | Rustc 1.26 | -| **Rust "ref-counted"** | **0.37** | x2.1 | **0.5** | 0.431 | Rustc 1.26 | -| Swift | 1.66 | x9.3 | 2.5 | 0.016 + Swift shared libraries | Swift 4.1 | - -(*) C# has a noticable VM start time (~0.4 seconds), but we still measure real +| Language | Expressiveness | Maintenance Complexity | Real Time, seconds | Slowdown Time | Memory, MB | Binary Size, MB | Compiler Version | +| --------------------------------------------- | -------------- | ---------------------- | ------------------ | ------------- | ---------- | ------------------------------- | --------------------------------- | +| *Best tuned solution* | | | **0.165** | x1 | **0.25** | | | +| Ada "naive unsafe raw pointers" | 6 | 8 | 0.24 | x1.45 | **0.4** | 0.278 | GCC Ada 8.1.0 | +| C++ "java-like" (clang) | 7 | 5 | 0.33 | x2 | **0.5** | 0.015 + libstdc++ | Clang 6.0.0 | +| C++ "java-like" (gcc) | 7 | 5 | 0.37 | x2.2 | **0.5** | 0.043 + libstdc++ | GCC 8.1.0 | +| C++ "naive unsafe raw pointers" (clang) | 6 | 8 | 0.20 | x1.21 | **0.4** | 0.011 + libstdc++ | Clang 6.0.0 | +| **C++ "naive unsafe raw pointers" (gcc)** | 6 | 8 | **0.19** | x1.15 | **0.4** | 0.023 + libstdc++ | GCC 8.1.0 | +| C++ "naive `shared_ptr`" (clang) | 6 | 6 | 0.36 | x2.2 | **0.5** | 0.015 + libstdc++ | Clang 6.0.0 | +| C++ "naive `shared_ptr`" (gcc) | 6 | 6 | 0.35 | x2.1 | **0.5** | 0.047 + libstdc++ | GCC 8.1.0 | +| C# | 9 | 1 | 0.70\* | x4.2 | 11 | N/A | .NET Core 2.0 | +| D "naive unsafe raw pointers" | 8 | 6 | 0.24 | x1.45 | 1.6 | 0.019 + D runtime | LDC 1.9.0 | +| Go "naive unsafe raw pointers" | 9 | 6 | 0.38 | x2.3 | 5.8 | 1.2 (static) | Go 1.10.2 | +| Haskell | ? | ? | 0.87 | x5.3 | 3.4 | 3.8 | GHC 8.2.2 | +| JavaScript | 10\*\* | 3\*\* | 1.12 | x6.8 | 52 | N/A | Node.js 10.1.0 | +| Java (no-limit / `-Xm*50M`) | 9 | 1 | 0.50 / 0.50 | x3 | 142 / 29 | N/A | OpenJDK 1.8.0 | +| Kotlin/JVM (no-limit / `-Xm*50M`) | 9 | 1 | 0.53 / 0.51 | x3.2 | 144 / 30 | N/A | Kotlinc 1.2.40 + OpenJDK 1.8.0 | +| Kotlin/Native | 9 | 1 | 5.88 | x36 | 1.2 | 0.239 | Kotlinc-native 0.7 | +| Nim | 9 | 1 | 1.00 | x6 | **0.5** | 0.051 | Nim 0.18 / GCC 8.1.0 | +| OCaml | ? | ? | 0.69 | x4.2 | 3.8 | N/A | OCaml 4.06 | +| Object Pascal "naive unsafe raw pointers" | 6 | 8 | 0.37 | x2.2 | 0.38 | **0.028 (static)** | FPC 3.0.4 | +| PHP | 9 | 3\*\* | 4.44 | x27 | 5.8 | N/A | PHP 7.2.5 | +| Python (CPython) | 10\*\* | 3\*\* | 12.25 | x74 | 5 | N/A | CPython 3.6 | +| Python (PyPy) | 10\*\* | 3\*\* | 3.20 | x19 | 48.5 | N/A | PyPy 6.0.0 | +| Rust "idiomatic" | 8 | 2 | 0.37\*\*\* | x2.2 | **0.5** | 0.427 | Rustc 1.26 | +| Rust "ref-counted" | 6 | 5 | 0.37 | x2.2 | **0.5** | 0.431 | Rustc 1.26 | +| Swift | 9 | 1 | 1.66 | x10 | 2.5 | 0.016 + Swift shared libraries | Swift 4.1 | + +(\*) C# has a noticeable VM start time (~0.4 seconds), but we still measure real execution time of the whole program. -#### Mac OS (Mac OS 10.13, Intel Core i7-4770HQ CPU) +(\*\*) Having no static types leaves the code clean, but makes it less reliable +from the maintenance perspective. + +(\*\*\*) With [a minor +update](https://barrielle.cedeela.fr/research_page/dropping-drops.html) +([PR #52](https://github.com/frol/completely-unscientific-benchmarks/pull/52)), +Rust solution gets a significant speedup while still keeping its safety +guarantees (see the result in the "Tuned Implementations Scoreboard" below). + +#### Mac OS (Mac OS 10.13, Intel Core i7-4770HQ CPU) (outdated) | Language | Real Time, seconds | Slowdown Time | Binary Size, MB | Compiler version | | --------------------------------- | ------------------ | ------------- | -------------------------------- | --------------------------------------------- | -| *Best tuned solution* | 0.25 | x1 | | | -| C++ `shared_ptr` ("ref-counted") | 0.72 | x2.9 | 0.019 + libstdc++ | Apple LLVM version 9.1.0 (clang-902.0.39.1) | +| *Best tuned solution* | **0.25** | x1 | | | +| **C++ "naive unsafe raw pointers" (clang)** | **0.25** | x1 | 0.009 + libstdc++ | Apple LLVM version 9.1.0 (clang-902.0.39.1) | +| C++ "naive `shared_ptr`" | 0.72 | x2.9 | 0.019 + libstdc++ | Apple LLVM version 9.1.0 (clang-902.0.39.1) | | C# | 0.79\* | x3.2 | 0.006 + .Net | .NET Core 2.1.200 | -| **Go** | **0.39** | x1.6 | 2.1 (static) | Go 1.10.2 | +| D "naive unsafe raw pointers" | 0.26 | x1.04 | 0.019 + D runtime | LDC 1.9.0 | +| Go "naive unsafe raw pointers" | 0.39 | x1.6 | 2.1 (static) | Go 1.10.2 | | Haskell | 1.15 | x4.6 | 1.3 | GHC 8.2.2 | | JavaScript | 1.47 | x5.9 | N/A | Node.js 6.11.1 | | Java (no-limit / `-Xm*50M`) | 0.69 / 0.59 | x2.8 / x2.4 | N/A | Oracle JDK 1.8.0 | | Kotlin/JVM (no-limit / `-Xm*50M`) | 0.69 / 0.62 | x2.8 / x2.5 | N/A | Kotlinc 1.2.41 + Oracle JDK 1.8.0 | | Kotlin/Native | 8.2 | x32.8 | 0.543 | Kotlinc-native 0.6.2 | | Nim | 1.0 | x4 | 0.293 | Nim 0.18 | +| Object Pascal | 0.36 | x1.4 | 0.272 | FPC 3.0.4 | | Python (CPython) | 15.9 | x63.6 | N/A | CPython 2.7.10 | | Python (PyPy) | 3.7 | x14.8 | N/A | PyPy 6.0.0 | -| **Rust "idiomatic"** | **0.41** | x1.6 | 0.415 | Rustc 1.26.0 | -| **Rust "ref-counted"** | **0.4** | x1.6 | 0.415 | Rustc 1.26.0 | +| Rust "idiomatic" | 0.41 | x1.6 | 0.415 | Rustc 1.26.0 | +| Rust "ref-counted" | 0.40 | x1.6 | 0.415 | Rustc 1.26.0 | | Swift | 1.72 | x6.9 | 0.019 + Swift shared libraries | Apple Swift version 4.1 | -#### Windows (Windows 10, x64, Intel Core i7-6700HQ CPU) +#### Windows (Windows 10, x64, Intel Core i7-6700HQ CPU) (outdated) | Language | Real Time, seconds | Slowdown Time | Binary Size, MB | Compiler version | | --------------------------------- | ------------------ | ------------- | -------------------------------- | --------------------------------------------- | -| *Best tuned solution* | 0.28 | x1 | | | -| C++ `shared_ptr` (msvc 2017) | 0.92 | x3.3 | 0.021 + libstdc++ | MSVC 2017 (19.13.26129) | -| C++ `shared_ptr` (clang) | 0.84 | x3 | 0.258 + libstdc++ | Clang 6.0.0 | -| C++ `shared_ptr` (mingw) | 0.65 | x2.3 | 0.031 + libstdc++ | GCC 6.3.0 | +| *Best tuned solution* | **0.28** | x1 | | | +| C++ "naive unsafe raw pointers" (msvc 2017) | 0.29 | x1.04 | 0.015 + libstdc++ | MSVC 2017 (19.13.26129) | +| C++ "naive unsafe raw pointers" (clang) | 0.29 | x1.04 | 0.254 + libstdc++ | Clang 6.0.0 | +| **C++ "naive unsafe raw pointers" (mingw)** | **0.28** | x1 | 0.039 + libstdc++ | GCC 6.3.0 | +| C++ "naive `shared_ptr`" (msvc 2017) | 0.92 | x3.3 | 0.021 + libstdc++ | MSVC 2017 (19.13.26129) | +| C++ "naive `shared_ptr`" (clang) | 0.84 | x3 | 0.258 + libstdc++ | Clang 6.0.0 | +| C++ "naive `shared_ptr`" (mingw) | 0.65 | x2.3 | 0.031 + libstdc++ | GCC 6.3.0 | | C# | 0.56\* | x2 | 0.006 + .Net | Visual Studio 2017 (Visual C# Compiler 2.7.0) | -| Go "pointers" | 0.43 | x1.5 | 2.0 (static) | Go 1.10.2 | +| D "naive unsafe raw pointers" | 0.31 | x1.1 | 0.681 + D runtime | LDC 1.9.0 | +| Go "naive unsafe raw pointers" | 0.43 | x1.5 | 2.0 (static) | Go 1.10.2 | | Haskell | 1.2 | x4.3 | 4.1 | GHC 8.2.2 | | JavaScript | 1.25 | x4.2 | N/A | Node.js 8.11.1 | | Java (no-limit / `-Xm*50M`) | 0.8 / 0.75 | x2.7 / x2.5 | N/A | Oracle JDK 10.0.1 | | Kotlin/JVM (no-limit / `-Xm*50M`) | 0.8 / 0.8 | x2.7 / x2.7 | N/A | Kotlinc 1.2.41 + Oracle JDK 10.0.1 | | Kotlin/Native | 7.8 | x26 | 0.46 | Kotlinc-native 0.7 | | Nim | 1.1 | x3.9 | 0.134 | Nim 0.18 | +| Object Pascal | 0.44 | x1.6 | 0.045 | FPC 3.0.4 | | Python (CPython) | 15.4 | x51.3 | N/A | CPython 2.7.13 | | Python (PyPy) | 3.4 | x11.3 | N/A | PyPy 6.0.0 | -| **Rust "idiomatic"** | **0.42** | x1.5 | 0.16 | Rustc 1.26.0 | +| Rust "idiomatic" | 0.42 | x1.5 | 0.16 | Rustc 1.26.0 | | Rust "ref-counted" | 0.46 | x1.6 | 0.16 | Rustc 1.26.0 | | Swift (Swift for Windows) | 2.1 | x7.5 | 0.019 + Swift shared libraries | Swift 4.0.3 (Swift for Windows 1.9.1) | @@ -122,52 +177,48 @@ execution time of the whole program. #### Linux (Arch Linux, x64, Intel Core i7-4710HQ CPU) -| Language | Real Time, seconds | Slowdown Time | Memory, MB | Binary Size, MB | Compiler Version | -| --------------------------------- | ------------------ | ------------- | ---------- | ------------------------------- | ------------------------------- | -| Ada | 0.241 | x1.35 | 0.38 | 0.278 | GCC Ada 8.1.0 | -| C++ "raw pointers" (clang / gcc) | 0.212 | x1.19 | 0.38 | 0.011 + libstdc++ | Clang 6.0.0 / GCC 8.1.0 | -| C++ "raw pointers" (static) | 0.208 | x1.16 | **0.25** | 1.7 (static) | Clang 6.0.0 / GCC 8.1.0 | -| C++ `unique_ptr` (clang / gcc) | 0.258 | x1.45 | 0.38 | 0.011 + libstdc++ | Clang 6.0.0 / GCC 8.1.0 | -| D | 0.242 | x1.36 | 1.6 | 0.019 + D runtime | LDC 1.9.0 | -| D "no D runtime" | 0.193 | x1.08 | 0.38 | 0.011 | LDC 1.9.0 | -| D "no D runtime" `-static` | 0.193 | x1.08 | **0.25** | 0.643 (static) | LDC 1.9.0 | -| Go "with-sync-pool" | 0.368 | x2.1 | 1.0 | 1.2 (static) | Go 1.10.2 | -| Haskell `+RTS -H128m` | 0.835 | x4.7 | 134 | 3.8 | GHC 8.2.2 | -| Nim `--gc:markAndSweep` | 0.655 | x3.7 | 5 | 0.055 | Nim 0.18 / GCC 8.1.0 | -| Nim "fast" | 0.359 | x2 | 0.5 | 0.047 | Nim 0.18 / GCC 8.1.0 | -| Nim "fast" `--gc:markAndSweep` | 0.186 | x1.04 | 5.1 | 0.043 | Nim 0.18 / GCC 8.1.0 | -| Nim "manual memory management" | 0.179 | x1 | 0.38 | 0.039 | Nim 0.18 / GCC 8.1.0 | -| **Nim "manual" (static)** | **0.178** | **x1** | 0.38 | 0.8 (static) | Nim 0.18 / GCC 8.1.0 | -| Object Pascal "raw pointers" | 0.369 | x2.1 | 0.38 | **0.028 (static)** | FPC 3.0.4 | -| Object Pascal "no-heap-cheating" | 0.327 | x1.8 | 8 | 0.027 (static) | FPC 3.0.4 | -| Rust "unsafe pointers" | 0.217 | x1.22 | 0.5 | 0.427 | Rustc 1.26.0 | -| Rust "safe mem::forget" | 0.239 | x1.34 | 0.5 | 0.427 | Rustc 1.26.0 | - - -#### Mac OS (Mac OS 10.13, Intel Core i7-4770HQ CPU) +| Language | Real Time, seconds | Slowdown Time | Memory, MB | Binary Size, MB | Compiler Version | +| ------------------------------------------------- | ------------------ | ------------- | ---------- | ------------------------------- | ------------------------------- | +| C++ "tuned raw pointers" (clang) | 0.182 | x1.10 | 0.5 | 0.011 + libstdc++ | Clang 6.0.0 | +| C++ "tuned raw pointers" (gcc) | 0.175 | x1.06 | 0.38 | 0.019 + libstdc++ | GCC 8.1.0 | +| C++ "tuned raw pointers" (gcc & static) | 0.172 | x1.04 | **0.25** | 1.7 (static) | GCC 8.1.0 | +| C++ "raw pointers with pool" (clang) | 0.174 | x1.05 | 0.38 | 0.011 + libstdc++ | Clang 6.0.0 | +| **C++ "raw pointers with pool" (gcc)** | **0.167** | x1.01 | 0.38 | 0.015 + libstdc++ | GCC 8.1.0 | +| **C++ "raw pointers with pool" (gcc & static)** | **0.165** | **x1** | **0.25** | 1.7 (static) | GCC 8.1.0 | +| C++ `unique_ptr` (clang) | 0.248 | x1.5 | 0.38 | 0.011 + libstdc++ | Clang 6.0.0 | +| C++ `unique_ptr` (gcc) | 0.248 | x1.5 | 0.38 | 0.043 + libstdc++ | GCC 8.1.0 | +| D "no D runtime" | 0.193 | x1.17 | 0.38 | 0.011 | LDC 1.9.0 | +| D "no D runtime" `-static` | 0.193 | x1.17 | **0.25** | 0.643 (static) | LDC 1.9.0 | +| Go "with-sync-pool" | 0.368 | x2.2 | 1.0 | 1.2 (static) | Go 1.10.2 | +| Haskell `+RTS -H128m` | 0.835 | x5.1 | 134 | 3.8 | GHC 8.2.2 | +| Nim `--gc:markAndSweep` | 0.655 | x4 | 5 | 0.055 | Nim 0.18 / GCC 8.1.0 | +| Nim "fast" | 0.359 | x2.2 | 0.5 | 0.047 | Nim 0.18 / GCC 8.1.0 | +| Nim "fast" `--gc:markAndSweep` | 0.186 | x1.13 | 5.1 | 0.043 | Nim 0.18 / GCC 8.1.0 | +| Nim "manual memory management" | 0.179 | x1.08 | 0.38 | 0.039 | Nim 0.18 / GCC 8.1.0 | +| Nim "manual" (static) | 0.178 | x1.08 | 0.38 | 0.8 (static) | Nim 0.18 / GCC 8.1.0 | +| Object Pascal "no-heap cheating" | 0.327 | x2 | 8 | 0.027 (static) | FPC 3.0.4 | +| Rust "unsafe pointers" | 0.217 | x1.32 | 0.5 | 0.427 | Rustc 1.26.0 | +| Rust "safe mem::forget" | 0.239 | x1.45 | 0.5 | 0.427 | Rustc 1.26.0 | + + +#### Mac OS (Mac OS 10.13, Intel Core i7-4770HQ CPU) (outdated) | Language | Real Time, seconds | Slowdown Time | Binary Size, MB | Compiler version | | --------------------------------- | ------------------ | ------------- | -------------------------------- | --------------------------------------------- | -| **C++ "raw pointers" (clang)** | **0.25** | x1 | 0.009 + libstdc++ | Apple LLVM version 9.1.0 (clang-902.0.39.1) | +| **C++ "naive unsafe raw pointers" (clang)** | **0.25** | x1 | 0.009 + libstdc++ | Apple LLVM version 9.1.0 (clang-902.0.39.1) | | C++ `unique_ptr` (clang) | 0.3 | x1.2 | 0.009 + libstdc++ | Apple LLVM version 9.1.0 (clang-902.0.39.1) | -| D | 0.26 | x1.04 | 0.019 + D runtime | LDC 1.9.0 | | Nim `--gc:markAndSweep` | 0.7 | x2.8 | 0.293 | Nim 0.18 | -| Object Pascal | 0.36 | x1.4 | 0.272 | FPC 3.0.4 | -#### Windows (Windows 10, x64, Intel Core i7-6700HQ CPU) +#### Windows (Windows 10, x64, Intel Core i7-6700HQ CPU) (outdated) | Language | Real Time, seconds | Slowdown Time | Binary Size, MB | Compiler version | | --------------------------------- | ------------------ | ------------- | -------------------------------- | --------------------------------------------- | -| C++ "raw pointers" (msvc 2017) | 0.29 | x1.04 | 0.015 + libstdc++ | MSVC 2017 (19.13.26129) | +| **C++ "naive unsafe raw pointers" (mingw)** | **0.28** | x1 | 0.039 + libstdc++ | GCC 6.3.0 | | C++ `unique_ptr` (msvc 2017) | 0.4 | x1.4 | 0.015 + libstdc++ | MSVC 2017 (19.13.26129) | -| C++ "raw pointers" (clang) | 0.29 | x1.04 | 0.254 + libstdc++ | Clang 6.0.0 | | C++ `unique_ptr` (clang) | 0.36 | x1.3 | 0.254 + libstdc++ | Clang 6.0.0 | -| **C++ "raw pointers" (mingw)** | **0.28** | x1 | 0.039 + libstdc++ | GCC 6.3.0 | | C++ `unique_ptr` (mingw) | 0.34 | x1.2 | 0.039 + libstdc++ | GCC 6.3.0 | -| D | 0.31 | x1.1 | 0.681 + D runtime | LDC 1.9.0 | | Nim `--gc:markAndSweep` | 0.83 | x3 | 0.143 | Nim 0.18 | -| Object Pascal | 0.44 | x1.6 | 0.045 | FPC 3.0.4 | ## Observations diff --git a/ada/main_fast.adb b/ada/main_naive_pointers.adb similarity index 81% rename from ada/main_fast.adb rename to ada/main_naive_pointers.adb index a9b2806..b2524a7 100644 --- a/ada/main_fast.adb +++ b/ada/main_naive_pointers.adb @@ -1,10 +1,10 @@ with Ada.Integer_Text_IO; with Ada.Text_IO; -with Tree_Fast; use Tree_Fast; +with Tree_Naive_Pointers; use Tree_Naive_Pointers; -procedure Main_Fast is - t: Tree_Fast.Tree; +procedure Main_Naive_Pointers is + t: Tree_Naive_Pointers.Tree; cur: Integer := 5; res: Integer := 0; mode: Integer; @@ -22,4 +22,4 @@ begin end if; end loop; PutI(res); New_Line(1); -end Main_Fast; +end Main_Naive_Pointers; diff --git a/ada/tree_fast.adb b/ada/tree_naive_pointers.adb similarity index 97% rename from ada/tree_fast.adb rename to ada/tree_naive_pointers.adb index a4223b2..81bfc2d 100644 --- a/ada/tree_fast.adb +++ b/ada/tree_naive_pointers.adb @@ -1,6 +1,6 @@ with Ada.Unchecked_Deallocation; -package body Tree_Fast is +package body Tree_Naive_Pointers is procedure initialize is begin @@ -96,4 +96,4 @@ begin delete_node(equal); end erase; -end Tree_Fast; +end Tree_Naive_Pointers; diff --git a/ada/tree_fast.ads b/ada/tree_naive_pointers.ads similarity index 94% rename from ada/tree_fast.ads rename to ada/tree_naive_pointers.ads index 8cb7126..0a72d14 100644 --- a/ada/tree_fast.ads +++ b/ada/tree_naive_pointers.ads @@ -1,6 +1,6 @@ with Ada.Numerics.Discrete_Random; -package Tree_Fast is +package Tree_Naive_Pointers is type Node is private; @@ -43,4 +43,4 @@ type Node is record y: Integer := Random(g); end record; -end Tree_Fast; +end Tree_Naive_Pointers; diff --git a/c++/Makefile b/c++/Makefile new file mode 100644 index 0000000..b25ae46 --- /dev/null +++ b/c++/Makefile @@ -0,0 +1,26 @@ +TARGETS = \ + main-naive-java-like \ + main-naive-raw \ + main-naive-shared_ptr \ + main-tuned-raw \ + main-tuned-raw-with-pool \ + main-tuned-unique_ptr + +.PHONY: all +all: $(TARGETS:%=%-clang) $(TARGETS:%=%-clang-static) $(TARGETS:%=%-gcc) $(TARGETS:%=%-gcc-static) + +$(TARGETS:%=%-clang): %-clang : %.cpp + clang++ -O3 --std=c++17 -flto -s -o $@ $< + +$(TARGETS:%=%-clang-static): %-clang-static : %.cpp + /usr/bin/clang++ -O3 --std=c++17 -flto -s -static -o $@ $< + +$(TARGETS:%=%-gcc): %-gcc : %.cpp + g++ -O3 --std=c++17 -flto -s -o $@ $< + +$(TARGETS:%=%-gcc-static): %-gcc-static : %.cpp + g++ -O3 --std=c++17 -flto -s -static -o $@ $< + +.PHONY: clean +clean: + rm $(TARGETS:%=%-clang) $(TARGETS:%=%-clang-static) $(TARGETS:%=%-gcc) $(TARGETS:%=%-gcc-static) diff --git a/c++/README.md b/c++/README.md index b9c0dce..ba5f1f2 100644 --- a/c++/README.md +++ b/c++/README.md @@ -1,6 +1,6 @@ # C++ -Author: Stas Minakov (@supermina999) +Authors: Stas Minakov (@supermina999), Paul Harvey (@prharvey) ## Compile diff --git a/c++/main-naive-java-like.cpp b/c++/main-naive-java-like.cpp new file mode 100644 index 0000000..8e7e70f --- /dev/null +++ b/c++/main-naive-java-like.cpp @@ -0,0 +1,99 @@ +#include +#include + +struct Node { + explicit Node(int value) : value(value) {} + + int value; + int priority = rand(); + + std::shared_ptr left; + std::shared_ptr right; +}; + +inline std::tuple, std::shared_ptr> SplitBinary( + std::shared_ptr&& input, int value) { + if (!input) { + return std::make_tuple(nullptr, nullptr); + } else if (input->value < value) { + auto [less, greater] = SplitBinary(std::move(input->right), value); + input->right = std::move(less); + return std::make_tuple(std::move(input), std::move(greater)); + } else { + auto [less, greater] = SplitBinary(std::move(input->left), value); + input->left = std::move(greater); + return std::make_tuple(std::move(less), std::move(input)); + } +} + +inline std::tuple, std::shared_ptr, + std::shared_ptr> +Split(std::shared_ptr&& input, int value) { + auto [less, greater_or_equal] = SplitBinary(std::move(input), value); + auto [equal, greater] = SplitBinary(std::move(greater_or_equal), value + 1); + return std::make_tuple(std::move(less), std::move(equal), std::move(greater)); +} + +inline std::shared_ptr&& Merge(std::shared_ptr&& less, + std::shared_ptr&& greater) { + if (!less | !greater) { + return std::move(less ? less : greater); + } else if (less->priority < greater->priority) { + less->right = Merge(std::move(less->right), std::move(greater)); + return std::move(less); + } else { + greater->left = Merge(std::move(less), std::move(greater->left)); + return std::move(greater); + } +} + +inline std::shared_ptr&& Merge(std::shared_ptr&& less, + std::shared_ptr&& equal, + std::shared_ptr&& greater) { + return Merge(Merge(std::move(less), std::move(equal)), std::move(greater)); +} + +class Treap { + public: + bool HasValue(int value) { + auto [less, equal, greater] = Split(std::move(root_), value); + const bool has_value = equal != nullptr; + root_ = Merge(std::move(less), std::move(equal), std::move(greater)); + return has_value; + } + + void Insert(int value) { + auto [less, equal, greater] = Split(std::move(root_), value); + if (!equal) equal = std::make_shared(value); + root_ = Merge(std::move(less), std::move(equal), std::move(greater)); + } + + void Erase(int value) { + auto [less, equal, greater] = Split(std::move(root_), value); + root_ = Merge(std::move(less), std::move(greater)); + } + + private: + std::shared_ptr root_; +}; + +int main() { + srand(time(0)); + + Treap treap; + int current = 5; + int result = 0; + for (int i = 1; i < 1000000; ++i) { + const int mode = i % 3; + current = (current * 57 + 43) % 10007; + if (mode == 0) { + treap.Insert(current); + } else if (mode == 1) { + treap.Erase(current); + } else if (mode == 2) { + result += treap.HasValue(current); + } + } + std::cout << result; + return 0; +} diff --git a/c++/main-raw.cpp b/c++/main-naive-raw.cpp similarity index 100% rename from c++/main-raw.cpp rename to c++/main-naive-raw.cpp diff --git a/c++/main-shared_ptr.cpp b/c++/main-naive-shared_ptr.cpp similarity index 100% rename from c++/main-shared_ptr.cpp rename to c++/main-naive-shared_ptr.cpp diff --git a/c++/main-tuned-raw-with-pool.cpp b/c++/main-tuned-raw-with-pool.cpp new file mode 100644 index 0000000..00f2317 --- /dev/null +++ b/c++/main-tuned-raw-with-pool.cpp @@ -0,0 +1,167 @@ +#include +#include + +#include +#include + +class Tree +{ +public: + Tree() = default; + ~Tree() {} + + bool hasValue(int x); + void insert(int x); + void erase(int x); + +private: + struct Node + { + Node(int x): x(x) {} + Node() {} + + int x = 0; + int y = rand(); + + Node* left = nullptr; + Node* right = nullptr; + }; + + using NodePtr = Node*; + + static void merge(NodePtr lower, NodePtr greater, NodePtr& merged); + static void merge(NodePtr lower, NodePtr equal, NodePtr greater, NodePtr& merged); + static void split(NodePtr orig, NodePtr& lower, NodePtr& greaterOrEqual, int val); + static void split(NodePtr orig, NodePtr& lower, NodePtr& equal, NodePtr& greater, int val); + + void clear(NodePtr node); + + NodePtr mRoot = nullptr; + boost::object_pool mNodePool = boost::object_pool(); +}; + +void Tree::clear(NodePtr node) +{ + if(!node) + return; + + clear(node->left); + clear(node->right); + mNodePool.destroy(node); +} + +bool Tree::hasValue(int x) +{ + NodePtr lower, equal, greater; + split(mRoot, lower, equal, greater, x); + bool res = equal != nullptr; + merge(lower, equal, greater, mRoot); + return res; +} + +void Tree::insert(int x) +{ + NodePtr lower, equal, greater; + split(mRoot, lower, equal, greater, x); + if(!equal) + equal = mNodePool.construct(x); + + merge(lower, equal, greater, mRoot); +} + +void Tree::erase(int x) +{ + NodePtr lower, equal, greater; + split(mRoot, lower, equal, greater, x); + merge(lower, greater, mRoot); + clear(equal); +} + +void Tree::merge(NodePtr lower, NodePtr greater, NodePtr& merged) +{ + if(!lower) + { + merged = greater; + return; + } + + if(!greater) + { + merged = lower; + return; + } + + if(lower->y < greater->y) + { + merged = lower; + merge(merged->right, greater, merged->right); + } + else + { + merged = greater; + merge(lower, merged->left, merged->left); + } +} + +void Tree::merge(NodePtr lower, NodePtr equal, NodePtr greater, NodePtr& merged) +{ + merge(lower, equal, merged); + merge(merged, greater, merged); +} + +void Tree::split(NodePtr orig, NodePtr& lower, NodePtr& greaterOrEqual, int val) +{ + if(!orig) + { + lower = greaterOrEqual = nullptr; + return; + } + + if(orig->x < val) + { + lower = orig; + split(lower->right, lower->right, greaterOrEqual, val); + } + else + { + greaterOrEqual = orig; + split(greaterOrEqual->left, lower, greaterOrEqual->left, val); + } +} + +void Tree::split(NodePtr orig, NodePtr& lower, NodePtr& equal, NodePtr& greater, int val) +{ + NodePtr equalOrGreater; + split(orig, lower, equalOrGreater, val); + split(equalOrGreater, equal, greater, val + 1); +} + +int main() +{ + srand(time(0)); + + Tree tree; + + int cur = 5; + int res = 0; + + for(int i = 1; i < 1000000; i++) + { + int mode = i % 3; + cur = (cur * 57 + 43) % 10007; + if(mode == 0) + { + tree.insert(cur); + } + else if(mode == 1) + { + tree.erase(cur); + } + else if(mode == 2) + { + res += tree.hasValue(cur); + } + } + std::cout << res; + return 0; +} diff --git a/c++/main-tuned-raw.cpp b/c++/main-tuned-raw.cpp new file mode 100644 index 0000000..f5d2cd2 --- /dev/null +++ b/c++/main-tuned-raw.cpp @@ -0,0 +1,160 @@ +#include +#include + +class Tree +{ +public: + Tree() = default; + ~Tree() + { + delete mRoot; + } + + bool hasValue(int x); + void insert(int x); + void erase(int x); + +private: + struct Node + { + Node(int x): x(x) {} + Node() {} + ~Node() + { + delete left; + delete right; + } + + int x = 0; + int y = rand(); + + Node* left = nullptr; + Node* right = nullptr; + }; + + using NodePtr = Node*; + + static void merge(NodePtr lower, NodePtr greater, NodePtr& merged); + static void merge(NodePtr lower, NodePtr equal, NodePtr greater, NodePtr& merged); + static void split(NodePtr orig, NodePtr& lower, NodePtr& greaterOrEqual, int val); + static void split(NodePtr orig, NodePtr& lower, NodePtr& equal, NodePtr& greater, int val); + static void clear(NodePtr node); + + NodePtr mRoot = nullptr; +}; + +bool Tree::hasValue(int x) +{ + NodePtr lower, equal, greater; + split(mRoot, lower, equal, greater, x); + bool res = equal != nullptr; + merge(lower, equal, greater, mRoot); + return res; +} + +void Tree::insert(int x) +{ + NodePtr lower, equal, greater; + split(mRoot, lower, equal, greater, x); + if(!equal) + equal = new Node(x); + + merge(lower, equal, greater, mRoot); +} + +void Tree::erase(int x) +{ + NodePtr lower, equal, greater; + split(mRoot, lower, equal, greater, x); + merge(lower, greater, mRoot); + delete equal; +} + +void Tree::merge(NodePtr lower, NodePtr greater, NodePtr& merged) +{ + if(!lower) + { + merged = greater; + return; + } + + if(!greater) + { + merged = lower; + return; + } + + if(lower->y < greater->y) + { + merged = lower; + merge(merged->right, greater, merged->right); + } + else + { + merged = greater; + merge(lower, merged->left, merged->left); + } +} + +void Tree::merge(NodePtr lower, NodePtr equal, NodePtr greater, NodePtr& merged) +{ + merge(lower, equal, merged); + merge(merged, greater, merged); +} + +void Tree::split(NodePtr orig, NodePtr& lower, NodePtr& greaterOrEqual, int val) +{ + if(!orig) + { + lower = greaterOrEqual = nullptr; + return; + } + + if(orig->x < val) + { + lower = orig; + split(lower->right, lower->right, greaterOrEqual, val); + } + else + { + greaterOrEqual = orig; + split(greaterOrEqual->left, lower, greaterOrEqual->left, val); + } +} + +void Tree::split(NodePtr orig, NodePtr& lower, NodePtr& equal, NodePtr& greater, int val) +{ + NodePtr equalOrGreater; + split(orig, lower, equalOrGreater, val); + split(equalOrGreater, equal, greater, val + 1); +} + +int main() +{ + srand(time(0)); + + Tree tree; + + int cur = 5; + int res = 0; + + for(int i = 1; i < 1000000; i++) + { + int mode = i % 3; + cur = (cur * 57 + 43) % 10007; + if(mode == 0) + { + tree.insert(cur); + } + else if(mode == 1) + { + tree.erase(cur); + } + else if(mode == 2) + { + res += tree.hasValue(cur); + } + } + std::cout << res; + return 0; +} diff --git a/c++/main-unique_ptr.cpp b/c++/main-tuned-unique_ptr.cpp similarity index 100% rename from c++/main-unique_ptr.cpp rename to c++/main-tuned-unique_ptr.cpp diff --git a/d/README.md b/d/README.md index c8a464e..6bd3649 100644 --- a/d/README.md +++ b/d/README.md @@ -7,13 +7,13 @@ Translated from C++ version by: Stas Minakov (@supermina999) ## Compile ``` -ldc2 main.d -O3 -release -Xcc -flto -of=main-ldc -ldc2 -betterC main_nort.d -O3 -release -Xcc -flto -of=main-nort-ldc -defaultlib= +ldc2 main_naive_pointers.d -O3 -release -Xcc -flto -of=main_naive_pointers-ldc +ldc2 -betterC main_tuned_no_rt.d -O3 -release -Xcc -flto -of=main_tuned_no_rt-ldc -defaultlib= ``` ``` -gdc main.d -O3 -frelease -flto -o main-gdc -gdc -betterC main_nort.d -O3 -frelease -flto -o main-nort-gdc +gdc main_naive_pointers.d -O3 -frelease -flto -o main_naive_pointers-gdc +gdc -betterC main_tuned_no_rt.d -O3 -frelease -flto -o main_tuned_no_rt-gdc ``` NOTE: To compile statically, add `-static` flag. @@ -21,8 +21,8 @@ NOTE: To compile statically, add `-static` flag. ## Execute ``` -./main-ldc -./main-gdc -./main-nort-ldc -./main-nort-gdc +./main_naive_pointers-ldc +./main_naive_pointers-gdc +./main_tuned_no_rt-ldc +./main_tuned_no_rt-gdc ``` diff --git a/d/main_pointers.d b/d/main_naive_pointers.d similarity index 100% rename from d/main_pointers.d rename to d/main_naive_pointers.d diff --git a/d/main_nort.d b/d/main_tuned_no_rt.d similarity index 100% rename from d/main_nort.d rename to d/main_tuned_no_rt.d diff --git a/pascal/main_noheap_cheating.pas b/pascal/main_cheating_noheap.pas similarity index 100% rename from pascal/main_noheap_cheating.pas rename to pascal/main_cheating_noheap.pas diff --git a/pascal/main_raw_pointers.pas b/pascal/main_naive_pointers.pas similarity index 100% rename from pascal/main_raw_pointers.pas rename to pascal/main_naive_pointers.pas