This repository has been archived by the owner on Feb 20, 2023. It is now read-only.
Build improvements #1026
Merged
Build improvements #1026
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Searches for Brew's clang++ on mac OS, which is required for builds.
Supports dynamic linking on non-mac OS platforms, but force-disables it on mac OS.
Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu>
Implements the changes that Matt requested.
…into TerrierShared
Address sanitizer checks for one-definition-rule violations. This ruins tests. This is technically a bandaid solution, but since the violations don't seem to matter, it's probably the best solution.
Avoids overwriting Mac OS ASAN options.
An attempt (here goes nothing) at bringing in spdlog as a git submodule and statically linking it.
…into TerrierShared
Sets the location of Clang on MacOS by reusing existing variables.
…)" This reverts commit 686eb69.
Migrate to remote fetching of spdlog.
Entirely manage spdlog through CMake.
…rt." This reverts commit d162eae.
This reverts commit ca80090.
Attempting to make this visible to Jenkins.
Fixes Release builds for macOS by slightly changing header order. May cause obscure AppleClang segfaults. In fact, this will almost certainly cause a segfault on the Jenkins machine during the build, but here's why I'm submitting this anyway: I've been attempting to debug AppleClang, but after a few hours of debugging, I'm at my wits end. There's a bug somewhere in this call to getRegClassWeight, which causes getRegClassWeight to dereference a null pointer: https://github.com/llvm/llvm-project-staging/blob/f0fd4f3af0acda5068acc5bbe10b3792c41bdfc4/llvm/lib/CodeGen/MachineLICM.cpp#L912 The really weird thing is that for regular Clang, getRegisterClassWeight should be essentially a no-op here--it's a const getter. There's really not much that should be going on unless a null registerclass is being passed, but even then, it shouldn't be crashing *here*. A null registerclass should cause catastrophic failure way earlier. So I'm thinking Apple's fork of Clang does something special here that Clang doesn't do, and that the Jenkins version of AppleClang might not incorporate this new feature, since it runs an older version of the macOS toolchain.
A long-standing pet peeve of mine is that FindClangTools.cmake doesn't work (at least not on Arch) when clang-{format,tidy} are installed through the system package manager. This fixes that.
mbutrovich
added
infrastructure
mbutrovich
approved these changes
Jul 27, 2020
|
If there's a breakage, how hard would it be to rollback? |
mbutrovich
added
the
blocked
|
We want to merge #994 first. |
mbutrovich
added
in-progress
# Conflicts: # util/CMakeLists.txt
mbutrovich
added
ready-to-merge
mbutrovich
added a commit
to mbutrovich/noisepage
that referenced
this pull request
Jul 29, 2020
This reverts commit d31d08e
mbutrovich
added a commit
that referenced
this pull request
Jul 29, 2020
skyfishjy
added a commit
to cmu-mse-cmudb/terrier
that referenced
this pull request
Jul 29, 2020
* Add support for compiling shared library * mac OS build fix Searches for Brew's clang++ on mac OS, which is required for builds. * Build with dynamic-linking on non-mac OS platforms Supports dynamic linking on non-mac OS platforms, but force-disables it on mac OS. * Make sure that OLTP-Bench results directory names are unique (cmu-db#1015) * Move Limit/Offset Fields to AbstractScanPlanNode (cmu-db#938) Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> * Merge Matt's requested changes Implements the changes that Matt requested. * Disable ODR violation detection for asan tests Address sanitizer checks for one-definition-rule violations. This ruins tests. This is technically a bandaid solution, but since the violations don't seem to matter, it's probably the best solution. * Fix ASAN options Avoids overwriting Mac OS ASAN options. * Statically link spdlog An attempt (here goes nothing) at bringing in spdlog as a git submodule and statically linking it. * Cleaner Clang initialization on MacOS Sets the location of Clang on MacOS by reusing existing variables. * Pull submodules from within Jenkinsfile * Just a test commit to try to coerce a proper build. Will revert. Ignore me. * Revert "Explicit instantiation of spdlog template functions (cmu-db#995)" This reverts commit 686eb69. * Update spdlog entry in versions.txt Migrate to remote fetching of spdlog. * Delete spdlog folder Entirely manage spdlog through CMake. * Revert "Just a test commit to try to coerce a proper build. Will revert." This reverts commit d162eae. * Revert "Pull submodules from within Jenkinsfile" This reverts commit ca80090. * Revert changes to .gitmodules * Revert src/CMakeLists.txt * Fetch and (properly) build spdlog * Literally just adding a file named "a" Attempting to make this visible to Jenkins. * Obscure bugfix for Release builds Fixes Release builds for macOS by slightly changing header order. May cause obscure AppleClang segfaults. In fact, this will almost certainly cause a segfault on the Jenkins machine during the build, but here's why I'm submitting this anyway: I've been attempting to debug AppleClang, but after a few hours of debugging, I'm at my wits end. There's a bug somewhere in this call to getRegClassWeight, which causes getRegClassWeight to dereference a null pointer: https://github.com/llvm/llvm-project-staging/blob/f0fd4f3af0acda5068acc5bbe10b3792c41bdfc4/llvm/lib/CodeGen/MachineLICM.cpp#L912 The really weird thing is that for regular Clang, getRegisterClassWeight should be essentially a no-op here--it's a const getter. There's really not much that should be going on unless a null registerclass is being passed, but even then, it shouldn't be crashing *here*. A null registerclass should cause catastrophic failure way earlier. So I'm thinking Apple's fork of Clang does something special here that Clang doesn't do, and that the Jenkins version of AppleClang might not incorporate this new feature, since it runs an older version of the macOS toolchain. * Fix FindClangTools.cmake A long-standing pet peeve of mine is that FindClangTools.cmake doesn't work (at least not on Arch) when clang-{format,tidy} are installed through the system package manager. This fixes that. * Cleaner ThirdPartyToolchain.cmake Removes some unneeded code that used to (but no longer) exists in the codebase. * Suppress clang-format warning We are actually required to keep the includes in this order (at least, spdlog.h MUST be at the top). clang-format is really bothered by this. * Remove stray file. Override compiler on macOS in Jenkinsfile. * Different way to set compiler on macOS. * Revert "Different way to set compiler on macOS." This reverts commit 8a06349 * Fix typo. Co-authored-by: Kevin Geng <khg@andrew.cmu.edu> Co-authored-by: Jordi Gonzalez <jordig@imac1.db.pdl.local.cmu.edu> Co-authored-by: Yao Yu <skyfish.jy@gmail.com> Co-authored-by: Joseph Koshakow <jkosh44@users.noreply.github.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> Co-authored-by: Matt Butrovich <mbutrovich@gmail.com>
skyfishjy
pushed a commit
to cmu-mse-cmudb/terrier
that referenced
this pull request
Jul 29, 2020
This reverts commit d31d08e
17zhangw
added a commit
to 17zhangw/terrier
that referenced
this pull request
Aug 4, 2020
commit 43fb57c Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Fri Jul 31 13:08:47 2020 -0400 Fix pipeline metrics, clean up some stray includes (cmu-db#1044) commit 643b480 Author: Yao Yu <skyfish.jy@gmail.com> Date: Wed Jul 29 21:07:21 2020 -0400 Refactor Python Test Harness to Improve Extensibility (cmu-db#1024) Co-authored-by: Daniel Biales <bialesdaniel@gmail.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> commit d9d1c42 Author: Wan Shen Lim <wanshen.lim@gmail.com> Date: Wed Jul 29 17:32:27 2020 -0400 Fix update counts, run TPCC in end-to-end performance on Jenkins. (cmu-db#1039) * Add execCtxAddRowsAffected builtin. * Fix TrafficCopTest. * Change Jenkinsfile End-to-End Performance to (1) timeout for tatp, (2) run tpcc. Co-authored-by: Wan Shen Lim <wanshenl@cs.cmu.edu> commit de55bfe Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Wed Jul 29 11:35:32 2020 -0400 Revert "Build improvements (cmu-db#1026)" (cmu-db#1036) This reverts commit d31d08e commit d31d08e Author: J. Gonzalez <gonzalezjo@users.noreply.github.com> Date: Wed Jul 29 11:06:37 2020 -0400 Build improvements (cmu-db#1026) * Add support for compiling shared library * mac OS build fix Searches for Brew's clang++ on mac OS, which is required for builds. * Build with dynamic-linking on non-mac OS platforms Supports dynamic linking on non-mac OS platforms, but force-disables it on mac OS. * Make sure that OLTP-Bench results directory names are unique (cmu-db#1015) * Move Limit/Offset Fields to AbstractScanPlanNode (cmu-db#938) Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> * Merge Matt's requested changes Implements the changes that Matt requested. * Disable ODR violation detection for asan tests Address sanitizer checks for one-definition-rule violations. This ruins tests. This is technically a bandaid solution, but since the violations don't seem to matter, it's probably the best solution. * Fix ASAN options Avoids overwriting Mac OS ASAN options. * Statically link spdlog An attempt (here goes nothing) at bringing in spdlog as a git submodule and statically linking it. * Cleaner Clang initialization on MacOS Sets the location of Clang on MacOS by reusing existing variables. * Pull submodules from within Jenkinsfile * Just a test commit to try to coerce a proper build. Will revert. Ignore me. * Revert "Explicit instantiation of spdlog template functions (cmu-db#995)" This reverts commit 686eb69. * Update spdlog entry in versions.txt Migrate to remote fetching of spdlog. * Delete spdlog folder Entirely manage spdlog through CMake. * Revert "Just a test commit to try to coerce a proper build. Will revert." This reverts commit d162eae. * Revert "Pull submodules from within Jenkinsfile" This reverts commit ca80090. * Revert changes to .gitmodules * Revert src/CMakeLists.txt * Fetch and (properly) build spdlog * Literally just adding a file named "a" Attempting to make this visible to Jenkins. * Obscure bugfix for Release builds Fixes Release builds for macOS by slightly changing header order. May cause obscure AppleClang segfaults. In fact, this will almost certainly cause a segfault on the Jenkins machine during the build, but here's why I'm submitting this anyway: I've been attempting to debug AppleClang, but after a few hours of debugging, I'm at my wits end. There's a bug somewhere in this call to getRegClassWeight, which causes getRegClassWeight to dereference a null pointer: https://github.com/llvm/llvm-project-staging/blob/f0fd4f3af0acda5068acc5bbe10b3792c41bdfc4/llvm/lib/CodeGen/MachineLICM.cpp#L912 The really weird thing is that for regular Clang, getRegisterClassWeight should be essentially a no-op here--it's a const getter. There's really not much that should be going on unless a null registerclass is being passed, but even then, it shouldn't be crashing *here*. A null registerclass should cause catastrophic failure way earlier. So I'm thinking Apple's fork of Clang does something special here that Clang doesn't do, and that the Jenkins version of AppleClang might not incorporate this new feature, since it runs an older version of the macOS toolchain. * Fix FindClangTools.cmake A long-standing pet peeve of mine is that FindClangTools.cmake doesn't work (at least not on Arch) when clang-{format,tidy} are installed through the system package manager. This fixes that. * Cleaner ThirdPartyToolchain.cmake Removes some unneeded code that used to (but no longer) exists in the codebase. * Suppress clang-format warning We are actually required to keep the includes in this order (at least, spdlog.h MUST be at the top). clang-format is really bothered by this. * Remove stray file. Override compiler on macOS in Jenkinsfile. * Different way to set compiler on macOS. * Revert "Different way to set compiler on macOS." This reverts commit 8a06349 * Fix typo. Co-authored-by: Kevin Geng <khg@andrew.cmu.edu> Co-authored-by: Jordi Gonzalez <jordig@imac1.db.pdl.local.cmu.edu> Co-authored-by: Yao Yu <skyfish.jy@gmail.com> Co-authored-by: Joseph Koshakow <jkosh44@users.noreply.github.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> Co-authored-by: Matt Butrovich <mbutrovich@gmail.com> commit ed6131b Author: Wan Shen Lim <wanshen.lim@gmail.com> Date: Tue Jul 28 16:28:13 2020 -0400 [TPL2] Update of execution engine (cmu-db#994) Co-authored-by: Tanuj Nayak <tanujnay112@live.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Matt Butrovich <mbutrovich@gmail.com> Co-authored-by: Wan Shen Lim <wanshenl@cs.cmu.edu> commit 0043d01 Author: abalakumar083 <30840848+abalakumar083@users.noreply.github.com> Date: Sat Jul 25 13:07:10 2020 -0500 Fix self joins which use index joins (cmu-db#1022) commit 32cff0d Author: Andy Pavlo <pavlo@cs.brown.edu> Date: Sat Jul 25 07:40:04 2020 -0400 Re-enable Microbenchmark Regression Checks (cmu-db#1034) commit 89292eb Author: coco <coconason@gmail.com> Date: Thu Jul 23 12:41:36 2020 +0800 Add Support for ORDER BY column# (cmu-db#964) Co-authored-by: wangguangyuan <wangguangyuan@sensetime.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> commit e8d309e Author: Jeff Niu <dniu@andrew.cmu.edu> Date: Wed Jul 22 21:09:14 2020 -0400 Add SQL Trace Files to Repository (cmu-db#1025) Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> commit 5beb32f Author: benliangw <benliangw@gmail.com> Date: Fri Jul 17 15:01:27 2020 -0400 Add Debug Log Timestamps to Python Test Harness (cmu-db#1010) Co-authored-by: Benliang Wang <benlianw@gmail.com> commit bc255fd Author: Andy Pavlo <pavlo@cs.brown.edu> Date: Thu Jul 16 19:56:10 2020 -0400 JUnit Trace File Testing Fixes/Cleanup (cmu-db#1021) * Simplify Jenkinsfile and make sure we only build the server binary for end-to-end tests * Make sure that ConnectionHandle does not contain any Postgres-specific includes * Minor tweaks to Jenkinsfile to reduce timeout for junit and explicitly set protocol mode commit 23e1157 Author: Jeff Niu <dniu@andrew.cmu.edu> Date: Thu Jul 16 08:33:01 2020 -0400 Dynamically Generate Junit Test Cases from SQLite Trace Files (cmu-db#993) Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> commit 6c7b8a6 Author: FFFelix <yuzeliao@gmail.com> Date: Mon Jul 13 20:43:26 2020 -0700 Fixed Arrow export test bugs & added more debug info (cmu-db#1019) Co-authored-by: Yuze Liao <yuzliao@yuzliao-mn1.linkedin.biz> commit 89ef5dc Author: bialesdaniel <bialesdaniel@gmail.com> Date: Fri Jul 10 15:32:26 2020 -0400 Use Extended Query Protocol for End-to-End Testing + Nightly Benchmarks (cmu-db#1005) commit 1ce0da3 Author: Joseph Koshakow <jkosh44@users.noreply.github.com> Date: Fri Jul 10 11:20:44 2020 -0400 Move Limit/Offset Fields to AbstractScanPlanNode (cmu-db#938) Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu>
Closed
stephaniewang100
added a commit
to stephaniewang100/terrier-steph
that referenced
this pull request
Aug 14, 2020
* Add Debug Log Timestamps to Python Test Harness (cmu-db#1010) Co-authored-by: Benliang Wang <benlianw@gmail.com> * Add SQL Trace Files to Repository (cmu-db#1025) Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> * Add Support for ORDER BY column# (cmu-db#964) Co-authored-by: wangguangyuan <wangguangyuan@sensetime.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> * Re-enable Microbenchmark Regression Checks (cmu-db#1034) * Fix self joins which use index joins (cmu-db#1022) * [TPL2] Update of execution engine (cmu-db#994) Co-authored-by: Tanuj Nayak <tanujnay112@live.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Matt Butrovich <mbutrovich@gmail.com> Co-authored-by: Wan Shen Lim <wanshenl@cs.cmu.edu> * Build improvements (cmu-db#1026) * Add support for compiling shared library * mac OS build fix Searches for Brew's clang++ on mac OS, which is required for builds. * Build with dynamic-linking on non-mac OS platforms Supports dynamic linking on non-mac OS platforms, but force-disables it on mac OS. * Make sure that OLTP-Bench results directory names are unique (cmu-db#1015) * Move Limit/Offset Fields to AbstractScanPlanNode (cmu-db#938) Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> * Merge Matt's requested changes Implements the changes that Matt requested. * Disable ODR violation detection for asan tests Address sanitizer checks for one-definition-rule violations. This ruins tests. This is technically a bandaid solution, but since the violations don't seem to matter, it's probably the best solution. * Fix ASAN options Avoids overwriting Mac OS ASAN options. * Statically link spdlog An attempt (here goes nothing) at bringing in spdlog as a git submodule and statically linking it. * Cleaner Clang initialization on MacOS Sets the location of Clang on MacOS by reusing existing variables. * Pull submodules from within Jenkinsfile * Just a test commit to try to coerce a proper build. Will revert. Ignore me. * Revert "Explicit instantiation of spdlog template functions (cmu-db#995)" This reverts commit 686eb69. * Update spdlog entry in versions.txt Migrate to remote fetching of spdlog. * Delete spdlog folder Entirely manage spdlog through CMake. * Revert "Just a test commit to try to coerce a proper build. Will revert." This reverts commit d162eae. * Revert "Pull submodules from within Jenkinsfile" This reverts commit ca80090. * Revert changes to .gitmodules * Revert src/CMakeLists.txt * Fetch and (properly) build spdlog * Literally just adding a file named "a" Attempting to make this visible to Jenkins. * Obscure bugfix for Release builds Fixes Release builds for macOS by slightly changing header order. May cause obscure AppleClang segfaults. In fact, this will almost certainly cause a segfault on the Jenkins machine during the build, but here's why I'm submitting this anyway: I've been attempting to debug AppleClang, but after a few hours of debugging, I'm at my wits end. There's a bug somewhere in this call to getRegClassWeight, which causes getRegClassWeight to dereference a null pointer: https://github.com/llvm/llvm-project-staging/blob/f0fd4f3af0acda5068acc5bbe10b3792c41bdfc4/llvm/lib/CodeGen/MachineLICM.cpp#L912 The really weird thing is that for regular Clang, getRegisterClassWeight should be essentially a no-op here--it's a const getter. There's really not much that should be going on unless a null registerclass is being passed, but even then, it shouldn't be crashing *here*. A null registerclass should cause catastrophic failure way earlier. So I'm thinking Apple's fork of Clang does something special here that Clang doesn't do, and that the Jenkins version of AppleClang might not incorporate this new feature, since it runs an older version of the macOS toolchain. * Fix FindClangTools.cmake A long-standing pet peeve of mine is that FindClangTools.cmake doesn't work (at least not on Arch) when clang-{format,tidy} are installed through the system package manager. This fixes that. * Cleaner ThirdPartyToolchain.cmake Removes some unneeded code that used to (but no longer) exists in the codebase. * Suppress clang-format warning We are actually required to keep the includes in this order (at least, spdlog.h MUST be at the top). clang-format is really bothered by this. * Remove stray file. Override compiler on macOS in Jenkinsfile. * Different way to set compiler on macOS. * Revert "Different way to set compiler on macOS." This reverts commit 8a06349 * Fix typo. Co-authored-by: Kevin Geng <khg@andrew.cmu.edu> Co-authored-by: Jordi Gonzalez <jordig@imac1.db.pdl.local.cmu.edu> Co-authored-by: Yao Yu <skyfish.jy@gmail.com> Co-authored-by: Joseph Koshakow <jkosh44@users.noreply.github.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> Co-authored-by: Matt Butrovich <mbutrovich@gmail.com> * Revert "Build improvements (cmu-db#1026)" (cmu-db#1036) This reverts commit d31d08e * Fix update counts, run TPCC in end-to-end performance on Jenkins. (cmu-db#1039) * Add execCtxAddRowsAffected builtin. * Fix TrafficCopTest. * Change Jenkinsfile End-to-End Performance to (1) timeout for tatp, (2) run tpcc. Co-authored-by: Wan Shen Lim <wanshenl@cs.cmu.edu> * Refactor Python Test Harness to Improve Extensibility (cmu-db#1024) Co-authored-by: Daniel Biales <bialesdaniel@gmail.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> * Fix pipeline metrics, clean up some stray includes (cmu-db#1044) * compiled_query_execution command line arg (cmu-db#1057) * Forward declare (operator and expression) translators, fix TPL AstPrettyPrint for array length. (cmu-db#1051) * Fix parallel codegen and parallel scan, force insert/update/delete to be serial. (cmu-db#1056) * Add Support for More than Two Disjunction Predicates (cmu-db#1064) Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> * VarlenEntry microbenchmark (cmu-db#1055) * Mini-Runners Update for TPLv2 Engine (cmu-db#1060) * Hyperbolic Trig SQL Functions (cmu-db#1078) Co-authored-by: rohanaggarwal <aggarwal.rohan@yahoo.co.in> Co-authored-by: gautam20197 <gautam20197@gmail.com> Co-authored-by: Preetansh Goyal <preetjee313@gmail.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: benliangw <benliangw@gmail.com> Co-authored-by: Benliang Wang <benlianw@gmail.com> Co-authored-by: Jeff Niu <dniu@andrew.cmu.edu> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> Co-authored-by: coco <coconason@gmail.com> Co-authored-by: wangguangyuan <wangguangyuan@sensetime.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: abalakumar083 <30840848+abalakumar083@users.noreply.github.com> Co-authored-by: Wan Shen Lim <wanshen.lim@gmail.com> Co-authored-by: Tanuj Nayak <tanujnay112@live.com> Co-authored-by: Matt Butrovich <mbutrovich@gmail.com> Co-authored-by: Wan Shen Lim <wanshenl@cs.cmu.edu> Co-authored-by: J. Gonzalez <gonzalezjo@users.noreply.github.com> Co-authored-by: Kevin Geng <khg@andrew.cmu.edu> Co-authored-by: Jordi Gonzalez <jordig@imac1.db.pdl.local.cmu.edu> Co-authored-by: Yao Yu <skyfish.jy@gmail.com> Co-authored-by: Joseph Koshakow <jkosh44@users.noreply.github.com> Co-authored-by: Daniel Biales <bialesdaniel@gmail.com> Co-authored-by: wuwenw <55009204+wuwenw@users.noreply.github.com> Co-authored-by: William Zhang <wz2@andrew.cmu.edu> Co-authored-by: rohanaggarwal <aggarwal.rohan@yahoo.co.in> Co-authored-by: gautam20197 <gautam20197@gmail.com> Co-authored-by: Preetansh Goyal <preetjee313@gmail.com>
stephaniewang100
added a commit
to stephaniewang100/terrier-steph
that referenced
this pull request
Aug 14, 2020
* Add Debug Log Timestamps to Python Test Harness (cmu-db#1010) Co-authored-by: Benliang Wang <benlianw@gmail.com> * Add SQL Trace Files to Repository (cmu-db#1025) Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> * Add Support for ORDER BY column# (cmu-db#964) Co-authored-by: wangguangyuan <wangguangyuan@sensetime.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> * Re-enable Microbenchmark Regression Checks (cmu-db#1034) * Fix self joins which use index joins (cmu-db#1022) * [TPL2] Update of execution engine (cmu-db#994) Co-authored-by: Tanuj Nayak <tanujnay112@live.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Matt Butrovich <mbutrovich@gmail.com> Co-authored-by: Wan Shen Lim <wanshenl@cs.cmu.edu> * Build improvements (cmu-db#1026) * Add support for compiling shared library * mac OS build fix Searches for Brew's clang++ on mac OS, which is required for builds. * Build with dynamic-linking on non-mac OS platforms Supports dynamic linking on non-mac OS platforms, but force-disables it on mac OS. * Make sure that OLTP-Bench results directory names are unique (cmu-db#1015) * Move Limit/Offset Fields to AbstractScanPlanNode (cmu-db#938) Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> * Merge Matt's requested changes Implements the changes that Matt requested. * Disable ODR violation detection for asan tests Address sanitizer checks for one-definition-rule violations. This ruins tests. This is technically a bandaid solution, but since the violations don't seem to matter, it's probably the best solution. * Fix ASAN options Avoids overwriting Mac OS ASAN options. * Statically link spdlog An attempt (here goes nothing) at bringing in spdlog as a git submodule and statically linking it. * Cleaner Clang initialization on MacOS Sets the location of Clang on MacOS by reusing existing variables. * Pull submodules from within Jenkinsfile * Just a test commit to try to coerce a proper build. Will revert. Ignore me. * Revert "Explicit instantiation of spdlog template functions (cmu-db#995)" This reverts commit 686eb69. * Update spdlog entry in versions.txt Migrate to remote fetching of spdlog. * Delete spdlog folder Entirely manage spdlog through CMake. * Revert "Just a test commit to try to coerce a proper build. Will revert." This reverts commit d162eae. * Revert "Pull submodules from within Jenkinsfile" This reverts commit ca80090. * Revert changes to .gitmodules * Revert src/CMakeLists.txt * Fetch and (properly) build spdlog * Literally just adding a file named "a" Attempting to make this visible to Jenkins. * Obscure bugfix for Release builds Fixes Release builds for macOS by slightly changing header order. May cause obscure AppleClang segfaults. In fact, this will almost certainly cause a segfault on the Jenkins machine during the build, but here's why I'm submitting this anyway: I've been attempting to debug AppleClang, but after a few hours of debugging, I'm at my wits end. There's a bug somewhere in this call to getRegClassWeight, which causes getRegClassWeight to dereference a null pointer: https://github.com/llvm/llvm-project-staging/blob/f0fd4f3af0acda5068acc5bbe10b3792c41bdfc4/llvm/lib/CodeGen/MachineLICM.cpp#L912 The really weird thing is that for regular Clang, getRegisterClassWeight should be essentially a no-op here--it's a const getter. There's really not much that should be going on unless a null registerclass is being passed, but even then, it shouldn't be crashing *here*. A null registerclass should cause catastrophic failure way earlier. So I'm thinking Apple's fork of Clang does something special here that Clang doesn't do, and that the Jenkins version of AppleClang might not incorporate this new feature, since it runs an older version of the macOS toolchain. * Fix FindClangTools.cmake A long-standing pet peeve of mine is that FindClangTools.cmake doesn't work (at least not on Arch) when clang-{format,tidy} are installed through the system package manager. This fixes that. * Cleaner ThirdPartyToolchain.cmake Removes some unneeded code that used to (but no longer) exists in the codebase. * Suppress clang-format warning We are actually required to keep the includes in this order (at least, spdlog.h MUST be at the top). clang-format is really bothered by this. * Remove stray file. Override compiler on macOS in Jenkinsfile. * Different way to set compiler on macOS. * Revert "Different way to set compiler on macOS." This reverts commit 8a06349 * Fix typo. Co-authored-by: Kevin Geng <khg@andrew.cmu.edu> Co-authored-by: Jordi Gonzalez <jordig@imac1.db.pdl.local.cmu.edu> Co-authored-by: Yao Yu <skyfish.jy@gmail.com> Co-authored-by: Joseph Koshakow <jkosh44@users.noreply.github.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> Co-authored-by: Matt Butrovich <mbutrovich@gmail.com> * Revert "Build improvements (cmu-db#1026)" (cmu-db#1036) This reverts commit d31d08e * Fix update counts, run TPCC in end-to-end performance on Jenkins. (cmu-db#1039) * Add execCtxAddRowsAffected builtin. * Fix TrafficCopTest. * Change Jenkinsfile End-to-End Performance to (1) timeout for tatp, (2) run tpcc. Co-authored-by: Wan Shen Lim <wanshenl@cs.cmu.edu> * Refactor Python Test Harness to Improve Extensibility (cmu-db#1024) Co-authored-by: Daniel Biales <bialesdaniel@gmail.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> * Fix pipeline metrics, clean up some stray includes (cmu-db#1044) * compiled_query_execution command line arg (cmu-db#1057) * Forward declare (operator and expression) translators, fix TPL AstPrettyPrint for array length. (cmu-db#1051) * Fix parallel codegen and parallel scan, force insert/update/delete to be serial. (cmu-db#1056) * Add Support for More than Two Disjunction Predicates (cmu-db#1064) Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> * VarlenEntry microbenchmark (cmu-db#1055) * Mini-Runners Update for TPLv2 Engine (cmu-db#1060) * Hyperbolic Trig SQL Functions (cmu-db#1078) Co-authored-by: rohanaggarwal <aggarwal.rohan@yahoo.co.in> Co-authored-by: gautam20197 <gautam20197@gmail.com> Co-authored-by: Preetansh Goyal <preetjee313@gmail.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: benliangw <benliangw@gmail.com> Co-authored-by: Benliang Wang <benlianw@gmail.com> Co-authored-by: Jeff Niu <dniu@andrew.cmu.edu> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> Co-authored-by: coco <coconason@gmail.com> Co-authored-by: wangguangyuan <wangguangyuan@sensetime.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: abalakumar083 <30840848+abalakumar083@users.noreply.github.com> Co-authored-by: Wan Shen Lim <wanshen.lim@gmail.com> Co-authored-by: Tanuj Nayak <tanujnay112@live.com> Co-authored-by: Matt Butrovich <mbutrovich@gmail.com> Co-authored-by: Wan Shen Lim <wanshenl@cs.cmu.edu> Co-authored-by: J. Gonzalez <gonzalezjo@users.noreply.github.com> Co-authored-by: Kevin Geng <khg@andrew.cmu.edu> Co-authored-by: Jordi Gonzalez <jordig@imac1.db.pdl.local.cmu.edu> Co-authored-by: Yao Yu <skyfish.jy@gmail.com> Co-authored-by: Joseph Koshakow <jkosh44@users.noreply.github.com> Co-authored-by: Daniel Biales <bialesdaniel@gmail.com> Co-authored-by: wuwenw <55009204+wuwenw@users.noreply.github.com> Co-authored-by: William Zhang <wz2@andrew.cmu.edu> Co-authored-by: rohanaggarwal <aggarwal.rohan@yahoo.co.in> Co-authored-by: gautam20197 <gautam20197@gmail.com> Co-authored-by: Preetansh Goyal <preetjee313@gmail.com> Co-authored-by: benliangw <benliangw@gmail.com> Co-authored-by: Benliang Wang <benlianw@gmail.com> Co-authored-by: Jeff Niu <dniu@andrew.cmu.edu> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> Co-authored-by: coco <coconason@gmail.com> Co-authored-by: wangguangyuan <wangguangyuan@sensetime.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: abalakumar083 <30840848+abalakumar083@users.noreply.github.com> Co-authored-by: Wan Shen Lim <wanshen.lim@gmail.com> Co-authored-by: Tanuj Nayak <tanujnay112@live.com> Co-authored-by: Matt Butrovich <mbutrovich@gmail.com> Co-authored-by: Wan Shen Lim <wanshenl@cs.cmu.edu> Co-authored-by: J. Gonzalez <gonzalezjo@users.noreply.github.com> Co-authored-by: Kevin Geng <khg@andrew.cmu.edu> Co-authored-by: Jordi Gonzalez <jordig@imac1.db.pdl.local.cmu.edu> Co-authored-by: Yao Yu <skyfish.jy@gmail.com> Co-authored-by: Joseph Koshakow <jkosh44@users.noreply.github.com> Co-authored-by: Daniel Biales <bialesdaniel@gmail.com> Co-authored-by: wuwenw <55009204+wuwenw@users.noreply.github.com> Co-authored-by: William Zhang <wz2@andrew.cmu.edu> Co-authored-by: rohanaggarwal <aggarwal.rohan@yahoo.co.in> Co-authored-by: gautam20197 <gautam20197@gmail.com> Co-authored-by: Preetansh Goyal <preetjee313@gmail.com>
mbutrovich
added a commit
that referenced
this pull request
Aug 19, 2020
* Add the Python logging levels. * Add the hardware info. * Checkpoint. * Comment one more lock for experiments. * Check in the new global model. * Checkpoint. * Add the caching. * Checkpointing. * Added the extractYear builtin function. * Add more variations. * Add q7 features. * Minor fix. * Add the query info. * Add a tqdm for reading. * Bug fix. * Trainer reorganization. * Refactor the type names. * Revert the latches. * Fix format and linter. * Clang-tidy fix. * Compilation fix... * Uncomment a query. * Update TPL generation scripts with the new table for mini runners. * Add the date function test. * Checkpoint. * A few qurey info fixes. * Small fix. * More info fix.. * Forgot to comment out the latch!!!! * Changelog - Removed NestedLoopJoin runners since just seq_scan + arith - Refactored some of the code/organization - Tweaked data numbers to match scripts - mini_runners now runs and emits stage-level CSVs * Add sampling to the global impact model training. * HASHJOIN_PROBE -> HASHJOIN_ITERATE * Commit that flattened all the directories and deleted all non-mini-trainer data * Should be reverted or reconciled with malin1993's modeling-metrics branch * Mini-Trainer updates to C++ runners * Boost min % error? * Remerged terrier/master * Updated query_info... * Boosted mini_runners data * Fixed AggregateRunners * Tweaked mini_runners limit * Tweaked HashJoin OP_INTEGER_COMPARE feature * Some patches * Format and pulled new query_info * Added non-self join * mini_runners hashjoin_probe recording * Modified ExecutableQuery constructor for TPL or file * OutputTranslator records OUTPUT * Output modelling * OutputRunners pre-dev4 * Add TODOs + more recording * Tweak to OUTPUT runners and mini_trainer prediction * Interpret + Compiled * Restored * query_info OUTPUT * DECIMAL tables which generate int64_t values; modified GenSeqScan * Purged OP_INTEGER_COMPARE from HASHJOIN_PROBE and added some type info * Some recorder fixes, still need refactoring after rows/cardinality is done * OUTPUT now models DECIMAL * Augmentation to DECIMAL * Updated .gitignore * Output model includes dummy writes * Fixed cardinality (off by 1) * Refix cardinality of tables and augment idxscan/insert/update * Output fixes * Shifted trackers for IndexScan * BIGINT switch * Generate multi-type tables * Fixed and verified cardinality for mixed tables * MultiTable runners * OUTPUT patches * Bump resource usage * GenOutputArguments * Lowered join limit * DECIMAL to SeqScan + Output * Changelist (build stabilization): - Removed PROJECTION from being recorded - Added DELETE to model type/runners - Augment query_info with OUTPUT - No write OUTPUT in runners and other fixes - Drop PerformGarbageCollection() -- which causues measurements to rise * Stabilize on mixed scans, integers only * Changes/patches to UPDATE/DELETE, IDX_SCAN, run order * UPDATE/DELETE + row ordering patches * MemoryPool::Deallocate n is off * Benchmark changes * OUTPUT changes; UPDATE/DELETE fixes * IndexFitCostModel * Removed std::cout * GenericHashTable/ConciseHashTable entries tracking * Memory scaling factor * Augmentation for INSERT/UPDATE/DELETE * Update fixes * Insert fixes * Adjusted rows * Fixed global_data_constructing_util * mini_trainer tiny bias * Tweaked bias * IndexFitCostModel * IdxJoin * Index Join implementation, just need tests * Fixed tests * make format * Remove EnableMetric * Test recording * doxygen * Fixed test * fixed compiler_test * Fixed forcedcostmodel * Fixed more test * tpcc_plan_stocklevel_test * format * accidentally disabled a test * fixed sort_iterate check * make format * query_info fix * Relocated some index_util into new cpp file * Added documentation * IndexSize() through plan nodes * Get StructDecl from SortBottom, HashJoinBottom, AggregateBottom * Memory scaling factor in recorder/runners * Fixes to computing memory scale factor * Fixed varchar size recording * Memory modifications to trainers * check-lint * quality of life fixes * fixed concise/generic_hash_table_test * make format * fixed compiler_test * clang-tidy fixes * clang-tidy * clang-tidy again * restart build * IndexScan as SQL + warmup * Warmup using same query * std::move() fix * Refactored signature; warmup delete * Twekaed InvokeGC * Runners update/delete with idx_scan * Removed compiled padding * Fixed AggregatePlanNode recording * Cleanup of state, warmup sort_build/agg_build * Updated doc * removed std::cout * Test fixes * clang-tiy * restart build? * Empty aggregate recording fix * Fixed compiler_test * Restart * Mac worker died * Bumped warmup limit * SQL-invokable functions * Fixed networkqueries recording for OUTPUT * Limit output * doxygen * check-lint * check-lint * Reduced rows * tweaked args * Fixed concat * Fix varchar recording * Squashed commit of the following: commit 4cc8b12053474451f384454f51e554c5d3cfb33d Merge: 7fc20e2 40d7228 Author: William Zhang <17zhangw@gmail.com> Date: Tue Jul 7 12:29:32 2020 -0400 Merge branch 'master' of https://github.com/cmu-db/terrier into master_merge commit 40d7228 Author: abalakumar083 <30840848+abalakumar083@users.noreply.github.com> Date: Tue Jul 7 07:26:33 2020 -0500 Adding support for NOT expression (#1006) commit 58aaad7 Author: Andy Pavlo <pavlo@cs.brown.edu> Date: Fri Jul 3 15:04:10 2020 -0400 Switched Jenkins-nightly OLTP-Bench TATP run to use 8 terminals instead of 6 commit 6158e15 Author: Andy Pavlo <pavlo@cs.brown.edu> Date: Fri Jul 3 14:44:28 2020 -0400 Quick fix for broken Jenkinsfile. I edited this file directly on GitHub without testing it first. Don't do what Andy did, kids. I'm not proud of this commit 45bfdcc Author: bialesdaniel <bialesdaniel@gmail.com> Date: Fri Jul 3 14:40:17 2020 -0400 Added performance stage in Jenkins pipeline to execute TATP through OLTP-Bench (#992) This currently does not check against previous results. We will provide more automated checks and data retention in the future. Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> commit 686eb69 Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Thu Jul 2 11:58:51 2020 -0400 Explicit instantiation of spdlog template functions (#995) commit 78d2946 Author: Chad Dougherty <crd@acm.org> Date: Thu Jul 2 09:41:58 2020 -0400 ensure the match is not empty before assigning ref_cycles_us_ (#1003) commit 923f6e9 Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Wed Jul 1 20:27:28 2020 -0400 Relax asserts in MetricsStore (#1002) commit 776674f Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Wed Jul 1 12:02:24 2020 -0400 Hook up per-component metrics settings to DBMain's Builder (#996) commit e525b6d Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Tue Jun 30 15:39:27 2020 -0400 Remove namespace oids where appropriate (#989) commit b0d2879 Author: Deepayan Patra <dpatra@andrew.cmu.edu> Date: Tue Jun 30 07:29:03 2020 -0400 Add support for VERSION() builtin (#988) Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> commit 799e9dc Author: Emmanuel J. Eppinger <ej.eppinger@gmail.com> Date: Mon Jun 29 09:05:23 2020 -0400 Improve Logging Efficiency and Performance (#941) commit ad0fe11 Author: abalakumar083 <30840848+abalakumar083@users.noreply.github.com> Date: Sun Jun 28 13:57:49 2020 -0500 Remove common/stat_registry.h and common/performance_counter.h (#991) commit 3470d9e Author: abalakumar083 <30840848+abalakumar083@users.noreply.github.com> Date: Sat Jun 27 17:53:06 2020 -0500 Changes to speed up compilation (#987) commit 2adaaa8 Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Fri Jun 26 16:17:18 2020 -0400 Clean up optimizer includes a bit (#986) commit a06a45f Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Fri Jun 26 11:11:12 2020 -0400 Memory usage estimates, miscellaneous #include cleanups (#984) commit e200467 Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Fri Jun 26 08:32:22 2020 -0400 Move IndexBuilder definition to cpp (#985) commit 83a9312 Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Thu Jun 25 09:48:14 2020 -0400 Error framework, including parser failures (#983) commit f5f44a5 Author: wuwenw <55009204+wuwenw@users.noreply.github.com> Date: Tue Jun 23 13:43:27 2020 -0400 fix for AVX512 on macOS (#981) commit 04a7022 Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Tue Jun 23 11:50:38 2020 -0400 Move index wrapper definitions to cpp files (#980) commit 24041a6 Author: Yao Yu <skyfish.jy@gmail.com> Date: Mon Jun 22 23:22:23 2020 -0400 Expanded nightly benchmarks to include longer run times (#975) Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> commit 4becdd2 Author: Yao Yu <skyfish.jy@gmail.com> Date: Mon Jun 22 19:51:08 2020 -0400 Delete when branch condition for nightly builds (#979) commit bf3748d Author: Richthofen <jazzplato@gmail.com> Date: Mon Jun 22 11:47:01 2020 -0400 JUnit: add QueryMode option from command line (#921) commit 016aeeb Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Sun Jun 21 12:28:58 2020 -0400 Dedicated test for BinderUtil::CheckAndTryPromoteType (#970) commit a7101c0 Author: Deepayan Patra <dpatra@andrew.cmu.edu> Date: Sat Jun 20 22:25:11 2020 -0400 Faster Timestamps (#963) commit cb9bbc2 Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Sat Jun 20 11:24:36 2020 -0400 PostgresPacketWriter cleanup (#974) commit 919da37 Author: Yao Yu <skyfish.jy@gmail.com> Date: Fri Jun 19 10:51:20 2020 -0400 Add OLTP-Bench Performance Tests for Nightly Build (#967) Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> commit fd819dd Author: abalakumar083 <30840848+abalakumar083@users.noreply.github.com> Date: Thu Jun 18 12:59:55 2020 -0500 Changes to improve compilation time (#968) commit c50a22a Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Wed Jun 17 11:32:27 2020 -0400 Per-connection CatalogCache prototype (#952) commit a8aa5aa Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Wed Jun 17 08:03:44 2020 -0400 Introduce a sleep to DisconnectAbort to try to get the abort to fully process before firing off any more queries that are supposed to succeed. (#966) commit bc19fb5 Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Tue Jun 16 19:07:05 2020 -0400 Binder error messages returned to client (#965) commit 17bad7d Author: J. Gonzalez <gonzalezjo@users.noreply.github.com> Date: Mon Jun 15 14:04:07 2020 -0400 Manually deregister exception frames (#962) LLVM can "forget" to properly cleanup its exception frames in certain cases when dealing with ELFs. An important such case is when an exception is thrown. We throw many exceptions, so this hurts us in an especially rough way. That's because when LLVM forgets to clean up its exception frames, exceptions thrown later are undefined behavior. (UAF in libunwind) Issue #961 (#961) is such an instance of this undefined behavior causing a crash. Hopefully fixes #961 by forcing LLVM to clear EH frames when the CompiledModule destructor is called. Note that (as long as we don't need the EH frames anymore) it is safe to call deregisterEHFrames even if it has been called before. This is because there's an LLVM-internal vectors that tracks registered EH frames. If you call it more than once, it will be empty on the next-call, making it essentially a no-op. When LLVM's JITLink project is finished and we move to an LLVM version that supports it, it would be best to migrate to it from RuntimeDyld, which has serious issues with EH frames. That would, in theory, fix this issue entirely. I have no idea if this solution is thread safe or not, but LLVM doesn't lock on it and it should only be called rarely, so I think it is. __deregister_frame has...interesting documentation, to say the least: https://gcc.gnu.org/onlinedocs/gcc-4.2.4/gccint/Exception-handling-routines.html Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> commit 77d1c27 Author: Emmanuel J. Eppinger <ej.eppinger@gmail.com> Date: Mon Jun 15 12:43:42 2020 -0400 Add support for formatted strings in exceptions (#959) commit f40e1af Author: abalakumar083 <30840848+abalakumar083@users.noreply.github.com> Date: Mon Jun 15 10:52:16 2020 -0500 Compile speed up changes (#957) commit b765cf2 Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Sat Jun 13 18:28:03 2020 -0400 StorageInterface instantiates PRI once (#953) * Keep the PRI around in StorageInterface rather than doing it for every access. * Change DeleteTranslator to create empty oids array, and change semantic check for StorageInterface to allow "unknown length" (0) array. commit 27534ba Author: Wan Shen Lim <wanshen.lim@gmail.com> Date: Thu Jun 11 20:32:47 2020 -0400 Update libcount to Prashanth's version. (#954) commit 929c8a6 Author: Andy Pavlo <pavlo@cs.brown.edu> Date: Wed Jun 10 16:04:00 2020 -0400 OLTP-Bench Jenkins Stage with DEBUG/ASAN (#927) commit 131a536 Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Wed Jun 10 13:52:41 2020 -0400 Fix TrafficCopTest.java. (#949) commit c3c9b5f Author: William Zhang <17zhangw@gmail.com> Date: Thu Jun 11 01:58:13 2020 +0900 MiniRunners Augmentations (#910) * COUNT(*) aggregate runners * grouped_pipeline.csv * TPCC Hack * TPCC data * Redefined stats_map/prediction_cache to multi point support * Multiple runs with post-merge based on opunit * Don't rerun hashjoins, dedup update/delete * Parameterize rerun_iterations * updel limit * no rerun arithmetic + tpl output * Revert "Redefined stats_map/prediction_cache to multi point support" This reverts commit 49dce49. * Global warmup_period + new tpcc numbers * Predict single row correctly... * Fix anomaly_clearance; add visibility to model flag * anomaly_clearance is % trimmage now * IndexScan extended to 300000 * num_iterations recorded * IdxJoin runners; tpcc id update * Mini-trainer updates to x% trim * --expose_all flag * Fix self joins which use index joins (#1022) * IDX_JOIN feature corrections * Delete invalid constraint * Correct loop feature? * Fixed features * Better features...sim cache 0.4 * Squashed commit of the following: commit 43fb57c Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Fri Jul 31 13:08:47 2020 -0400 Fix pipeline metrics, clean up some stray includes (#1044) commit 643b480 Author: Yao Yu <skyfish.jy@gmail.com> Date: Wed Jul 29 21:07:21 2020 -0400 Refactor Python Test Harness to Improve Extensibility (#1024) Co-authored-by: Daniel Biales <bialesdaniel@gmail.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> commit d9d1c42 Author: Wan Shen Lim <wanshen.lim@gmail.com> Date: Wed Jul 29 17:32:27 2020 -0400 Fix update counts, run TPCC in end-to-end performance on Jenkins. (#1039) * Add execCtxAddRowsAffected builtin. * Fix TrafficCopTest. * Change Jenkinsfile End-to-End Performance to (1) timeout for tatp, (2) run tpcc. Co-authored-by: Wan Shen Lim <wanshenl@cs.cmu.edu> commit de55bfe Author: Matt Butrovich <mbutrovich@users.noreply.github.com> Date: Wed Jul 29 11:35:32 2020 -0400 Revert "Build improvements (#1026)" (#1036) This reverts commit d31d08e commit d31d08e Author: J. Gonzalez <gonzalezjo@users.noreply.github.com> Date: Wed Jul 29 11:06:37 2020 -0400 Build improvements (#1026) * Add support for compiling shared library * mac OS build fix Searches for Brew's clang++ on mac OS, which is required for builds. * Build with dynamic-linking on non-mac OS platforms Supports dynamic linking on non-mac OS platforms, but force-disables it on mac OS. * Make sure that OLTP-Bench results directory names are unique (#1015) * Move Limit/Offset Fields to AbstractScanPlanNode (#938) Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> * Merge Matt's requested changes Implements the changes that Matt requested. * Disable ODR violation detection for asan tests Address sanitizer checks for one-definition-rule violations. This ruins tests. This is technically a bandaid solution, but since the violations don't seem to matter, it's probably the best solution. * Fix ASAN options Avoids overwriting Mac OS ASAN options. * Statically link spdlog An attempt (here goes nothing) at bringing in spdlog as a git submodule and statically linking it. * Cleaner Clang initialization on MacOS Sets the location of Clang on MacOS by reusing existing variables. * Pull submodules from within Jenkinsfile * Just a test commit to try to coerce a proper build. Will revert. Ignore me. * Revert "Explicit instantiation of spdlog template functions (#995)" This reverts commit 686eb69. * Update spdlog entry in versions.txt Migrate to remote fetching of spdlog. * Delete spdlog folder Entirely manage spdlog through CMake. * Revert "Just a test commit to try to coerce a proper build. Will revert." This reverts commit d162eae. * Revert "Pull submodules from within Jenkinsfile" This reverts commit ca80090. * Revert changes to .gitmodules * Revert src/CMakeLists.txt * Fetch and (properly) build spdlog * Literally just adding a file named "a" Attempting to make this visible to Jenkins. * Obscure bugfix for Release builds Fixes Release builds for macOS by slightly changing header order. May cause obscure AppleClang segfaults. In fact, this will almost certainly cause a segfault on the Jenkins machine during the build, but here's why I'm submitting this anyway: I've been attempting to debug AppleClang, but after a few hours of debugging, I'm at my wits end. There's a bug somewhere in this call to getRegClassWeight, which causes getRegClassWeight to dereference a null pointer: https://github.com/llvm/llvm-project-staging/blob/f0fd4f3af0acda5068acc5bbe10b3792c41bdfc4/llvm/lib/CodeGen/MachineLICM.cpp#L912 The really weird thing is that for regular Clang, getRegisterClassWeight should be essentially a no-op here--it's a const getter. There's really not much that should be going on unless a null registerclass is being passed, but even then, it shouldn't be crashing *here*. A null registerclass should cause catastrophic failure way earlier. So I'm thinking Apple's fork of Clang does something special here that Clang doesn't do, and that the Jenkins version of AppleClang might not incorporate this new feature, since it runs an older version of the macOS toolchain. * Fix FindClangTools.cmake A long-standing pet peeve of mine is that FindClangTools.cmake doesn't work (at least not on Arch) when clang-{format,tidy} are installed through the system package manager. This fixes that. * Cleaner ThirdPartyToolchain.cmake Removes some unneeded code that used to (but no longer) exists in the codebase. * Suppress clang-format warning We are actually required to keep the includes in this order (at least, spdlog.h MUST be at the top). clang-format is really bothered by this. * Remove stray file. Override compiler on macOS in Jenkinsfile. * Different way to set compiler on macOS. * Revert "Different way to set compiler on macOS." This reverts commit 8a06349 * Fix typo. Co-authored-by: Kevin Geng <khg@andrew.cmu.edu> Co-authored-by: Jordi Gonzalez <jordig@imac1.db.pdl.local.cmu.edu> Co-authored-by: Yao Yu <skyfish.jy@gmail.com> Co-authored-by: Joseph Koshakow <jkosh44@users.noreply.github.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> Co-authored-by: Matt Butrovich <mbutrovich@gmail.com> commit ed6131b Author: Wan Shen Lim <wanshen.lim@gmail.com> Date: Tue Jul 28 16:28:13 2020 -0400 [TPL2] Update of execution engine (#994) Co-authored-by: Tanuj Nayak <tanujnay112@live.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Matt Butrovich <mbutrovich@gmail.com> Co-authored-by: Wan Shen Lim <wanshenl@cs.cmu.edu> commit 0043d01 Author: abalakumar083 <30840848+abalakumar083@users.noreply.github.com> Date: Sat Jul 25 13:07:10 2020 -0500 Fix self joins which use index joins (#1022) commit 32cff0d Author: Andy Pavlo <pavlo@cs.brown.edu> Date: Sat Jul 25 07:40:04 2020 -0400 Re-enable Microbenchmark Regression Checks (#1034) commit 89292eb Author: coco <coconason@gmail.com> Date: Thu Jul 23 12:41:36 2020 +0800 Add Support for ORDER BY column# (#964) Co-authored-by: wangguangyuan <wangguangyuan@sensetime.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> commit e8d309e Author: Jeff Niu <dniu@andrew.cmu.edu> Date: Wed Jul 22 21:09:14 2020 -0400 Add SQL Trace Files to Repository (#1025) Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> commit 5beb32f Author: benliangw <benliangw@gmail.com> Date: Fri Jul 17 15:01:27 2020 -0400 Add Debug Log Timestamps to Python Test Harness (#1010) Co-authored-by: Benliang Wang <benlianw@gmail.com> commit bc255fd Author: Andy Pavlo <pavlo@cs.brown.edu> Date: Thu Jul 16 19:56:10 2020 -0400 JUnit Trace File Testing Fixes/Cleanup (#1021) * Simplify Jenkinsfile and make sure we only build the server binary for end-to-end tests * Make sure that ConnectionHandle does not contain any Postgres-specific includes * Minor tweaks to Jenkinsfile to reduce timeout for junit and explicitly set protocol mode commit 23e1157 Author: Jeff Niu <dniu@andrew.cmu.edu> Date: Thu Jul 16 08:33:01 2020 -0400 Dynamically Generate Junit Test Cases from SQLite Trace Files (#993) Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> commit 6c7b8a6 Author: FFFelix <yuzeliao@gmail.com> Date: Mon Jul 13 20:43:26 2020 -0700 Fixed Arrow export test bugs & added more debug info (#1019) Co-authored-by: Yuze Liao <yuzliao@yuzliao-mn1.linkedin.biz> commit 89ef5dc Author: bialesdaniel <bialesdaniel@gmail.com> Date: Fri Jul 10 15:32:26 2020 -0400 Use Extended Query Protocol for End-to-End Testing + Nightly Benchmarks (#1005) commit 1ce0da3 Author: Joseph Koshakow <jkosh44@users.noreply.github.com> Date: Fri Jul 10 11:20:44 2020 -0400 Move Limit/Offset Fields to AbstractScanPlanNode (#938) Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Andy Pavlo <pavlo@cs.brown.edu> * Fix registration of pipeline drivers. * Fix parallel scan. * Refactoring and adjustment fixes for TPLv2 * friend class to ExecutionSettings * No parallel query execution * Fixed memory tracking and start/end trackers * Reverted some code * Cleaned up start-up arguments slightly * make format and make check-lint * Undo changes that should not be committed * new TPCC ids * Actually fix TPCC IDs * Address some comments, fix small bugs * Ripped out TPCC IDs * Updated script for absolute error * Mini-runners patch * Merged * Fixed mini_runners compile * Filters format * SetSize test fixes * Fixed tests * clang * Suggested fix from Wan * Index updates to Update/Delete * Fixed updel single-index possibly, need extensive test * All lookups by index due to build/destroy * Fixes to visibility for runners * Rerun OUTPUT * Hack around static_aggregates * Training model tpcc hack * q33 and q34 feature hacks * OURecorder VARCHAR fixes * Strip prepared * Doxygen * compiler_test * Comment updates Co-authored-by: Lin Ma <malin1993ml@gmail.com> Co-authored-by: Matt Butrovich <mbutrovich@users.noreply.github.com> Co-authored-by: Wan Shen Lim <wanshenl@cs.cmu.edu>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a continuation of #1017, which Jenkins refuses to pull from. This PR does a number of things.
Builds should be far faster on Linux through dynamic linking of the terrier and test binaries.
Builds should "just work" work on the latest version of mac OS
This was a pain to troubleshoot, but builds work flawlessly on a dev machine with all tests passing.
Whatever issues existed with address sanitizer not doing anything on macOS appear to be gone.
(07-29-2020) Jenkins uses clang-8 instead of AppleClang (thanks, Matt!)
(07-29-2020) Jenkins correctly refers to the new compiler as clang-8 instead of AppleClang in the macOS stage names.
(07-29-2020) Jenkins resolves LLVM_DIR using brew.
(07-29-2020) Note: LLVM_DIR still has to be set via an environment variable, but I believe that that's desired behavior. I can make it automatic, but that would involve assuming that brew is installed and/or making it only automatic.
Clang tooling (tidy and format) are found automatically on more Linux distributions
We now statically link spdlog
We now use spdlog version 1.7.0 instead of 1.0.0
We now pull in an up-to-date version of CMake on Linux machines
We now delegate dependency management for spdlog to CMake
We now compile gtest with address sanitizer when TERRIER_USE_ASAN=ON