Add tests to verify assembler output -- Fix DoNotOptimize.#530
Add tests to verify assembler output -- Fix DoNotOptimize.#530EricWF merged 5 commits intogoogle:masterfrom
Conversation
0456059 to
b82654b
Compare
|
✅ Build benchmark 1038 completed (commit ea9de3658c by @EricWF) |
822f76d to
b2d8318
Compare
1fd7610 to
d207257
Compare
|
✅ Build benchmark 1047 completed (commit 4b063052b6 by @EricWF) |
|
✅ Build benchmark 1055 completed (commit afed53d504 by @EricWF) |
|
❌ Build benchmark 1056 failed (commit 907711ef1d by @EricWF) |
|
✅ Build benchmark 1057 completed (commit 428d4ca88c by @EricWF) |
3d87069 to
942cfd1
Compare
|
✅ Build benchmark 1061 completed (commit 630e0e8506 by @EricWF) |
|
✅ Build benchmark 1063 completed (commit af7ce5fdec by @EricWF) |
|
✅ Build benchmark 1065 completed (commit 7738ee1ce5 by @EricWF) |
328869a to
cc1136d
Compare
|
✅ Build benchmark 1069 completed (commit fb2552a0c2 by @EricWF) |
|
Can you add a markdown documentation file about AssemblyTests or whatever name you choose, essentially with the contents of your commit message. Maybe more detail. And then cross-link to it from the README? |
|
TIL about LLVM FileCheck. Pretty cool! |
|
@dominichamon Yep. Can do.
Indeed. It's an amazing tool to have when you need to test string output. |
|
@dominichamon I added documentation as you requested. Please take a look at your leisure. |
|
✅ Build benchmark 1070 completed (commit 52fd2436f5 by @EricWF) |
docs/AssemblyTests.md
Outdated
| @@ -0,0 +1,147 @@ | |||
| # Assembly Tests | |||
|
|
|||
| The Benchmark library provides a number of functions who's primary | |||
|
|
||
| ``` | ||
|
|
||
| #### LLVM Filecheck |
There was a problem hiding this comment.
this should be ### i think, as you're only at ## above.
There was a problem hiding this comment.
If it's no matter, I actually prefer having the smaller section headers for these bits. IMHO it's looks nicer and flows better.
|
We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for the commit author(s). If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google. |
For things like `DoNotOptimize`, `ClobberMemory`, and even `KeepRunning()`, it is important exactly what assembly they generate. However, we currently have no way to test this. Instead it must be manually validated every time a change occurs -- including a change in compiler version. This patch attempts to introduce a way to test the assembled output automatically. It's mirrors how LLVM verifies compiler output, and it uses LLVM FileCheck to run the tests in a similar way. The tests function by generating the assembly for a test in CMake, and then using FileCheck to verify the // CHECK lines in the source file are found in the generated assembly. Currently, the tests only run on 64-bit x86 systems under GCC and Clang, and when FileCheck is found on the system. Additionally, this patch tries to improve the code gen from DoNotOptimize. This should probably be a separate change, but I needed something to test.
dd39f2b to
ca7f89c
Compare
|
CLAs look good, thanks! |
|
@dominichamon Any final words? |
|
✅ Build benchmark 1130 completed (commit 408113083e by @EricWF) |
|
✅ Build benchmark 1132 completed (commit 57aa030a13 by @EricWF) |
|
I'd prefer if the bazel TODO linked to a github issue so we can track it, but otherwise, Ship It. |
269be46 to
8e43961
Compare
8e43961 to
0f2f95f
Compare
|
✅ Build benchmark 1136 completed (commit ab6fe60e62 by @EricWF) |
* Add tests to verify assembler output -- Fix DoNotOptimize. For things like `DoNotOptimize`, `ClobberMemory`, and even `KeepRunning()`, it is important exactly what assembly they generate. However, we currently have no way to test this. Instead it must be manually validated every time a change occurs -- including a change in compiler version. This patch attempts to introduce a way to test the assembled output automatically. It's mirrors how LLVM verifies compiler output, and it uses LLVM FileCheck to run the tests in a similar way. The tests function by generating the assembly for a test in CMake, and then using FileCheck to verify the // CHECK lines in the source file are found in the generated assembly. Currently, the tests only run on 64-bit x86 systems under GCC and Clang, and when FileCheck is found on the system. Additionally, this patch tries to improve the code gen from DoNotOptimize. This should probably be a separate change, but I needed something to test. * Disable assembly tests on Bazel for now * Link FIXME to github issue * Fix Tests on OS X * fix strip_asm.py to work on both Linux and OS X like targets
For things like
DoNotOptimize,ClobberMemory, and evenKeepRunning(),it is important exactly what assembly they generate. However, we currently
have no way to test this. Instead it must be manually validated every
time a change occurs -- including a change in compiler version.
This patch attempts to introduce a way to test the assembled output automatically.
It's mirrors how LLVM verifies compiler output, and it uses LLVM FileCheck to run
the tests in a similar way.
The tests function by generating the assembly for a test in CMake, and then
using FileCheck to verify the // CHECK lines in the source file are found
in the generated assembly.
Currently, the tests only run on 64-bit x86 systems under GCC and Clang,
and when FileCheck is found on the system.
Additionally, this patch tries to improve the code gen from DoNotOptimize. This should probably be a separate change, but I needed something to test.
This PR is largely just to run the new patch against the bots to ensure nothing has broken.