build: add "make runregression" replacement

[derekbruening]

From derek.br...@gmail.com on March 03, 2009 18:17:08

split from base cmake feature issue #19 once we have issue #65 we should add a replacement for "make runregression"
so developers can easily run a short pre-commit test suite

Original issue: http://code.google.com/p/dynamorio/issues/detail?id=66

[derekbruening]

From derek.br...@gmail.com on March 06, 2009 12:24:59

we should also look into hooking up the results to "make review" (xref issue #64 )

[derekbruening]

From derek.br...@gmail.com on April 05, 2009 14:09:55

support for Linux is in r126 as suite/runsuite.cmake:

• created a script for use with ctest -S that does multiple builds and test
  runs for each build
• separate short (pre-commit) and long (nightly: issue tests: set up nightly regression tests #11 ) sets of builds
  and runs
• for short, added code to pull out a summary of results, including
  info on crashes and asserts
• for nightly we'll use the ctest remote reporting (still todo as issue tests: set up nightly regression tests #11 )
• fixed core bugs that had cropped in while we didn't have a test suite
  • warnings in debug-external builds
  • don't build samples for non-CI builds or docs for non-VMAP builds
    (never put in VMSAFE docs support in switch to cmake)
  • fixed build errors in non-VMAP builds
  • added LINKCOUNT top-level option
• test fixes:
  • don't set rpath for test apps
  • disabled decode{,-stress} until issue CRASH (TOT suite/tests/common/decode) linux 32-bit test_modrm16 #105 fixed
• removed unused speccpu scripts
• updated TestSuite .wiki on how to use the new script

[derekbruening]

From derek.br...@gmail.com on April 12, 2009 21:22:29

in r135 :
issue #66: Windows test suite

• tweak env vars to swap between 32-bit and 64-bit compiler
• search for tools instead of hardcoding paths
• switched to cp instead of scp due to cygwin scp not supporting colon
• only supporting cygwin for now: Unix Makefiles, make, cp

we now have a pre-commit test suite for both Windows and Linux.
still a number of missing tests and more test failures than we'd like, but the suite
infrastructure is in place. on Windows it assumes amd64 subdir names for env var
swapping and assumes cygwin but it's only for developers.

closing this issue and separating unfinished nice-to-have features as issue #121 .

Status: Verified
Labels: -Priority-High Priority-Critical