Integrate Google Test and port all tests in `testeverything` #179

utensil · 2014-09-08T05:57:05Z

This PR is the work on #10 as per discussed at #10 (comment) (@rhyolight , @scottpurdy , @chetan51), using the Google Test framework . Replaces #150 .

This PR completed the following tasks:

generate JUnit-style report for nupic.tools to consume as the original C++ unit tests generate xUnit style reports #10 requests
porting original tests in testeverything with minimal modification
silent gcov so it will not generate long outputs to console
add tests for the re-implemented Tester

Decision TODOs:

decide a goal name for unit test, for now I'm using tests_unit
decide where the tests go, for now I've moved from src/test/testeverything to src/test/unit

Conflicts: .travis.yml

…thout touching the original code base

…test

Regexes used: TESTEQUAL$([^\,]*"[^\,]*)\,([^\,$]*)\) TESTEQUAL$([^\,]*str[^\,]*)\,([^\,$]*)\) TESTEQUAL$([^\,]*)\,([^\,$]*"[^\,\)]*)\) TESTEQUAL$([^\,]*)\,([^\,$]*str[^\,\)]*)\) TESTEQUAL2$([^\,]*)\,([^\,]*"[^\,]*)\,([^\,$]*)\) TESTEQUAL2$([^\,]*)\,([^\,]*str[^\,]*)\,([^\,$]*)\) TESTEQUAL2$([^\,]*)\,([^\,]*)\,([^\,]*"[^\,]*)$ TESTEQUAL2$([^\,]*)\,([^\,]*)\,([^\,]*str[^\,]*)$

utensil · 2014-09-21T11:49:05Z

@rhyolight @oxtopus The problem is solved at numenta/nupic-legacy#1299 (comment) . The problem is actually caused by:

I've placed gtest-all.cpp at $NUPIC_CORE/src/test/unit/, along with the tests, so when compiling unit tests in nupic.core, the CMake macro generate_executable(which assumes all the source file for a binary are in the same directory) will be happy
but $NUPIC/extensions/cpp_region/unittests/ also used Tester which in turn used gtest, then its linker will require gtest-all.cpp to be part of the source, or gtest has to be a library to be linked
and yet this requirement is not satisfied

I don't understand why it's not causing the same problem for linux, but this is the reason causing mac builds to fail.

My naive solution is to copy $NUPIC_CORE/src/test/unit/gtest-all.cpp to $NUPIC/extensions/cpp_region/unittests/, this is successful but somehow ugly. I need your input to tell what is a better solution:

keep this naive strategy
put gtest-all.cpp in $NUPIC_CORE/external/common, when building nupic.core, also build a gtest.a only for nupic.core and cpp_region unit tests to link
none of the above

oxtopus · 2014-09-21T13:59:12Z

Thanks for the detailed analysis! I prefer the second option.

rhyolight · 2014-09-21T16:36:23Z

Nice job tracking this down, @utensil! Thanks a bunch. I was clueless.

Matt Taylor
OS Community Flag-Bearer
Numenta

On Sun, Sep 21, 2014 at 6:59 AM, Austin Marshall notifications@github.com
wrote:

Thanks for the detailed analysis! I prefer the second option.

—
Reply to this email directly or view it on GitHub
#179 (comment).

utensil · 2014-10-03T10:51:08Z

Will carry on with the second option.

…0-google-test-rewrite

rhyolight · 2014-10-03T15:47:04Z

@utensil Let me know when you are ready and I will test again on OS X.

utensil · 2014-10-04T09:38:27Z

@rhyolight I think it's now ready both here and at numenta/nupic-legacy#1299

utensil · 2014-10-04T09:41:21Z

There's one weird thing: coverall is not commenting coverage report...

utensil · 2014-10-04T09:44:02Z

But I can see coverall functional at my branch:

utensil · 2014-10-04T09:47:17Z

The error output of nupic is:

Successfully installed python-coveralls requests sh
Cleaning up...
Traceback (most recent call last):
  File "/home/travis/build/numenta/nupic/bin/coveralls", line 9, in <module>
    load_entry_point('python-coveralls==2.4.2', 'console_scripts', 'coveralls')()
  File "/home/travis/build/numenta/nupic/local/lib/python2.7/site-packages/coveralls/__init__.py", line 73, in wear
    from coveralls.control import coveralls
  File "/home/travis/build/numenta/nupic/local/lib/python2.7/site-packages/coveralls/control.py", line 1, in <module>
    from coverage.control import coverage
ImportError: cannot import name coverage
Requirement already satisfied (use --upgrade to upgrade): python-coveralls in ./lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): PyYAML in ./lib/python2.7/site-packages (from python-coveralls)
Requirement already satisfied (use --upgrade to upgrade): requests in ./lib/python2.7/site-packages (from python-coveralls)
Requirement already satisfied (use --upgrade to upgrade): coverage in ./lib/python2.7/site-packages (from python-coveralls)
Requirement already satisfied (use --upgrade to upgrade): six in ./lib/python2.7/site-packages (from python-coveralls)
Requirement already satisfied (use --upgrade to upgrade): sh in ./lib/python2.7/site-packages (from python-coveralls)
Cleaning up...
Traceback (most recent call last):
  File "/home/travis/build/numenta/nupic/bin/coveralls", line 9, in <module>
    load_entry_point('python-coveralls==2.4.2', 'console_scripts', 'coveralls')()
  File "/home/travis/build/numenta/nupic/local/lib/python2.7/site-packages/coveralls/__init__.py", line 73, in wear
    from coveralls.control import coveralls
  File "/home/travis/build/numenta/nupic/local/lib/python2.7/site-packages/coveralls/control.py", line 1, in <module>
    from coverage.control import coverage
ImportError: cannot import name coverage
Done. Your build exited with 0.

This seems to be a separate issue brought in by other PRs. Will investigate and open a separate issue. This PR remains ready, @rhyolight .

rhyolight · 2014-10-06T14:18:48Z

@utensil Great news! Ok @scottpurdy / @chetan51 / @oxtopus can one of you review this for the (hopefully) last time? Let's see if we can finally put this issue to bed today! 😄

oxtopus · 2014-10-06T14:55:15Z

Will do. Stand by.

Integrate Google Test and port all tests in `testeverything`

rhyolight · 2014-10-06T16:57:14Z

💥 Thanks Austin!

utensil added 30 commits September 7, 2014 03:56

Add google test 1.7.0 license

e8f8f72

Add gtest header and the one-file source

3299e82

Add google test sample test code

f078b92

Add google tests to Travis

fbff004

Conflicts: .travis.yml

Relocate gtest-all.cc

87d9935

Add README.md

1ea0f09

Add build for google tests, based on @fergalbyrne's work

a4fb831

Add google_tests target

ab09372

Add the main() for google tests

7bdd1fe

.cc are not included in build, renaming to .cpp

c292534

Temporarily copying the Tester for reimplementing with Google Test wi…

b2fe3c4

…thout touching the original code base

Temporarily disable testeverything build

73870dd

Using directory engine as the entry point to port testeverything to g…

677a8b3

…test

Create file with unit tests used by Google Test

b3dc9bb

Rename generated header files

37f4323

Re-implement Tester using gtest

aa1ac97

Use GTEST_TEST to avoid macro redefinition

e256d5e

Fix travis errors

d9ece1f

Correct the usage of #

076ea74

Correct the usage of \

ea07e58

Fix the usage of ##

f48d048

Fix includes

5a827dd

Fix the usage of argv

aaa4119

Make gtest and apr both happy

838c825

Copy test directory first

1257b74

Prefer EXPECT_DOUBLE_EQ

bb68b1f

Add macro for char * tests

7c4d75f

Add the rest of tests in testeverything to google tests

f64d4c7

Prefer ASSERT_NEAR

244ceea

rhyolight added the priority:2 label Oct 2, 2014

Merge branch 'master' of https://github.com/numenta/nupic.core into 1…

de1812d

…0-google-test-rewrite

utensil force-pushed the 10-google-test-rewrite branch from 3a98a78 to de1812d Compare October 3, 2014 10:58

Move gtest-all.cpp to external

9d37067

utensil force-pushed the 10-google-test-rewrite branch from 0c8a281 to 46996ae Compare October 3, 2014 14:12

Compile google test from source for linking with tests

b049174

utensil force-pushed the 10-google-test-rewrite branch from 46996ae to b049174 Compare October 3, 2014 14:39

oxtopus added a commit that referenced this pull request Oct 6, 2014

Merge pull request #179 from utensil/10-google-test-rewrite

adb53f7

Integrate Google Test and port all tests in `testeverything`

oxtopus merged commit adb53f7 into numenta:master Oct 6, 2014

rhyolight removed status:in progress priority:2 labels Oct 6, 2014

utensil deleted the 10-google-test-rewrite branch October 7, 2014 03:50

utensil mentioned this pull request Oct 7, 2014

C++ unit tests generate xUnit style reports #10

Closed

rhyolight mentioned this pull request Oct 14, 2014

Remove Tester class from unit test framework #208

Closed

rhyolight added type:tests and removed type:tests labels Oct 14, 2014

rhyolight modified the milestone: Testing Oct 14, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate Google Test and port all tests in `testeverything` #179

Integrate Google Test and port all tests in `testeverything` #179

utensil commented Sep 8, 2014

utensil commented Sep 21, 2014

oxtopus commented Sep 21, 2014

rhyolight commented Sep 21, 2014

utensil commented Oct 3, 2014

rhyolight commented Oct 3, 2014

utensil commented Oct 4, 2014

utensil commented Oct 4, 2014

utensil commented Oct 4, 2014

utensil commented Oct 4, 2014

rhyolight commented Oct 6, 2014

oxtopus commented Oct 6, 2014

rhyolight commented Oct 6, 2014

Integrate Google Test and port all tests in testeverything #179

Integrate Google Test and port all tests in testeverything #179

Conversation

utensil commented Sep 8, 2014

utensil commented Sep 21, 2014

oxtopus commented Sep 21, 2014

rhyolight commented Sep 21, 2014

utensil commented Oct 3, 2014

rhyolight commented Oct 3, 2014

utensil commented Oct 4, 2014

utensil commented Oct 4, 2014

utensil commented Oct 4, 2014

utensil commented Oct 4, 2014

rhyolight commented Oct 6, 2014

oxtopus commented Oct 6, 2014

rhyolight commented Oct 6, 2014

Integrate Google Test and port all tests in `testeverything` #179

Integrate Google Test and port all tests in `testeverything` #179