New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2 minor Mango test failures on Windows only #1732

Closed
wohali opened this Issue Nov 13, 2018 · 2 comments

Comments

Projects
None yet
1 participant
@wohali
Member

wohali commented Nov 13, 2018

@tonysun83 @garrensmith any ideas on this one?

I've finally got the Mango test suite running on Windows. Here is the current output:

C:\relax\couchdb\src\mango>..\..\dev\run -n 1 --admin=testuser:testpass nosetests
[ * ] Setup environment ... ok
[ * ] Ensure CouchDB is built ... ok
[ * ] Prepare configuration files ... ok
[ * ] Start node node1 ... ok
[ * ] Check node at http://127.0.0.1:15984/ ... ok
[ * ] Running cluster setup ... ok
[ * ] Exec command nosetests ... ...............SSS...................................................................SSSSSSSSSSSSSSSSSSSSSSSSSS.................SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS.SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS...................................F.FSS.......SSSSSSSS...........
======================================================================
FAIL: test_quorum_json_index (15-execution-stats-test.ExecutionStatsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\relax\couchdb\src\mango\test\15-execution-stats-test.py", line 39, in test_quorum_json_index
    self.assertGreater(resp["execution_stats"]["execution_time_ms"], 0)
AssertionError: 0.0 not greater than 0
-------------------- >> begin captured logging << --------------------
urllib3.connectionpool: DEBUG: http://127.0.0.1:15984 "POST /mango_test_9a3431add7ae4911b713cf9333377743/_find HTTP/1.1" 200 None
--------------------- >> end captured logging << ---------------------

======================================================================
FAIL: test_simple_json_index (15-execution-stats-test.ExecutionStatsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\relax\couchdb\src\mango\test\15-execution-stats-test.py", line 26, in test_simple_json_index
    self.assertGreater(resp["execution_stats"]["execution_time_ms"], 0)
AssertionError: 0.0 not greater than 0
-------------------- >> begin captured logging << --------------------
urllib3.connectionpool: DEBUG: http://127.0.0.1:15984 "POST /mango_test_9a3431add7ae4911b713cf9333377743/_find HTTP/1.1" 200 None
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 284 tests in 90.943s

FAILED (SKIP=128, failures=2)

These two errors occur every time, easily repeatable, it's not a heisenbug.

Smells like possibly a real Windows-specific bug to me, but not sure how to diagnose/help fix it. I don't just want to disable the test, and I'd like these all to pass cleanly before we release 2.3.0.

It could be as simple as this:

C:\relax\couchdb>erl
Eshell V8.3  (abort with ^G)
1> os:timestamp().
{1542,92419,103000}
2> os:timestamp().
{1542,92421,478000}

In other words, os:timestamp() on Windows appears to be accurate only to the millisecond, not the microsecond. Could these tests be running so fast that the delta is indeed 0.0?

@wohali

This comment has been minimized.

Member

wohali commented Nov 13, 2018

@willholley Garren mentioned you might have an idea here as well?

@wohali

This comment has been minimized.

Member

wohali commented Nov 13, 2018

So I think the problem is what I previously mentioned, that basically the test completes too fast (<1ms) and the result is 0.0. The only fix I see possible here is to not test for that assertion on Windows platforms.

I'll file a PR.

wohali added a commit that referenced this issue Nov 13, 2018

Mango test suite Windows fixes, closes #1732
Unfortuantely, os:timestamp() on Windows only has millisecond
accuracy. That means that the two Mango tests checking for a positive
value of execution_time fail, since these tests appear to be running
in <1ms on my test setup (a rather anemic Windows VM!)

This change disables only the check for execution_time in two tests,
and leaves the remainder of the execution_stats checks in place
on Windows.

It also introduces a convenience "make.cmd" file so you can
"make check" without typing "make -f Makefile.win check" all the time.

@wohali wohali referenced this issue Nov 13, 2018

Merged

Test suite fixes for Windows #1736

2 of 2 tasks complete

@wohali wohali closed this in #1736 Nov 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment