Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test failure on Mac: //tensorflow/python:directory_watcher_test #964

Closed
caisq opened this issue Feb 2, 2016 · 4 comments
Closed

Test failure on Mac: //tensorflow/python:directory_watcher_test #964

caisq opened this issue Feb 2, 2016 · 4 comments

Comments

@caisq
Copy link
Contributor

caisq commented Feb 2, 2016

The following Python unit test fails on Mac: //tensorflow/python:directory_watcher_test

Below is the log with detailed error info:

..F...F...
======================================================================
FAIL: testFinishesLoadingFileWhenSwitchingToNewFile (__main__.DirectoryWatcherTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jenkins/workspace/experimental-cais-tensorflow-mac-python27-copt_pip_install-test/pip_install_tests/directory_watcher_test.py", line 102, in testFinishesLoadingFileWhenSwitchingToNewFile
    self.assertWatcherYields(['b', 'c'])
  File "/Users/jenkins/workspace/experimental-cais-tensorflow-mac-python27-copt_pip_install-test/pip_install_tests/directory_watcher_test.py", line 63, in assertWatcherYields
    self.assertEqual(list(self._watcher.Load()), values)
AssertionError: Lists differ: ['c'] != ['b', 'c']

First differing element 0:
c
b

Second list contains 1 additional elements.
First extra element 1:
c

- ['c']
+ ['b', 'c']

======================================================================
FAIL: testMultipleWrites (__main__.DirectoryWatcherTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jenkins/workspace/experimental-cais-tensorflow-mac-python27-copt_pip_install-test/pip_install_tests/directory_watcher_test.py", line 82, in testMultipleWrites
    self.assertWatcherYields(['x', 'y', 'z'])
  File "/Users/jenkins/workspace/experimental-cais-tensorflow-mac-python27-copt_pip_install-test/pip_install_tests/directory_watcher_test.py", line 63, in assertWatcherYields
    self.assertEqual(list(self._watcher.Load()), values)
AssertionError: Lists differ: [] != ['x', 'y', 'z']

Second list contains 3 additional elements.
First extra element 0:
x

- []
+ ['x', 'y', 'z']

----------------------------------------------------------------------
Ran 10 tests in 0.051s

FAILED (failures=2)
@caisq
Copy link
Contributor Author

caisq commented Feb 2, 2016

Machine machine details:
OS: "Darwin",
Kernel: "15.3.0",
Architecture: "i386",
Bazel_version: "Build label: 0.1.4-homebrew",
Java_version: "1.8.0_65",
Python_version: "2.7.10",
gpp_version: "Apple LLVM version 7.0.2 (clang-700.1.81)"

@vrv
Copy link

vrv commented Feb 3, 2016

Okay, I think I have a fix for this on Mac. @danmane in case he's interested.

The problem seems to be that in the DirectoryWatcherTest, the ByteLoader implementation assumes that you can read EOF from a file, have another thread append to the file, and then continue reading from the last known seek position. Apparently this breaks under mac os x but not under linux.

I changed the ByteLoader code to keep track of the number of bytes read, and when you see EOF, to try seeking to the current number of bytes read and trying again to read from the file. Doing this allows the test to pass. I couldn't find any documentation on this behavior difference between OS X and Linux though. I'll try to send a review in tomorrow.

@caisq
Copy link
Contributor Author

caisq commented Feb 3, 2016

Thanks, @vrv. Looking forward to seeing the fix.

vrv pushed a commit that referenced this issue Feb 3, 2016
Fix directory_watcher_test on mac. Fixes #964
6cb4f30 
Keeps track of seek position and seeks to current byte offset prior to reading
from the file.  Apparently this is necessary on Mac, because once you've read EOF,
you need to seek to your current position to read new data that has since been
appended to the file.
@vrv vrv closed this as completed in f8741f1 Feb 3, 2016
@teamdandelion
Copy link
Contributor

Thanks @vrv!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@vrv @teamdandelion @caisq