First-time installation issues: can't run video_to_features.py

[mfshao]

Hello,

I have installed OWAT on my computer and configured by following the instruction. I have also downloaded the example data from Google Drive. The tools/test_setup.py can run without any issues and I can also get some of the tests to run. However, when I try to run video_to_features.py, it output the following error messages:

C:\Programs\Python\Python37-32\lib\site-packages\scipy\signal_arraytools.py:45: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result.
b = a[a_slice]
C:\Programs\Python\Python37-32\lib\site-packages\numpy\core\fromnumeric.py:83: RuntimeWarning: invalid value encountered in reduce
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram.py:253: RuntimeWarning: invalid value encountered in ceil
max_boundary = np.ceil(max_data / bin_width) * bin_width
Traceback (most recent call last):
File "D:/PycharmProjects/open-worm-analysis-toolbox/examples/video_to_features.py", line 142, in
main()
File "D:/PycharmProjects/open-worm-analysis-toolbox/examples/video_to_features.py", line 98, in main
experiment_histograms = mv.HistogramManager(experiment_files)
File "D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram_manager.py", line 104, in init
new_histogram_set = self.init_histograms(worm_features)
File "D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram_manager.py", line 203, in init_histograms
return np.array([Histogram.create_histogram(f) for f in worm_features])
File "D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram_manager.py", line 203, in
return np.array([Histogram.create_histogram(f) for f in worm_features])
File "D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram.py", line 149, in create_histogram
return cls(feature)
File "D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram.py", line 113, in init
self.compute_covering_bins()
File "D:\PycharmProjects\open-worm-analysis-toolbox\open_worm_analysis_toolbox\statistics\histogram.py", line 278, in compute_covering_bins
step=bin_width)
ValueError: arange: cannot compute length

I am running Python 3.7.0 on a Windows 7 machine and using PyCharm 2018.2.1, with the following packages installed:
atlas 0.27.0
nose 1.3.7
pandas 0.23.2
statsmodels 0.9.0
h5py 2.8.0
seaborn 0.9.0

I'm not sure what is wrong here. Any help would be appreciated.

Thank you!

[JimHokanson]

@tshao87

I'll try and replicate your result. Thanks.

[JimHokanson]

@tshao87

I fixed a couple of bugs that were due to changes in numpy behavior. I still need to fix a few things with getting proper setup but hopefully things will be ready in the next couple of days.

[JimHokanson]

@tshao87

Just checking in ...

I'm ending up needing to rewrite a lot of the post-processing statistics code. I'm almost done but there is still a bit more work to do. I'm not sure why these bugs weren't found earlier in testing ... I'll let you know as soon as it is ready to go.

[mfshao]

@JimHokanson

No problem. Thanks for keeping me posted.

[JimHokanson]

@tshao87

The examples should work now, including the referenced example. I've marked a couple as not working as well at the top of the files. If it works for you please close this issue.

For reference to everyone on changes made:

1. The previous code had poor nan support. It makes me wonder if the numpy interface changed ...
2. The referenced example was working with un-extended features, which was causing some problems. I've tried to make fewer assumptions regarding what types of features are being processed statistically, as well as fewer assumptions about the consistency in features processed across each video. I've also changed the histograms into a clearly defined matrix where each row consists of the same feature. This was not well handled before.
3. Finally, I was getting an error in the false discovery adjustment code (p-values to q-values). I ended up rewriting that code based on the Matlab version and everything seems to be working now.