Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows test errors #22

Closed
jlmelville opened this issue Apr 14, 2019 · 0 comments
Closed

Windows test errors #22

jlmelville opened this issue Apr 14, 2019 · 0 comments

Comments

@jlmelville
Copy link
Collaborator

I am seeing some unit test errors on Windows, which are replicated on AppVeyor. The Linux travis-ci builds are passing, though.

Type Errors

In test_threaded.py, the following three tests fail: test_init_current_graph, test_new_build_candidates, test_nn_descent. They all have the same error which originates with a call to:

 for _ in executor.map(shuffle, range(n_tasks)):

The error is:

TypeError: No matching definition for argument type(s) array(float64, 3d, C), array(int32, 1d, C), array(int32, 2d, C), int64, int64, int64

There are various type signatures on jitted functions in threaded.py, e.g. this one for shuffle_jit (which I think is the relevant function for shuffle):

@numba.njit("void(f8[:, :, :], i8[:], i8[:, :], i8, i8, i8)", nogil=True)

Based on my inexpert reading of the error and the type signature, it looks like there's a mismatch of int32 and int64 types being passed to shuffle_jit. There's several places in the threaded routines where numpy arrays are declared with dtype=int:

        offsets = np.zeros((n_tasks, max_count), dtype=int)

If those are all changed to:

        offsets = np.zeros((n_tasks, max_count), dtype=np.int64)

the tests pass.

I am not sure what these type signatures exactly do in numba: type-checking? Performance? Both? Something else? Anyway, should the numba type signatures change or should the type of the integers going into the arrays change?

Recursion Error

There is also a recursion problem in the test_sparse_angular_nn_descent_neighbor_accuracy of test_pynndescent_.py. When run via nosetest, I get the error:

Fatal Python error: Cannot recover from stack overflow.

This is probably related to lmcinnes/umap#99 and comparing floating points to zero in angular distance. However, as well as fixing the margin == 0 code, it's also necessary to check if hyperplane_norm is zero. This is done in the non-sparse angular code, as well checking left_norm and right_norm. If setting these values to 1.0 is also ok in the sparse case, I will go about fixing this as part of a wider check for lurking float-to-0 comparisons and submit a PR.

jlmelville added a commit to jlmelville/pynndescent that referenced this issue Apr 16, 2019
jlmelville added a commit to jlmelville/pynndescent that referenced this issue Apr 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant