Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not work InvertedMultiIndexSearcherTest. #1

Open
heinrichI opened this issue Sep 28, 2020 · 3 comments
Open

Not work InvertedMultiIndexSearcherTest. #1

heinrichI opened this issue Sep 28, 2020 · 3 comments

Comments

@heinrichI
Copy link

I run all test, but InvertedMultiIndexSearcherTest not execute.
image
More precisely on the line

py_imi = pyimi.PyInvertedMultiIndexBuilder().buildInvertedMultiIndex(x_ids, x_pqcodes,
                                                                                 cluster_centers)

.pyd files are in the folder
image

@DIMAthe47
Copy link
Owner

DIMAthe47 commented Oct 9, 2020

I have just successfully run test_inverted_multi_index_searcher.

Steps to find the reason of failure:

>>venv\Scripts\activate
>>cd cbir_framework\core\search\inverted_multi_index
>>python
>>import py_inverted_multi_index

This gave me an error "ImportError: DLL load failed".
With help of DependenciesGui.exe (https://github.com/lucasg/Dependencies) (be sure to run it from your virtualenv's command prompt) find what is missing in py_inverted_multi_index library.
In my case it was python35.dll, libgfortran-3.dll, libopenblass.dll.

Steps I have made to fix it.
Create virtual env with python 3.5.
Download openblas, add it's bin directory to path.
Download and install mingw-w64 x86_64-6.1.0-win32-seh-rt_v5-rev1, add it's bin folder to path (or another mingw64 with libgfortran-3.dll).

After these steps "import py_inverted_multi_index" had success.
And test_inverted_multi_index_searcher.py run had success too.

@DIMAthe47
Copy link
Owner

DIMAthe47 commented Oct 9, 2020

My recommendations on project.
I tried to make stream-like loading, computing, transformation, saving.
I even tried to encode these steps in json-configs: https://github.com/DIMAthe47/cbir_47/tree/master/model_generators.
But it is all in raw state and in big mess.

I think most valuable part of this project are c++ modules, cython wraps, and examples of how to use them.
But if your main app is in python and you need c++ only for perfomance I would advise to use https://numba.pydata.org/ instead.

For storing data I would advise:
numpy arrays: .npy, .npz, or .hdf5.
images: file hierarchy inside .zip(zip is not for compression because most images are already compressed, but for grouping it inside one file).
I think I have good examples for storing: https://gitlab.com/Digipathology/Pathadin/-/tree/master/src/main/python/ndarray_persist.

@heinrichI
Copy link
Author

heinrichI commented Oct 10, 2020

Thanks, I already guessed that the problem is in the libraries. Added them.
image
But still the test is aborted without errors on

        searcher_ = imis.InvertedMultiIndexSearcher(x=base_vectors, x_ids=ids_ndarray,
                                                    cluster_centers=cluster_centers)

image
I think, reason in python_wrap. Create another Issue: DIMAthe47/inverted_multi_index#1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants