question - installation process

[marcoaaz]

Dear Norbert,

I hope you are well. I found your paper very interesting and wanted to train the model locally.

I tried to install MinDet in a new environment (first I tried Conda then Venv in Visual Studio Code). I got GCC compiler installed.

In the last installation line I got the following error:

pip install git+https://git@github.com/norberttoth398/MinDet

Command line:

PS C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main> pip install git+https://git@github.com/norberttoth398/MinDet
Collecting git+https://@github.com/norberttoth398/MinDet
Cloning https://@github.com/norberttoth398/MinDet to c:\users\n10832084\appdata\local\temp\pip-req-build-o4xjse8u
Running command git clone --filter=blob:none --quiet 'https://@github.com/norberttoth398/MinDet' 'C:\Users\n10832084\AppData\Local\Temp\pip-req-build-o4xjse8u'
Resolved https://@github.com/norberttoth398/MinDet to commit fcb409c
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [19 lines of output]
Traceback (most recent call last):
File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv3\lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 353, in
main()
File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv3\lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv3\lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "C:\Users\n10832084\AppData\Local\Temp\pip-build-env-etik9xn_\overlay\Lib\site-packages\setuptools\build_meta.py", line 332, in get_requires_for_build_wheel
return self.get_build_requires(config_settings, requirements=[])
File "C:\Users\n10832084\AppData\Local\Temp\pip-build-env-etik9xn\overlay\Lib\site-packages\setuptools\build_meta.py", line 302, in get_build_requires
self.run_setup()
File "C:\Users\n10832084\AppData\Local\Temp\pip-build-env-etik9xn\overlay\Lib\site-packages\setuptools\build_meta.py", line 502, in run_setup
super().run_setup(setup_script=setup_script)
File "C:\Users\n10832084\AppData\Local\Temp\pip-build-env-etik9xn_\overlay\Lib\site-packages\setuptools\build_meta.py", line 318, in run_setup
exec(code, locals())
File "", line 12, in
File "c:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv2\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 2731: character maps to
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

I hope you can help me with this issue. The installation was trialled on Windows 10 Enterprise.

Thanks for your patience.

Cheers,
Marco Acevedo

Bonus: I am attaching a MatLab script to read Norbert's plagioclase annotations and use them elsewhere.
matlab_interpreter_saveAs_m.txt

[norberttoth398]

Hi Marco

First of all, thank you for reading the paper and I hope you have huge success in applying the method to your work! I am very sorry for the error! It did not come to my attention as it was not an issue in the automatic build github does for testing!

It's all fixed - it was an issue with encoding in the README that windows didn't like for some reason. Please do get in touch with any questions.

I've just finished installing it on my windows laptop so let me know if you have any success so I can close the thread!

Best
Norbert

[marcoaaz]

Hi Norbert,

I was able to build the wheel. I had to use python 3.8 because with 3.7 the installation halts when trying to run:

pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.0/index.html

To ensure it will install successfully the following line had to be modified to:

python.exe -m pip install git+https://git@github.com/norberttoth398/MinDet

Although, it installed correctly (minor issues below):

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
rich 13.4.2 requires pygments<3.0.0,>=2.13.0, but you have pygments 2.12.0 which is incompatible.
openxlab 0.1.1 requires packaging~=24.0, but you have packaging 21.3 which is incompatible.
openxlab 0.1.1 requires pytz~=2023.3, but you have pytz 2022.1 which is incompatible.
openxlab 0.1.1 requires requests~=2.28.2, but you have requests 2.27.1 which is incompatible.

, when running the example notebook I got the following when trying to import MinDet:

Following a google search, I have tried installing several versions of mmcv-full but the same error persist, so I cannot run the follow up notebook cells. We are almost there :).

Thanks for your support.
Cordially,
Marco

[norberttoth398]

Hi Marco

It won't work with python 3.8 - all the package versions are specified explicitly for 3.7. Running the line below should not require you to build a wheel but it should download a .whl file to install:

pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.0/index.html

Potentially the problem is a lack of CUDA support on your machine - please try the CPU only install in a freshly created clean environment:

pip install torch==1.7.0 torchvision==0.8.0 torchaudio==0.7.0

pip install openmim

pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.7.0/index.html

mim install "mmdet<3.0.0"

I do apologise, MMCV is a pain to get working and if I could start the project over I wouldn't have used it to be honest. It is imperative you start with a brand new environment and Python 3.7 otherwise it will most likely run into problems - mmcv struggles with persistent errors once it failed to run for some reason.

Please do let me know if it works! I'm confident the above (CPU-only) install will do the job.

Best
Norbert

[marcoaaz]

Dear Norbert,

I tried a fresh environment with Python 3.7.7. I have not tried other 3.7.* versions yet. This line:

pip install torch==1.7.0 torchvision==0.8.0 torchaudio==0.7.0 results in a

(MinDetEnv10) PS C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main> python -m pip install torch==1.7.0 torchvision==0.8.0 torchaudio==0.7.0
Collecting torch==1.7.0
Using cached https://files.pythonhosted.org/packages/24/44/2b66a46b8ce7895a96db1a6dc525d9a9910353f7ca323bf04d0753603f18/torch-1.7.0-cp37-cp37m-win_amd64.whl
ERROR: Exception:
Traceback (most recent call last):
File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv10\lib\site-packages\pip_internal\cli\base_command.py", line 188, in main
status = self.run(options, args)
File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv10\lib\site-packages\pip_internal\commands\install.py", line 345, in run
resolver.resolve(requirement_set)
File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv10\lib\site-packages\pip_internal\legacy_resolve.py", line 196, in resolve
self._resolve_one(requirement_set, req)
File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv10\lib\site-packages\pip_internal\legacy_resolve.py", line 362, in _resolve_one
dist = abstract_dist.get_pkg_resources_distribution()
File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv10\lib\site-packages\pip_internal\distributions\wheel.py", line 14, in get_pkg_resources_distribution
self.req.source_dir))[0]
IndexError: list index out of range

, whereas this line (exactly as specified in the pytorch doc):

pip install torch==1.7.0+cu110 torchvision==0.8.0 torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

results in:

(MinDetEnv10) PS C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main> pip install torch==1.7.0+cu110 torchvision==0.8.0 torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
Looking in links: https://download.pytorch.org/whl/torch_stable.html
Collecting torch==1.7.0+cu110
Using cached https://download.pytorch.org/whl/cu110/torch-1.7.0%2Bcu110-cp37-cp37m-win_amd64.whl
Collecting torchvision==0.8.0
Using cached https://download.pytorch.org/whl/cpu/torchvision-0.8.0-cp37-cp37m-win_amd64.whl
Collecting torchaudio==0.7.0
Using cached https://download.pytorch.org/whl/torchaudio-0.7.0-cp37-none-win_amd64.whl
Collecting numpy (from torch==1.7.0+cu110)
Using cached https://files.pythonhosted.org/packages/97/9f/da37cc4a188a1d5d203d65ab28d6504e17594b5342e0c1dc5610ee6f4535/numpy-1.21.6-cp37-cp37m-win_amd64.whl
Collecting future (from torch==1.7.0+cu110)
Using cached https://files.pythonhosted.org/packages/da/71/ae30dadffc90b9006d77af76b393cb9dfbfc9629f339fc1574a1c52e6806/future-1.0.0-py3-none-any.whl
Collecting dataclasses (from torch==1.7.0+cu110)
Using cached https://files.pythonhosted.org/packages/26/2f/1095cdc2868052dd1e64520f7c0d5c8c550ad297e944e641dbf1ffbb9a5d/dataclasses-0.6-py3-none-any.whl
Collecting typing-extensions (from torch==1.7.0+cu110)
Using cached https://download.pytorch.org/whl/typing_extensions-4.9.0-py3-none-any.whl
ERROR: Package 'typing-extensions' requires a different Python: 3.7.7 not in '>=3.8'

Any steps beyond that have not worked this time. Would you suggest using an early python 3.8?

I am working in a Alienware X15 R1.

[norberttoth398]

Hi Marco I think you may be running into the issue due to the prior installation attempts with 3.8 using cached versions that will not work with the correct 3.7 environment. The top attempt looks like there is something wrong with the installation of the wheel file itself - this is odd and it's completely to do with Pytorch. You could perhaps attempt the CPU-only install with no caching and building own wheel files instead of downloading it from link. I would suggest trying to Google around with the specific error messages you're getting as these are all so far to do with Pytorch install - some common problems can be averted by upgrading Pip for example. Please do stuck to Python 3.7 though as it works - the auto build in the GitHub workflows folder builds just fine every time I push my repo so please do take a look at that if you're unsure. Norbert

…

On Mon, 26 Aug 2024, 01:42 Marco AcZ, ***@***.***> wrote: Dear Norbert, I tried a fresh environment with Python 3.7.7. I have not tried other 3.7.* versions yet. This line: pip install torch==1.7.0 torchvision==0.8.0 torchaudio==0.7.0 results in a (MinDetEnv10) PS C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main> python -m pip install torch==1.7.0 torchvision==0.8.0 torchaudio==0.7.0 Collecting torch==1.7.0 Using cached https://files.pythonhosted.org/packages/24/44/2b66a46b8ce7895a96db1a6dc525d9a9910353f7ca323bf04d0753603f18/torch-1.7.0-cp37-cp37m-win_amd64.whl ERROR: Exception: Traceback (most recent call last): File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv10\lib\site-packages\pip_internal\cli\base_command.py", line 188, in main status = self.run(options, args) File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv10\lib\site-packages\pip_internal\commands\install.py", line 345, in run resolver.resolve(requirement_set) File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv10\lib\site-packages\pip_internal\legacy_resolve.py", line 196, in resolve self._resolve_one(requirement_set, req) File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv10\lib\site-packages\pip_internal\legacy_resolve.py", line 362, in _resolve_one dist = abstract_dist.get_pkg_resources_distribution() File "C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main\MinDetEnv10\lib\site-packages\pip_internal\distributions\wheel.py", line 14, in get_pkg_resources_distribution self.req.source_dir))[0] IndexError: list index out of range , whereas this line (exactly as specified in the pytorch doc <https://pytorch.org/get-started/previous-versions/>): pip install torch==1.7.0+cu110 torchvision==0.8.0 torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html results in: (MinDetEnv10) PS C:\Users\n10832084\OneDrive - Queensland University of Technology\Desktop\Toth and Maclennan_supp.data\MinDet-main> pip install torch==1.7.0+cu110 torchvision==0.8.0 torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html Looking in links: https://download.pytorch.org/whl/torch_stable.html Collecting torch==1.7.0+cu110 Using cached https://download.pytorch.org/whl/cu110/torch-1.7.0%2Bcu110-cp37-cp37m-win_amd64.whl Collecting torchvision==0.8.0 Using cached https://download.pytorch.org/whl/cpu/torchvision-0.8.0-cp37-cp37m-win_amd64.whl Collecting torchaudio==0.7.0 Using cached https://download.pytorch.org/whl/torchaudio-0.7.0-cp37-none-win_amd64.whl Collecting numpy (from torch==1.7.0+cu110) Using cached https://files.pythonhosted.org/packages/97/9f/da37cc4a188a1d5d203d65ab28d6504e17594b5342e0c1dc5610ee6f4535/numpy-1.21.6-cp37-cp37m-win_amd64.whl Collecting future (from torch==1.7.0+cu110) Using cached https://files.pythonhosted.org/packages/da/71/ae30dadffc90b9006d77af76b393cb9dfbfc9629f339fc1574a1c52e6806/future-1.0.0-py3-none-any.whl Collecting dataclasses (from torch==1.7.0+cu110) Using cached https://files.pythonhosted.org/packages/26/2f/1095cdc2868052dd1e64520f7c0d5c8c550ad297e944e641dbf1ffbb9a5d/dataclasses-0.6-py3-none-any.whl Collecting typing-extensions (from torch==1.7.0+cu110) Using cached https://download.pytorch.org/whl/typing_extensions-4.9.0-py3-none-any.whl ERROR: Package 'typing-extensions' requires a different Python: 3.7.7 not in '>=3.8' Any steps beyond that have not worked this time. Would you suggest using an early python 3.8? I am working in a Alienware X15 R1. — Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADDLQPCRNMAUSRYULAHBNFDZTJ2WFAVCNFSM6AAAAABM3J4Q4CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBZGA4DKOBUGQ> . You are receiving this because you commented.Message ID: ***@***.***>

[marcoaaz]

It is all good now. I figured out that the following sequence works (in Visual Studio Code, Windows 10, laptop with GPU) for a GPU install:
.\.venv\Scripts\python.exe -m venv MinDetEnv11
MinDetEnv11\Scripts\Activate.ps1
python.exe -m pip install --upgrade pip
python.exe -m pip install torch==1.7.0+cu110 torchvision==0.8.0 torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cpu/torch1.7.0/index.html
pip install openmim
mim install "mmdet<3.0.0"
python.exe -m pip install git+https://git@github.com/norberttoth398/MinDet

, the CPU install for 'torchvision==0.8.0+cpu' seems not to be available.

I was working with 3.7. One cannot mess with the torch installation. In my system (admin rights managed by institution), I needed to add 'python.exe -m ' before the torch installation command, which was causing the blockage. Also, upgrading pip probably helped. The following minor issue remains:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behavior is the source of the following dependency conflicts.
openxlab 0.0.10 requires pytz~=2023.3, but you have pytz 2022.1 which is incompatible.
openxlab 0.0.10 requires requests~=2.28.2, but you have requests 2.27.1 which is incompatible.
rich 13.8.0 requires pygments<3.0.0,>=2.13.0, but you have pygments 2.12.0 which is incompatible.

Also, the example code provided installs
!pip install Pillow==9.0.0

but it was be already installed as 9.2 version producing no conflicts in the script.

Regarding timing, these are 32-36 sec for prediction (with GPU install). I would be happy if you accept that I close the issue. Thank you!

Note: Just a quick question, how can I directly use or import nms_remove, mask_nms, and perform_nms from MinDet.nms module if they are already defined? I seem to require defining them in the script (as shown in the example).

[norberttoth398]

That's great to hear!

The only reason I define them in the example script explicitly was because I initially wrote that to showcase the code for a talk/reading group. It's all in nms.py that are called explicitly by the main workflows (_run and tile_only) in run.py.

Best
Norbert