test scripts failing after native windows code merge

[chauhang]

The development install and tests scripts are broken after the recent Native windows PR code merge changes.

Following are not working:

1. Install from source on fresh machine (Install from source not working #796)
2. Test script for model archiver
3. Test script for TorchServe
4. Test script for sanity tests
5. Tests for API

Detailed error logs

python ./scripts/test_modelarchiver.py

Started model archiver linting
In directory: /home/ubuntu/serve/model-archiver | Executing command: pylint -rn --rcfile=./model_archiver/tests/pylintrc model_archiver/.

---

Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

Started model archiver pytests - unit tests
In directory: /home/ubuntu/serve/model-archiver | Executing command: python -m pytest --cov-report html:result_units --cov=. model_archiver/tests/unit_tests
ERROR: usage: main.py [options] [file_or_dir] [file_or_dir] [...]
main.py: error: unrecognized arguments: --cov-report --cov=. model_archiver/tests/unit_tests
inifile: None
rootdir: /home/ubuntu/serve/model-archiver

Started model archiver pytests - integration tests
In directory: /home/ubuntu/serve/model-archiver | Executing command: python -m pytest model_archiver/tests/integ_tests
=========================================================== test session starts ===========================================================
platform linux -- Python 3.6.10, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /home/ubuntu/serve/model-archiver
collected 2 items

model_archiver/tests/integ_tests/test_integration_model_archiver.py .. [100%]

============================================================ 2 passed in 0.99s ============================================================
Model archiver Unit Pytests Failed !

python ./scripts/test_torchserve.py

Started torchserve linting
In directory: /home/ubuntu/serve | Executing command: pylint -rn --rcfile=./ts/tests/pylintrc ts/.

---

Your code has been rated at 10.00/10

Started torchserve pytests
In directory: /home/ubuntu/serve | Executing command: python -m pytest --cov-report html:result_units --cov=ts ts/tests/unit_tests
ERROR: usage: main.py [options] [file_or_dir] [file_or_dir] [...]
main.py: error: unrecognized arguments: --cov-report --cov=ts ts/tests/unit_tests
inifile: None
rootdir: /home/ubuntu/serve

TorchServe Pytests Failed !

python ./scripts/test_torchserve.py

Started torchserve linting
In directory: /home/ubuntu/serve | Executing command: pylint -rn --rcfile=./ts/tests/pylintrc ts/.

---

Your code has been rated at 10.00/10

Started torchserve pytests
In directory: /home/ubuntu/serve | Executing command: python -m pytest --cov-report html:result_units --cov=ts ts/tests/unit_tests
ERROR: usage: main.py [options] [file_or_dir] [file_or_dir] [...]
main.py: error: unrecognized arguments: --cov-report --cov=ts ts/tests/unit_tests
inifile: None
rootdir: /home/ubuntu/serve

TorchServe Pytests Failed !
(pytorch_p36) ubuntu@ip-172-31-43-138:~/serve$ python ./scripts/test_sanity.py
Traceback (most recent call last):
File "./scripts/test_sanity.py", line 3, in
import nvgpu
ModuleNotFoundError: No module named 'nvgpu'

python ./scripts/test_api.py management

Starting TorchServe
Console logs redirected to file: ts_console.log
In directory: /home/ubuntu/serve/test | Executing command: torchserve --start --model-store=model_store --ncs >> ts_console.log
Successfully started TorchServe
sh: 1: newman: not found
Stopping TorchServe
In directory: /home/ubuntu/serve/test | Executing command: torchserve --stop
TorchServe has stopped.
Successfully stopped TorchServe
Newman API Tests Failed !

[maaquib]

@chauhang This seems to be working after following install steps from #796 (comment) (and newman install)

[chauhang]

@maaquib @dhaniram-kshirsagar Tests are still failing at different places on different machines after the merge of #798

CPU logs (ubuntu 18.04)

test-frontend-cpu-hung-after-79pc.log

GPU Logs (ubuntu 18_04)

test-frontend-gpu.log

Rerun install from src on GPU machine after creating a new conda env

test-frontend-gpu-conda.log

Please add:

1. Proper checks for the versions numbers of the base software - Pip and Python 3.x (min version on which things will work)
2. There are few "Permission denied" errors for the installs and the scripts are failing if one runs with sudo python xxx. Manually running sudo apt-get install resolved some of the items but not all
3. Add a script for printing version numbers of all the relevant software (python ,java, torch etc) that can be used for torubleshooting

[chauhang]

test_torchserve.py is still failing even after manually installing pytest and pylint on CPU (ubuntu 18.04)

`ubuntu@ip-172-31-40-192:/serve$ pip freeze| grep py
numpy==1.19.4
py==1.9.0
pyasn1==0.4.2
pyasn1-modules==0.2.1
pycrypto==2.6.1
pygobject==3.26.1
pylint==2.6.0
pyOpenSSL==17.5.0
pyparsing==2.4.7
pyserial==3.4
pytest==6.1.2
python-apt==1.6.5+ubuntu0.3
python-debian==0.1.32
pyxdg==0.25
systemd-python==234
ubuntu@ip-172-31-40-192:/serve$ which python
/usr/bin/python
ubuntu@ip-172-31-40-192:/serve$ python --version
Python 3.6.9
ubuntu@ip-172-31-40-192:/serve$ python ./ts_scripts/test_torchserve.py

Started torchserve linting
In directory: /home/ubuntu/serve | Executing command: pylint -rn --rcfile=./ts/tests/pylintrc ts/.
sh: 1: pylint: not found
Started torchserve pytests
In directory: /home/ubuntu/serve | Executing command: python -m pytest --cov-report html:result_units --cov=ts ts/tests/unit_tests
/usr/bin/python: No module named pytest
TorchServe Linting Failed !

[codinnvrends]

@chauhang Only install_from_src.py should be executed from ts_scripts folder. However, I understood, why somebody may try to directly execute other scripts in that folder and end up with this kind of issue i.e. Along with install_from_src.py other scripts are also self-executable i.e. they have main in it.
I will fix this so that only install_from_src.py and install_dependencies.py can be executed. And also correct the related doc.

[chauhang]

@dhaniram-kshirsagar Please see #824 still many issues with older folder name that need to be resolved

[harshbafna]

This has been fixed and corresponding documentation has been updated as part of #836. Closing.