[FIX] apply pylint to tests and examples

[kemingy]

We have changed to pylint in #134. But only the code under mosec/ directory has been applied.

Tasks

Beta Give feedback

No tasks being tracked yet.

[thinkcache]

I ran pylint for the files/folders in $PY_SOURCE_FILES

This is the output. So the PR is expected to have all these issues fixed, Right? New to Open Source and new to Mosec. Hope I am going in the right direction.

make lint

apple@apples-MacBook-Pro ~/D/g/mosec (main)> make lint
isort --check --diff --project=mosec mosec tests examples scripts setup.py
black --check --diff mosec tests examples scripts setup.py
All done! ✨ 🍰 ✨
28 files would be left unchanged.
pylint -j 8 --recursive=y mosec tests examples scripts setup.py
************* Module tests.square_service
tests/square_service.py:21:0: C0115: Missing class docstring (missing-class-docstring)
tests/square_service.py:26:12: W0707: Consider explicitly re-raising using 'raise ValidationError(err) from err' (raise-missing-from)
************* Module tests.square_service_shm
tests/square_service_shm.py:18:0: E0401: Unable to import 'pyarrow' (import-error)
tests/square_service_shm.py:25:0: C0115: Missing class docstring (missing-class-docstring)
tests/square_service_shm.py:30:12: W0707: Consider explicitly re-raising using 'raise ValidationError(err) from err' (raise-missing-from)
************* Module tests.mock_socket
tests/mock_socket.py:24:4: C0116: Missing function or method docstring (missing-function-docstring)
tests/mock_socket.py:24:28: W0613: Unused argument 'flags' (unused-argument)
tests/mock_socket.py:29:24: C0103: Argument name "mv" doesn't conform to snake_case naming style (invalid-name)
tests/mock_socket.py:36:4: C0116: Missing function or method docstring (missing-function-docstring)
tests/mock_socket.py:39:4: C0116: Missing function or method docstring (missing-function-docstring)
tests/mock_socket.py:42:4: C0116: Missing function or method docstring (missing-function-docstring)
tests/mock_socket.py:45:4: C0116: Missing function or method docstring (missing-function-docstring)
tests/mock_socket.py:45:28: W0613: Unused argument 'flags' (unused-argument)
tests/mock_socket.py:49:4: C0116: Missing function or method docstring (missing-function-docstring)
tests/mock_socket.py:49:4: R6301: Method could be a function (no-self-use)
tests/mock_socket.py:52:4: C0116: Missing function or method docstring (missing-function-docstring)
tests/mock_socket.py:55:4: C0116: Missing function or method docstring (missing-function-docstring)
tests/mock_socket.py:59:0: C0115: Missing class docstring (missing-class-docstring)
tests/mock_socket.py:59:0: C0103: Class name "socket" doesn't conform to PascalCase naming style (invalid-name)
tests/mock_socket.py:64:4: C0116: Missing function or method docstring (missing-function-docstring)
tests/mock_socket.py:64:28: W0622: Redefining built-in 'type' (redefined-builtin)
tests/mock_socket.py:64:28: W0613: Unused argument 'type' (unused-argument)
tests/mock_socket.py:64:39: W0613: Unused argument 'proto' (unused-argument)
tests/mock_socket.py:59:0: R0903: Too few public methods (1/2) (too-few-public-methods)
************* Module tests.utils
tests/utils.py:23:0: C0116: Missing function or method docstring (missing-function-docstring)
************* Module plasma_shm_ipc
examples/plasma_shm_ipc.py:17:0: E0401: Unable to import 'pyarrow' (import-error)
examples/plasma_shm_ipc.py:24:0: C0115: Missing class docstring (missing-class-docstring)
examples/plasma_shm_ipc.py:29:12: W0707: Consider explicitly re-raising using 'raise ValidationError(err) from err' (raise-missing-from)
examples/plasma_shm_ipc.py:33:0: C0115: Missing class docstring (missing-class-docstring)
examples/plasma_shm_ipc.py:39:4: W0105: String statement has no effect (pointless-string-statement)
************* Module distil_bert_server_pytorch
examples/distil_bert_server_pytorch.py:18:0: E0401: Unable to import 'torch' (import-error)
examples/distil_bert_server_pytorch.py:19:0: E0401: Unable to import 'transformers' (import-error)
examples/distil_bert_server_pytorch.py:40:0: C0115: Missing class docstring (missing-class-docstring)
examples/distil_bert_server_pytorch.py:57:0: C0115: Missing class docstring (missing-class-docstring)
examples/distil_bert_server_pytorch.py:63:8: W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation)
************* Module echo
examples/echo.py:31:0: C0115: Missing class docstring (missing-class-docstring)
examples/echo.py:36:12: W0621: Redefining name 'time' from outer scope (line 16) (redefined-outer-name)
examples/echo.py:33:8: W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation)
examples/echo.py:38:12: W0707: Consider explicitly re-raising using 'raise ValidationError(f'cannot find key {err}') from err' (raise-missing-from)
examples/echo.py:42:0: C0115: Missing class docstring (missing-class-docstring)
examples/echo.py:44:8: W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation)
examples/echo.py:49:0: C0115: Missing class docstring (missing-class-docstring)
examples/echo.py:51:8: W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation)
************* Module custom_env
examples/custom_env.py:31:0: C0115: Missing class docstring (missing-class-docstring)
examples/custom_env.py:36:8: W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation)
examples/custom_env.py:41:8: W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation)
examples/custom_env.py:46:4: C0103: Constant name "num_device" doesn't conform to UPPER_CASE naming style (invalid-name)
examples/custom_env.py:19:0: W0611: Unused ValidationError imported from mosec.errors (unused-import)
************* Module resnet50_server_pytorch
examples/resnet50_server_pytorch.py:20:0: E0401: Unable to import 'cv2' (import-error)
examples/resnet50_server_pytorch.py:21:0: E0401: Unable to import 'numpy' (import-error)
examples/resnet50_server_pytorch.py:22:0: E0401: Unable to import 'torch' (import-error)
examples/resnet50_server_pytorch.py:23:0: E0401: Unable to import 'torchvision' (import-error)
examples/resnet50_server_pytorch.py:40:0: C0115: Missing class docstring (missing-class-docstring)
examples/resnet50_server_pytorch.py:41:4: W0237: Parameter 'data' has been renamed to 'req' in overridden 'Preprocess.forward' method (arguments-renamed)
examples/resnet50_server_pytorch.py:46:12: C0103: Variable name "im" doesn't conform to snake_case naming style (invalid-name)
examples/resnet50_server_pytorch.py:47:12: C0103: Variable name "im" doesn't conform to snake_case naming style (invalid-name)
examples/resnet50_server_pytorch.py:49:12: W0707: Consider explicitly re-raising using 'raise ValidationError(f'cannot find key {err}') from err' (raise-missing-from)
examples/resnet50_server_pytorch.py:51:12: W0707: Consider explicitly re-raising using 'raise ValidationError(f'cannot decode as image data: {err}') from err' (raise-missing-from)
examples/resnet50_server_pytorch.py:53:8: C0103: Variable name "im" doesn't conform to snake_case naming style (invalid-name)
examples/resnet50_server_pytorch.py:63:0: C0115: Missing class docstring (missing-class-docstring)
examples/resnet50_server_pytorch.py:69:8: W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation)
examples/resnet50_server_pytorch.py:80:8: W1203: Use lazy % formatting in logging functions (logging-fstring-interpolation)
examples/resnet50_server_pytorch.py:88:0: C0115: Missing class docstring (missing-class-docstring)
examples/resnet50_server_pytorch.py:96:13: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
examples/resnet50_server_pytorch.py:96:37: C0103: Variable name "f" doesn't conform to snake_case naming style (invalid-name)
************* Module gen_docs_args
scripts/gen_docs_args.py:22:0: C0103: Constant name "stdout" doesn't conform to UPPER_CASE naming style (invalid-name)
scripts/gen_docs_args.py:23:0: C0103: Constant name "stdout" doesn't conform to UPPER_CASE naming style (invalid-name)
scripts/gen_docs_args.py:19:10: R1732: Consider using 'with' for resource-allocating operations (consider-using-with)
************* Module tests.test_protocol
tests/test_protocol.py:29:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_protocol.py:29:9: C0103: Argument name "p" doesn't conform to snake_case naming style (invalid-name)
tests/test_protocol.py:37:11: R1729: Use a generator instead 'all(bytes(got_payloads[i]) == sent_payloads[i] for i in range(len(sent_payloads)))' (use-a-generator)
tests/test_protocol.py:47:11: R1729: Use a generator instead 'all(bytes(got_payloads[i]) == sent_payloads[i] for i in range(len(sent_payloads)))' (use-a-generator)
tests/test_protocol.py:53:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_protocol.py:55:4: C0103: Variable name "p" doesn't conform to snake_case naming style (invalid-name)
tests/test_protocol.py:72:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_protocol.py:72:14: W0621: Redefining name 'mock_protocol' from outer scope (line 53) (redefined-outer-name)
************* Module python_side_metrics
examples/python_side_metrics.py:42:0: E0401: Unable to import 'prometheus_client' (import-error)
examples/python_side_metrics.py:42:0: C0413: Import "from prometheus_client import CONTENT_TYPE_LATEST, CollectorRegistry, Counter, generate_latest, multiprocess" should be placed at the top of the module (wrong-import-position)
examples/python_side_metrics.py:55:0: C0116: Missing function or method docstring (missing-function-docstring)
examples/python_side_metrics.py:55:15: W0613: Unused argument 'environ' (unused-argument)
examples/python_side_metrics.py:64:0: C0116: Missing function or method docstring (missing-function-docstring)
examples/python_side_metrics.py:69:0: C0115: Missing class docstring (missing-class-docstring)
examples/python_side_metrics.py:79:12: W0707: Consider explicitly re-raising using 'raise ValidationError(err) from err' (raise-missing-from)
examples/python_side_metrics.py:42:0: C0411: third party import "from prometheus_client import CONTENT_TYPE_LATEST, CollectorRegistry, Counter, generate_latest, multiprocess" should be placed before "from mosec import Server, Worker" (wrong-import-order)
************* Module setup
setup.py:34:43: W0621: Redefining name 'f' from outer scope (line 12) (redefined-outer-name)
setup.py:28:8: C0207: Use subprocess.check_output(['cargo', 'pkgid']).decode().strip().rsplit('#', maxsplit=1)[-1] instead (use-maxsplit-arg)
setup.py:31:18: C0209: Formatting a regular string which could be a f-string (consider-using-f-string)
setup.py:34:9: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
setup.py:34:43: C0103: Variable name "f" doesn't conform to snake_case naming style (invalid-name)
setup.py:39:0: R0903: Too few public methods (0/2) (too-few-public-methods)
setup.py:56:4: R1710: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
************* Module tests.test_service
tests/test_service.py:32:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_service.py:39:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_service.py:41:14: R1732: Consider using 'with' for resource-allocating operations (consider-using-with)
tests/test_service.py:64:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_service.py:64:24: W0621: Redefining name 'mosec_service' from outer scope (line 39) (redefined-outer-name)
tests/test_service.py:64:39: W0621: Redefining name 'http_client' from outer scope (line 32) (redefined-outer-name)
tests/test_service.py:64:24: W0613: Unused argument 'mosec_service' (unused-argument)
tests/test_service.py:95:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_service.py:95:27: W0621: Redefining name 'mosec_service' from outer scope (line 39) (redefined-outer-name)
tests/test_service.py:95:42: W0621: Redefining name 'http_client' from outer scope (line 32) (redefined-outer-name)
tests/test_service.py:98:8: C0103: Variable name "t" doesn't conform to snake_case naming style (invalid-name)
tests/test_service.py:104:8: C0103: Variable name "t" doesn't conform to snake_case naming style (invalid-name)
tests/test_service.py:95:27: W0613: Unused argument 'mosec_service' (unused-argument)
tests/test_service.py:110:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_service.py:110:46: C0103: Argument name "x" doesn't conform to snake_case naming style (invalid-name)
tests/test_service.py:110:28: W0621: Redefining name 'http_client' from outer scope (line 32) (redefined-outer-name)
tests/test_service.py:115:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_service.py:115:33: W0621: Redefining name 'http_client' from outer scope (line 32) (redefined-outer-name)
tests/test_service.py:117:4: C0103: Variable name "bs" doesn't conform to snake_case naming style (invalid-name)
tests/test_service.py:118:17: C3001: Lambda expression assigned to a variable. Define a function using the "def" keyword instead. (unnecessary-lambda-assignment)
tests/test_service.py:122:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_service.py:122:23: W0621: Redefining name 'http_client' from outer scope (line 32) (redefined-outer-name)
************* Module tests.test_coordinator
tests/test_coordinator.py:38:0: C0103: Constant name "stage" doesn't conform to UPPER_CASE naming style (invalid-name)
tests/test_coordinator.py:45:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_coordinator.py:50:0: C0115: Missing class docstring (missing-class-docstring)
tests/test_coordinator.py:60:0: C0115: Missing class docstring (missing-class-docstring)
tests/test_coordinator.py:61:4: W0235: Useless super delegation in method '__init__' (useless-super-delegation)
tests/test_coordinator.py:68:0: C0115: Missing class docstring (missing-class-docstring)
tests/test_coordinator.py:70:4: W0221: Number of parameters was 2 in 'Worker.deserialize' and is now 1 in overridden 'EchoWorkerMSGPACK.deserialize' method (arguments-differ)
tests/test_coordinator.py:74:4: W0221: Number of parameters was 2 in 'Worker.serialize' and is now 1 in overridden 'EchoWorkerMSGPACK.serialize' method (arguments-differ)
tests/test_coordinator.py:79:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_coordinator.py:88:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_coordinator.py:106:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_coordinator.py:106:39: W0621: Redefining name 'base_test_config' from outer scope (line 79) (redefined-outer-name)
tests/test_coordinator.py:129:0: C0116: Missing function or method docstring (missing-function-docstring)
tests/test_coordinator.py:129:39: W0621: Redefining name 'base_test_config' from outer scope (line 79) (redefined-outer-name)
tests/test_coordinator.py:141:8: R1732: Consider using 'with' for resource-allocating operations (consider-using-with)
tests/test_coordinator.py:141:8: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
tests/test_coordinator.py:204:0: R0914: Too many local variables (20/15) (too-many-locals)
tests/test_coordinator.py:204:28: W0621: Redefining name 'base_test_config' from outer scope (line 79) (redefined-outer-name)
tests/test_coordinator.py:251:19: R1729: Use a generator instead 'all(deserializer(x) == deserializer(y) for (x, y) in zip(got_payloads, sent_payloads))' (use-a-generator)
tests/test_coordinator.py:260:8: C0103: Variable name "e" doesn't conform to snake_case naming style (invalid-name)
tests/test_coordinator.py:204:20: W0613: Unused argument 'mocker' (unused-argument)

-----------------------------------
Your code has been rated at 8.41/10

make: *** [lint] Error 30

[kemingy]

Hi @thinkcache, thanks for your interest. You're right. You can split it into two PRs, one for the tests, and one for the examples.

[thinkcache]

@kemingy
I have started fixing linting issues in examples. Found another issue with error: Unused "type: ignore" comment when executing mypy.

Any suggestion to overcome that.

mypy --install-types --non-interactive mosec tests examples scripts setup.py
examples/resnet50_server_pytorch.py:21: error: Unused "type: ignore" comment
    import numpy as np  # type: ignore
    ^
Found 1 error in 1 file (checked 27 source files)
make: *** [lint] Error 1

[kemingy]

Hi @thinkcache, you can create the WIP pull request first. I'll take a look. For the mypy issue, it's very annoying. Sometimes it's related to the Python or mypy version.

[thinkcache]

@kemingy #180 is a WIP PR.