Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fixing pytest discovery issues #28158

Merged
merged 20 commits into from
Feb 7, 2024

Conversation

vismaysur
Copy link
Contributor

@vismaysur vismaysur commented Feb 2, 2024

PR Description

Fixing pytest discovery issues

Specific edge cases addressed:

  • ivy.functional.frontends.jax.numpy.indexing.CClass is a class object invoked as a function call, parsed in ast extraction.
  • jnp_frontend namespace causing string manipulation error in retrieving dtypes/device for nested functions.
  • dict containing ivy.functional.frontends supported versions does not contain a supported mindspore version.

Related Issue

Closes #28138

Checklist

  • Did you add a function?
  • Did you add the tests?
  • Did you run your tests and are your tests passing?
  • Did pre-commit not fail on any check?
  • Did you follow the steps we provided?

Socials

@vismaysur vismaysur changed the title fixFixing pytest discovery issues fix: Fixing pytest discovery issues Feb 2, 2024
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Compliance Checks Passed!

@ivy-leaves ivy-leaves added JAX Frontend Developing the JAX Frontend, checklist triggered by commenting add_frontend_checklist TensorFlow Frontend Developing the TensorFlow Frontend, checklist triggered by commenting add_frontend_checklist Ivy Functional API labels Feb 2, 2024
Copy link
Contributor

@vedpatwardhan vedpatwardhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @Vismay-dev, thanks for creating the PR, just requested a couple of changes!

@vismaysur
Copy link
Contributor Author

vismaysur commented Feb 2, 2024

@vedpatwardhan i’ve made the changes 🚀

@@ -1,7 +1,7 @@
import ivy
from ivy.func_wrapper import with_unsupported_dtypes
from ivy.functional.frontends.jax.array import Array
import ivy.functional.frontends.jax.numpy as jnp_frontend
import ivy.functional.frontends.jax.numpy as jnp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we have to change the name of the import at either of the 2 places, I'd much rather do that in the test file instead. Sorry for the change of mind, could you please change it so that we do the jnp import in the test file and jnp_frontend import in the frontend file? Thanks @Vismay-dev 😄

Copy link
Contributor Author

@vismaysur vismaysur Feb 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vedpatwardhan it's the front end file import that's causing the error rather than the test file. Not the import itself but the use of this namespace jnp_frontend in the function setdiff1d".

This line https://github.com/unifyai/ivy/blob/d382882aafb0e19cccc8145cfcaa6cc4f53a77c4/ivy/functional/ivy/data_type.py#L192 checks for the keyword "frontend" in the base names of all the functions invoked in the frontend function setdiff1d. I'm not sure if this is an intentional design caveat of the current use of an AST, or just an unprecedented edge case.

Changing the import in the test file, as you originally pointed out, is not related to the error.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's probably change the import in the frontend file to something like from ivy.functional.frontends.jax import where, unique similar to the other imports in the jax frontend if that fixes the issue 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vedpatwardhan that's causing a circular import error. the reason there's no circular import error for the other imports in the jax frontend is because those imported functions (for eg. promote_types_of_jax_inputs) have been defined in __init__.py for the frontends/jax/numpy frontend directory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just pushed a change to data_type.py, does test collection now work with jnp_frontend in the frontend file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the test collection works with jnp_frontend now 😄

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome! We can get the PR merged after the changes to the demos submodule are reverted. Feel free to request a review once you're done with that, thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vedpatwardhan all done!

Copy link
Contributor

@vedpatwardhan vedpatwardhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! Feel free to merge the PR @NripeshN, thanks @Vismay-dev 😄

@NripeshN
Copy link
Contributor

NripeshN commented Feb 6, 2024

Hey @Vismay-dev @vedpatwardhan
I am still facing errors locally with pytest discovery.

Find the entire log below

2024-02-06 10:21:31.595 [info] Experiment 'pythonRecommendTensorboardExt' is active
2024-02-06 10:21:31.595 [info] Experiment 'pythonREPLSmartSend' is active
2024-02-06 10:21:31.595 [info] Experiment 'pythonTerminalEnvVarActivation' is active
2024-02-06 10:21:31.595 [info] Experiment 'pythonTestAdapter' is active
2024-02-06 10:21:31.701 [info] Default formatter is set to ms-python.autopep8 for workspace /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api
2024-02-06 10:21:31.711 [info] Test server listening.
2024-02-06 10:21:31.715 [info] > conda info --json
2024-02-06 10:21:31.716 [info] Python interpreter path: ./ivy_dev/bin/python
2024-02-06 10:21:31.753 [error] Error in worker /Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/out/client/plainExec.worker.js Error: spawn ENOTDIR
    at ChildProcess.spawn (node:internal/child_process:413:11)
    at spawn (node:child_process:806:9)
    at t._workerPlainExecImpl (/Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/out/client/plainExec.worker.js:1:2069)
    at /Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/out/client/plainExec.worker.js:1:140045
    at /Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/out/client/plainExec.worker.js:1:140318
    at Object.<anonymous> (/Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/out/client/plainExec.worker.js:1:140339)
    at Module._compile (node:internal/modules/cjs/loader:1318:14)
    at Object..js (node:internal/modules/cjs/loader:1373:10)
    at Module.load (node:internal/modules/cjs/loader:1173:32)
    at node:internal/modules/cjs/loader:1014:12 {
  errno: -20,
  code: 'ENOTDIR',
  syscall: 'spawn'
}
2024-02-06 10:21:31.775 [info] > ~/.anaconda/bin/conda info --json
2024-02-06 10:21:31.798 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2024-02-06 10:21:31.798 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2024-02-06 10:21:31.812 [info] > . "~/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_dev/bin/activate" && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.0.0/pythonFiles/printEnvVariables.py
2024-02-06 10:21:31.812 [info] shell: bash
2024-02-06 10:21:31.813 [info] > ~/.conda/bin/conda info --json
2024-02-06 10:21:31.817 [info] Send text to terminal: echo /opt/homebrew/bin/fish
2024-02-06 10:21:31.819 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2024-02-06 10:21:31.819 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2024-02-06 10:21:31.819 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2024-02-06 10:21:31.819 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2024-02-06 10:21:31.819 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2024-02-06 10:21:31.844 [info] > ~/.condarc/bin/conda info --json
2024-02-06 10:21:31.892 [error] Error in worker /Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/out/client/plainExec.worker.js Error: spawn ENOTDIR
    at ChildProcess.spawn (node:internal/child_process:413:11)
    at spawn (node:child_process:806:9)
    at t._workerPlainExecImpl (/Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/out/client/plainExec.worker.js:1:2069)
    at /Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/out/client/plainExec.worker.js:1:140045
    at /Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/out/client/plainExec.worker.js:1:140318
    at Object.<anonymous> (/Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/out/client/plainExec.worker.js:1:140339)
    at Module._compile (node:internal/modules/cjs/loader:1318:14)
    at Object..js (node:internal/modules/cjs/loader:1373:10)
    at Module.load (node:internal/modules/cjs/loader:1173:32)
    at node:internal/modules/cjs/loader:1014:12 {
  errno: -20,
  code: 'ENOTDIR',
  syscall: 'spawn'
}
2024-02-06 10:21:31.893 [info] > ~/anaconda3/bin/conda info --json
2024-02-06 10:21:32.476 [info] Starting Jedi language server for test_array_api.
2024-02-06 10:21:34.821 [info] Shell integration is not working for fish
2024-02-06 10:21:34.821 [warning] Shell integration is not active, environment activated maybe overriden by the shell.
2024-02-06 10:21:34.824 [info] > . "~/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_dev/bin/activate.fish" && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.0.0/pythonFiles/printEnvVariables.py
2024-02-06 10:21:34.824 [info] shell: fish
2024-02-06 10:21:34.867 [info] > ~/.pyenv/versions/3.11.1/bin/python ~/.vscode/extensions/ms-python.python-2024.0.0/pythonFiles/printEnvVariables.py
2024-02-06 10:21:34.867 [info] shell: fish
2024-02-06 10:21:34.965 [info] Send text to terminal: /Users/nripeshniketan/.pyenv/versions/3.11.1/bin/python /Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/pythonFiles/printEnvVariablesToFile.py /Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/pythonFiles/deactivate/fish/envVars.txt
2024-02-06 12:16:07.158 [info] Discover tests for workspace name: test_array_api - uri: /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api
2024-02-06 12:16:09.342 [info] Discover tests for workspace name: test_array_api - uri: /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api
2024-02-06 12:16:12.894 [info] > . "~/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_dev/bin/activate" && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.0.0/pythonFiles/printEnvVariables.py
2024-02-06 12:16:12.894 [info] shell: bash
2024-02-06 12:16:12.938 [info] > "~/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_dev/bin/python" -c "import pytest"
2024-02-06 12:16:13.411 [info] Discover tests for workspace name: test_array_api - uri: /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/.vscode/settings.json
2024-02-06 12:16:13.411 [info] Running discovery for pytest using the new test adapter.
2024-02-06 12:16:13.411 [info] All environment variables set for pytest discovery for workspace /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api: {"CAML_LD_LIBRARY_PATH":"/Users/nripeshniketan/.opam/4.13.1/lib/stublibs:/Users/nripeshniketan/.opam/4.13.1/lib/ocaml/stublibs:/Users/nripeshniketan/.opam/4.13.1/lib/ocaml","COMMAND_MODE":"unix2003","HOME":"/Users/nripeshniketan","LC_CTYPE":"en_US.UTF-8","LOCAL_GIT_DIRECTORY":"/Applications/GitHub Desktop.app/Contents/Resources/app/git","LOGNAME":"nripeshniketan","LaunchInstanceID":"2CC3A606-5D6F-4A87-90D5-8680130A019A","MallocNanoZone":"0","OCAML_TOPLEVEL_PATH":"/Users/nripeshniketan/.opam/4.13.1/lib/toplevel","OPAM_SWITCH_PREFIX":"/Users/nripeshniketan/.opam/4.13.1","ORIGINAL_XDG_CURRENT_DESKTOP":"undefined","PWD":"/","SECURITYSESSIONID":"186a2","SHELL":"/opt/homebrew/bin/fish","SHLVL":"2","SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.15R946m7QE/Listeners","TMPDIR":"/var/folders/j_/nbyvxq0j7hl4t1t2px8t3hlm0000gn/T/","USER":"nripeshniketan","VSCODE_AMD_ENTRYPOINT":"vs/workbench/api/node/extensionHostProcess","VSCODE_CODE_CACHE_PATH":"/Users/nripeshniketan/Library/Application Support/Code/CachedData/05047486b6df5eb8d44b2ecd70ea3bdf775fd937","VSCODE_CRASH_REPORTER_PROCESS_TYPE":"extensionHost","VSCODE_CWD":"/","VSCODE_HANDLES_UNCAUGHT_ERRORS":"true","VSCODE_IPC_HOOK":"/Users/nripeshniketan/Library/Application Support/Code/1.86-main.sock","VSCODE_NLS_CONFIG":"{\"locale\":\"en-us\",\"osLocale\":\"en-ae\",\"availableLanguages\":{},\"_languagePackSupport\":true}","VSCODE_PID":"53903","XPC_FLAGS":"0x0","XPC_SERVICE_NAME":"application.com.microsoft.VSCode.68464367.68464373","__CFBundleIdentifier":"com.microsoft.VSCode","__CF_USER_TEXT_ENCODING":"0x1F5:0x0:0x0","ELECTRON_RUN_AS_NODE":"1","APPLICATION_INSIGHTS_NO_DIAGNOSTIC_CHANNEL":"1","VSCODE_L10N_BUNDLE_LOCATION":"","PATH":"/Users/nripeshniketan/.opam/4.13.1/bin:/opt/homebrew/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/Applications/VMware Fusion.app/Contents/Public:/usr/local/share/dotnet:~/.dotnet/tools:/Library/TeX/texbin:/Users/nripeshniketan/Downloads/google-cloud-sdk/bin:/Users/nripeshniketan/.opam/4.13.1/bin:/users/nripeshniketan/flutter/bin:/Users/nripeshniketan/.pub-cache/bin:/opt/homebrew/bin/python3.11:/users/nripeshniketan/flutter/bin:/Users/nripeshniketan/.pub-cache/bin:/opt/homebrew/bin/python3.11","PYTHONPATH":"/Users/nripeshniketan/.vscode/extensions/ms-python.python-2024.0.0/pythonFiles","TEST_UUID":"5831953b-275c-470d-839d-986a5efd774d","TEST_PORT":"57689"} 

2024-02-06 12:16:13.414 [info] > . "~/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_dev/bin/activate" && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.0.0/pythonFiles/printEnvVariables.py
2024-02-06 12:16:13.414 [info] shell: bash
2024-02-06 12:16:13.441 [info] > "~/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_dev/bin/python" -m pytest -p vscode_pytest --collect-only ivy_tests
2024-02-06 12:16:13.441 [info] cwd: ~/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api
2024-02-06 12:16:13.886 [info] Discover tests for workspace name: test_array_api - uri: /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api
2024-02-06 12:16:13.886 [info] Running discovery for pytest using the new test adapter.
2024-02-06 12:16:13.886 [error] Test discovery already in progress, not starting a new one.
2024-02-06 12:16:18.823 [info] Use Database in ReadOnly Mode with local caching !

2024-02-06 12:16:18.841 [info] ============================= test session starts ==============================
platform darwin -- Python 3.10.13, pytest-7.4.4, pluggy-1.3.0

2024-02-06 12:16:18.843 [info] backend(s): all
device: cpu
number of Hypothesis examples: 25
rootdir: /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api

2024-02-06 12:16:18.843 [info] plugins: hypothesis-6.92.2, anyio-4.2.0

2024-02-06 12:16:19.070 [info] collected 0 items / 1 error

2024-02-06 12:16:19.071 [info] 

2024-02-06 12:16:19.071 [error] WARNING:root:	Some binaries seem to be missing in your system. This could be either because we don't have compatible binaries for your system or that newer binaries were available. In the latter case, calling ivy.utils.cleanup_and_fetch_binaries() should fetch the binaries binaries. Feel free to create an issue on https://github.com/unifyai/ivy.git in case of the former

WARNING:root:
Following are the supported configurations :
compiler : cp38-cp38-manylinux_2_17_x86_64, cp39-cp39-manylinux_2_17_x86_64, cp310-cp310-manylinux_2_17_x86_64, cp311-cp311-manylinux_2_17_x86_64, cp310-cp310-macosx_12_0_arm64, cp311-cp311-macosx_12_0_arm64

WARNING:root:	/Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy/compiler/utils/IIM.so not found.

2024-02-06 12:16:19.087 [info] Test server connected to a client.
2024-02-06 12:16:19.089 [error] pytest test discovery error for workspace:  /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api 
 _pytest.config.ConftestImportFailure: AttributeError: module 'ivy.functional.backends.numpy' has no attribute 'bool' (from /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_tests/array_api_testing/test_array_api/conftest.py)
 Check Python Test Logs for more details.

Interrupted: 1 error during collection
 Check Python Test Logs for more details.
2024-02-06 12:16:19.089 [error] pytest test discovery error for workspace:  /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api 
 _pytest.config.ConftestImportFailure: AttributeError: module 'ivy.functional.backends.numpy' has no attribute 'bool' (from /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_tests/array_api_testing/test_array_api/conftest.py)
 Check Python Test Logs for more details.

Interrupted: 1 error during collection
 Check Python Test Logs for more details.
2024-02-06 12:16:19.089 [info] ResultResolver EOT received for discovery.
2024-02-06 12:16:19.089 [info] 
==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
/opt/homebrew/Cellar/python@3.10/3.10.13_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1050: in _gcd_import
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
ivy_dev/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:186: in exec_module
    exec(co, module.__dict__)
ivy_tests/array_api_testing/test_array_api/conftest.py:13: in <module>
    from reporting import pytest_metadata, pytest_json_modifyreport, add_extra_json_metadata # noqa
ivy_tests/array_api_testing/test_array_api/reporting.py:1: in <module>
    from array_api_tests.dtype_helpers import dtype_to_name
ivy_tests/array_api_testing/test_array_api/array_api_tests/dtype_helpers.py:138: in <module>
    all_dtypes = (xp.bool,) + numeric_dtypes
E   AttributeError: module 'ivy.functional.backends.numpy' has no attribute 'bool'
=============================== warnings summary ===============================
ivy_tests/__init__.py:2
  /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_tests/__init__.py:2: DeprecationWarning: Accessing jax.config via the jax.config submodule is deprecated.
    from jax.config import config

ivy_tests/array_api_testing/test_array_api/array-api/spec/API_specification/signatures/creation_functions.py:38
  /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_tests/array_api_testing/test_array_api/array-api/spec/API_specification/signatures/creation_functions.py:38: DeprecationWarning: invalid escape sequence '\s'
    """

ivy_tests/array_api_testing/test_array_api/array-api/spec/API_specification/signatures/elementwise_functions.py:601
  /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_tests/array_api_testing/test_array_api/array-api/spec/API_specification/signatures/elementwise_functions.py:601: DeprecationWarning: invalid escape sequence '\s'
    """

ivy_tests/array_api_testing/test_array_api/array-api/spec/API_specification/signatures/linalg.py:447
  /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_tests/array_api_testing/test_array_api/array-api/spec/API_specification/signatures/linalg.py:447: DeprecationWarning: invalid escape sequence '\*'
    """

ivy_tests/array_api_testing/test_array_api/array_api_tests/__init__.py:49
  /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api/ivy_tests/array_api_testing/test_array_api/array_api_tests/__init__.py:49: HypothesisWarning: Could not determine whether module ivy.functional.backends.numpy is an Array API library
    xps = array_api.make_strategies_namespace(_xp, api_version=api_version)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
ERROR  - AttributeError: module 'ivy.functional.backends.numpy' has no attrib...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
===================== no tests collected, 1 error in 0.25s =====================

2024-02-06 12:16:19.380 [error] Subprocess exited unsuccessfully with exit code 2 and signal null on workspace /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api.
2024-02-06 12:16:19.381 [error] Subprocess exited unsuccessfully with exit code 2 and signal null on workspace /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api. Creating and sending error discovery payload
2024-02-06 12:16:19.381 [error] pytest test discovery error for workspace:  /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api 
  
 The python test process was terminated before it could exit on its own, the process errored with: Code: 2, Signal: null for workspace /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api
2024-02-06 12:16:19.381 [info] ResultResolver EOT received for discovery.
2024-02-06 12:16:19.381 [info] Disposing data receiver for /Users/nripeshniketan/Documents - Nripesh’s MacBook Pro/python_programs/ivy/ivy_tests/array_api_testing/test_array_api and deleting UUID; pytest discovery.

@vismaysur
Copy link
Contributor Author

vismaysur commented Feb 6, 2024

Hey @vedpatwardhan @NripeshN

This error is related to array_api_testing suite.

I've fixed the issue with numpy bool attribute on my local machine. How do I go about pushing changes to array_api_testing suite? As per the deep dive, this cannot be done without a submodule update.

Pytest discovery works for ivy_tests/test_ivy subdirectory. Array_api_testing also works on my local machine after fixing the issues, but requires the changes I've made locally to be pushed.

Also, I still face this error, though I presume this is because of my setup.

INTERNALERROR> Traceback (most recent call last): INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1645, in getoption INTERNALERROR> val = getattr(self.option, name) INTERNALERROR> AttributeError: 'Namespace' object has no attribute 'skips_file' INTERNALERROR> INTERNALERROR> The above exception was the direct cause of the following exception: INTERNALERROR> INTERNALERROR> Traceback (most recent call last): INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/_pytest/main.py", line 272, in wrap_session INTERNALERROR> session.exitstatus = doit(config, session) or 0 INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/_pytest/main.py", line 325, in _main INTERNALERROR> config.hook.pytest_collection(session=session) INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_hooks.py", line 501, in __call__ INTERNALERROR> return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_manager.py", line 119, in _hookexec INTERNALERROR> return self._inner_hookexec(hook_name, methods, kwargs, firstresult) INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_callers.py", line 138, in _multicall INTERNALERROR> raise exception.with_traceback(exception.__traceback__) INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_callers.py", line 121, in _multicall INTERNALERROR> teardown.throw(exception) # type: ignore[union-attr] INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/_pytest/logging.py", line 783, in pytest_collection INTERNALERROR> return (yield) INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_callers.py", line 121, in _multicall INTERNALERROR> teardown.throw(exception) # type: ignore[union-attr] INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/_pytest/warnings.py", line 118, in pytest_collection INTERNALERROR> return (yield) INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_callers.py", line 121, in _multicall INTERNALERROR> teardown.throw(exception) # type: ignore[union-attr] INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1365, in pytest_collection INTERNALERROR> return (yield) INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_callers.py", line 102, in _multicall INTERNALERROR> res = hook_impl.function(*args) INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/_pytest/main.py", line 336, in pytest_collection INTERNALERROR> session.perform_collect() INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/_pytest/main.py", line 802, in perform_collect INTERNALERROR> hook.pytest_collection_modifyitems( INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_hooks.py", line 501, in __call__ INTERNALERROR> return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult) INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_manager.py", line 119, in _hookexec INTERNALERROR> return self._inner_hookexec(hook_name, methods, kwargs, firstresult) INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_callers.py", line 138, in _multicall INTERNALERROR> raise exception.with_traceback(exception.__traceback__) INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_callers.py", line 121, in _multicall INTERNALERROR> teardown.throw(exception) # type: ignore[union-attr] INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/_pytest/cacheprovider.py", line 413, in pytest_collection_modifyitems INTERNALERROR> res = yield INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_callers.py", line 121, in _multicall INTERNALERROR> teardown.throw(exception) # type: ignore[union-attr] INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/_pytest/cacheprovider.py", line 339, in pytest_collection_modifyitems INTERNALERROR> res = yield INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/pluggy/_callers.py", line 102, in _multicall INTERNALERROR> res = hook_impl.function(*args) INTERNALERROR> File "/workspaces/ivy/ivy_tests/array_api_testing/test_array_api/conftest.py", line 103, in pytest_collection_modifyitems INTERNALERROR> skips_file = skips_path = config.getoption('--skips-file') INTERNALERROR> File "/opt/miniconda/envs/multienv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1656, in getoption INTERNALERROR> raise ValueError(f"no option named {name!r}") from e INTERNALERROR> ValueError: no option named 'skips_file'

Copy link
Contributor

@vedpatwardhan vedpatwardhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @NripeshN, as @Vismay-dev mentioned, setting

    "python.testing.pytestArgs": [
        "ivy_tests/test_ivy"
    ],

in the settings.json should help ignore the array api tests 😄

Copy link
Contributor

@NripeshN NripeshN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!!!

@NripeshN NripeshN merged commit 0c3a4a9 into ivy-llc:main Feb 7, 2024
128 of 140 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ivy Functional API JAX Frontend Developing the JAX Frontend, checklist triggered by commenting add_frontend_checklist TensorFlow Frontend Developing the TensorFlow Frontend, checklist triggered by commenting add_frontend_checklist
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fixing pytest discovery issues on vscode
4 participants