Benchmarks not running in JsonSchema after fix

[AbhimanyuBahree]

When i run tox -e perf as mentioned in the readme then I get the following error.

/Users/abhimanyubahree/PycharmProjects/untitled1/venv/bin/python -m tox -e perf
perf installed: appdirs==1.4.4,attrs==20.3.0,Automat==20.2.0,bcrypt==3.2.0,behave==1.2.6,certifi==2020.12.5,cffi==1.14.5,chardet==4.0.0,constantly==15.1.0,cryptography==3.4.6,distlib==0.3.1,filelock==3.0.12,gila==0.5.0,hyperlink==21.0.0,idna==2.10,incremental==21.3.0,jsonschema==3.2.0,packaging==20.9,paramiko==2.7.2,parse==1.19.0,parse-type==0.5.2,pluggy==0.13.1,ply==3.11,py==1.10.0,pycparser==2.20,pyhcl==0.3.13,PyNaCl==1.4.0,pyparsing==2.4.7,pyperf==2.2.0,pyrsistent==0.17.3,python-dotenv==0.10.3,PyYAML==5.1.2,requests==2.25.1,requests-toolbelt==0.9.1,scp==0.13.3,six==1.15.0,toml==0.10.0,tox==3.23.0,Twisted==21.2.0,urllib3==1.26.3,virtualenv==20.4.3,zope.interface==5.3.0
perf run-test-pre: PYTHONHASHSEED='1935722257'
perf run-test: commands[0] | /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/bin/python -m pip install '/Users/abhimanyubahree/Documents/personal_project/jsonschema[format]'
Processing /Users/abhimanyubahree/Documents/personal_project/jsonschema
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing wheel metadata: started
Preparing wheel metadata: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/bin/python /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/tmpzxa4kdjo
cwd: /private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-req-build-p3_x04zm
Complete output (37 lines):
Traceback (most recent call last):
File "/Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in
main()
File "/Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 166, in prepare_metadata_for_build_wheel
self.run_setup()
File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 150, in run_setup
exec(compile(code, file, 'exec'), locals())
File "setup.py", line 1, in
File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 434, in init
_Distribution.init(self, {
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/dist.py", line 292, in init
self.finalize_options()
File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 743, in finalize_options
ep(self)
File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/integration.py", line 59, in infer_version
dist.metadata.version = _get_version(config)
File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/init.py", line 173, in _get_version
parsed_version = _do_parse(config)
File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/init.py", line 127, in _do_parse
version = _version_from_entrypoints(config) or _version_from_entrypoints(
File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/init.py", line 65, in _version_from_entrypoints
version = _call_entrypoint_fn(root, config, ep.load())
File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/init.py", line 46, in _call_entrypoint_fn
return fn(root, config=config)
File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/git.py", line 115, in parse
require_command("git")
File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/utils.py", line 142, in require_command
raise OSError("%r was not found" % name)
OSError: 'git' was not found
----------------------------------------
WARNING: Discarding file:///Users/abhimanyubahree/Documents/personal_project/jsonschema. Command errored out with exit status 1: /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/bin/python /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/tmpzxa4kdjo Check the logs for full command output.
ERROR: Command errored out with exit status 1: /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/bin/python /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/tmpzxa4kdjo Check the logs for full command output.
ERROR: InvocationError for command /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/bin/python -m pip install '/Users/abhimanyubahree/Documents/personal_project/jsonschema[format]' (exited with code 1)
___________________________________ summary ____________________________________
ERROR: perf: commands failed

Process finished with exit code 1

[Julian]

Looks like from the error at the bottom there that your system is missing git

…

On Tue, Apr 13, 2021, 03:40 Abhimanyu Sai Bahree ***@***.***> wrote: When i run tox -e perf as mentioned in the readme then I get the following error. /Users/abhimanyubahree/PycharmProjects/untitled1/venv/bin/python -m tox -e perf perf installed: appdirs==1.4.4,attrs==20.3.0,Automat==20.2.0,bcrypt==3.2.0,behave==1.2.6,certifi==2020.12.5,cffi==1.14.5,chardet==4.0.0,constantly==15.1.0,cryptography==3.4.6,distlib==0.3.1,filelock==3.0.12,gila==0.5.0,hyperlink==21.0.0,idna==2.10,incremental==21.3.0,jsonschema==3.2.0,packaging==20.9,paramiko==2.7.2,parse==1.19.0,parse-type==0.5.2,pluggy==0.13.1,ply==3.11,py==1.10.0,pycparser==2.20,pyhcl==0.3.13,PyNaCl==1.4.0,pyparsing==2.4.7,pyperf==2.2.0,pyrsistent==0.17.3,python-dotenv==0.10.3,PyYAML==5.1.2,requests==2.25.1,requests-toolbelt==0.9.1,scp==0.13.3,six==1.15.0,toml==0.10.0,tox==3.23.0,Twisted==21.2.0,urllib3==1.26.3,virtualenv==20.4.3,zope.interface==5.3.0 perf run-test-pre: PYTHONHASHSEED='1935722257' perf run-test: commands[0] | /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/bin/python -m pip install '/Users/abhimanyubahree/Documents/personal_project/jsonschema[format]' Processing /Users/abhimanyubahree/Documents/personal_project/jsonschema Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing wheel metadata: started Preparing wheel metadata: finished with status 'error' ERROR: Command errored out with exit status 1: command: /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/bin/python /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/tmpzxa4kdjo cwd: /private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-req-build-p3_x04zm Complete output (37 lines): Traceback (most recent call last): File "/Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in main() File "/Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel return hook(metadata_directory, config_settings) File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 166, in prepare_metadata_for_build_wheel self.run_setup() File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 150, in run_setup exec(compile(code, *file*, 'exec'), locals()) File "setup.py", line 1, in File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools/ *init*.py", line 153, in setup return distutils.core.setup(**attrs) File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/core.py", line 108, in setup _setup_distribution = dist = klass(attrs) File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 434, in *init* _Distribution.*init*(self, { File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/distutils/dist.py", line 292, in *init* self.finalize_options() File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 743, in finalize_options ep(self) File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/integration.py", line 59, in infer_version dist.metadata.version = _get_version(config) File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/ *init*.py", line 173, in _get_version parsed_version = _do_parse(config) File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/ *init*.py", line 127, in _do_parse version = _version_from_entrypoints(config) or _version_from_entrypoints( File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/ *init*.py", line 65, in _version_from_entrypoints version = _call_entrypoint_fn(root, config, ep.load()) File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/ *init*.py", line 46, in _call_entrypoint_fn return fn(root, config=config) File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/git.py", line 115, in parse require_command("git") File "/private/var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/pip-build-env-t_ue8en9/overlay/lib/python3.8/site-packages/setuptools_scm/utils.py", line 142, in require_command raise OSError("%r was not found" % name) OSError: 'git' was not found ---------------------------------------- WARNING: Discarding file:///Users/abhimanyubahree/Documents/personal_project/jsonschema. Command errored out with exit status 1: /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/bin/python /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/tmpzxa4kdjo Check the logs for full command output. ERROR: Command errored out with exit status 1: /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/bin/python /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /var/folders/83/3t7qm4nn5jx2p5f4ync9cv_m0000gn/T/tmpzxa4kdjo Check the logs for full command output. ERROR: InvocationError for command /Users/abhimanyubahree/Documents/personal_project/jsonschema/.tox/perf/bin/python -m pip install '/Users/abhimanyubahree/Documents/personal_project/jsonschema[format]' (exited with code 1) ___________________________________ summary ____________________________________ ERROR: perf: commands failed Process finished with exit code 1 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#800>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACQQXW67V5AQTNTTVGCJXDTIPYQNANCNFSM422ZBGZQ> .

[AbhimanyuBahree]

my system has git

[AbhimanyuBahree]

try making a new virtual environment with python 3.9 and run the benchmarks

[AbhimanyuBahree]

not able to run unit tests also in my mac. Unit test and benchmarks are not working. If it is working for you provide step-by-step instructions for what to do after creating a new python virtual environment.

[Julian]

⊙  tox -e perf -r                                                                                                                                                                                                julian@Airm
perf recreate: /Users/julian/Development/jsonschema/.tox/perf
perf installdeps: pyperf
perf installed: /Users/julian/Development/jsonschema/.tox/perf/lib/python3.9/site-packages/pip/_vendor/packaging/version.py:127: DeprecationWarning: Creating a LegacyVersion has been deprecated and will be removed in the next major release,  warnings.warn(,pyperf==2.2.0
perf run-test-pre: PYTHONHASHSEED='3956464195'
perf run-test: commands[0] | /Users/julian/Development/jsonschema/.tox/perf/bin/python -m pip install '/Users/julian/Development/jsonschema[format]'
Processing /Users/julian/Development/jsonschema
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting attrs>=17.4.0
  Using cached attrs-20.3.0-py2.py3-none-any.whl (49 kB)
Collecting pyrsistent>=0.14.0
  Using cached pyrsistent-0.17.3-cp39-cp39-macosx_11_0_arm64.whl
Collecting strict-rfc3339
  Using cached strict_rfc3339-0.7-py3-none-any.whl
Collecting fqdn
  Using cached fqdn-1.5.1-py3-none-any.whl (9.1 kB)
Collecting rfc3987
  Using cached rfc3987-1.3.8-py2.py3-none-any.whl (13 kB)
Collecting webcolors
  Using cached webcolors-1.11.1-py3-none-any.whl (9.9 kB)
Collecting jsonpointer>1.13
  Using cached jsonpointer-2.1-py2.py3-none-any.whl (7.4 kB)
Collecting idna
  Using cached idna-3.1-py3-none-any.whl (58 kB)
Building wheels for collected packages: jsonschema
  Building wheel for jsonschema (PEP 517) ... done
  Created wheel for jsonschema: filename=jsonschema-3.2.1.dev368+ga241799-py2.py3-none-any.whl size=60962 sha256=fba1f2442ea6385bea7af86a52aa429286ec43dba32da03885458403a269fe43
  Stored in directory: /private/var/folders/4p/mgyph8ds28sgxk21xrn2f5r80000gn/T/pip-ephem-wheel-cache-ge28cbnx/wheels/26/d3/2e/c32f31d0f270381163a0eb2df7f02a3eae37bef349125f7a4a
Successfully built jsonschema
Installing collected packages: pyrsistent, attrs, webcolors, strict-rfc3339, rfc3987, jsonschema, jsonpointer, idna, fqdn
Successfully installed attrs-20.3.0 fqdn-1.5.1 idna-3.1 jsonpointer-2.1 jsonschema-3.2.1.dev368+ga241799 pyrsistent-0.17.3 rfc3987-1.3.8 strict-rfc3339-0.7 webcolors-1.11.1
perf run-test: commands[1] | mkdir /Users/julian/Development/jsonschema/.tox/perf/tmp/benchmarks/
perf run-test: commands[2] | /Users/julian/Development/jsonschema/.tox/perf/bin/python /Users/julian/Development/jsonschema/jsonschema/benchmarks/issue232.py --inherit-environ JSON_SCHEMA_TEST_SUITE --output /Users/julian/Development/jsonschema/.tox/perf/tmp/benchmarks/issue232.json
............^CBenchmark worker interrupted: exit

etc.

There's nothing more I'm running. The installation error you found was simply that the package wasn't being installed anymore due to some refactoring (and due to the benchmarks not running in CI, so they're untested :/ but which is #524).

I apologize that I don't have time to walk step by step through diagnosing your system, but both e.g. tox -e py39-format-tests and tox -e perf should work at this point (as should all the tox envs). If they don't, follow the error messages. Something seems up with your PATH, because tox isn't finding git on your system.

[AbhimanyuBahree]

I managed to get the benchmarks to work by

removing the below line in pyproproject.toml

[tool.setuptools_scm]

Apparently this causes some issues with git.

[Julian]

Setuptools scm is what will try to access git. So removing that "fixes" the issue in the sense that it hides the fact that your git isn't working :) But if it got things working for the interim that's good.

[Julian]

What do you get when you run type -a git?

[AbhimanyuBahree]

(jsonschema-oss-2) abhimanyubahree@Abhimanyus-MacBook-Pro docs % type -a git

git is /usr/local/bin/git
git is /usr/bin/git

[Julian]

And does that work even when you're not in an interactive shell but just a regular one?

[AbhimanyuBahree]

it works in terminal. I am using pycharm

[AbhimanyuBahree]

I have created a new VM and verified that the tests are working as expected.