[Failing Test]: beam_PreCommit_Python_Coverage suite fails with ModuleNotFoundError: No module named 'pydantic._hypothesis_plugin' and `pip check` failures #30852

tvalentyn · 2024-04-04T17:48:14Z

What happened?

The 'coverage' suite runs some Beam unit tests in environments with different versions a particular dependency, for example we test severalversions of pyarrow or tft. The py38-tft-113 suite currently fails, likely due to a incompatible dependencies in tox environment:

============================= test session starts ==============================
Plugin: terminalreporter, Hook: pytest_sessionfinish
ModuleNotFoundError: No module named 'pydantic._hypothesis_plugin'
For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning
  config.hook.pytest_sessionfinish(
Traceback (most recent call last):
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/bin/pytest", line 10, in <module>
    sys.exit(console_main())
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/_pytest/config/__init__.py", line 192, in console_main
    code = main()
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/_pytest/config/__init__.py", line 169, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/pluggy/_callers.py", line 138, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/_pytest/main.py", line 318, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/_pytest/main.py", line 306, in wrap_session
    config.hook.pytest_sessionfinish(
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/pluggy/_callers.py", line 155, in _multicall
    teardown[0].send(outcome)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/_pytest/terminal.py", line 867, in pytest_sessionfinish
    self.config.hook.pytest_terminal_summary(
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/pluggy/_callers.py", line 181, in _multicall
    return outcome.get_result()
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/pluggy/_result.py", line 99, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/_hypothesis_pytestplugin.py", line 391, in pytest_terminal_summary
    from hypothesis.internal.observability import _WROTE_TO
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/_pytest/assertion/rewrite.py", line 186, in exec_module
    exec(co, module.__dict__)
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/hypothesis/__init__.py", line 57, in <module>
    run()
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/hypothesis/entry_points.py", line 35, in run
    hook = entry.load()
  File "/runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 208, in load
    module = import_module(match.group('module'))
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'pydantic._hypothesis_plugin'
platform linux -- Python 3.8.18, pytest-7.4.4, pluggy-1.4.0
cachedir: target/.tox-py38-tft-113/py38-tft-113/.pytest_cache
rootdir: /runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python
configfile: pytest.ini
plugins: xdist-3.5.0, timeout-2.3.1, requests-mock-1.12.1, hypothesis-6.100.0
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 129 items / 129 deselected / 0 selected

- generated xml file: /runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/pytest_py38-tft-113_no_xdist.xml -
py38-tft-113: exit 1 (86.04 seconds) /runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python> bash /runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_pytest.sh py38-tft-113 'apache_beam/ml/transforms apache_beam/examples/snippets/transforms/elementwise/mltransform_test.py' pid=63936
py38-tft-113: commands_post[0]> bash /runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh
  py38-tft-113: FAIL code 1 (420.51=setup[332.64]+cmd[0.01,0.33,1.44,0.03,86.04,0.03] seconds)
  evaluation failed :( (420.70 seconds)

Clues are in dependencies that were installed:

py38-tft-113: install_package_deps> target/.tox-py38-tft-113/py38-tft-113/bin/python /runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/bin/pip install --retries 10 --pre 'cloudpickle~=2.2.1' 'crcmod<2.0,>=1.7' 'cryptography>=41.0.2' 'dill<0.3.2,>=0.3.1.1' 'docstring-parser<1.0,>=0.15' 'fastavro<2,>=0.23.6' 'fasteners<1.0,>=0.3' 'freezegun>=0.3.12' 'grpcio!=1.48.0,<2,>=1.33.1' 'hdfs<3.0.0,>=2.1.0' 'httplib2<0.23.0,>=0.8' 'hypothesis<=7.0.0,>5.0.0' 'joblib>=1.0.1' 'js2py<1,>=0.74; python_version < "3.12"' 'jsonpickle<4.0.0,>=3.0.0' 'jsonschema<5.0.0,>=4.0.0' 'mock<6.0.0,>=1.0.1' 'numpy<1.27.0,>=1.14.3' 'objsize<0.8.0,>=0.6.1' 'orjson<4,>=3.9.7' 'packaging>=22.0' 'pandas!=1.5.0,!=1.5.1,<2.1,>=1.4.3; python_version >= "3.8"' 'pandas<2.1.0' 'parameterized<0.10.0,>=0.7.1' 'proto-plus<2,>=1.7.1' 'protobuf!=4.0.*,!=4.21.*,!=4.22.0,!=4.23.*,!=4.24.*,<4.26.0,>=3.20.3' 'psycopg2-binary<3.0.0,>=2.8.5' 'pyarrow-hotfix<1' 'pyarrow<15.0.0,>=3.0.0' 'pydot<2,>=1.2.0' 'pyhamcrest!=1.10.0,<3.0.0,>=1.9' 'pymongo<5.0.0,>=3.8.0' 'pytest-timeout<3,>=2.1.0' 'pytest-xdist<4,>=2.5.0' 'pytest<8.0,>=7.1.2' 'python-dateutil<3,>=2.8.0' 'pytz>=2018.3' 'pyyaml<7.0.0,>=3.12' 'redis<6,>=5.0.0' 'regex>=2020.6.8' 'requests-mock<2.0,>=1.7' 'requests<3.0.0,>=2.24.0' 'scikit-learn>=0.20.0' 'sqlalchemy<2.0,>=1.3' 'tenacity<9,>=8.0.0' 'testcontainers[mysql]<4.0.0,>=3.0.3' 'typing-extensions>=3.7.0' 'zstandard<1,>=0.18.0'
py38-tft-113: install_package> target/.tox-py38-tft-113/py38-tft-113/bin/python /runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/bin/pip install --retries 10 --pre --force-reinstall --no-deps /runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/.tmp/package/1/apache-beam-2.56.0.dev0.tar.gz
py38-tft-113: freeze> target/.tox-py38-tft-113/py38-tft-113/bin/python /runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/py38-tft-113/bin/pip freeze
py38-tft-113: absl-py==1.4.0,annotated-types==0.6.0,apache-beam @ file:///runner/_work/beam/beam/sdks/python/test-suites/tox/py38/build/srcs/sdks/python/target/.tox-py38-tft-113/.tmp/package/1/apache-beam-2.56.0.dev0.tar.gz#sha256=af9d8c12f56cce760e8ef3014f6edc25ec24d2a84ffa708ff45ca19e2fe866bd,astunparse==1.6.3,async-timeout==4.0.3,attrs==23.2.0,backports.zoneinfo==0.2.1,cachetools==5.3.3,certifi==2024.2.2,cffi==1.16.0,charset-normalizer==3.3.2,cloudpickle==2.2.1,crcmod==1.7,cryptography==42.0.5,Deprecated==1.2.14,deprecation==2.1.0,dill==0.3.1.1,dnspython==2.6.1,docker==7.0.0,docopt==0.6.2,docstring_parser==0.16,exceptiongroup==1.2.0,execnet==2.0.2,fastavro==1.9.4,fasteners==0.19,flatbuffers==24.3.25,freezegun==1.4.0,gast==0.4.0,google-api-core==2.18.0,google-api-python-client==1.12.11,google-apitools==0.5.31,google-auth==2.29.0,google-auth-httplib2==0.1.1,google-auth-oauthlib==1.0.0,google-cloud-aiplatform==1.46.0,google-cloud-bigquery==3.20.1,google-cloud-bigquery-storage==2.24.0,google-cloud-bigtable==2.23.0,google-cloud-core==2.4.1,google-cloud-datastore==2.19.0,google-cloud-dlp==3.16.0,google-cloud-language==2.13.3,google-cloud-pubsub==2.21.0,google-cloud-pubsublite==1.9.0,google-cloud-recommendations-ai==0.10.10,google-cloud-resource-manager==1.12.3,google-cloud-spanner==3.44.0,google-cloud-storage==2.16.0,google-cloud-videointelligence==2.13.3,google-cloud-vision==3.7.2,google-crc32c==1.5.0,google-pasta==0.2.0,google-resumable-media==2.7.0,googleapis-common-protos==1.63.0,greenlet==3.0.3,grpc-google-iam-v1==0.13.0,grpc-interceptor==0.15.4,grpcio==1.62.1,grpcio-status==1.62.1,h5py==3.10.0,hdfs==2.7.3,httplib2==0.22.0,hypothesis==6.100.0,idna==3.6,importlib_metadata==7.1.0,importlib_resources==6.4.0,iniconfig==2.0.0,joblib==1.3.2,Js2Py==0.74,jsonpickle==3.0.3,jsonschema==4.21.1,jsonschema-specifications==2023.12.1,keras==2.13.1,libclang==18.1.1,Markdown==3.6,MarkupSafe==2.1.5,mock==5.1.0,numpy==1.22.4,oauth2client==4.1.3,oauthlib==3.2.2,objsize==0.7.0,opt-einsum==3.3.0,orjson==3.10.0,overrides==7.7.0,packaging==24.0,pandas==1.5.3,parameterized==0.9.0,pkgutil_resolve_name==1.3.10,pluggy==1.4.0,proto-plus==1.24.0.dev0,protobuf==4.25.3,psycopg2-binary==2.9.9,pyarrow==6.0.1,pyarrow-hotfix==0.6,pyasn1==0.6.0,pyasn1_modules==0.4.0,pycparser==2.22,pydantic==2.0a4,pydantic_core==0.30.0,pydot==1.4.2,PyHamcrest==2.1.0,pyjsparser==2.7.1,pymongo==4.6.3,PyMySQL==1.1.0,pyparsing==3.1.2,pytest==7.4.4,pytest-timeout==2.3.1,pytest-xdist==3.5.0,python-dateutil==2.9.0.post0,pytz==2024.1,PyYAML==6.0.1,redis==5.1.0b4,referencing==0.34.0,regex==2023.12.25,requests==2.31.0,requests-mock==1.12.1,requests-oauthlib==2.0.0,rpds-py==0.18.0,rsa==4.9,scikit-learn==1.3.2,scipy==1.10.1,shapely==2.0.3,six==1.16.0,sortedcontainers==2.4.0,SQLAlchemy==1.4.52,sqlparse==0.4.4,tenacity==8.2.3,tensorboard==2.13.0,tensorboard-data-server==0.7.2,tensorflow==2.13.1,tensorflow-estimator==2.13.0,tensorflow-io-gcs-filesystem==0.34.0,tensorflow-metadata==1.13.1,tensorflow-serving-api==2.13.1,tensorflow-transform==1.13.0,termcolor==2.4.0,testcontainers==3.7.1,tfx-bsl==1.13.0,threadpoolctl==3.4.0,tomli==2.0.1,typing_extensions==4.5.0,tzlocal==5.2,uritemplate==3.0.1,urllib3==2.2.1,Werkzeug==3.0.2,wrapt==1.16.0,zipp==3.18.1,zstandard==0.22.0

Issue Failure

Failure: Test is continually failing

Issue Priority

Priority: 1 (unhealthy code / failing or flaky postcommit so we cannot be sure the product is healthy)

Issue Components

The text was updated successfully, but these errors were encountered:

tvalentyn · 2024-04-04T17:48:37Z

Sample error: https://github.com/apache/beam/actions/runs/8548288264/job/23421775819?pr=30843

tvalentyn · 2024-04-04T19:05:24Z

Not sure how pydantic comes in the picture here yet, but for 'pydantic._hypothesis_plugin' to be available, it seems that we need to have 1.0.0<pydantic<2.0.0

tvalentyn · 2024-04-04T20:10:07Z

Beam test infra installs pre-released dependencies to detect possible issues ahead of releases. The comand:

 pip install --pre "tensorflow_transform>=1.13.0,<1.14.0" apache-beam[gcp,test]

installs pydantic==2.0a4

The command

 pip install  "tensorflow_transform>=1.13.0,<1.14.0" apache-beam[gcp,test]

installs pydantic==1.10.15

The tft requirement comes from:

beam/sdks/python/tox.ini

Line 316 in 21129a4

113: tensorflow_transform>=1.13.0,<1.14.0

tvalentyn · 2024-04-04T20:23:36Z

from pipdeptree:

* google-cloud-aiplatform==1.46.0
 - proto-plus [required: >=1.22.0,<2.0.0dev, installed: 1.24.0.dev0]
 - pydantic [required: <3, installed: 2.0a4]

this seems to be an incorrect installer behavior. 2.0a4 shouln't be installed under these constraints.

not sure how that happens.

Likely what triggered the error for us was a recent release in https://pypi.org/project/google-cloud-aiplatform/#history , which added the pydantic dependency.

tvalentyn · 2024-04-04T20:24:56Z

pydantic [required: <3, installed: 2.0a4]
this seems to be an incorrect installer behavior. 2.0a4 shouln't be installed under these constraints.

Actually I misread this, it still fits the range but the chosen version is strange, there might be more constraints.

tvalentyn · 2024-04-04T20:45:27Z

even though pip selects a bizzare version for pydantic, pydantic 2, this pydantic-hypothesis plugin seems broken

from pydantic import _hypothesis_plugin fails, the more correct import from pydantic.v1 import _hypothesis_plugin also fails with recent versions of hypthesis

tvalentyn · 2024-04-04T20:47:35Z

https://stackoverflow.com/questions/71394400/how-to-block-the-hypothesis-pytest-plugin has some discussion how to disable it

tvalentyn · 2024-04-04T21:01:48Z

The hypothesis plugin might be provided by hypothesis itself, and we might need it for tests that use hypothesis.
But looks like Pydantic 2 doesn't work with hypothesis: pydantic/pydantic#5979 , and somehow pydantic (which i believe we didn't have in our dependency chain before), now intefers with pytest/hypothesis. the fact that old version of pydantic gets installed due to --pre flag may or may not be a factor.

tvalentyn · 2024-04-04T22:51:11Z

the fact that old version of pydantic gets installed due to --pre flag may or may not be a factor.

that is the factor. The pydantic-2.0a4 distribution has the following:

(py38b) :py38b$ cat lib/python3.8/site-packages/pydantic-2.0a4.dist-info/entry_points.txt
[hypothesis]
_ = pydantic._hypothesis_plugin

tvalentyn · 2024-04-05T18:51:34Z

One other problem that causes additional failures in this suite is that in the test environment we first install test environment dependencies (e.g., tensorflow), then install the Beam package. This has implication on dependency resolution, and pip fails to resolve the conflicts.

We might be able to prevent that if we install both deps in the same command, asked on tox-dev/tox#2386 (comment) if that is possible.

tvalentyn added bug failing test awaiting triage labels Apr 4, 2024

github-actions bot added python tests P1 permared labels Apr 4, 2024

tvalentyn mentioned this issue Apr 4, 2024

[Python] Add a couple quality-of-life improvemenets to testing.util.assert_that #30771

Open

tvalentyn mentioned this issue Apr 4, 2024

Adds a bound on pydantic to exclude incompatible versions in the compat test. #30863

Merged

3 tasks

tvalentyn closed this as completed in #30863 Apr 5, 2024

github-actions bot added this to the 2.56.0 Release milestone Apr 5, 2024

This was referenced Apr 9, 2024

Update a restriction for another test suite #30907

Merged

[Feature Request]: Beam compat test suites should create compatible environments that installs Beam and Test environment dependencies #30909

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Failing Test]: beam_PreCommit_Python_Coverage suite fails with ModuleNotFoundError: No module named 'pydantic._hypothesis_plugin' and `pip check` failures #30852

[Failing Test]: beam_PreCommit_Python_Coverage suite fails with ModuleNotFoundError: No module named 'pydantic._hypothesis_plugin' and `pip check` failures #30852

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024 •

edited

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024 •

edited

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 5, 2024

[Failing Test]: beam_PreCommit_Python_Coverage suite fails with ModuleNotFoundError: No module named 'pydantic._hypothesis_plugin' and pip check failures #30852

[Failing Test]: beam_PreCommit_Python_Coverage suite fails with ModuleNotFoundError: No module named 'pydantic._hypothesis_plugin' and pip check failures #30852

Comments

tvalentyn commented Apr 4, 2024

What happened?

Issue Failure

Issue Priority

Issue Components

tvalentyn commented Apr 4, 2024 • edited

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024 • edited

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 4, 2024

tvalentyn commented Apr 5, 2024

[Failing Test]: beam_PreCommit_Python_Coverage suite fails with ModuleNotFoundError: No module named 'pydantic._hypothesis_plugin' and `pip check` failures #30852

[Failing Test]: beam_PreCommit_Python_Coverage suite fails with ModuleNotFoundError: No module named 'pydantic._hypothesis_plugin' and `pip check` failures #30852

tvalentyn commented Apr 4, 2024 •

edited

tvalentyn commented Apr 4, 2024 •

edited