Revert to support torch>=1.11.0

[francois-rozet]

See #3239

[fritzo]

Thanks, LGTM as long as you can confirm tests still pass locally using torch==1.11.0

[francois-rozet]

I will now run (make test) the tests with different versions of PyTorch (1.11, 1.12 and 1.13). I don't plan on modifying pyro's code so I don't need to lint it or build the documentation. I therefore modified the Makefile's test recipe to test: doctest FORCE. The testing environment(s) are built with conda. Note that the funsor package is not installed by pyro-ppl[test] so I do it manually.

$ conda create -n pyro
$ conda activate pyro
(pyro) $ conda install python=3.9 jupyter lap -c conda-forge
(pyro) $ conda install pytorch=<version> torchvision cpuonly -c pytorch -c conda-forge
(pyro) $ pip install -e .[test]
(pyro) $ pip install funsor
(pyro) $ make test

[francois-rozet]

There is an issue with the Makefile' s test recipe. Running the command pytest -vx -n auto --stage unit produces the following error.

ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: -n
  inifile: /mnt/c/Users/franc/Documents/Git/pyro/setup.cfg
  rootdir: /mnt/c/Users/franc/Documents/Git/pyro

@fritzo do you know what the -n flag means?

[fritzo]

@francois-rozet you can enable the pytest -n flag via pip install pytest-xdist or more broadly pip install .[dev] or just make install

[francois-rozet]

There were still a few dependencies missing (g++ and ninja). The final setup is

$ conda create -n pyro
$ conda activate pyro
(pyro) $ conda install python=3.9 gxx jupyter lap ninja -c conda-forge
(pyro) $ conda install pytorch=<version> torchvision cpuonly -c pytorch -c conda-forge
(pyro) $ pip install -e .[test]
(pyro) $ pip install funsor pytest-xdist
(pyro) $ pytest -v -n auto --stage unit

With torch 2.0.1, it leads to

=========================================================== warnings summary ============================================================
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
  /mnt/c/Users/franc/Documents/Git/pyro/pyro/generic.py:9: DeprecationWarning: pyro.generic has moved to the pyroapi package
    warnings.warn("pyro.generic has moved to the pyroapi package", DeprecationWarning)

tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
  /mnt/c/Users/franc/Documents/Git/pyro/tests/distributions/test_stable.py:8: DeprecationWarning: Please use `IntegrationWarning` from the `scipy.integrate` namespace, the `scipy.integrate.quadpack` namespace is deprecated.
    from scipy.integrate.quadpack import IntegrationWarning

tests/ops/test_contract.py: 165 warnings
  /mnt/c/Users/franc/Documents/Git/pyro/pyro/ops/contract.py:433: DeprecationWarning: 'ubersum' is deprecated, use 'pyro.ops.contract.einsum' instead
    warnings.warn(

tests/infer/test_predictive.py::test_posterior_predictive_svi_auto_delta_guide[False]
tests/infer/test_predictive.py::test_posterior_predictive_svi_auto_delta_guide[True]
tests/infer/test_predictive.py::test_posterior_predictive_svi_auto_diag_normal_guide[False]
tests/infer/test_predictive.py::test_posterior_predictive_svi_one_hot
  /mnt/c/Users/franc/Documents/Git/pyro/pyro/infer/predictive.py:284: DeprecationWarning: The method `.get_samples` has been deprecated in favor of `.forward`.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================= 19339 passed, 595 skipped, 374 xfailed, 3 xpassed, 185 warnings in 687.84s (0:11:27) ==========================

With torch < 2.0.1 (1.11.0, 1.12.1 and 1.13.0), it leads to

=========================================================== warnings summary ============================================================pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
  /mnt/c/Users/franc/Documents/Git/pyro/pyro/generic.py:9: DeprecationWarning: pyro.generic has moved to the pyroapi package
    warnings.warn("pyro.generic has moved to the pyroapi package", DeprecationWarning)

tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
  /mnt/c/Users/franc/Documents/Git/pyro/tests/distributions/test_stable.py:8: DeprecationWarning: Please use `IntegrationWarning` from the `scipy.integrate` namespace, the `scipy.integrate.quadpack` namespace is deprecated.
    from scipy.integrate.quadpack import IntegrationWarning

tests/infer/test_predictive.py::test_posterior_predictive_svi_auto_delta_guide[False]
tests/infer/test_predictive.py::test_posterior_predictive_svi_auto_delta_guide[True]
tests/infer/test_predictive.py::test_posterior_predictive_svi_auto_diag_normal_guide[False]
tests/infer/test_predictive.py::test_posterior_predictive_svi_one_hot
  /mnt/c/Users/franc/Documents/Git/pyro/pyro/infer/predictive.py:284: DeprecationWarning: The method `.get_samples` has been deprecated in favor of `.forward`.
    warnings.warn(

tests/ops/test_contract.py: 165 warnings
  /mnt/c/Users/franc/Documents/Git/pyro/pyro/ops/contract.py:433: DeprecationWarning: 'ubersum' is deprecated, use 'pyro.ops.contract.einsum' instead
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================== short test summary info ========================================================FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_mean-Trace_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_mean-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_mean-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_inspect.py::test_get_dependencies[True] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_mean-TraceEnum_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_inspect.py::test_get_dependencies[False] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_inspect.py::test_docstring_example_1 - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_inspect.py::test_docstring_example_2 - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_mean-TraceEnum_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_inspect.py::test_docstring_example_3 - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_inspect.py::test_factor - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_median-Trace_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_inspect.py::test_discrete_obs - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_inspect.py::test_discrete - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_inspect.py::test_plate_coupling - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_median-Trace_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_inspect.py::test_plate_coupling_2 - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_inspect.py::test_plate_coupling_3 - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_valid_models.py::test_variable_clash_in_model_error[TraceGraph_ELBO] - AssertionError: Regex pattern did not match.
FAILED tests/infer/test_inspect.py::test_plate_collider - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_median-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_variable_clash_in_guide_error[TraceGraph_ELBO] - AssertionError: Regex pattern did not match.
FAILED tests/infer/test_inspect.py::test_plate_dependency - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_inspect.py::test_nested_plate_collider - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_valid_models.py::test_set_has_rsample_ok[TraceGraph_ELBO-False] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_median-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_not_has_rsample_ok[TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_valid_models.py::test_iplate_ok[TraceGraph_ELBO-full] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_iplate_ok[TraceGraph_ELBO-subsample] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_median-TraceEnum_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_iplate_variable_clash_error[TraceGraph_ELBO] - AssertionError: Regex pattern did not match.FAILED tests/infer/test_valid_models.py::test_plate_ok[TraceGraph_ELBO-full] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_ok[TraceGraph_ELBO-subsample] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_subsample_param_ok[TraceGraph_ELBO-full] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_subsample_param_ok[TraceGraph_ELBO-subsample] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_subsample_primitive_ok[TraceGraph_ELBO-full] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_subsample_primitive_ok[TraceGraph_ELBO-subsample] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_median-TraceEnum_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_param_size_mismatch_error[shape0-True-TraceGraph_ELBO-full] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_param_size_mismatch_error[shape0-True-TraceGraph_ELBO-subsample] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_param_size_mismatch_error[shape1-True-TraceGraph_ELBO-full] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_param_size_mismatch_error[shape1-True-TraceGraph_ELBO-subsample] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_param_size_mismatch_error[shape2-True-TraceGraph_ELBO-full] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_sample-Trace_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_gradient.py::test_subsample_gradient[TraceGraph_ELBO-False-full-reparam-False-unscaled] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_param_size_mismatch_error[shape2-True-TraceGraph_ELBO-subsample] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_param_size_mismatch_error[shape3-True-TraceGraph_ELBO-full] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_gradient.py::test_subsample_gradient[TraceGraph_ELBO-False-full-reparam-False-scaled] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_param_size_mismatch_error[shape3-True-TraceGraph_ELBO-subsample] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_param_size_mismatch_error[shape4-True-TraceGraph_ELBO-full] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_param_size_mismatch_error[shape4-True-TraceGraph_ELBO-subsample] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_gradient.py::test_subsample_gradient[TraceGraph_ELBO-False-full-nonreparam-unscaled] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_sample-Trace_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_no_size_ok[TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_gradient.py::test_subsample_gradient[TraceGraph_ELBO-False-full-nonreparam-scaled] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_iplate_iplate_ok[TraceGraph_ELBO-full-0] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_iplate_iplate_ok[TraceGraph_ELBO-full-inf] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_gradient.py::test_subsample_gradient[TraceGraph_ELBO-False-subsample-reparam-False-unscaled] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_iplate_iplate_ok[TraceGraph_ELBO-subsample-0] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_gradient.py::test_subsample_gradient[TraceGraph_ELBO-False-subsample-reparam-False-scaled] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_iplate_iplate_ok[TraceGraph_ELBO-subsample-inf] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_sample-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_gradient.py::test_subsample_gradient[TraceGraph_ELBO-False-subsample-nonreparam-unscaled] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_factor[AutoGaussian-Trace_ELBO] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_valid_models.py::test_iplate_iplate_swap_ok[TraceGraph_ELBO-full-0] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_discrete_helpful_error[init_to_mean-AutoGaussian] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_iplate_iplate_swap_ok[TraceGraph_ELBO-full-inf] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_gradient.py::test_subsample_gradient[TraceGraph_ELBO-False-subsample-nonreparam-scaled] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_sample-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_iplate_iplate_swap_ok[TraceGraph_ELBO-subsample-0] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_discrete_helpful_error[init_to_median-AutoGaussian] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_iplate_iplate_swap_ok[TraceGraph_ELBO-subsample-inf] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_factor[AutoGaussian-TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_discrete_helpful_error[init_to_sample-AutoGaussian] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_iplate_in_model_not_guide_ok[TraceGraph_ELBO-full] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_iplate_in_model_not_guide_ok[TraceGraph_ELBO-subsample] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_sample-TraceEnum_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_sphere_helpful_error[init_to_feasible-AutoGaussian] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_iplate_in_guide_not_model_error[False-TraceGraph_ELBO-full] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_valid_models.py::test_iplate_in_guide_not_model_error[False-TraceGraph_ELBO-subsample] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_sphere_helpful_error[init_to_mean-AutoGaussian] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_plate_iplate_ok[TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_factor[AutoGaussian-TraceEnum_ELBO] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_valid_models.py::test_iplate_plate_ok[TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_sphere_helpful_error[init_to_median-AutoGaussian] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_gradient.py::test_plate[TraceGraph_ELBO-nonreparam] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDelta-init_to_feasible-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_plate_stack_ok[sizes0-TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDelta-init_to_feasible-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_plate_stack_ok[sizes1-TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDelta-init_to_mean-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_sample-TraceEnum_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoRegressiveMessenger-init_to_feasible-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoRegressiveMessenger-init_to_feasible-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDelta-init_to_mean-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_plate_stack_ok[sizes2-TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_sphere_helpful_error[init_to_sample-AutoGaussian] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDelta-init_to_median-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_plate_stack_and_plate_ok[sizes0-TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDelta-init_to_median-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_gradient.py::test_plate_elbo_vectorized_particles[TraceGraph_ELBO-nonreparam] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoRegressiveMessenger-init_to_mean-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_plate_stack_and_plate_ok[sizes1-TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDelta-init_to_sample-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoRegressiveMessenger-init_to_mean-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_plate_stack_and_plate_ok[sizes2-TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDelta-init_to_sample-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoRegressiveMessenger-init_to_median-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoRegressiveMessenger-init_to_median-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDiagonalNormal-init_to_feasible-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_nested_plate_plate_ok[TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDiagonalNormal-init_to_feasible-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_plate_reuse_ok[TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDiagonalNormal-init_to_mean-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoRegressiveMessenger-init_to_sample-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoRegressiveMessenger-init_to_sample-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDiagonalNormal-init_to_mean-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_nested_plate_plate_dim_error_1[TraceGraph_ELBO] - AssertionError: Regex pattern did not match.
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDiagonalNormal-init_to_median-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDiagonalNormal-init_to_median-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_sphere_reparam_ok[init_to_feasible-AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDiagonalNormal-init_to_sample-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoDiagonalNormal-init_to_sample-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_nested_plate_plate_dim_error_2[TraceGraph_ELBO] - AssertionError: Regex pattern did not match.
FAILED tests/infer/test_autoguide.py::test_shapes[AutoMultivariateNormal-init_to_feasible-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoMultivariateNormal-init_to_feasible-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_nested_plate_plate_dim_error_3[TraceGraph_ELBO] - AssertionError: Regex pattern did not match.
FAILED tests/infer/test_autoguide.py::test_shapes[AutoMultivariateNormal-init_to_mean-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoMultivariateNormal-init_to_mean-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoMultivariateNormal-init_to_median-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_nested_plate_plate_dim_error_4[TraceGraph_ELBO] - AssertionError: Regex pattern did not match.
FAILED tests/infer/test_autoguide.py::test_shapes[AutoMultivariateNormal-init_to_median-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_nested_plate_plate_subsample_param_ok[TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoMultivariateNormal-init_to_sample-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoMultivariateNormal-init_to_sample-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_sphere_reparam_ok[init_to_mean-AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_valid_models.py::test_nonnested_plate_plate_ok[TraceGraph_ELBO] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_valid_models.py::test_three_indep_plate_at_different_depths_ok - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoNormal-init_to_feasible-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_enum_discrete_misuse_warning[TraceGraph_ELBO-None] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoNormal-init_to_feasible-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_enum_discrete_misuse_warning[TraceGraph_ELBO-sequential] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoNormal-init_to_mean-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_enum_discrete_misuse_warning[TraceGraph_ELBO-parallel] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoNormal-init_to_mean-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoNormal-init_to_median-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoNormal-init_to_median-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoNormal-init_to_sample-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoNormal-init_to_sample-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLowRankMultivariateNormal-init_to_feasible-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLowRankMultivariateNormal-init_to_feasible-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_sphere_reparam_ok[init_to_median-AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLowRankMultivariateNormal-init_to_mean-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLowRankMultivariateNormal-init_to_mean-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLowRankMultivariateNormal-init_to_median-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLowRankMultivariateNormal-init_to_median-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLowRankMultivariateNormal-init_to_sample-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLowRankMultivariateNormal-init_to_sample-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoIAFNormal-init_to_feasible-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoIAFNormal-init_to_feasible-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoIAFNormal-init_to_mean-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoIAFNormal-init_to_mean-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_sphere_reparam_ok[init_to_sample-AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_shapes[AutoIAFNormal-init_to_median-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoIAFNormal-init_to_median-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_dim_allocation_ok[TraceGraph_ELBO-False] - RecursionError: maximum recursion depth exceededFAILED tests/infer/test_autoguide.py::test_shapes[AutoIAFNormal-init_to_sample-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoIAFNormal-init_to_sample-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLaplaceApproximation-init_to_feasible-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLaplaceApproximation-init_to_feasible-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLaplaceApproximation-init_to_mean-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLaplaceApproximation-init_to_mean-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLaplaceApproximation-init_to_median-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLaplaceApproximation-init_to_median-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLaplaceApproximation-init_to_sample-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoLaplaceApproximation-init_to_sample-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_valid_models.py::test_dim_allocation_error[TraceGraph_ELBO-False] - AssertionError: Regex pattern did not match.
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_feasible-Trace_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_feasible-Trace_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_exact[AutoGaussian] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_feasible-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_median[JitTrace_ELBO-AutoStructured] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_feasible-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_gradient.py::test_subsample_gradient_sequential[TraceGraph_ELBO-full-nonreparam] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_median[JitTrace_ELBO-AutoGaussian] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_feasible-TraceEnum_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_feasible-TraceEnum_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_gradient.py::test_subsample_gradient_sequential[TraceGraph_ELBO-subsample-nonreparam] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_mean-Trace_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_mean-Trace_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_exact_batch[AutoGaussian] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_mean-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_mean-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_mean-TraceEnum_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_mean-TraceEnum_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_median-Trace_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_exact_tree[AutoStructured] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_median[JitTraceGraph_ELBO-AutoStructured] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_median-Trace_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_exact_tree[AutoGaussian] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_median-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_median[JitTraceGraph_ELBO-AutoGaussian] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_median-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-True-True-True-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-True-True-True-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_median-TraceEnum_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-True-True-False-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-True-True-False-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_median-TraceEnum_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-True-False-True-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-True-False-True-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-True-False-False-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_sample-Trace_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-True-False-False-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-False-True-True-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-False-True-True-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_sample-Trace_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-False-True-False-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-False-True-False-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_sample-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-False-False-True-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-False-False-True-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-False-False-False-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_sample-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[True-False-False-False-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-True-True-True-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-True-True-True-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_sample-TraceEnum_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-True-True-False-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-True-True-False-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-True-False-True-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured-init_to_sample-TraceEnum_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured_shapes-init_to_feasible-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-True-False-True-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured_shapes-init_to_feasible-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured_shapes-init_to_mean-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured_shapes-init_to_mean-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-True-False-False-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured_shapes-init_to_median-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured_shapes-init_to_median-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-True-False-False-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured_shapes-init_to_sample-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoStructured_shapes-init_to_sample-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-False-True-True-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-False-True-True-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-False-True-False-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_feasible-Trace_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-False-True-False-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-False-False-True-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-False-False-True-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_feasible-Trace_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-False-False-False-True] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_median[JitTraceEnum_ELBO-AutoStructured] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_big_model_guide_pair[False-False-False-False-False] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_feasible-TraceGraph_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_duplicates[2] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_duplicates[3] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_duplicates[7] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_feasible-TraceGraph_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_duplicates[11] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_plate_in_iplate[2] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_median[JitTraceEnum_ELBO-AutoGaussian] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_feasible-TraceEnum_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_plate_in_iplate[5] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_plate_in_iplate[9] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_feasible-TraceEnum_ELBO-10] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_iplate_in_plate[3-2] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_iplate_in_plate[3-5] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_iplate_in_plate[4-2] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_shapes[AutoGaussian-init_to_mean-Trace_ELBO-1] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_iplate_in_plate[4-5] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_plate_reuse[3-2] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_plate_reuse[3-5] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_plate_reuse[4-2] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_compute_downstream_costs.py::test_compute_downstream_costs_plate_reuse[4-5] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_conjugate_gradients.py::ConjugateChainGradientTests::test_gradients - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/autoguide/test_gaussian.py::test_structure_0[dense] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/autoguide/test_gaussian.py::test_structure_1[dense] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/autoguide/test_gaussian.py::test_structure_2[dense] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/autoguide/test_gaussian.py::test_structure_3[dense] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/autoguide/test_gaussian.py::test_structure_4[dense] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/autoguide/test_gaussian.py::test_structure_5[dense] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/autoguide/test_gaussian.py::test_structure_6[dense] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/autoguide/test_gaussian.py::test_structure_7[dense] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/autoguide/test_gaussian.py::test_structure_8[dense] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_discrete_parallel[AutoGaussian] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/autoguide/test_gaussian.py::test_broken_plates_smoke[dense] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_init_loc_fn[AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/autoguide/test_gaussian.py::test_intractable_smoke[dense] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_median_module[Trace_ELBO-auto_class11] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/autoguide/test_gaussian.py::test_pyrocov_smoke[dense-pyrocov_model-AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_median_module[TraceGraph_ELBO-auto_class11] - RecursionError: maximum recursion depth exceededFAILED tests/infer/autoguide/test_gaussian.py::test_pyrocov_smoke[dense-pyrocov_model_relaxed-AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_median_module[TraceEnum_ELBO-auto_class11] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/autoguide/test_gaussian.py::test_pyrocov_smoke[dense-pyrocov_model_plated-AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_linear_regression_smoke[Trace_ELBO-AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/autoguide/test_gaussian.py::test_pyrocov_smoke[dense-pyrocov_model_poisson-PoissonGuide] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/autoguide/test_gaussian.py::test_pyrocov_reparam[dense-pyrocov_model-AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_linear_regression_smoke[TraceGraph_ELBO-AutoGaussian] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/autoguide/test_gaussian.py::test_pyrocov_reparam[dense-pyrocov_model_relaxed-AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_linear_regression_smoke[TraceEnum_ELBO-AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/autoguide/test_gaussian.py::test_pyrocov_reparam[dense-pyrocov_model_plated-AutoGaussian] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_predictive[AutoStructured] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/test_autoguide.py::test_predictive[AutoGaussian] - RecursionError: maximum recursion depth exceeded while calling a Python object
FAILED tests/infer/autoguide/test_gaussian.py::test_pyrocov_reparam[dense-pyrocov_model_poisson-PoissonGuide] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/autoguide/test_gaussian.py::test_profile[dense-nojit] - RecursionError: maximum recursion depth exceeded in comparisonFAILED tests/infer/autoguide/test_gaussian.py::test_profile[dense-jit] - RecursionError: maximum recursion depth exceeded in comparison
FAILED tests/infer/test_autoguide.py::test_replay_plates[AutoStructured-()] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_replay_plates[AutoStructured-(6,)] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_replay_plates[AutoStructured-(5, 4)] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_replay_plates[AutoGaussian-()] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_replay_plates[AutoGaussian-(6,)] - RecursionError: maximum recursion depth exceeded
FAILED tests/infer/test_autoguide.py::test_replay_plates[AutoGaussian-(5, 4)] - RecursionError: maximum recursion depth exceeded
=================== 313 failed, 19026 passed, 595 skipped, 374 xfailed, 3 xpassed, 185 warnings in 696.52s (0:11:36) ====================

To summarize, there seems to be an infinite recursion error within TraceGraph_ELBO which gets propagated to many "auto-guide" features. Apart from that, all tests pass. I am not familiar with that part of the API, but I can check what what has changed between Pyro 1.8.4 and 1.8.5.

If necessary, an assert that checks for torch>=2.0.1 can be put inside the code such that this part of the API is only available for newer torch versions.

[francois-rozet]

Oh, interesting. The tests actually pass with torch 1.11.0 for the commit of the release (ae124d5) but not for the latest commit on dev. The tests stop working with torch 1.11.0 from commit b131230 (PR #3223).

[fritzo]

@ordabayevy do you recall the motivation behind #3223? Was it a fix to support PyTorch 2, or merely a cleanup made possible by PyTorch 2? If it's merely backwards-incompatible refactoring, we might be better off reverting it so as to continue supporting PyTorch 1.11. 🤔

[ordabayevy]

Hi @fritzo . It was a cleanup made possible by PyTorch 2.0. It can be reverted if necessary.

[francois-rozet]

Is there a way to make it compatible with 1.11 instead of fully reverting it or maybe add a condition if torch.__version__.startswith('2'): ?

[ordabayevy]

It seems the issue is that in torch==1.11.0 tree_map doesn't work with torch.Size and torch.return_types (pytorch/pytorch#75218). One solution that I'm testing right now is to add to track_provenance:

# avoid max-recursion depth error
flat_args, _ = tree_flatten(x)
if flat_args[0] is x:
    return x

[ordabayevy]

@francois-rozet can I push my proposed changes to this branch? I'm running pytest tests/infer -n auto and so far 90% of tests have passed (I can make a push after 100% tests pass). I don't think I can run all pyro tests on my computer since it would be slow.

[francois-rozet]

Sure, i'll run them on my computer when I get back home!

[ordabayevy]

I'm not sure how to push to your branch :)
Can you add this diff to your branch yourself? Feel free to modify it anyway you want.

The idea behind this code is that in an older version of PyTorch tree_flatten (used by tree_map) wasn't implemented for some of the types which leads to an infinite recursion. The code below assumes that these types are leafs that don't need to be tracked (e.g. torch.Size and torch.return_types).

I can confirm that all tests in tests/infer pass.

diff --git a/pyro/ops/provenance.py b/pyro/ops/provenance.py
index a6902a60..6ca6c0c6 100644
--- a/pyro/ops/provenance.py
+++ b/pyro/ops/provenance.py
@@ -93,6 +93,10 @@ def _track_provenance_set(x, provenance: frozenset):
 @track_provenance.register(tuple)
 @track_provenance.register(dict)
 def _track_provenance_pytree(x, provenance: frozenset):
+    # avoid max-recursion depth error for torch<=2.0
+    flat_args, _ = tree_flatten(x)
+    if flat_args[0] is x:
+        return x
     return tree_map(partial(track_provenance, provenance=provenance), x)
 
 
@@ -138,6 +142,10 @@ def _extract_provenance_set(x):
 @extract_provenance.register(tuple)
 @extract_provenance.register(dict)
 def _extract_provenance_pytree(x):
+    # avoid max-recursion depth error for torch<=2.0
+    flat_args, _ = tree_flatten(x)
+    if flat_args[0] is x:
+        return x, frozenset()
     flat_args, spec = tree_flatten(x)
     xs = []
     provenance = frozenset()

[francois-rozet]

With the fix of @ordabayevy, all the tests (pytest -v -n auto --stage unit) pass with torch 1.11, 1.12 and 1.13! Good job 💪

=================================================== warnings summary ===================================================
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
pyro/generic.py:9
  /mnt/c/Users/franc/Documents/Git/pyro/pyro/generic.py:9: DeprecationWarning: pyro.generic has moved to the pyroapi package
    warnings.warn("pyro.generic has moved to the pyroapi package", DeprecationWarning)

tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
tests/distributions/test_stable.py:8
  /mnt/c/Users/franc/Documents/Git/pyro/tests/distributions/test_stable.py:8: DeprecationWarning: Please use `IntegrationWarning` from the `scipy.integrate` namespace, the `scipy.integrate.quadpack` namespace is deprecated.
    from scipy.integrate.quadpack import IntegrationWarning

tests/ops/test_contract.py: 165 warnings
  /mnt/c/Users/franc/Documents/Git/pyro/pyro/ops/contract.py:433: DeprecationWarning: 'ubersum' is deprecated, use 'pyro.ops.contract.einsum' instead
    warnings.warn(

tests/infer/test_predictive.py::test_posterior_predictive_svi_auto_delta_guide[False]
tests/infer/test_predictive.py::test_posterior_predictive_svi_auto_delta_guide[True]
tests/infer/test_predictive.py::test_posterior_predictive_svi_auto_diag_normal_guide[False]
tests/infer/test_predictive.py::test_posterior_predictive_svi_one_hot
  /mnt/c/Users/franc/Documents/Git/pyro/pyro/infer/predictive.py:284: DeprecationWarning: The method `.get_samples` has been deprecated in favor of `.forward`.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================= 19339 passed, 595 skipped, 374 xfailed, 3 xpassed, 185 warnings in 596.35s (0:09:56) =================

[francois-rozet]

Unfortunately @ordabayevy, this fix breaks the tests for torch 2.0 😢 I think this arises when flat_args is empty (but not the TreeSpec).

[ordabayevy]

LGTM. I think it can be merged pending any further comments.

[ordabayevy]

This is a heuristic solution but we can always patch bug fixes if any bugs come up in the future. So I am happy with it.

[fritzo]

Glad to have this working, thanks @francois-rozet and @ordabayevy!

If nobody objects, I'd like to release a Pyro 1.8.6 as soon as this merges, treating the semver breakage issue of 1.8.5 as a bug. (I may try to get #3243 in to the release as well.)