Issues with Setting up Aesara

[anirudhacharya]

I am running into a few issues while setting up aesara on my laptop, I am raising this issue to see if others have faced the same problem and if there is a fix, or if I am doing anything wrong while setting it up.

I am using this guide to setup and test the installation.

On the latest main branch commit( c7ff283de) following things are failing -

• After running python setup.py install I ran pytest tests/ from the aesara home directory and the following tests failed. I recently raised a PR and the CI system passed with all green, so clearly, these tests are only failing locally on my laptop. I am not sure if I am doing anything wrong while running the tests -

FAILED tests/test_breakpoint.py::TestPdbBreakpoint::test_infer_shape - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_breakpoint.py::TestPdbBreakpoint::test_grad - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_breakpoint.py::TestPdbBreakpoint::test_fprop - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_gradient.py::TestGrad::test_grad_int - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_gradient.py::test_subgraph_grad - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_gradient.py::TestConsiderConstant::test_op_removed - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_gradient.py::TestConsiderConstant::test_grad - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_gradient.py::TestZeroGrad::test_op_removed - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_gradient.py::TestZeroGrad::test_grad - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_gradient.py::TestDisconnectedGrad::test_op_removed - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_gradient.py::TestDisconnectedGrad::test_grad - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_gradient.py::test_undefined_grad_opt - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_gradient.py::test_jacobian_matrix - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_printing.py::test_pydotprint_profile - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_printing.py::test_debugprint - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_rop.py::TestRopLop::test_max - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_rop.py::TestRopLop::test_conv - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_rop.py::TestRopLop::test_flatten - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_rop.py::TestRopLop::test_sum - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/test_rop.py::TestRopLop::test_multiple_outputs - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
...
FAILED tests/tensor/test_elemwise.py::TestBroadcast::test_c - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/tensor/test_elemwise.py::TestBroadcast::test_c_inplace - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/tensor/test_elemwise.py::TestBroadcast::test_fill - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/tensor/test_elemwise.py::TestBroadcast::test_weird_strides - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/tensor/test_elemwise.py::TestBroadcast::test_same_inputs - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/tensor/test_elemwise.py::TestCAReduce::test_c_noopt - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/tensor/test_elemwise.py::TestCAReduce::test_infer_shape - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/tensor/test_elemwise.py::TestBitOpReduceGrad::test_all_grad - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
FAILED tests/tensor/test_elemwise.py::TestBitOpReduceGrad::test_any_grad - aesara.link.c.exceptions.CompileError: Compilation failed (return status=1):
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
/Users/anirudhacharya/miniconda3/envs/aesara-dev/lib/python3.10/codecs.py:309: KeyboardInterrupt
(to show a full traceback on KeyboardInterrupt use --full-trace)
================================================== 231 failed, 1879 passed, 84 skipped, 25 xfailed, 1 xpassed, 2591 warnings in 2875.66s (0:47:55) ===================================================

These are not the only failures, there are more which I did not update here, because the list would be too long.

• When I try to install the documentation dependencies with this command pip install -r requirements-rtd.txt it throws the following error - no longer an issue due to this comment

Collecting pydot2
  Using cached pydot2-1.0.33.tar.gz (19 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [1 lines of output]
      error in pydot2 setup command: use_2to3 is invalid.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Versions and main components

• Aesara version: 1.0.10
• Python version: Python 3.9.12
• Operating system: macOS Monterey version 12.0.1
• How did you install Aesara: (conda/pip) python setup.py install ( built with the latest commit c7ff283de)

[rlouf]

Can you give us specifics about you local setup? It's impossible for me to give you an answer otherwise; following the steps works on my machine.

[rlouf]

I'm also confused because the developer guide does not recommend running python setup.py install

[anirudhacharya]

@rlouf Output of python -c "import aesara; print(aesara.config)" is in this gist - https://gist.github.com/anirudhacharya/ef24e5f056690e3c9dfb820383b14804

Can you give us specifics about you local setup? It's impossible for me to give you an answer otherwise; following the steps works on my machine.

The steps I followed for my local setup -

1. conda env create -n aesara-dev -f environment.yml
2. conda activate aesara-dev
3. pip install -r requirements.txt
4. pre-commit install
5. Installed numba and jax based on this - https://aesara.readthedocs.io/en/latest/install.html#installing-aesara
   1. conda install numba
   2. conda install jax -c conda-forge
6. Build and install from source - python setup.py install. How else do you suggest I build and install from the source code? I have also tried pip install -e .

For running aesara tests I followed this - how to test that aesara works properly.

1. python -c "import numpy; numpy.test()"

====================================================================================== short test summary info =======================================================================================
ERROR array_api/tests/test_set_functions.py - hypothesis.errors.InvalidArgument: Array module numpy.array_api has no attribute __array_api_version__, which is required when inferring api_version....
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1295 deselected, 5 warnings, 1 error in 5.81s

2. python -c "import scipy; scipy.test()"

====================================================================================== short test summary info =======================================================================================
ERROR linalg/tests/test_decomp.py::test_aligned_mem - TypeError: Skip.__init__() got multiple values for argument 'reason' - maybe you meant pytest.mark.skipif?
ERROR optimize/tests/test_linprog.py::TestLinprogSimplexDefault::test_bug_10466 - TypeError: Skip.__init__() got multiple values for argument 'reason' - maybe you meant pytest.mark.skipif?
ERROR optimize/tests/test_linprog.py::TestLinprogSimplexBland::test_bug_10466 - TypeError: Skip.__init__() got multiple values for argument 'reason' - maybe you meant pytest.mark.skipif?
ERROR optimize/tests/test_linprog.py::TestLinprogSimplexNoPresolve::test_bug_10466 - TypeError: Skip.__init__() got multiple values for argument 'reason' - maybe you meant pytest.mark.skipif?
ERROR optimize/tests/test_linprog.py::TestLinprogIPDense::test_bug_10466 - TypeError: Skip.__init__() got multiple values for argument 'reason' - maybe you meant pytest.mark.skipif?
ERROR optimize/tests/test_linprog.py::TestLinprogIPSparse::test_bug_10466 - TypeError: Skip.__init__() got multiple values for argument 'reason' - maybe you meant pytest.mark.skipif?
ERROR optimize/tests/test_linprog.py::TestLinprogIPSparsePresolve::test_bug_10466 - TypeError: Skip.__init__() got multiple values for argument 'reason' - maybe you meant pytest.mark.skipif?
ERROR optimize/tests/test_linprog.py::TestLinprogRSCommon::test_bug_10466 - TypeError: Skip.__init__() got multiple values for argument 'reason' - maybe you meant pytest.mark.skipif?
ERROR optimize/tests/test_linprog.py::TestLinprogRSBland::test_bug_10466 - TypeError: Skip.__init__() got multiple values for argument 'reason' - maybe you meant pytest.mark.skipif?
ERROR optimize/tests/test_linprog.py::TestLinprogHiGHSSimplexDual::test_bug_10466 - TypeError: Skip.__init__() got multiple values for argument 'reason' - maybe you meant pytest.mark.skipif?
ERROR optimize/tests/test_linprog.py::TestLinprogHiGHSIPM::test_bug_10466 - TypeError: Skip.__init__() got multiple values for argument 'reason' - maybe you meant pytest.mark.skipif?
======================================= 32567 passed, 2077 skipped, 11134 deselected, 105 xfailed, 9 xpassed, 148179 warnings, 11 errors in 441.60s (0:07:21) ========================================

3. AESARA_FLAGS='' pytest tests/ - It fails with the log pasted in the issue description.

[anirudhacharya]

When I try to install the documentation dependencies with this command pip install -r requirements-rtd.txt it throws the following error

Collecting pydot2
  Using cached pydot2-1.0.33.tar.gz (19 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [1 lines of output]
      error in pydot2 setup command: use_2to3 is invalid.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

This is no longer an issue. I am able to build the documentation without installing pydot2. I work with Python version 3.9. And pydot library and not pydot2 is used for that version of Python.

[anirudhacharya]

The other test failure due ompilation issue still exists, I am trying to triage it.