[BUG] Dependency Management Issues in Documentation Workflow

[JerryChen97]

Expected behavior

• Dependencies should be defined in a single source of truth (e.g., pyproject.toml, or setup.py)
• Documentation builds should use an isolated virtual environment
• Package versions should be consistent across different requirement files
• Dependencies should be properly isolated to prevent conflicts

Actual behavior

• Multiple dependency definition sources exist:
• • setup.py with core dependencies
• • Root requirements.txt with different version constraints
• • doc/requirements.txt with documentation-specific requirements
• No virtual environment is created in the workflow before installing packages
• Installation is performed in layers, potentially causing version conflicts:

pip3 install -r doc/requirements.txt
&& pip3 install .
&& pip3 install openfermionpyscf
&& pip3 install aiohttp fsspec h5py

• This results in a "scrambled environment" where the final state depends on installation order, see https://github.com/PennyLaneAI/pennylane/actions/runs/13838317452/job/38718984043 as a concrete example for confusing results regarding a missing deps cachetools

Additional information

The workflow does capture frozen dependencies after build with pip freeze, which helps diagnose issues
The scheduled job creates PRs to update stable dependencies, addressing symptoms rather than causes
Test files show extensive use of pytest fixtures and parametrization, suggesting complex testing requirements

Source code

# Problematic section in docs.yml
- uses: PennyLaneAI/sphinx-action@master
  with:
    docs-folder: "doc/"
    pre-build-command: >
        apt install -y graphviz
        && pip3 install -r doc/requirements.txt
        && pip3 install .
        && pip3 install openfermionpyscf
        && pip3 install aiohttp fsspec h5py
    build-command: "sphinx-build -b html . _build -W --keep-going"

Tracebacks

Running Sphinx v3.5.4

Configuration error:
There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/sphinx/config.py", line 327, in eval_config_file
    execfile_(filename, namespace)
  File "/usr/local/lib/python3.9/site-packages/sphinx/util/pycompat.py", line 88, in execfile_
    exec(code, _globals)
  File "/github/workspace/doc/conf.py", line 115, in <module>
    import pennylane
  File "/github/workspace/pennylane/__init__.py", line 98, in <module>
    from pennylane.workflow import QNode, qnode, execute
  File "/github/workspace/pennylane/workflow/__init__.py", line 46, in <module>
    from .construct_batch import construct_batch, get_transform_program
  File "/github/workspace/pennylane/workflow/construct_batch.py", line 25, in <module>
    from .qnode import QNode, _make_execution_config
  File "/github/workspace/pennylane/workflow/qnode.py", line 25, in <module>
    from cachetools import Cache, LRUCache
ModuleNotFoundError: No module named 'cachetools'

[sphinx-action] Starting sphinx-action build.
Traceback (most recent call last):
  File "/entrypoint.py", line 22, in <module>
    action.build_all_docs(github_env, [os.environ.get("INPUT_DOCS-FOLDER")])
  File "/sphinx_action/action.py", line 167, in build_all_docs
    raise RuntimeError("Build failed")
RuntimeError: Build failed
Running: apt install -y graphviz && pip3 install -r doc/requirements.txt && pip3 install . && pip3 install openfermionpyscf && pip3 install aiohttp fsspec h5py

====================================
Building docs in doc/
====================================
[sphinx-action] Running: ['sphinx-build', '-b', 'html', '.', '_build', '-W', '--keep-going', '--keep-going', '--no-color', '-w', '/tmp/sphinx-log']
[sphinx-action] Build failed with 0 warnings

System information

See https://github.com/PennyLaneAI/pennylane/actions/runs/13838317452/job/38718984043 for further information

Existing GitHub issues

• I have searched existing GitHub issues to make sure the issue does not already exist.