DOC: Update testing documentation to dev.py

[melissawm]

Update the developer documentation to mention dev.py test instead of runtests.py.

This PR is still a draft - it was made during the Newcomer's meeting.

Reference issue

Partially addresses #17373

What does this implement/fix?

Additional information

[Kai-Striega]

@melissawm how are you going with this? It's looking pretty good to me, are there any further changes you would like to make?

[melissawm]

I think I got everything now - a few files are still mentioning runtests:

$ grep -R "runtests" --exclude-dir=build/
MANIFEST.in:include runtests.py dev.py
ci/azure-travis-template.yaml:- script: 'python -u runtests.py -g -j2 --build-only ${COVERAGE:-} ${USE_WHEEL_BUILD:-}'
ci/azure-travis-template.yaml:    python -u runtests.py -g -j2 -m ${{ parameters.test_mode }} ${COVERAGE:-} ${USE_WHEEL_BUILD:-} -- -rfEX --durations=10 --timeout=60 2>&1 | tee runtests.log
ci/azure-travis-template.yaml:    tools/validate_runtests_log.py ${{ parameters.test_mode }} < runtests.log
ci/azure-travis-template.yaml:  - script: python runtests.py -g --refguide-check
setup.py:              - `python runtests.py`              (to build and test)
setup.py:              - `python runtests.py --no-build`   (to test installed scipy)
benchmarks/README.rst:``runtests.py`` also tell ASV to use the SciPy compiled by
benchmarks/README.rst:``runtests.py``. To run the benchmarks, you do not need to install a
benchmarks/README.rst:    python runtests.py --bench sparse.Arithmetic
benchmarks/README.rst:    python runtests.py --bench-compare main sparse.Arithmetic
benchmarks/README.rst:    python runtests.py -n --bench sparse.Arithmetic
benchmarks/README.rst:and disabling BLAS/LAPACK multi-threading, as ``runtests.py`` does)::
tools/validate_runtests_log.py:In an ideal world, this should be done directly in runtests.py  using the
tools/validate_runtests_log.py:command, which (i) aborts the py process so that runtests.py does not finish,
azure-pipelines.yml:            python3.8 runtests.py -n --mode=full -- -n auto -s --junitxml=junit/test-results.xml --cov-config=.coveragerc --cov-report=xml --cov-report=html --durations=10 --timeout=60"
azure-pipelines.yml:        python runtests.py -n --mode=$(TEST_MODE) -- -n 2 --junitxml=junit/test-results.xml --durations=10 --timeout=60
runtests.py:runtests.py [OPTIONS] [-- ARGS]
runtests.py:    $ python runtests.py
runtests.py:    $ python runtests.py -s {SAMPLE_SUBMODULE}
runtests.py:    $ python runtests.py -t {SAMPLE_TEST}
runtests.py:    $ python runtests.py --ipython
runtests.py:    $ python runtests.py --python somescript.py
runtests.py:    $ python runtests.py --bench
runtests.py:    $ python runtests.py --no-build --bench signal.LTI
runtests.py:    $ gdb --args python runtests.py [...other args...]
runtests.py:    $ python runtests.py --gcov [...other args...]
runtests.py:    $ python runtests.py --lcov-html
runtests.py:        print("To build the project, run runtests.py in "
.github/workflows/macos.yml:        SCIPY_USE_PYTHRAN=`test ${{ matrix.python-version }} != 3.9; echo $?` python -u runtests.py -- --durations=10 --timeout=60
.github/workflows/linux.yml:        run: python3.8-dbg -u runtests.py -g -j2 --build-only
.github/workflows/linux.yml:          python3.8-dbg -u runtests.py -n -g -j2 -m fast -- -rfEX --durations=10 --timeout=60 2>&1 | tee runtests.log
.github/workflows/linux.yml:          python3.8-dbg tools/validate_runtests_log.py fast < runtests.log
.github/CODEOWNERS:runtests.py  @larsoner
tox.ini:    {envpython} {toxinidir}/runtests.py {posargs: --mode full}
doc/release/1.8.0-notes.rst:* `#14501 <https://github.com/scipy/scipy/issues/14501>`__: runtests.py overrides \`$PYTHONPATH\`
doc/release/1.8.0-notes.rst:* `#14844 <https://github.com/scipy/scipy/pull/14844>`__: DOC: Add refguide_check option details to runtests.rst
doc/release/1.8.0-notes.rst:* `#15144 <https://github.com/scipy/scipy/pull/15144>`__: BUG: Add missing import of 'errno' to runtests.py
doc/release/1.7.0-notes.rst:* `#13445 <https://github.com/scipy/scipy/pull/13445>`__: DOC: fix runtests guidelines.
doc/release/1.4.0-notes.rst:Documentation can now be built with ``runtests.py --doc``
doc/release/1.4.0-notes.rst:* `#10724 <https://github.com/scipy/scipy/pull/10724>`__: DEV: Add doc option to runtests.py
doc/release/0.16.0-notes.rst:run the suite locally via ``python runtests.py --bench``. For more
doc/release/1.2.0-notes.rst:* `#6498 <https://github.com/scipy/scipy/issues/6498>`__: runtests.py is missing in pypi distfile
doc/release/1.10.0-notes.rst:* `#16732 <https://github.com/scipy/scipy/pull/16732>`__: DOC: Replace runtests.py with dev.py where appropriate
doc/release/1.1.0-notes.rst:* `#8703 <https://github.com/scipy/scipy/issues/8703>`__: Q: Does runtests.py --refguide-check need env (or other) variables...
doc/release/0.14.0-notes.rst:- `#2986 <https://github.com/scipy/scipy/pull/2986>`__: BUG: runtests: chdir away from root when running tests
doc/release/0.17.0-notes.rst:``runtests.py``, the development script to build and test Scipy, now allows
doc/release/0.17.0-notes.rst:- `#4972 <https://github.com/scipy/scipy/pull/4972>`__: MAINT: impove the error message in validate_runtests_log
doc/release/0.17.0-notes.rst:- `#5491 <https://github.com/scipy/scipy/pull/5491>`__: MAINT: Expose parallel build option to runtests.py
doc/release/1.9.0-notes.rst:* `#16065 <https://github.com/scipy/scipy/issues/16065>`__: BUG: Gitpod build with \`python runtests.py\` fails; move to...
doc/release/1.9.0-notes.rst:* `#15255 <https://github.com/scipy/scipy/pull/15255>`__: MAINT: Remove \`distutils\` usage in \`runtests.py\` to fix deprecation...
doc/release/1.9.0-notes.rst:* `#15440 <https://github.com/scipy/scipy/pull/15440>`__: MAINT: Try to detect scipy path in \`runtests.py\` while not...
doc/release/1.5.0-notes.rst:* `#11601 <https://github.com/scipy/scipy/issues/11601>`__: linalg tests failing in runtests.py
doc/release/1.5.0-notes.rst:* `#11706 <https://github.com/scipy/scipy/pull/11706>`__: ENH: mypy: add \`--mypy\` option to \`runtests.py\`
doc/release/0.18.0-notes.rst:- `#5659 <https://github.com/scipy/scipy/pull/5659>`__: DOC: Define BEFORE/AFTER in runtests.py -h for bench-compare
doc/release/0.18.0-notes.rst:- `#5852 <https://github.com/scipy/scipy/pull/5852>`__: MAINT: runtests.py: ensure Ctrl-C interrupts the build
doc/release/1.6.0-notes.rst:* `#12293 <https://github.com/scipy/scipy/pull/12293>`__: MAINT: Fix a DeprecationWarning in validate_runtests_log.py.
doc/source/dev/contributor/meson.rst:same way as ``python runtests.py`` worked before. What it does is rebuild if
doc/source/dev/contributor/meson.rst:1. ``python runtests.py``

I think we shouldn't remove the runtests mention from the meson docs because they are actually talking about how the workflow has changed, and we can ignore the release notes. But I'm not sure what to do with the benchmark docs. I also assume CI should not be changed before runtests is effectively removed.

[tupui]

Thanks for the update Melissa. LGTM. Ok to not touch the meson part until we remove this. And yes AFAIK we are still missing a few things to migrate CI. For benchmarking, I think we should be able to update though (can be done in a follow up).

@rgommers any other comments before we get this in?

[rgommers]

LGTM too, thanks @melissawm