Development Documentation Overhaul #10286

mdhaber · 2019-06-10T00:46:54Z

Closes #8353
Closes #8966
Closes #9951
Closes #10355
Partially addresses #8575, but information about using Cython as a bridge to other compiled code should be added.
development_workflow.rst and all files in the /gitwash adapted from NumPy documentation.

Most existing SciPy documentation has been preserved, but may have been moved (e.g. parts from HACKING.rst.txt have been redistributed). Please see self-review for more information.

Questions:

Why does HACKING.rst.tst exist separately in the scipy root directory? Why not just put the contents in \doc\source\hacking.rst? Just curious.
The extra slow tests are run by adding SCIPY_XSLOW=1 as an OS environment variable?
How should I credit NumPy for the Git and Development Workflow documentation?

TODO:

Fix CI failure.
link to CONTRIBUTING.rst somewhere. (I don't know that I've run across this before, so if it had a link before perhaps it was buried?)
Read through everything. I know there are a lot of issues due to things moving around.

mdhaber

Some comments to help reviewers.

mdhaber · 2019-06-10T00:48:26Z

HACKING.rst.txt

@@ -6,7 +8,7 @@ This document aims to give an overview of how to contribute to SciPy.  It
 tries to answer commonly asked questions, and provide some insight into how the
 community process works in practice.  Readers who are familiar with the SciPy
 community and are experienced Python coders may want to jump straight to the
-`git workflow`_ documentation.
+:ref:`contributor-toc`.


New contributors were not ready to jump to NumPy's git workflow documentation.
Now they are taken to a new Contributor Table of Contents (CTOC).

mdhaber · 2019-06-10T00:50:07Z

HACKING.rst.txt

@@ -114,6 +116,29 @@ other users and yourself.  Aiming for consensus in the discussion is important
 rare cases that agreement cannot be reached, the maintainers of the module
 in question can decide the issue.

+.. _license-considerations:


Moved from general FAQ (which has been completely redistributed) to here, soon after it is needed. In this future, this could be written in a narrative style that matches the rest of the document.

mdhaber · 2019-06-10T00:50:22Z

HACKING.rst.txt

@@ -194,289 +219,6 @@ project and SciPy the community, and it can always use a new pair of hands.
 The sources for the website live in their own separate repo:
 https://github.com/scipy/scipy.org

-
-Recommended development setup
-=============================


Moved to new recommended_development_setup.rst linked from CTOC. Should probably be removed and included in individual build/install instructions, as the information in here is a useful part of setting up a development environment but is not nearly complete.

mdhaber · 2019-06-10T00:54:11Z

HACKING.rst.txt

-
-SciPy structure
-===============
-


Moved to /doc/API.rst.txt

mdhaber · 2019-06-10T00:55:00Z

HACKING.rst.txt

-
-Checklist before submitting a PR
--------------------------------
-


Moved to Development Workflow

mdhaber · 2019-06-10T01:28:05Z

doc/source/dev/contributor/runtests.rst

+
+===========================
+Running SciPy Tests Locally
+===========================


mdhaber · 2019-06-10T01:28:38Z

doc/source/dev/gitwash/configure_git.rst

+
+=================
+Git configuration
+=================


This and the rest of doc/source/dev/gitwash adapted from NumPy documentation.

mdhaber · 2019-06-10T01:33:04Z

doc/source/dev/gitwash/gitwash.rst

+
+=====================
+ Git for development
+=====================


NumPy removed the Development Workflow documentation from this list (https://www.numpy.org/devdocs/dev/gitwash/index.html) so I followed suit. It feels sort of incomplete without it, though. Maybe will fill it out with the basic commands (like git commit) at some point.

mdhaber · 2019-06-10T01:33:30Z

doc/source/dev/gitwash/useful_git.rst

+
+========
+Git tips
+========


Moved from Development Workflow.

mdhaber · 2019-06-10T01:35:50Z

doc/source/dev/index.rst

+
+==========================
+SciPy Core Developer Guide
+==========================


As a new contributor, I thought I should start with the "SciPy Developer Guide". Most of this isn't really appropriate for new contributors, though. Some of it could be useful, so it's linked from the CTOC, but hopefully now this looks less like it's supposed to be starting point for new contributors.

mdhaber · 2019-06-10T01:43:58Z

CI failure presumably for:

API.rst.txt:175: WARNING: Unknown target name: "how to document".
/home/circleci/repo/doc/source/building/faq.rst:10: WARNING: Unknown target name: "virtualenv".
/home/circleci/repo/doc/source/building/faq.rst:10: WARNING: Unknown target name: "virtualenvwrapper".
/home/circleci/repo/doc/source/building/faq.rst:39: WARNING: Unknown target name: "spyder".
/home/circleci/repo/doc/source/dev/contributor/quickstart_mac.rst:59: WARNING: Title underline too short.

Installing SciPy
--------------
/home/circleci/repo/doc/source/dev/contributor/quickstart_mac.rst:59: WARNING: Title underline too short.

Installing SciPy
--------------
/home/circleci/repo/doc/source/dev/contributor/quickstart_mac.rst:5: WARNING: Duplicate explicit target name: "here".
/home/circleci/repo/doc/source/dev/gitwash/useful_git.rst:227: WARNING: Unknown target name: "scipy/master".

Will fix.

rgommers · 2019-06-10T16:52:45Z

Why does HACKING.rst.tst exist separately in the scipy root directory? Why not just put the contents in \doc\source\hacking.rst? Just curious.

It predates more extensive html docs, and that file name is (used to be) commonly used by projects for developer docs.

The extra slow tests are run by adding SCIPY_XSLOW=1 as an OS environment variable?

yes

How should I credit NumPy for the Git and Development Workflow documentation?

don't, in the docs themselves. just put it in the description of this PR. that applies to other credits too (e.g. I saw a "thanks to Yarik ..." note somewhere).

link to CONTRIBUTING.rst somewhere. (I don't know that I've run across this before, so if it had a link before perhaps it was buried?)

I don't think you should link back to that. It's a very concise intro, and is a file that GitHub recognizes so it needs to be there. It's an intro point to link to the real docs, no need to reverse that flow of information/attention.

Contributing to SciPy
Contributor Table of Contents

This naming is a bit odd. How about something like "Ways to contribute to SciPy" and "Contributor Guide"?

mdhaber · 2019-06-12T20:00:20Z

Re: "Thanks to Yarik", I didn't add that. It was in the NumPy version, but I'll remove it.
Yes, I'll change the names.

mdhaber · 2019-06-15T20:14:12Z

Changed names and fixed some errors, but somehow my changes are making Sphinx complain about 100 files not being included in any toctree. I compared to another doc build, and these warnings are the only messages not present there, so I assume they are why Makefile:115: recipe for target 'html-scipyorg' failed. Some of the warnings are for my files, but most of them aren't. How could the changes I made to the development documentation break all of this content generated from docstrings?

I will try adding all the new stuff to an invisible toctree to see if that helps. (Everything is already linked from contributor_toc.rst, which is intended to be a table of contents, so I don't want a visible toctree there.)

Update: I had to add all my new files to a visible toctree to get Sphinx to stop complaining about them. Now the warnings are all about auto-generated files:
Update: Fixed warnings about auto-generated files by rebasing on master.

mdhaber · 2019-06-16T20:24:04Z

I'm trying to improve the checklist for submitting a PR:
https://13487-1460385-gh.circle-artifacts.com/0/html-scipyorg/dev/contributor/development_workflow.html#checklist-before-submitting-a-pr

Is the wiki the place to add to the release notes? Update: I'm assuming so.
What is the easiest way to find the version number of the next release, if not running scipy.__version__ from the development version? The checklist says to get it from setup.py, but I don't see it there. Update: I'm recommending getting it from the version number of the newest release notes on the wiki.
If compiled code is not "integrated correctly via setup.py", could it still pass tests? Since there is not yet documentation for setting up compiled code correctly (except the Cython instructions I'm adding), I'm leaving this off the list.
I don't know what the "Is the new functionality added to the reference guide?" means if not "Do all public function have docstrings including examples?" I know there are some cases in which the reference guide will not be updated automatically based on the docstrings (adding a new method to an existing function?, but what are others?

mdhaber · 2019-06-20T21:57:10Z

@larsoner Hi Eric, you mentioned you'd be willing to review this. I added notes about finding a reviewer to Deciding on New Features, and I link to that in HACKING.rst.txt:

...the first step is to
discuss on the scipy-dev mailing list.  All new features, as well as changes to
existing code, are discussed and decided on there.  You can, and probably
should, already start this discussion before your code is finished. *Remember
that in order to be added to SciPy your code will need to be reviewed by
someone else, so try to find someone willing to review your work while you're
at it.*

Before reviewing, I was wondering if you had any idea what is causing the Sphinx build to fail. My comment above:

Changed names and fixed some errors...

describes the issue in more detail. I see there are also some new errors:

source/dev/core-dev/github.rst:10: WARNING: duplicate label labels-and-milestones, other instance in /home/circleci/repo/doc/source/dev/core-dev/github.rst
source/dev/core-dev/github.rst:61: WARNING: duplicate label backporting, other instance in /home/circleci/repo/doc/source/dev/core-dev/github.rst
source/dev/core-dev/deprecations.rst:4: WARNING: duplicate label deprecations, other instance in /home/circleci/repo/doc/source/dev/core-dev/deprecations.rst
source/dev/core-dev/distributing.rst:4: WARNING: duplicate label distributing-a-release, other instance in /home/circleci/repo/doc/source/dev/core-dev/distributing.rst
source/dev/core-dev/distributing.rst:88: WARNING: duplicate label supported-py-numpy-versions, other instance in /home/circleci/repo/doc/source/dev/core-dev/distributing.rst
/home/circleci/repo/doc/source/dev/core-dev/newfeatures.rst:5: WARNING: duplicate label deciding-on-new-features, other instance in /home/circleci/repo/doc/source/dev/core-dev/index.rst
/home/circleci/repo/doc/source/dev/core-dev/releasing.rst:4: WARNING: duplicate label making-a-release, other instance in /home/circleci/repo/doc/source/dev/core-dev/index.rst
/home/circleci/repo/doc/source/dev/core-dev/versioning.rst:4: WARNING: duplicate label version-numbering, other instance in /home/circleci/repo/doc/source/dev/core-dev/index.rst

They just showed up when I moved the Core Developer documentation files into their own folder. I didn't make changes to all those files, and I don't see the duplicate labels it's complaining about.

larsoner · 2019-06-21T00:31:02Z

Should be able to look tomorrow. In the meantime can you rebase on latest master to make sure it hasn't been fixed there?

mdhaber · 2019-06-21T01:42:30Z

@larsoner Genius! (or I'm a noob.)
I'm sure I rebased on master before branching, so I didn't suspect there was something wrong with it.

Now all that's left is the duplicate labels:

source/dev/core-dev/github.rst:10: WARNING: duplicate label labels-and-milestones, other instance in /home/circleci/repo/doc/source/dev/core-dev/github.rst
source/dev/core-dev/github.rst:61: WARNING: duplicate label backporting, other instance in /home/circleci/repo/doc/source/dev/core-dev/github.rst
source/dev/core-dev/deprecations.rst:4: WARNING: duplicate label deprecations, other instance in /home/circleci/repo/doc/source/dev/core-dev/deprecations.rst
source/dev/core-dev/distributing.rst:4: WARNING: duplicate label distributing-a-release, other instance in /home/circleci/repo/doc/source/dev/core-dev/distributing.rst
source/dev/core-dev/distributing.rst:88: WARNING: duplicate label supported-py-numpy-versions, other instance in /home/circleci/repo/doc/source/dev/core-dev/distributing.rst
/home/circleci/repo/doc/source/dev/core-dev/newfeatures.rst:5: WARNING: duplicate label deciding-on-new-features, other instance in /home/circleci/repo/doc/source/dev/core-dev/index.rst
/home/circleci/repo/doc/source/dev/core-dev/releasing.rst:4: WARNING: duplicate label making-a-release, other instance in /home/circleci/repo/doc/source/dev/core-dev/index.rst
/home/circleci/repo/doc/source/dev/core-dev/versioning.rst:4: WARNING: duplicate label version-numbering, other instance in /home/circleci/repo/doc/source/dev/core-dev/index.rst

That seems to be the same as this, so perhaps I need to edit exclude_patterns. Update. Yeah, that's it. Fix incoming.

fletcheaston

Nearly all of the instructions were clear, and easy to follow. The accompanying videos were also tremendously helpful in the few places where the documentation was difficult to follow.

While I was unable to add our root SciPy directory to our PYTHONPATH, there appears to be a simpler solution than creating directories and editing files. The potential solution is detailed in the comments.

fletcheaston · 2019-06-21T02:38:26Z

doc/source/dev/contributor/quickstart_mac.rst

+
+#. Enter ``conda install cython numpy matplotlib pytest spyder``. |br| Note that we're only installing SciPy's dependencies (and Spyder so we can use the IDE), but not SciPy itself.
+
+#. Our goal now is to add our root SciPy directory to the ``PYTHONPATH`` environment variable whenever this virtual environment is activated. This will ensure that Python can find the SciPy code we are trying to `import`. This requires adding a few files and folders deep inside your Anaconda installation directory. I suggest watching `the video <https://youtu.be/Faz29u5xIZc?t=35>`_ for this part. To summarize, you want to create:


I encountered some problems when adding our root development-SciPy directory to the PYTHONPATH environment variable.

First, the video tutorial differs from the written tutorial. In the video, the /conda/activate.d/env_vars.sh and /conda/deactivate.d/env_vars.sh were located within the /anaconda3/envs/scipydev/etc directory, not within the /anaconda3/envs/scipydev directory.

Second, trying either approach (with ~/env_vars.sh in either ~/scipydev or ~/scipydev/etc) failed to make our development-SciPy library available in Spyder. While the Conda installed SciPy library was available in the base environment, neither the Conda installed library nor our development library were available in the scipydev environment. Unsure if anaconda has changed this in the last few months, or if I'm missing something entirely. Two solutions were found, however.

The first is a quick workaround, and not feasible for long-term development. This includes using the Spyder app (in either the base or scipydev environment) to edit our PYTHONPATH. The downside is that Spyder seemingly uses this PYTHONPATH for all development environments, and will override the Conda installed SciPy library in the base environment.

The second solution is built into Anaconda, and appears to be the 'correct' way of doing this. Using the command "conda develop /path-to-scipy", Anaconda creates a .pth file within the ~/anaconda3/envs/scipydev/lib/pythonX.X/site-packages/ directory. On startup, Anaconda adds any newline-separated paths found in any .pth files to our environment path. Spyder will then use our development-SciPy library when in our scipydev environment, and the Conda installed SciPy library when in our base environment.

I would look into replacing this section on '...adding our root SciPy directory...' with the second solution.

More information on the "conda develop..." command can be found here.

Good find! I created the video about a year ago, maybe more. It looks like I made a typo when I (recently) typed the document based on the video.
I knew about changing PYTHONPATH from within Spyder, and you can toggle that on and off, but it's not as convenient as the solution I chose to present (once you get it set up right). Anyway, I didn't find this conda develop solution then. I'll look into it.

Can you write instructions for this and indicate which existing instructions they should replace?

Definitely!

fletcheaston · 2019-06-21T02:44:03Z

doc/source/dev/contributor/quickstart_mac.rst

+
+#. Install `Homebrew`_. Enter into the terminal |br| ``/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"`` |br| or follow the installation instructions listed on the Homebrew website. Homebrew is a package manager for macOS that will help you download `gcc`, the software we will use to compile C, C++, and Fortran code included in SciPy.
+
+#. Use Homebrew to install ``gcc`` by entering the command ``brew install gcc``.


Compared to the old documentation on getting the correct compilers (because I attempted to follow those directions very closely... for scientific reasons of course..), this is far simpler, and much harder to mess up.

Great to hear!

fletcheaston · 2019-06-21T02:58:31Z

doc/source/dev/contributor/quickstart_mac.rst

+* performing an in-place build of SciPy; and 
+* creating a virtual environment that adds this development version of SciPy to the Python path on macOS.
+
+Its companion videos `Anaconda SciPy Dev: Part I (macOS)`_ and `Anaconda SciPy Dev: Part I (macOS)`_ show many of the steps being performed. This guide may diverge slightly from the videos over time with the goal of keeping this guide the simplest, up-to-date procedure.


The companion videos are extremely helpful! Watching and pausing them as I went through each respective step helped ensure I was doing the correct things.

fletcheaston · 2019-06-21T03:01:59Z

doc/source/dev/contributor/quickstart_mac.rst

+
+#. (Optional) Check your present working directory by entering ``pwd`` at the terminal. You should be in the root ``/scipy`` directory, not in a directory ending ``/scipy/scipy``.
+
+#. Do an in-place build: enter ``python3 setup.py build_ext --inplace``. |br| This will compile the C, C++, and Fortran code that comes with SciPy. We installed ``python3`` with Anaconda. ``setup.py`` is a script in the root directory of SciPy, which is why you have to be in the SciPy root directory to call it. ``build_ext`` is a command defined in ``setup.py``, and ``--inplace`` is an option we'll use to ensure that the compiling happens in the SciPy directory you already have rather than some other folder on your computer.


Explaining what the command does and what the flags do is a good move, especially for those who haven't built SciPy or similar libraries before.

doc/source/dev/contributor/quickstart_mac.rst

doc/source/dev/contributor/contributor_toc.rst

fletcheaston

The steps for rendering documentation locally with Sphinx are fairly straightforward, and they work well.

doc/source/dev/contributor/rendering_documentation.rst

fletcheaston · 2019-06-21T19:25:07Z

doc/source/dev/contributor/rendering_documentation.rst

+
+.. note::
+
+   Changes to certain documents do not take effect when Sphinx documentation


Do we know what kinds of changes exhibit this behavior, or is it seemingly arbitrary?

I know it happens when updating documentation of solver methods, but I'm not sure if that's the only time, so I didn't speculate. If someone else has an answer it would be good to know.

mdhaber · 2019-06-22T17:40:34Z

@larsoner CI is happy and the toctrees are gone. I still need to make the modifications Fletcher suggested, but they are relatively small. Update: all set.

larsoner

Looks like a useful refactoring, I added a few sphinx-related suggestions that might make it a bit more streamlined (highlight and intersphinx use) and easier to understand for the next person to edit (rename included files as .inc and remove them from excludes).

larsoner · 2019-06-24T18:32:12Z

doc/source/conf.py

-    'dev/releasing.rst',
-    'dev/versioning.rst',
+    # these are all included directly in dev/core-dev/index.rst:
+    'dev/core-dev/decisions.rst',


Maybe the thing to do would be to rename the files with .rst.inc extension to make it clear that they are just meant to be included and not rendered/used on their own. Then they don't even need to be in this list because they won't match the doc extension (rst)

After renaming the files, removing the exclude patterns, modifying the include directives, e.g.

.. include:: decisions.rst.inc

and waiting for it to rebuild from scratch, I found /dev/core-dev/index.html to be empty. Reverting and waiting again : (
I'd prefer not to try again with this PR, if that's OK? Maybe we can make this an issue?

Saw your PR so I tried again.

The problem was that when I attempted to change the filename extension, macOS appended .rst onto it again and then automatically turned on "Hide filename extension" for that file. So my files were actually .rst.inc.rst but it only showed me .rst.inc, as though I had successfully changed the extension.

No ./ because all the files are in the same directory as index.rst.

doc/source/dev/contributor/benchmarking.rst

doc/source/dev/contributor/contributor_toc.rst

larsoner · 2019-06-24T18:40:34Z

doc/source/dev/contributor/contributor_toc.rst

+
+.. _NumPy/SciPy Testing Guidelines: https://docs.scipy.org/doc/numpy/reference/testing.html
+
+.. _numpydoc docstring guide: https://numpydoc.readthedocs.io/en/latest/format.html


After making sure numpydoc is in our intersphinx this can be

:label:`numpydoc:format`

How about :ref:numpy:howto-document?

doc/source/dev/contributor/recommended_development_setup.rst

doc/source/dev/contributor/rendering_documentation.rst

larsoner · 2019-06-24T19:06:55Z

doc/source/dev/contributor/rendering_documentation.rst

+
+#. Log into CircleCI with your GitHub account.
+#. Visit the “ci/circleci: build_docs” job page as explained above.
+#. Select the “Artifacts” tab and find ``index.html``.


All of this can be removed, given the above.

FYI any GitHub user can also access https://circleci.com/gh/scipy to see all builds, but they probably don't care in most cases.

Maybe that's new? At some point I think that URL trick was needed. Thanks.

doc/source/dev/gitwash/development_setup.rst

doc/source/dev/gitwash/dot2_dot3.rst

tylerjereddy · 2019-06-25T03:21:43Z

What is the easiest way to find the version number of the next release, if not running scipy.version from the development version? The checklist says to get it from setup.py, but I don't see it there. Update: I'm recommending getting it from the version number of the newest release notes on the wiki.

It is in setup.py (see below) for development branch, but you'd be right that a released version of SciPy probably wouldn't have the next version string around IIRC. I suppose you could also check the milestones available on GitHub, but programmatically probing the next SciPy stable release from an older stable release would indeed seem confusing to me.

 56 MAJOR = 1
 57 MINOR = 4
 58 MICRO = 0
 59 ISRELEASED = False
 60 VERSION = '%d.%d.%d' % (MAJOR, MINOR, MICRO)

mdhaber · 2019-06-27T05:05:06Z

Thanks @tylerjreddy didn't see that before. I'm adding it to the checklist.

Adapted existing NumPy documentation for SciPy. Changes are mostly trivial, but should be helpful to new contributors.

Added contributor-toc.rst; link to it from source/index.rst and HACKING.rst.txt. Remove "Recommended development setup", "SciPy structure", checklist, links, and FAQ from HACKING.rst.txt. They will be added to other sections as appropriate. HACKING.rst.txt is a good introduction to how to contribute, but it does not prepare the new reader for actually using all the rest of this information.

Added macOS Development Setup Quickstart guide Moved ecommended Development Setup from HACKING.rst.txt to new document

…inks to CTOC

New SciPy contributors are likely to think of themselves as "Developers", but this guide is geared toward Core Developers. The name made it sound like it was a good place to start learning about developing for SciPy, but it isn't. I hope this slight change helps to distinguish it from the rest of the contributor documentation.

… CTOC

…ild FAQ

mdhaber · 2019-07-01T00:33:04Z

@Kai-Striega @larsoner I've responded to comments and cleaned commit history. Can we merge before the sprint at the SciPy conference (July 13/14)?

Kai-Striega · 2019-07-01T11:03:27Z

Sounds good to me.

…

On Mon., 1 Jul. 2019, 08:33 Matt Haberland, ***@***.***> wrote: @Kai-Striega <https://github.com/Kai-Striega> @larsoner <https://github.com/larsoner> I've responded to comments and cleaned commit history. Can we merge before the sprint at the SciPy conference (July 13/14)? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10286?email_source=notifications&email_token=AHFOJNIFRBSBIC2NH6UDTBTP5FGEZA5CNFSM4HWPVCI2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY4XAMQ#issuecomment-507080754>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHFOJNMF2X43PZM4HSJQFFTP5FGEZANCNFSM4HWPVCIQ> .

pvanmulbregt · 2019-07-02T02:13:53Z

I added gh-10386 with instructions on adding a new scipy.stats distribution. I kept it separate so that it wouldn't get lost in the conversation of this PR. Similarly the process to add a function to scipy.special is sufficiently complicated that it should be written down. (There are some old minimal docs in scipy/doc/source/dev/modules.rst but developers would probably appreciate step-by-step instructions for the ~10 steps needed.)

Does this Overhaul have an obvious place for such instructions?

mdhaber · 2019-07-02T02:32:05Z

@pvanmulbregt adding those is a great idea. There is definitely a good place for them, but yeah - this PR is already massive, so let's get this merged and then you can fit these documents into the new table of contents.

@Kai-Striega @larsoner I tried to clean the commit history after responding to your comments. Look ready to go? If so, would one of you hit the green button?

Kai-Striega · 2019-07-02T02:39:41Z

@mdhaber just confirming merge or squash & merge?

mdhaber · 2019-07-02T03:28:06Z

@Kai-Striega I've tried to keep the commits logical. If you think they look OK to you, regular merge is good.

WarrenWeckesser · 2019-07-15T00:37:15Z

@mdhaber, a belated thanks for this!

Earlier in the discussion you asked about the file HACKING.rst.txt. I still have a question about it. I'd like to see that file changed to CONTRIBUTING.rst (and merge anything from the existing CONTRIBUTING.rst into that file if it is not already there). Having both is clearly redundant, and the current CONTRIBUTING.rst has just a couple paragraphs. The first section title in HACKING.rst.txt is "Ways to contribute", so the name CONTRIBUTING.rst seems the most appropriate for the merged file, and these days it just seems like the right name to use.

Is there any reason to not merge these? I can do it if there is no reason not to.

mdhaber · 2019-07-15T03:59:30Z

@WarrenWeckesser note that @rgommers wrote on June 10:

Why does HACKING.rst.tst exist separately in the scipy root directory? Why not just put the contents in \doc\source\hacking.rst? Just curious.

It predates more extensive html docs, and that file name is (used to be) commonly used by projects for developer docs.

link to CONTRIBUTING.rst somewhere. (I don't know that I've run across this before, so if it had a link before perhaps it was buried?)

I don't think you should link back to that. It's a very concise intro, and is a file that GitHub recognizes so it needs to be there. It's an intro point to link to the real docs, no need to reverse that flow of information/attention.

If it's OK to break from history and not have a HACKING.rst.txt, it sounds like putting that file's contents into CONTRIBUTING.rst would satisfy GitHub, and it makes more sense to me, too.

mdhaber commented Jun 10, 2019

View reviewed changes

mdhaber mentioned this pull request Jun 19, 2019

A 6 x 6 linprog testcase: 3 methods x sparse / dense give different results #10288

Closed

mdhaber force-pushed the dev_doc_overhaul branch 2 times, most recently from 3f08ca4 to 51ecec1 Compare June 20, 2019 20:12

mdhaber requested a review from larsoner June 20, 2019 21:39

mdhaber force-pushed the dev_doc_overhaul branch from 4c679e7 to 5bec83e Compare June 21, 2019 00:42

fletcheaston suggested changes Jun 21, 2019

View reviewed changes

larsoner reviewed Jun 21, 2019

View reviewed changes

doc/source/dev/contributor/contributor_toc.rst Outdated Show resolved Hide resolved

fletcheaston reviewed Jun 21, 2019

View reviewed changes

larsoner reviewed Jun 24, 2019

View reviewed changes

larsoner mentioned this pull request Jun 25, 2019

DOC: Add pybind11 requirment to HACKING.rst.txt #10355

Closed

mdhaber changed the title ~~WIP: Development Documentation Overhaul~~ Development Documentation Overhaul Jun 25, 2019

mdhaber added 7 commits June 27, 2019 01:13

DOC: adapted NumPy git tutorial and Development Workflow documentation

ff0b646

Adapted existing NumPy documentation for SciPy. Changes are mostly trivial, but should be helpful to new contributors.

DOC: added macOS Quickstart, moved Recommended Development Setup

c8e6a28

Added macOS Development Setup Quickstart guide Moved ecommended Development Setup from HACKING.rst.txt to new document

DOC: added Development Workflow to CTOC

960cf55

DOC: added SciPy Structure info to API.rst.txt, added Editing SciPy l…

28d1689

…inks to CTOC

DOC: added Unit Tests, Documentation, Benchmarks, and Cython links to…

9cd35a3

… CTOC

mdhaber added 13 commits June 30, 2019 14:05

DOC: simplified macOS install instructions

7302534

DOC: added Ubuntu quickstart guide

a2c6197

DOC: linked to Licensing from HACKING.rst.txt

f96e8e2

DOC: added important Git setup instructions to Development Workflow

e7d06ee

DOC: added Reviewing Pull Requests document

1b3c510

DOC: added conda install command for Sphinx to Rendering Documentation

2c1f42c

DOC: fixed Detailed Roadmap bullet formatting

4d7d793

DOC: added note to Development Workflow about version number in setup.py

e82ac89

DOC: made adjustments per @larsoner suggestions in scipy#10286

82dbc6f

DOC: removed manual way of checking PR documentation artifacts

7728329

DOC: added pybind11 build-time dependency to quickstart guides and Bu…

b244b9c

…ild FAQ

DOC: renamed Core Dev documents to remove from exclude_patterns

c4e4a75

DOC: changed apt-get to apt in Ubuntu quickstart

709d4d7

mdhaber force-pushed the dev_doc_overhaul branch from 7b1b785 to 709d4d7 Compare June 30, 2019 21:09

pvanmulbregt mentioned this pull request Jul 2, 2019

DOC: Instructions for adding a new stats distribution #10386

Closed

Kai-Striega merged commit cbf7bca into scipy:master Jul 2, 2019

This was referenced Jul 2, 2019

Documentation: how to contribute mixed-language code (build; deployment)? #8575

Closed

Add pybind11 to the other part of quickstart guides #10412

Closed

pv added this to the 1.4.0 milestone Jul 8, 2019

This was referenced Jul 21, 2019

TST, MAINT: adjustments for pytest 5.0 #10377

Merged

MAINT: backports / preparation for SciPy 1.3.1 #10496

Merged

TST: optimize: fixed linprog "disp": True bug #10498

Merged


		Checklist before submitting a PR
		--------------------------------


		#. Enter ``conda install cython numpy matplotlib pytest spyder``. \|br\| Note that we're only installing SciPy's dependencies (and Spyder so we can use the IDE), but not SciPy itself.

		#. Our goal now is to add our root SciPy directory to the ``PYTHONPATH`` environment variable whenever this virtual environment is activated. This will ensure that Python can find the SciPy code we are trying to `import`. This requires adding a few files and folders deep inside your Anaconda installation directory. I suggest watching `the video <https://youtu.be/Faz29u5xIZc?t=35>`_ for this part. To summarize, you want to create:


		#. Install `Homebrew`_. Enter into the terminal \|br\| ``/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"`` \|br\| or follow the installation instructions listed on the Homebrew website. Homebrew is a package manager for macOS that will help you download `gcc`, the software we will use to compile C, C++, and Fortran code included in SciPy.

		#. Use Homebrew to install ``gcc`` by entering the command ``brew install gcc``.


		#. (Optional) Check your present working directory by entering ``pwd`` at the terminal. You should be in the root ``/scipy`` directory, not in a directory ending ``/scipy/scipy``.

		#. Do an in-place build: enter ``python3 setup.py build_ext --inplace``. \|br\| This will compile the C, C++, and Fortran code that comes with SciPy. We installed ``python3`` with Anaconda. ``setup.py`` is a script in the root directory of SciPy, which is why you have to be in the SciPy root directory to call it. ``build_ext`` is a command defined in ``setup.py``, and ``--inplace`` is an option we'll use to ensure that the compiling happens in the SciPy directory you already have rather than some other folder on your computer.


		.. note::

		Changes to certain documents do not take effect when Sphinx documentation


		.. _NumPy/SciPy Testing Guidelines: https://docs.scipy.org/doc/numpy/reference/testing.html

		.. _numpydoc docstring guide: https://numpydoc.readthedocs.io/en/latest/format.html

Development Documentation Overhaul #10286

Development Documentation Overhaul #10286

Conversation

mdhaber commented Jun 10, 2019 • edited Loading

mdhaber left a comment

Choose a reason for hiding this comment

mdhaber Jun 10, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mdhaber Jun 10, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mdhaber Jun 10, 2019 • edited Loading

Choose a reason for hiding this comment

mdhaber commented Jun 10, 2019

rgommers commented Jun 10, 2019

mdhaber commented Jun 12, 2019

mdhaber commented Jun 15, 2019 • edited Loading

mdhaber commented Jun 16, 2019 • edited Loading

mdhaber commented Jun 20, 2019 • edited Loading

larsoner commented Jun 21, 2019

mdhaber commented Jun 21, 2019 • edited Loading

fletcheaston left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mdhaber Jun 21, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fletcheaston left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mdhaber commented Jun 22, 2019 • edited Loading

larsoner left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mdhaber Jun 27, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tylerjereddy commented Jun 25, 2019

mdhaber commented Jun 27, 2019

mdhaber commented Jul 1, 2019

Kai-Striega commented Jul 1, 2019 via email

pvanmulbregt commented Jul 2, 2019

mdhaber commented Jul 2, 2019

Kai-Striega commented Jul 2, 2019

mdhaber commented Jul 2, 2019

WarrenWeckesser commented Jul 15, 2019

mdhaber commented Jul 15, 2019

mdhaber commented Jun 10, 2019 •

edited

Loading

mdhaber Jun 10, 2019 •

edited

Loading

mdhaber Jun 10, 2019 •

edited

Loading

mdhaber Jun 10, 2019 •

edited

Loading

mdhaber commented Jun 15, 2019 •

edited

Loading

mdhaber commented Jun 16, 2019 •

edited

Loading

mdhaber commented Jun 20, 2019 •

edited

Loading

mdhaber commented Jun 21, 2019 •

edited

Loading

mdhaber Jun 21, 2019 •

edited

Loading

mdhaber commented Jun 22, 2019 •

edited

Loading

mdhaber Jun 27, 2019 •

edited

Loading