[WIP] Create initial sphinx docs

[willingc]

An initial pass at sphinx docs for JupyterHub.

To do:

• Integrate README content
• Add content from folders other then docs

[minrk]

cc @ellisonbg

I was trying out markdown docs here as an experiment, since it removes any need to build the docs, and makes the docs easier to read and write. There are, of course, disadvantages (cross-linking, etc.). Have we decided that the markdown docs don't work?

[willingc]

Min, Good to know about the experiment with markdown. Nope, no decision that markdown doesn't work. I think the ideal would be to cleanly support both markdown and restructured text in Sphinx and rendered/hosted on RTD.

As an FYI, I spent some time yesterday digging into nbconvert, pandoc, and the creating combinations of builds in the Makefile for the Notebook docs. I'm trying out combinations of conversions from ipynb files to markdown or rst to use for the examples section which is generated from ipynb files using ipy2rst (nbconvert) to go from ipynb to rst files. Right now, the converted docs look yucky. I'm planning to experiment with conversion of ipynb to markdown or rst to try to get a better looking doc when converted. So long story short, I'm going to continue to experiment on the conversions and markdown may have the edge over rst in conversion.

Let me rework this WIP PR to use the existing markdown for the source files. Any objections to me adding building the markdown files with Sphinx and hosting the JupyterHub docs on RTD?

[minrk]

No objection, really. I do like having the docs on GitHub instead of a separate site, but it probably makes sense to move to RTD.

[willingc]

Min, Just for my own clarification, when you say that you like the docs on GitHub, are you using anything in the GitHub universe, other than viewing the doc markdown files in the GitHub file view window?

[minrk]

Just viewing the docs on GitHub itself next to source. Many (typically not Python) projects do this, and I often like it better than RTD, so I thought I'd give it a try. We've used JuptyerHub as a testing ground to do a few things differently than our other projects:

1. pytest instead of nose
2. markdown docs on GitHub instead of rst/rtd (started with GitHub Wiki, but that didn't seem to go well)
3. Python 3-only

Not necessarily that we should do all of these things, but it being a small project, it seemed appropriate as an experimental place. If we have determined that the markdown docs on GitHub are causing problems (e.g. lacking coherence with the Jupyter RTD setup), then it's AOK to change that. Personally, I suspect that the build process and rst sources of sphinx docs are a significant impediment to contributing/maintaining docs, so I thought it was worth a try for something else, for a while, at least.

[willingc]

Just viewing the docs on GitHub itself next to source. Many (typically not Python) projects do this, and I often like it better than RTD, so I thought I'd give it a try.

As a developer, I agree with your preference. As a user of a project, RTD is a more user friendly experience.

We've used JuptyerHub as a testing ground to do a few things differently than our other projects:

I completely respect that. I suspect that pytest will be preferred but time will tell.

Not necessarily that we should do all of these things, but it being a small project, it seemed appropriate as an experimental place. If we have determined that the markdown docs on GitHub are causing problems (e.g. lacking coherence with the Jupyter RTD setup), then it's AOK to change that.

Sounds reasonable to me. I have used sphinx/RTD with markdown sources before on a Systers project with no real downside.

Personally, I suspect that the build process and rst sources of sphinx docs are a significant impediment to contributing/maintaining docs, so I thought it was worth a try for something else, for a while, at least.

Honestly, I think that the ipython/jupyter projects have, for documentation, had an overly complicated and inconsistent workflow for contributing to the docs. Different workflows across the projects, underdocumented when deviating from a typical Sphinx/RTD workflow (i.e. autogen_config and ipynb conversion), tightly coupled to installing the jupyter project successfully to contribute to the docs, under automated doc builds for linkchecks, spurious travis failures due to js tests timing out, as well as community negativity toward the state of the docs have all played a part in making contributing to the docs a challenging experience (which we are going to change).

Personally, I value clean code and simple, yet effective, workflows. Hang with me on the PRs and continue giving me honest feedback, and I am confident that we will get docs to work well for developers and consumers 🌅

[ellisonbg]

@minrk no particular decision was made

I am really torn.

In general I don't like Sphinx and RTD. I think it is difficult for
developers (esp compared to plain markdown on GitHub) and makes life
difficult for users (especially for projects like ours than span multiple
repos). From that perspective, I am completely +1 on having JupyterHub use
its current approach.

OTOH, as more of our subprojects have started to use RTD, I do think it is
helpful for users to have everything there. It allows us to provide a more
uniform experience with common organization, cross-subprojects index pages,
etc.

But I don't think we have made any decisions in either direction - I am not
even sure how it is that we started to go with RTD so uniformly across
subprojects. Then again, I do think that if we use Sphinx, RTD is a
reasonable choice.

Now that @willingc is working with us and focusing on documentation, I
would like to soon start a documentation working group to begin working on
these things in a systematic manner. One of the main tasks of that working
group would be to assess existing tools and figure out what things we need
to build to fill the gaps.

This doesn't answer the question of what we should do in the meantime with
JupyterHub though...

On Wed, Dec 30, 2015 at 2:35 AM, Min RK notifications@github.com wrote:

cc @ellisonbg https://github.com/ellisonbg

I was trying out markdown docs here as an experiment, since it removes any
need to build the docs, and makes the docs easier to read and write. There
are, of course, disadvantages (cross-linking, etc.). Have we decided that
the markdown docs don't work?

—
Reply to this email directly or view it on GitHub
#360 (comment).

Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger@calpoly.edu and ellisonbg@gmail.com

[minrk]

I think it makes sense to unify JupyterHub with the rest of the docs. We've got enough fragmentation as it is. I certainly like writing docs like we have here more than sphinx, but I think it's more important (now more than ever, with the repo split) that our projects be cohesive and consistent than being a little more pleasant for me to write.

Looks like is conflicting now, but +1 in general. If you want to take a stab at markdown sources in sphinx here, I would be interested in that, but I've never tried it.

[ellisonbg]

OK, sounds good. Moving forward, I would like to bring up the issue you
raise here about the pain of Sphinx. I much prefer using Markdown for our
docs as well.

On Wed, Dec 30, 2015 at 11:33 AM, Min RK notifications@github.com wrote:

I think it makes sense to unify JupyterHub with the rest of the docs.
We've got enough fragmentation as it is. I certainly like writing docs like
we have here more than sphinx, but I think it's more important (now more
than ever, with the repo split) that our projects be cohesive and
consistent than being a little more pleasant for me to write.

Looks like is conflicting now, but +1 in general. If you want to take a
stab at markdown sources in sphinx here, I would be interested in that, but
I've never tried it.

—
Reply to this email directly or view it on GitHub
#360 (comment).

Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgranger@calpoly.edu and ellisonbg@gmail.com

[ssanderson]

Random 2 cents on RTD vs GitHub docs:

As a consumer of documentation, I much prefer readthedocs. The sidebar provided by the readthedocs theme is very helpful, both for finding something I'm specifically looking for, and for browsing to find features that I wouldn't have otherwise known about. Intersphinx linking to functions and classes in other libraries is also often helpful. (I also personally think RTD just looks nicer than GitHub's rendered markdown, but that's a more subjective point.)

[ssanderson]

Also, I actually wasn't aware of most of the documentation in the /docs directory, which doesn't bode well for its discoverability for others. Anecdotally, when I'm looking at a Python repo on GitHub for the first time, I scan the repo description first, then the first few lines of the README, then CTRL+f and search for "documentation". It's very unlikely that I'd think to check the /docs directory unless I was planning on writing docs myself.

Wherever we decide to host the docs, we should probably add a prominent link on the main GitHub page. The way traitlets is set up right now is more or less optimal for me as a user:

[minrk]

@ssanderson good points about prominent links to docs in the README and/or GitHub homepage, wherever they are.

It's more important that docs be discoverable than pleasant to write, so I'm on board with the move to sphinx.

[willingc]

I'm going to close this pull request as #368 is a better implementation.