DOC: Creating top-level user guide section, and moving pages inside

[datapythonista]

• xref DOC: Reorganization of the documentation #24499
• tests added / passed
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

This is part of #24499, will be moving the pages in different PRs, so it's easier to discuss.

[jorisvandenbossche]

Would it be possible to show a preview? (long time ago I eg just pushed the build docs to github repo to be able to show it : https://github.com/jorisvandenbossche/example-pandas-docs)

[datapythonista]

https://datapythonista.github.io/pandas-doc-preview/

[jreback]

should make a language section with the comparison ones (4), plus let’s remove rpy entirely

[datapythonista]

I agree on both things, both are proposed in #24499.

But I'd prefer to make the changes in different PRs, so the discussions can be focussed. So far I just created the PRs for the things I find more obvious. And I think grouping the comparison pages is quite obvious, but where to have it will require some discussion (I'd probably have it under the "Getting started" session, with all the new-comers documentation).

[jreback]

is it possible to make clicking on a top-level open AND close the section? (it just will open it now)

[datapythonista]

That's a good point. I think that should be implemented in the theme (as javascript). In the new theme we've been playing with, the navigation by default doesn't even have all the sections in the left sidebar, but just the current one (e.g. https://pandas-dev.github.io/pandas-sphinx-theme/pr-datapythonista_base/user_guide/io.html)

My conclusion after trying to customize those sidebars to do what we want with sphinx, is that we should not use sphinx (#23763). So, I don't think we'll be able to have a layout and navigation we're really happy with in the short term, but hopefully something significantly better than now.

[jreback]

@jorisvandenbossche do external links need to be updated for this one?

[datapythonista]

This PR (and the similar ones) break external links. I proposed to create redirect pages in #24705.

If we agree on that approach, I'd merge this PR and the rest, as they are quite likely to conflict soon, and I'll add all the redirects for the pages moved here when implementing the system.

[datapythonista]

Added the redirects in #24715

[datapythonista]

Would be nice to have this merged before merging other documentation changes. This will conflict with most changes to the docs.

[jreback]

does this need the re-directs?

[datapythonista]

They are implemented in #24715, I don't want to have both things in the same PR.

This should be merged first.

[jreback]

I am ok with this, @jorisvandenbossche @TomAugspurger

[TomAugspurger]

A bit of introductory text here would be nice, explaining this section of the docs.

---

The User Guide covers all of pandas by topic area. Each of the subsections introduces
a topic (such as "working with missing data"), and discusses how pandas approaches the
problem, with many examples throughout.

Users brand-new to pandas should start with ... (10 min?)

Further information on any specific method can be obtained in the API reference.

---

We might also want to link to this page from a few places

• overview
• 10min
• dsintro
• basics

[datapythonista]

I 100% agree, but it makes more sense to me to work on those in separate PRs after the whole restructure. There is already another PR for a top level "Getting started" section, that probably should be the linked for the "Users brand-new..." comment.

I think moving all the pages and finding an agreement on all the structures will actually take quite a lot of time, and will require a decent amount of discussion. That's why I'm proposing smaller PRs, to be able to keep some progress. Also, I think that will let us see the progress at every step, and make us move in the right direction (kind of agile methodology).

I mentioned it somewhere, but it was already work on trying to migrate all the documentation in a one-shot (see https://pandas-dev.github.io/pandas-sphinx-theme/pr-datapythonista_base/). After all the work on that, I think the approach will fail. If anyone else thinks a single PR or bigger PRs are better, I'm more than happy to leave this for someone else.

[TomAugspurger]

Agreed that smaller PRs are preferable. I worry a bit about things like this slipping through in the smaller PRs (e.g. I don't think this index page will be touched again until things are entirely done). In the meantime it'd be nice to not lose some of the few benefits of the flat structure.

…

On Mon, Jan 14, 2019 at 8:10 AM Marc Garcia ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In doc/source/user_guide/index.rst <#24677 (comment)>: > @@ -0,0 +1,30 @@ +{{ header }} + +.. _user_guide: + +========== +User Guide +========== + I 100% agree, but it makes more sense to me to work on those in separate PRs after the whole restructure. There is already another PR for a top level "Getting started" section, that probably should be the linked for the "Users brand-new..." comment. I think moving all the pages and finding an agreement on all the structures will actually take quite a lot of time, and will require a decent amount of discussion. That's why I'm proposing smaller PRs, to be able to keep some progress. Also, I think that will let us see the progress at every step, and make us move in the right direction (kind of agile methodology). I mentioned it somewhere, but it was already work on trying to migrate all the documentation in a one-shot (see https://pandas-dev.github.io/pandas-sphinx-theme/pr-datapythonista_base/). After all the work on that, I think the approach will fail. If anyone else thinks a single PR or bigger PRs are better, I'm more than happy to leave this for someone else. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24677 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIn0J2-L3O3X5rs7nmREiQFEueUQkks5vDI_FgaJpZM4Z2lfX> . On Mon, Jan 14, 2019 at 8:10 AM Marc Garcia ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In doc/source/user_guide/index.rst <#24677 (comment)>: > @@ -0,0 +1,30 @@ +{{ header }} + +.. _user_guide: + +========== +User Guide +========== + I 100% agree, but it makes more sense to me to work on those in separate PRs after the whole restructure. There is already another PR for a top level "Getting started" section, that probably should be the linked for the "Users brand-new..." comment. I think moving all the pages and finding an agreement on all the structures will actually take quite a lot of time, and will require a decent amount of discussion. That's why I'm proposing smaller PRs, to be able to keep some progress. Also, I think that will let us see the progress at every step, and make us move in the right direction (kind of agile methodology). I mentioned it somewhere, but it was already work on trying to migrate all the documentation in a one-shot (see https://pandas-dev.github.io/pandas-sphinx-theme/pr-datapythonista_base/). After all the work on that, I think the approach will fail. If anyone else thinks a single PR or bigger PRs are better, I'm more than happy to leave this for someone else. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24677 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIn0J2-L3O3X5rs7nmREiQFEueUQkks5vDI_FgaJpZM4Z2lfX> .

[datapythonista]

Sorry, I don't think I understand. What are the benefits of the flat structure that we'd loose?

Are you proposing not to make these changes, or just adding that explanation and links from the mentioned pages you'd be happy with this?

[TomAugspurger]

Sorry for being unclear. The benefit I forgot to state is "always being able to see where you are in the (previously linear flow through the docs. But I see in, e.g., https://datapythonista.github.io/pandas-doc-preview/user_guide/indexing.html#different-choices-for-indexing that that's still the case, so feel free to ignore me.

…

On Mon, Jan 14, 2019 at 8:29 AM Marc Garcia ***@***.***> wrote: Sorry, I don't think I understand. What are the benefits of the flat structure that we'd loose? Are you proposing not to make these changes, or just adding that explanation and links from the mentioned pages you'd be happy with this? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24677 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIiSgrFd4BRUAvjyDOjnTdkVPu4PQks5vDJRggaJpZM4Z2lfX> .

[datapythonista]

The navigation with the new template is different, and customization doesn't seem trivial (you saw #23763 with my experience trying). That part won't be easy.

But that will come later, so far, the PRs I'm opening don't change more than grouping some sections together, and everything else should be the same.

[jorisvandenbossche]

This looks good to me.

I agree that it is good to do those splits in multiple PRs, but, I think it still can't hurt to already add the paragraph Tom is proposing in user_guide/index.rst (the part about explaining the scope of the User Guide, without any explicit links to the other parts, as those are being created in parallel)

[codecov]

Codecov Report

Merging #24677 into master will increase coverage by 49.49%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master   #24677       +/-   ##
===========================================
+ Coverage   42.88%   92.38%   +49.49%     
===========================================
  Files         166      166               
  Lines       52411    52310      -101     
===========================================
+ Hits        22478    48327    +25849     
+ Misses      29933     3983    -25950

Flag	Coverage Δ
#multiple	90.8% <ø> (?)
#single	43.06% <ø> (+0.17%)	⬆️

Impacted Files	Coverage Δ
pandas/core/computation/pytables.py	92.37% <0%> (+0.32%)	⬆️
pandas/io/pytables.py	92.31% <0%> (+0.92%)	⬆️
pandas/util/_test_decorators.py	90.54% <0%> (+4.05%)	⬆️
pandas/compat/__init__.py	57.91% <0%> (+8.1%)	⬆️
pandas/core/config_init.py	99.24% <0%> (+9.84%)	⬆️
pandas/core/api.py	100% <0%> (+13.33%)	⬆️
pandas/compat/numpy/__init__.py	92.85% <0%> (+14.28%)	⬆️
pandas/core/computation/common.py	85.71% <0%> (+14.28%)	⬆️
pandas/core/indexes/api.py	99% <0%> (+14.85%)	⬆️
pandas/core/reshape/util.py	100% <0%> (+15%)	⬆️
... and 123 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 597f9f3...175ef47. Read the comment docs.

[codecov]

Codecov Report

Merging #24677 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #24677      +/-   ##
==========================================
- Coverage   92.39%   92.38%   -0.01%     
==========================================
  Files         166      166              
  Lines       52411    52411              
==========================================
- Hits        48423    48422       -1     
- Misses       3988     3989       +1

Flag	Coverage Δ
#multiple	90.81% <ø> (ø)	⬆️
#single	42.89% <ø> (ø)	⬆️

Impacted Files	Coverage Δ
pandas/util/testing.py	88.04% <0%> (-0.1%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 597f9f3...97f427e. Read the comment docs.

[datapythonista]

Fixed the conflicts and added the text from Tom to the user guide index. This should be ready to be merged, and #24715 immediately after to create the redirects from the old urls to the new ones.

[TomAugspurger]

+1

…

________________________________ From: Marc Garcia <notifications@github.com> Sent: Tuesday, January 22, 2019 6:32:11 PM To: pandas-dev/pandas Cc: Tom Augspurger; Mention Subject: Re: [pandas-dev/pandas] DOC: Creating top-level user guide section, and moving pages inside (#24677) Fixed the conflicts and added the text from Tom to the user guide index. This should be ready to be merged, and #24715<#24715> immediately after to create the redirects from the old urls to the new ones. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#24677 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ABQHIsgdSedgC_U_ljS2uRsnJrdXNLXOks5vF62LgaJpZM4Z2lfX>.

[TomAugspurger]

@datapythonista are you ready on #24691 and #24678 as well? I'd like to merge these all at the same time, and then #24715.

https://github.com/pandas-dev/pandas/pull/24691/files has a merge conficlit.

[datapythonista]

My idea is to get this merge and the redirect system, and then add the redirects in those same PRs (can't create the redirects until the redirecting system is merged, and would prefer to not have the redirects of the 3 restructuring PRs in the same changes as the redirecting code).

I can have everything ready in minutes since this and the redirect system are merged if that makes sense.

[TomAugspurger]

Yeah by "same time" I meant same dayish :) Feel free to merge the three restructuring PRs when you're ready, and I'll take another look at the redirect PR, but I think that looked good as well.