Add tutorials from PREP workshops to standard documentation

[kcrisman]

We've long needed a substantial, graded set of tutorials starting from absolute scratch geared toward new mathematical users of Sage who do not have any particular programming background. In the PREP workshops the last three years, such a series of tutorials, including eight disciplinary "quickstarts" and an intro to interacts, have taken good shape, and are ready for inclusion.

Apply attachment: trac_13260-tutorials.patch and attachment: trac_13260-images.patch.

The first patch has just the main tutorials, and the second should only have the image files except as noted below. The quickstarts have been moved to #13381.

Recommended reviewing method is to apply patches, build the documentation, and then look at it. Then one can refer to the relevant patch in review.

sage -docbuild prep html
sage -docbuild website html

is what I usually do.

CC: @jasongrout @rbeezer @dandrake @wdjoyner @sagetrac-mhampton @benjaminfjones @wypong @dcernst @sagetrac-travis @novoselt

Component: documentation

Author: Karl-Dieter Crisman

Reviewer: André Apitzsch, Andrey Novoseltsev, John Palmieri, Benjamin Jones

Merged: sage-5.3.rc0

Issue created by migration from https://trac.sagemath.org/ticket/13260

[kcrisman]

comment:1

Apply attachment: trac_13260-tutorials.patch, attachment: trac_13260-quickstarts.patch, and attachment: trac_13260-images.patch.

Recommended reviewing method is to apply all three patches, build the documentation, and then look at it. Then one can refer to the relevant patch in review.

sage -docbuild prep html
sage -docbuild website html

is what I usually do.

Putting author as me because I did the work on actually turning it all into .rst files, but the project authors in conf.py are all three of the original authors from the 2010 PREP. For positive review, I would want at least one person NOT involved as a leader in the 2010 or 2011 workshops to review this, but of course they are also welcome to give this partial review! Participants in said workshops would be more than welcome to help give positive review.

[kcrisman]

Author: Karl-Dieter Crisman

[kcrisman]

comment:2

By the way, I'm not sure why the attachment: trac_13260-tutorials.patch patch is previewing really weird, while the attachment: trac_13260-quickstarts.patch patch previews fine. However, they should (?) apply and build fine. This is a huge patch which I used hg qref -X foo -X bar to break up into a few reviewable pieces, so hopefully that didn't cause any problems.

Also, this should pass all tests, but you never know about numerical issues, so please report the results of

sage -t devel/sage/doc/en/prep

as well.

[a-andre]

comment:4

There are quite a lot of empty lines (especially before ::) are they all necessary?

Remove trailing white spaces and replace the tabs.

The maximum line length should be about 79 characters, at least for the text parts.

I haven't looked at the content yet. Maybe someone else could do this.

[kcrisman]

comment:5

Yes, the empty lines are all necessary. This is not the same as Sage code, it's the documentation.

Unless you mean that one could do

Do this thing::

    sage: do_it()

instead of

Do this thing.

::

    sage: do_it()

I'm not sure which looks nicer right now in the built doc.

There shouldn't be any trailing whitespace or tabs left.

Line length noted - this was from some other stuff, but you are right about this, of course.

[rbeezer]

comment:6

I routinely do. ::

Do this thing. ::

    sage: foo.bar()

[a-andre]

comment:7

$ grep -P "\t" trac_13260-quickstarts.patch

says that there are three tabs in quickstart.patch left

+    	Kruskal-Wallis rank sum test
+    ...   plot(x,y)	# make a plot 
+    ...   abline(lm(y ~ x))	# plot the regression line

There must be trailing white spaces otherwise patchbot wouldn't complain about them.

I have had a look at some other rst files. There is only one empty line before and one empty line after :: and not up to 3 empty lines.

[kcrisman]

comment:8

Oh, great, thanks for finding those! They are from some stuff I copied from some worksheet of mine from R examples.

I see what you mean by the extra empty lines. I can eventually fix that too, I thought you were referring to their being any empty lines at all. The sws2rst script - full props to #10637, where this is done! - adds a lot of extra new lines, because that has no effect on the final output, and it ensures that new lines are placed where they are needed.

I am only slowly looking at patchbot stuff - yikes,

ValueError: Trailing whitespace inserted on 153 non-empty lines.

and it lists them. Well, I don't use a real editor per se. I'll try to fix these. Thanks for the heads-up!

[nthiery]

comment:10

I am all +1 on that project. You may also want to have a look at

http://combinat.sagemath.org/doc/thematic_tutorials/index.html

which contains the tutorial we are using during Sage-Combinat days and that we want to get progressively merged into Sage too.

Cheers,
Nicolas

[kcrisman]

comment:11

http://combinat.sagemath.org/doc/thematic_tutorials/index.html

which contains the tutorial we are using during Sage-Combinat days and that we want to get progressively merged into Sage too.

Hmm, looks like a more expanded version of some of this stuff in the more detailed topics, and a shorter version of this in the intro versions.

I personally think that either of these sets are more appropriate than the current "tutorial", which dates from the days when the Sage user could be expected to have familiarity with the command line etc. Feel free to cc: me on some of your things as they become ready for review.

Meanwhile, I'm working on slowly removing the whitespace and justifying... a little tedious since I'm not a power editor user :8

[novoselt]

comment:12

Replying to @kcrisman:

I personally think that either of these sets are more appropriate than the current "tutorial", which dates from the days when the Sage user could be expected to have familiarity with the command line etc. Feel free to cc: me on some of your things as they become ready for review.

On a related note: is it possible to write documentation with typeset output? I.e. I still want the output to be computed automatically and checked against what's in the file, but for long polynomials and complicated/decorated arrays text output can be very difficult to read and annoying to produce... When jsMath fonts are not an issue, I don't see any reason why anyone would not want to use typeset output in notebooks, so why not use in the documentation?

[kcrisman]

comment:13

I personally think that either of these sets are more appropriate than the current "tutorial", which dates from the days when the Sage user could be expected to have familiarity with the command line etc. Feel free to cc: me on some of your things as they become ready for review.

On a related note: is it possible to write documentation with typeset output? I.e. I still want the output to be computed automatically and checked against what's in the file, but for long polynomials and complicated/decorated arrays text output can be very difficult to read and annoying to produce... When jsMath fonts are not an issue, I don't see any reason why anyone would not want to use typeset output in notebooks, so why not use in the documentation?

I don't know. That would be a good different ticket. I mean, I can make the output look nice, but I don't know whether it would doctest correctly. I only .. skip ones I really don't want tested (random output or interactive stuff).

[novoselt]

comment:14

It seems to me that one image is missing:

novoselt@sage ~/sage/devel/sage-main $ sage -docbuild prep html
sphinx-build -b html -d /home/novoselt/sage-5.2.beta1/devel/sage/doc/output/doctrees/en/prep    /home/novoselt/sage-5.2.beta1/devel/sage/doc/en/prep /home/novoselt/sage-5.2.beta1/devel/sage/doc/output/html/en/prep
Running Sphinx v1.1.2
loading pickled environment... not yet created
loading intersphinx inventory from /home/novoselt/sage-5.2.beta1/devel/sage/doc/common/python.inv...
loading intersphinx inventory from /home/novoselt/sage-5.2.beta1/devel/sage/doc/output/html/en/reference/objects.inv...
building [html]: targets for 17 source files that are out of date
updating environment: 17 added, 0 changed, 0 removed
reading sources... [100%] quickstart                                                                                
/home/novoselt/sage-5.2.beta1/devel/sage/doc/en/prep/Logging-On.rst:None: WARNING: image file not readable: media/HaveSignIn.png

and it is indeed missing in the browser.

It would be also cool to actually have plots in tutorials on plotting, but I don't know how difficult it is to implement.

[kcrisman]

comment:15

Huh, I thought I had added all of those images. I'll check it out. Rob had already told me about some missing files, so I thought I'd gotten them all.

It would be also cool to actually have plots in tutorials on plotting, but I don't know how difficult it is to implement.

Actually, I HAD all of those plots and removed them! That's what #10637 does - it keeps as many images as possible. But I got rid of them because they aren't present in any other Sage documentation.

The problem is that there is the "live" documentation as well as the "static" documentation, and putting images in one means that in the other you would get two images when you evaluated it. I didn't want to try to fight that, though maybe I should have at least kept the images!

[jhpalmieri]

comment:16

A question and a comment about the images: first, why aren't they all in the media subdirectory? Some are in Quickstarts instead. Also, more importantly, you have to make sure that the images will be included when Sage is packaged for distribution, so you have to list them in SAGE_ROOT/devel/sage/MANIFEST.in. You can either list them individually, or include entire directories (for example the media directory).

[kcrisman]

comment:17

A question and a comment about the images: first, why aren't they all in the media subdirectory? Some are in Quickstarts instead.

I put the main tutorial images in media, but kept the two for quickstarts in the quickstarts. The other docs in doc/en seems to switch back and forth. Also because I don't like typing ../filename because it's more breakable. But either is fine, if it's important.

Also, more importantly, you have to make sure that the images will be included when Sage is packaged for distribution, so you have to list them in SAGE_ROOT/devel/sage/MANIFEST.in. You can either list them individually, or include entire directories (for example the media directory).

Okay, I have heard of this file but don't really understand it. Why wouldn't I have to do anything for the .rst files, just the images? Precisely which files have to be in there? Thanks.

[jhpalmieri]

comment:18

Replying to @kcrisman:

Okay, I have heard of this file but don't really understand it. Why wouldn't I have to do anything for the .rst files, just the images? Precisely which files have to be in there? Thanks.

MANIFEST.in is documented here. The second line of the file is

recursive-include doc *.rst *.py *.inc

so all .py and .rst files are included automatically. I'm not sure why we don't do this for png files; I guess that building the documentation without the jsMath option can generate lots of png files that we don't need to include. So anyway, you have to include any png files in there, with a syntax like

include doc/en/prep/Quickstarts/Rplot001.png
include doc/en/prep/Quickstarts/heawood-graph-latex.png
recursive-include doc/en/prep/media *

probably somewhere near the bottom with the rest of the lines related to the doc directory. (The order doesn't matter, but it's nice if the file is somewhat organized.) You can test it by running sage --sdist ... and making sure that the appropriate files are included in the Sage spkg.

[kcrisman]

comment:19

/home/novoselt/sage-5.2.beta1/devel/sage/doc/en/prep/Logging-On.rst:None: WARNING: image file not readable: media/HaveSignIn.png

and it is indeed missing in the browser.

The problem was that it should be HaveSignin.png - I have fixed this.

So anyway, you have to include any png files in there, with a syntax like

Thank you so much for helping with this.

Remove trailing white spaces and replace the tabs.

Done. As far as I can tell with my modest grep skillz. Any very minor remaining ones could be fixed in a reviewer patch, of course.

Next version of patches coming up.

[kcrisman]

Attachment: trac_13260-quickstarts.patch.gz

[kcrisman]

Reviewer: André Apitzsch, Andrey Novoseltsev, John Palmieri

[kcrisman]

comment:20

Okay, all that is done. I ended up moving all the images to the media directory for MANIFEST.in ease of use.

I still don't know why the tutorials patch doesn't preview correctly. The manifest changes is in the images patch.

Patchbot, apply trac_13260-tutorials.patch, trac_13260-quickstarts.patch, and trac_13260-images.patch.

One remaining issues might be that the images showing the worksheets are quite large compared to many browser window sizes. On the other hand,

• it's a real pain to generate those images, and
• if they are too small, it will be hard to see detail.

More comments welcome.

Also, I was originally going to add log plots now that #4529 is in, but maybe that should go on another ticket. It's not like these can't be enhanced in the future :)

[kcrisman]

comment:21

Ping. It would be really good to have this available at least on the website in time for fall semester usage. Consider favors called in/promised if that motivates people ;-)

[jhpalmieri]

comment:22

Attachment: trac_13260-images-rebased.patch.gz

[jhpalmieri]

comment:23

I get a doctest failure, presumably because of the change in deprecations due to #13109:

sage -t  devel/sage-main/doc/en/prep/Symbolics-and-Basic-Plotting.rst
**********************************************************************
File "/Users/palmieri/Desktop/Sage_stuff/sage_builds/sage-5.3.beta1/devel/sage-main/doc/en/prep/Symbolics-and-Basic-Plotting.rst", line 128:
    sage: FV(5,.05)
Expected:
    doctest:...: DeprecationWarning: Substitution using function-call syntax and unnamed arguments is deprecated and will be removed from a future release of Sage; you can use named arguments instead, like EXPR(x=..., y=...)
    128.402541668774
Got:
    doctest:1172: DeprecationWarning: Substitution using function-call syntax and unnamed arguments is deprecated and will be removed from a future release of Sage; you can use named arguments instead, like EXPR(x=..., y=...)
    See http://trac.sagemath.org/5930 for details.
    128.402541668774

[benjaminfjones]

Changed reviewer from André Apitzsch, Andrey Novoseltsev, John Palmieri to André Apitzsch, Andrey Novoseltsev, John Palmieri, Benjamin Jones

[kcrisman]

comment:25

Replying to @jhpalmieri:

I get a doctest failure, presumably because of the change in deprecations due to #13109:

Hmm, good point. Modest reviewer patches welcome :)

[benjaminfjones]

comment:26

I made it through the first three intro tutorials so far. I checked all the links, they work. I found only two typos:

Logging-On.rst:

• Line 116 First sentence is redundant.

Intro-Tutorial.rst:

• Line 117 'fix me by putting in some arithmetic!'

Symbolics-and-Basic-Plotting.rst: looks good!

... more tomorrow.

[benjaminfjones]

comment:27

more comments...

Calculus.rst:

• Line 623: the command here goes off the edge of my screen. Maybe it should be wrapped manually?
• Line 737-738: bolding the first letter of each word like this doesn't seem to render properly in HTML.

Programming.rst:

• Line 272: "It would be best ..." sentence needs rewording
• Line 546: the warning here appears in the HTML docs, but the two blocks below don't, so that's confusing. I don't recall what the skip directive actually does, but in this case it seems to leave the blocks omitted.
• Line 338: I think you mean (all) here, not (any). Any can mean "forall" or "there exists" depending on context.
• Line 460: there is an extra A at the end of the sentence.

Advanced-2DPlotting.rst:

• Line 173: This plot command also goes off the right side of my browser window
• Line 373: ditto
• Line 409: typsetting mistake
• Line 524: "multiples" -> "multiple"
• Line 592: "finetune" -> "fine-tune"

Other than minor fixes, the into set of prep tutorials looks great. These will be a great addition to the official documentation. I may not have much more time to look at the quickstart tutorials, but perhaps another reviewer can.

[kcrisman]

comment:28

I made it through the first three intro tutorials so far. I checked all the links, they work. I found only two typos:

Thanks so much for doing this!

I get a doctest failure, presumably because of the change in deprecations due to #13109:

Fixed.

• Line 116 First sentence is redundant.

Ok, fixed.

• Line 117 'fix me by putting in some arithmetic!'

This is actually intentional, but I tried to make it clearer - the cell is talking to the reader, for what it's worth.

Symbolics-and-Basic-Plotting.rst: looks good!

Great.

• Line 623: the command here goes off the edge of my screen. Maybe it should be wrapped manually?

Fixed in the way the contour plot docs do it, hopefully this will work.

• Line 737-738: bolding the first letter of each word like this doesn't seem to render properly in HTML.
• Line 272: "It would be best ..." sentence needs rewording

I ended up removing some parentheses as well, since even though it's parenthetical, it just didn't work well.

• Line 546: the warning here appears in the HTML docs, but the two blocks below don't, so that's confusing. I don't recall what the skip directive actually does, but in this case it seems to leave the blocks omitted.

The skip directive is only supposed to skip it for testing. I seem to remember that they did indeed show up in the right place, but otherwise the fix is to just move the examples to the "usual" spot.

• Line 338: I think you mean (all) here, not (any). Any can mean "forall" or "there exists" depending on context.

I guess this is okay.

• Line 460: there is an extra A at the end of the sentence.

This is supposed to be an A in backticks. I changed the wording to avoid this.

• Line 173: This plot command also goes off the right side of my browser window

I tried the same thing as I did above.

• Line 373: ditto

Wow, that one really is long!

• Line 409: typsetting mistake

I assume there is no irony intended with the typo in this bullet. I don't see what the problem is right now, but I'll try to fix it.

• Line 524: "multiples" -> "multiple"

Thanks, and this was also not properly justified so I fixed that.

• Line 592: "finetune" -> "fine-tune"

If you say so.

Other than minor fixes, the into set of prep tutorials looks great. These will be a great addition to the official documentation.

Wonderful!

I may not have much more time to look at the quickstart tutorials, but perhaps another reviewer can.

I'll just separate these out for now, then, and create a new ticket for them. That will be a big enough change in the images patch that it will need another brief review just to make sure I got everything I was supposed to, and might as well check that the other changes did indeed work.

[kcrisman]

comment:29

Programming.rst:

• Line 546: the warning here appears in the HTML docs, but the two blocks below don't, so that's confusing. I don't recall what the skip directive actually does, but in this case it seems to leave the blocks omitted.

The skip directive is only supposed to skip it for testing. I seem to remember that they did indeed show up in the right place, but otherwise the fix is to just move the examples to the "usual" spot.

This was trickier. Moving the "skips" fixed the typesetting problem, but then the cells weren't skipped in doctesting. I fixed this by just adding the # not tested business.

Advanced-2DPlotting.rst:

• Line 409: typsetting mistake

I assume there is no irony intended with the typo in this bullet. I don't see what the problem is right now, but I'll try to fix it.

Ok, I see what you mean. I had single backticks instead of doubles.

---

As long as no one minds, although I removed the quickstart stuff from this ticket, I left the (two) images, because I wasn't sure how to hg rm a file from within the middle of a queue. I seriously doubt this will cause any problems.

---

For my own reference:

In the :doc:`Quickstart tutorials <quickstart>`, we assume
familiarity with the topics at the level of a student who has just
completed a course in the subject, or of a faculty member who is about
to teach it.
-   Quickstart Tutorials <quickstart>

[kcrisman]

comment:30

Here's the non-image changes in the image patch. I accidentally changed fixed these last two issues in that patch, shouldn't be a problem.

#! diff
diff --git a/MANIFEST.in b/MANIFEST.in
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -35,6 +35,7 @@
 include doc/tr/a_tour_of_sage/sin_plot.png
 graft   doc/en/reference/media
 graft   doc/en/thematic_tutorials/media
+graft   doc/en/prep/media
 prune   doc/en/reference/sage
 prune   doc/en/reference/sagenb
 prune   doc/output
diff --git a/doc/en/prep/Advanced-2DPlotting.rst b/doc/en/prep/Advanced-2DPlotting.rst
--- a/doc/en/prep/Advanced-2DPlotting.rst
+++ b/doc/en/prep/Advanced-2DPlotting.rst
@@ -408,7 +408,7 @@
 Here are the options for contour plots.
 
 - They are given as an "attribute" \- no parentheses \- of the
-  `contour_plot` object.
+  ``contour_plot`` object.
 
 - They are given as a dictionary (see :ref:`the programming tutorial
   <Advanced>`).
diff --git a/doc/en/prep/Programming.rst b/doc/en/prep/Programming.rst
--- a/doc/en/prep/Programming.rst
+++ b/doc/en/prep/Programming.rst
@@ -547,21 +547,17 @@
    There *is* a way to get around this, but it unleashes a horde of
    potential misuse. See the cells below if you are interested in this.
 
-.. skip
+   ::
+
+       sage: automatic_names(True) # not tested
+       sage: trig_expand((2*x + 4*y + sin(2*theta))^2) # not tested
+       4*(sin(theta)*cos(theta) + x + 2*y)^2
+
+   This only works in the notebook.  Now we'll turn it off.
 
    ::
 
-       sage: automatic_names(True)
-       sage: trig_expand((2*x + 4*y + sin(2*theta))^2)
-       4*(sin(theta)*cos(theta) + x + 2*y)^2
-
-   This only works in the notebook.
-
-.. skip
-
-   ::
-
-       sage: automatic_names(False)
+       sage: automatic_names(False) # not tested
 
 Another related issue is that a few names are "reserved" by Python/Sage,
 and which aren't allowed as variable names.

[kcrisman]

Based on 5.3.beta1

[kcrisman]

Attachment: trac_13260-images.patch.gz

Based on 5.3.beta1

[kcrisman]

comment:31

Attachment: trac_13260-tutorials.patch.gz

Ready for re-review. Quickstarts are now #13381, which should be ready for review.

[kcrisman]

comment:32

Patchbot, apply trac_13260-tutorials.patch and trac_13260-images.patch.

[benjaminfjones]

comment:33

Looks good to be merged!

[kcrisman]

comment:34

Thanks!

[jdemeyer]

Merged: sage-5.3.rc0