Add section in the developers manual about sagenb development

[jdemeyer]

Since some Sage tickets will require patches to sagenb, there should be a section in the Sage developers manual about how to contribute to the flask-sagenb project. We really want every Sage developer to be able to contribute to sagenb. It's okay if this requires some work (like installing git), but this should be documented well.

Context: http://groups.google.com/group/sage-notebook/t/3fa33efda8db5c94

Apply: attachment: trac_12229-sagenb-developer-doc.3.patch and attachment: trac-12229-manifest.patch and attachment: trac_12229-reviewer.patch.

Depends on #11080

Dependencies: to be merged with #11080

CC: @jasongrout @kcrisman

Component: documentation

Author: Jason Grout, Karl-Dieter Crisman

Reviewer: William Stein, Jeroen Demeyer, Punarbasu Purkayastha, Karl-Dieter Crisman, Keshav Kini

Merged: sage-5.2.beta0

Issue created by migration from https://trac.sagemath.org/ticket/12229

[jasongrout]

comment:3

This is a preliminary patch, based on the gitwash project for generating documentation for github projects: https://github.com/matthew-brett/gitwash

Any general improvements to this documentation should be contributed back to the gitwash project. Also, we should probably add the paragraph or two in the Development section in the sagenb README.rst about converting between hg and git repositories: https://github.com/sagemath/sagenb/blob/master/README.rst

Ideally, sagenb would be documented totally separately in its source tree, and that documentation would somehow be available to the users.

[jasongrout]

comment:4

Okay, I added a little bit about switching to git, almost straight from the sagenb readme. I think it's ready for review, though we should eventually move this to sagenb.

[williamstein]

comment:5

After hours of work and nitpicking... positive review!

[jdemeyer]

Author: Jason Grout

[jdemeyer]

Reviewer: William Stein

[jdemeyer]

comment:7

Replying to @jasongrout:

We should eventually move this to sagenb.

Hmm, I'm not sure about this. The documentation should be obviously findable for non-sagenb developers.

[jasongrout]

comment:8

Replying to @jdemeyer:

Replying to @jasongrout:

We should eventually move this to sagenb.

Hmm, I'm not sure about this. The documentation should be obviously findable for non-sagenb developers.

Yes, that is the reason why we are not doing that right now---something needs to be figured out to make that doc easily accessible. This ticket just adds the docs to the sage developer's guide for now.

[jdemeyer]

comment:9

Currently, the commit message of this patch has some very long lines. Could you wrap these long lines? Make sure the first line (shown by hg log) makes sense by itself.

[jdemeyer]

comment:11

make doc-html-jsmath fails:

sphinx-build -b html -d /home/buildbot/build/sage/eno-1/eno_full/build/sage-5.0.beta0/devel/sage/doc/output/doctrees/en/developer   -A hide_pdf_links=1 /home/buildbot/build/sage/eno-1/eno_full/build/sage-5.0.beta0/devel/sage/doc/en/developer /home/buildbot/build/sage/eno-1/eno_full/build/sage-5.0.beta0/devel/sage/doc/output/html/en/developer
Running Sphinx v1.1.2
loading pickled environment... not yet created
building [html]: targets for 27 source files that are out of date
updating environment: 27 added, 0 changed, 0 removed
reading sources... [  3%] coding_in_cython
reading sources... [  7%] coding_in_other
reading sources... [ 11%] coding_in_python
reading sources... [ 14%] conventions
reading sources... [ 18%] disseminating_code
reading sources... [ 22%] doctesting
reading sources... [ 25%] inclusion
reading sources... [ 29%] index
reading sources... [ 33%] patching_spkgs
reading sources... [ 37%] producing_patches
reading sources... [ 40%] producing_spkgs
reading sources... [ 44%] sage_manuals
reading sources... [ 48%] sagenb/configure_git
reading sources... [ 51%] sagenb/development_workflow
reading sources... [ 55%] sagenb/following_latest
reading sources... [ 59%] sagenb/forking_hell
reading sources... [ 62%] sagenb/git_development
reading sources... [ 66%] sagenb/git_install
reading sources... [ 70%] sagenb/git_intro
reading sources... [ 74%] sagenb/git_resources
reading sources... [ 77%] sagenb/index
reading sources... [ 81%] sagenb/maintainer_workflow
reading sources... [ 85%] sagenb/patching
reading sources... [ 88%] sagenb/set_up_fork
reading sources... [ 92%] trac
reading sources... [ 96%] walk_through
reading sources... [100%] writing_code

/home/buildbot/build/sage/eno-1/eno_full/build/sage-5.0.beta0/devel/sage/doc/en/developer/sagenb/configure_git.rst:158: SEVERE: Problems with "include" directive path:
IOError: [Errno 2] No such file or directory: '../../../../sage/doc/en/developer/sagenb/links.inc'.
/home/buildbot/build/sage/eno-1/eno_full/build/sage-5.0.beta0/devel/sage/doc/en/developer/sagenb/configure_git.rst:58: ERROR: Unknown target name: "git".
/home/buildbot/build/sage/eno-1/eno_full/build/sage-5.0.beta0/devel/sage/doc/en/developer/sagenb/development_workflow.rst:415: SEVERE: Problems with "include" directive path:
IOError: [Errno 2] No such file or directory: '../../../../sage/doc/en/developer/sagenb/links.inc'.
/home/buildbot/build/sage/eno-1/eno_full/build/sage-5.0.beta0/devel/sage/doc/en/developer/sagenb/development_workflow.rst:20: ERROR: Undefined substitution referenced: "emdash".
/home/buildbot/build/sage/eno-1/eno_full/build/sage-5.0.beta0/devel/sage/doc/en/developer/sagenb/development_workflow.rst:137: ERROR: Undefined substitution referenced: "emdash".
[...many more...]

[jasongrout]

comment:12

I'm trying to reproduce this, but everything works for me. I deleted the output for the developer's guide and ran make-doc-jsmath, and there were no complaints.

Is the error reproducible on eno? That's on skynet? I might be able to log in and try to reproduce it there.

[jdemeyer]

comment:13

Replying to @jasongrout:

I'm trying to reproduce this, but everything works for me. I deleted the output for the developer's guide and ran make-doc-jsmath, and there were no complaints.

Is the error reproducible on eno? That's on skynet? I might be able to log in and try to reproduce it there.

It's reproducible on every machine on the Buildbot. But I just think of something: are you adding files which are not in devel/sage/MANIFEST.in?

[jasongrout]

comment:14

I didn't touch the MANIFEST.in. That probably explains the problem. I'll post a patch.

[jasongrout]

comment:15

The attached patch should fix the MANIFEST.in issues.

[kcrisman]

comment:44

Attachment: trac_12229-reviewer.patch.gz

Patchbot: apply trac_12229-sagenb-developer-doc.3.patch and trac-12229-manifest.patch and trac_12229-reviewer.patch

All others: I think this is now ready for review. It was very tempting to completely rewrite this but having to get something done today won out. All that requires review is the reviewer patch, including that documentation builds correctly.

[kini]

comment:45

This is good enough for now. We already have stuff lined up for the successor to #11080, so let's at least get #11080 in ASAP :)

[kini]

Changed reviewer from William Stein, Jeroen Demeyer, Punarbasu Purkayastha, Karl-Dieter Crisman to William Stein, Jeroen Demeyer, Punarbasu Purkayastha, Karl-Dieter Crisman, Keshav Kini

[kini]

comment:46

To quote my recent comment on #11080:

John Palmieri, who is sitting next to me at sage days 41, suggests that we just not package a repository with sagenb at all. What do you think? After all, sagenb is considered an upstream package at this point, and we don't package repositories for those. Plus it's using a revision control system that isn't included with sage anyway (yet). Also, due to various decisions by people over the lifetime of the repository, the repository is really large - 14 MB at maximum compression, at the current master. So removing it would slim down the sage source and binary distributions by a non-negligible amount. The docs added at #12229 already give instructions on how to clone the repository from github anyway. So why do we need to ship a repo?

[jasongrout]

comment:47

I guess the question is if we want it to be easy for sage developers to jump into sagenb development. I say yes, but I'm a bit biased.

[kini]

comment:48

Well, we always want things to be easy, so that question has a clear answer, but there are other questions too, such as whether we want to double the size of the sagenb SPKG, or whether we want to ship more binary data, etc. etc. In any case it's not really much easier to develop for sagenb if we ship a repository, because you still need to install git on your system, or install the git SPKG. After that, cloning is a single command, and how to run that command is even explained in the patches on this ticket.

[jasongrout]

comment:49

But then you need to switch the sagenb symbolic link and do another setup.py develop.

Maybe an optional sagenb-dev spkg is the solution?

[kini]

comment:50

IMO switching a symbolic link and doing setup.py develop is not an unreasonable thing to expect a potential sagenb developer to do. At the same time as making things easy for sage developers who want to hack on sagenb, I also want to make things easy for whoever is making sagenb SPKGs (currently me, so that's a kind of selfish desire :) ).

[jdemeyer]

Merged: sage-5.2.beta0

[jdemeyer]

comment:52

Unmerging this from sage-5.2 due to the serious security issue at #13270.

[jdemeyer]

Changed merged from sage-5.2.beta0 to none

[jdemeyer]

Dependencies: to be merged with #11080

[jdemeyer]

Merged: sage-5.2.beta0