Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

D4.13: Refactorisation of SageMath's Sphinx documentation system #87

Closed
minrk opened this issue Sep 8, 2015 · 43 comments
Closed

D4.13: Refactorisation of SageMath's Sphinx documentation system #87

minrk opened this issue Sep 8, 2015 · 43 comments

Comments

@minrk
Copy link
Contributor

@minrk minrk commented Sep 8, 2015

To do:

General

Docbuilder

  • Reduce disk space requirement: https://trac.sagemath.org/ticket/26115
  • Make the Sage documentation build without requiring make doc-clean:
  • Build Sage documentation in $SAGE_SHARE: http://trac.sagemath.org/ticket/19963
  • Profile Sphinx compilation of Sage's doc for time and memory and seek for optimizations.
    Indeed Sphinx currently takes up to 3Gb and a lot of time which feels too much.
  • Cut down the large documents in the reference manual (in particular combinat) into smaller pieces for more efficient parallel build.
  • Reproducible documentation build:
  • Allow for crosslinks between any two documents
  • Make the Sage crosslink database accessible online.
    • This would enable crosslinking from outside the Sage sources, like we do in the Sage documentation for referencing Python's documentation.
  • Use upstream's parallel build system instead of our own. Or, if the parallel build feature of upstream fails to meet our needs: generalize the parallel build feature, and push it upstream. Related: #60 and #109.
  • Generalize the multiple-documents-with-crosslinks build feature, and push it upstream

autodoc

Package upgrades

@minrk minrk added this to the D4.13 milestone Sep 8, 2015
@minrk minrk self-assigned this Nov 3, 2015
@nthiery nthiery assigned hivert and unassigned minrk Mar 22, 2016
@nthiery nthiery modified the milestones: Month 24: 2017-08-31, D4.13 Mar 22, 2016
@jdemeyer jdemeyer changed the title D4.13: Refactorisation of Sage 's Sphinx documentation system D4.13: Refactorisation of SageMath's Sphinx documentation system Jun 16, 2016
@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Mar 2, 2017

I am going to look again at this now.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Mar 2, 2017

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Mar 3, 2017

Feel free to ping again the Sphinx devs to encourage them to come again to Cernay.

I think it was @hivert who invited them, certainly not me.

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Apr 13, 2017

I am currently working on improving support for Cython functions in Sphinx: sphinx-doc/sphinx#3623.

I'm using cysignals as test case: http://cysignals.readthedocs.io/en/latest/pysignals.html#cysignals.pysignals.setsignal

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Apr 14, 2017

As a second use-case, I will improve the documentation for fpylll too: fplll/fpylll#22

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Apr 15, 2017

I reorganized the issue description a bit. Personally, I worked mostly on the "autodoc" part and I think there is already more than enough stuff to do in that part.

Maybe we should sit together with the people interested in this issue and see what can be done?

@jdemeyer jdemeyer self-assigned this Apr 24, 2017
@bpilorget
Copy link
Contributor

@bpilorget bpilorget commented Aug 16, 2017

@jdemeyer @hivert This deliverable is officially due for 31/08/2017. Can you please report on the progress you have achieved?
Thanks in advance

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 29, 2017

Would be okay to delay this deliverable?

@bpilorget bpilorget added UGe and removed PS labels Jan 12, 2018
@nthiery
Copy link
Contributor

@nthiery nthiery commented Aug 18, 2018

Hi Jeroen!
How is the report writing going? Could you use some help, maybe from @minrk embray?
Cheers,

@embray
Copy link
Collaborator

@embray embray commented Aug 20, 2018

There's so much work yet to do here; in fact I'd say the work has barely begun, and Jeroen hasn't received much support on it. I have a few specific fixes I would like to try out that I think will improve memory usage in Sage's documentation build, but it hardly constitutes "refactoring". I'm not sure what you want us to do with this. At best we can say "minor progress has been made towards improvments".

In particular Jeroen's done a lot of work "unforking" parts of Sage's docbuild--that is, reintegrating changes we've made to code from Sphinx back into upstream. But that's very technical. I suppose it can be worded in terms of making upstream contributions that are mutually beneficial to both projects.

Unfortunately some of the work I've done for D3.5 (#64) proposes forking a new bit of Sphinx that I needed to fork in order to get everything working right: https://trac.sagemath.org/ticket/25914 I did this a month ago but haven't gotten anyone to look at it yet. I believe the changes I needed to make should mostly be easy to get accepted upstream though, and it's a high priority for me to do just that, but only once my changes are accepted into Sage in the first place.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Aug 20, 2018

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 21, 2018

Working on the report...

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 21, 2018

Concerning work to do: nobody has tried to assign me to any >M36 deliverable, so I'm essentially free after august :-) I certainly do plan to continue working on this and I will write that in the report.

Also PEP 580 is partially motivated by this deliverable. Unfortunately, due to the current crisis in CPython governance, that's unlikely to be accepted (or rejected) in 2018.

@embray
Copy link
Collaborator

@embray embray commented Aug 21, 2018

Also PEP 580 is partially motivated by this deliverable. Unfortunately, due to the current crisis in CPython governance, that's unlikely to be accepted (or rejected) in 2018.

Right, I meant to mention that. That would probably be worth adding to the report.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Aug 24, 2018

I just finished reviewing the draft of report. That's a good start, thanks again!

I made some edits, and left TODO's; for some of them, you may want to ping me back if unsure what to do.

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 27, 2018

I'll first finish a draft for D4.10 and then come back here.

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 28, 2018

Quick question: do we have an "official" way to refer to ODK workshops? Should I just write "the April 2016 OpenDreamKit workshop in Cernay"?

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 28, 2018

You wrote

\TODO{Sphinx configuration for Sage packages: template in sage-sample
  and utilities in sage-package}

but I disagree. The whole point is to use a standard Sphinx setup, so we don't need "Sphinx configuration for Sage packages".

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 28, 2018

\TODO{Any relevant visual?} -> I don't think so

@nthiery
Copy link
Contributor

@nthiery nthiery commented Aug 28, 2018

@nthiery
Copy link
Contributor

@nthiery nthiery commented Aug 28, 2018

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 28, 2018

Just pushed another update. I also added a reference to T3.3 (packaging) since this is actually quite relevant for packaging too.

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 28, 2018

Nevertheless Sage will still use some custom configuration

I'm not sure about this actually.

@nthiery
Copy link
Contributor

@nthiery nthiery commented Aug 28, 2018

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 28, 2018

OK, I see your point now. But what (if anything) should we write about it?

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 28, 2018

Regarding

\TODO{Compiling the documentation to Jupyter notebooks:
  rst2ipynb, contributions to https://github.com/QuantEcon/sphinxcontrib-jupyter/
  https://trac.sagemath.org/ticket/25909;
  NT can write that if needed; I believe it's not reported elsewhere;
  question: I don't know whether it fits the scope of the deliverable}
  1. Can we claim it as part of ODK?

  2. What it does it quite related to ThebeLab. So, if we report on it, it makes sense to do that on D4.7 (#96), where we already report on ThebeLab. (EDIT: on the other hand, we could report it here to give a better impression that we actually did something for D4.13)

@nthiery
Copy link
Contributor

@nthiery nthiery commented Aug 28, 2018

@nthiery
Copy link
Contributor

@nthiery nthiery commented Aug 28, 2018

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 28, 2018

@nthiery I pushed another update adding some of the stuff we discussed today. Please review in particular the last section.

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 31, 2018

I made some final fixes to the report. For me, it is done.

@embray
Copy link
Collaborator

@embray embray commented Aug 31, 2018

I'll give it another read over. Based on my last look at it I don't think I'll have much to add on the content; I'm mostly just proof-reading.

@embray
Copy link
Collaborator

@embray embray commented Aug 31, 2018

Unfortunately, the process of
getting a PEP accepted is quite slow: at the time of writing this report, the PEP had not been
accepted (nor rejected).

I wonder if it's worth mentioning in a footnote that Guido was mostly positive about the PEP, which might have been helpful in getting it done faster if not for the unfortunate (for your PEP) timing of his retirement from the BDFL role.

@embray
Copy link
Collaborator

@embray embray commented Aug 31, 2018

In other words, the current wording leaves some doubt as to whether or not it will be accepted. I think, from what I've seen so far, it likely will be accepted, eventually, in some form or other. Even if some of the details change I don't think anyone can disagree (well, I'm sure 1 or 2 people do) that refactoring CPython's function types to have a more consistent interface is a good thing.

@jdemeyer
Copy link
Contributor

@jdemeyer jdemeyer commented Aug 31, 2018

In other words, the current wording leaves some doubt as to whether or not it will be accepted. I think, from what I've seen so far, it likely will be accepted, eventually, in some form or other.

Personally, I'm more pessimistic (or realistic?). I think that there is also a real possibility of it just being ignored, neither accepted or rejected (this is certainly the case for many Python PRs).

@nthiery
Copy link
Contributor

@nthiery nthiery commented Aug 31, 2018

Thanks Jeroen and Erik for the polishing!
I just finished reviewing and pushed some changes; most minor, and a bit of refactoring here and there. I also included the pep as appendix. Let me know if you want to change something (e.g. about the PEP) or whether I should just submit as is.

@embray
Copy link
Collaborator

@embray embray commented Aug 31, 2018

OK, I remain optimistic about it in the long term, but you're in the thick of it so I know how frustrating it's been and can easily see how you can be pessimistic (there is precedent for PEPs that looked good in principle but just stalled out indefinitely). So if you're not comfortable making any claims with confidence then let's just let it be.

@nthiery nthiery added the Submitted label Aug 31, 2018
@nthiery
Copy link
Contributor

@nthiery nthiery commented Aug 31, 2018

Submitted!
Thanks Jeroen for the long term work on this project and the report; thanks Erik for the proofreading!

@IzabelaFaguet
Copy link
Contributor

@IzabelaFaguet IzabelaFaguet commented Dec 7, 2018

Accepted by the EU on December 5th.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.