Jupyter-book demo project

[jstac]

We aim to complete and deploy a relatively fully featured demo project exploiting EBP tools and built using the CLI.

This will supplement existing repos https://github.com/ExecutableBookProject/myst-nb.example-project and https://github.com/ExecutableBookProject/myst-parser.example-project.

The CLI documentation will show in simple steps how to

• Download the project (perhaps w/o assuming familiarity with git)
• Build the project locally
• View the html build locally (and perhaps the PDF, with comments that high quality PDF is coming)
• Edit the source files and rebuild (edit-compile-view loop)
• Give suggestions on how to publish

The project should contain

• Code, both inputs and outputs
• inline and displayed maths
• citations
• cross-refs within and between pages, including numbered equations
• figures with captions and cross-refs
• examples of glue functionality in MyST-NB

This relates to #40, where quickstart is mentioned. In my view, it will remove the need to add quickstart functionality beyond create.

This is one of the key components needing action from the discussion in #44.

We agree that #39 (blog) can proceed once this issue is addressed.

[jstac]

This is also related to the project in #45, which might serve as such a demo. That's just an option. (CC @najuzilu and @mtiley).

@gregcaporaso might prefer a demo based on QIIME2 material so we at QuantEcon have to race him and his team to put something good together :-).

[choldgraf]

Thanks for kicking this off!

For reference, here is the book template that will be created when people run jupyter-book create mybookname:

https://github.com/ExecutableBookProject/cli/tree/master/jupyter_book/book_template

The template files will all be copied into mybookname/.

One thing I was considering was moving the contents of book_template/ into the documentation, and adding those pages as a sub-section of the docs called "demo book" or something like that. Then, in the "getting started" guide, we can say something like

browse the 'demo book' section for some inspirations for major functionality. If you'd like to have your own demo book locally that you can build, run jupyter-book create mybookname/.

Do folks think that'd be a good pattern to follow?

[jstac]

Sounds good although we should keep a plan clear in our minds.

My understanding right now is that jb create book creates a nearly empty template that someone can start entering material in. It's not a fully fledged demo that shows off all the features.

On the other hand, the project referred to in this issue will be a fully fledged demo that does show off all features.

In my view, the docs should point to this fully fledged demo right at the start, as described above.

They should mention the empty template later, as an additional handy feature.

[choldgraf]

Right - though as you mentioned in the call, we could probably show off all of the major features we wish to using only a couple of pages, so I don't know that a "full demo" would be more than 2 or 3 files extra.

Right now, jupyter book has two options for create:

jupyter-book create mybookname/

will create a minimal folder that people can use to get started

jupyter-book create mybookname/ --demo

will create a more fully-featured folder.

@jstac what do you think about that pattern?

Also, I wanna direct your attention to https://github.com/ExecutableBookProject/cli/issues/44, I had a conversation with a friend that works with Microsoft and who is really into GH actions - she seemed to think we could make it pretty easy for people to auto-build and push HTML to gh-pages ✨

[jstac]

@jstac what do you think about that pattern?

I think we should change it :-)

I think we should ditch the --demo option entirely and instead have the documentation point to moderately sized, realistic, fully fledged project in the way that I suggested above.

The first step is for users to learn to download, compile and view this project, as stepped through in the docs. The second step is for them to run jupyter-book create mybookname/ and off they go.

The reason I say this is that the demo I'm suggesting is less artificial and hence closer to what people actually want to do.

I like the gh-pages idea.

[choldgraf]

I think I am a little confused about how exactly you want users to create a nearly empty template vs. a fully-fledged demo. Can you lay out the two pathways you imagine for how users will accomplish each? Or, do you think it's fine to scrap the "nearly empty template" and only create full demo-style books with create?

and I realized I didn't quite answer your question above - my intention was for jupyter-book create mybookname/ to output a small book, but one that shows off the major features that we want people to use, so they have an example to go from. I think this can be accomplished with just 3-4 pages.

[jstac]

First, we set up fully-fledged-demo as a repo in GH. It contains what the name suggests.

Then, right at the very start of the docs, we point people to a rendered/published version and say "here's how you create this." Then we give them clear instructions on how to

• download fully-fledged-demo
• compile it
• view the html output locally
• view the source files, briefly discuss edit-compile-view loop
• explain how output is published / can be published.
• touch on PDF output

Next, we say, OK, so you want to create your own project. Just type jupyter-book create foo and there's a blank template.

Also here's the docs on CLI, syntax, etc.

Other points

1. We should ditch --demo.
2. We should, over time, collect a few alternatives to fully-fledged-demo, hopefully from a few fields, show published versions, and point people to the source files. Just say "and you can do the same with these demos too".

That's my proposal.

[choldgraf]

Ah I see - so you're suggesting modifying the current docs so that they step through a fully-fledged book that's in a different repository, rather than the book that's created after running jupyter-book create mybookname/ (which is what the docs currently do: https://beta.jupyterbook.org/guide/02_create.html). Yeah?

re: your second point, I agree - I was starting to collect interesting books with the original jupyter book here: https://jupyterbook.org/features/gallery.html

I think we should have something like this for the new build system (in addition to a few specific ones that we focus on in the docs)

[jstac]

Ah I see - so you're suggesting modifying the current docs so that they step through a fully-fledged book that's in a different repository, rather than the book that's created after running jupyter-book create mybookname/ (which is what the docs currently do: https://beta.jupyterbook.org/guide/02_create.html). Yeah?

Yes, according to my proposal, that's the first thing for the docs to discuss. I think it's more realistic.

After that, we should discuss the basic template in jupyter-book create mybookname as one way to kick off your own project.

The user learns to compile-edit-view an existing project and then starts on their own.

[choldgraf]

Sounds good to me - I think that's a reasonable flow of information!

The one thing we should make sure not to do is overload people straight away. It'll be a balance between showing a fancy professional-looking final output, and confusing people by asking them to look at a ton of new markup they've never seen before

[jstac]

How are you feeling about using the quantecon example in the documentation for the CLI as a primary example?

It's quite big but does a nice job of showing off the tools.

My proposal is that we encourage people to download it and build it as a first step of getting to know the tool set. That would mean some edits to the CLI docs, where the "getting started" page encourages people to download it and work through the compile-view-edit loop, spelling out all the steps. If you agree then I can put in a PR.

[choldgraf]

@jstac I think from a content standpoint, it's great. I think I have two main concerns with that repo as it is now:

• complexity of the code, and the potential (at least, in the early days before all the bugs have been ironed out), for it to behave in unexpected ways on other people's machines. E.g., doesn't the quantecon example have Numba code, parallel computation, and stuff like that? Can we guarantee that it would work in 99.9% of brand new user cases?
• Amount of time it takes to run the book. Ideally users could get a "quick win", and my understanding from speaking to your team is that the quantecon code takes a long time to build. Has that been improved?

[jstac]

complexity of the code, and the potential (at least, in the early days before all the bugs have been ironed out), for it to behave in unexpected ways on other people's machines. E.g., doesn't the quantecon example have Numba code, parallel computation, and stuff like that? Can we guarantee that it would work in 99.9% of brand new user cases?

Yes, that's an issue. It will work with the latest version of Anaconda. But it does use quite a few libraries.

Amount of time it takes to run the book. Ideally users could get a "quick win", and my understanding from speaking to your team is that the quantecon code takes a long time to build. Has that been improved?

There's not really anything obvious to "improve". We demonstrate speed gains, so there's some code we don't want to optimize.

Anyway, I share your concerns. I think this application would be nice to point as an example of what can be done, but not ideal for referencing as a basic use case.

How about we extract three early lectures from that example, which use only numpy and matplotlib? These are so standard just about everyone using Jupyter with Python will have them. They should build quickly. And it's nice to generate a few figures. I can coordinate this with @najuzilu if you agree it's a better option.

[choldgraf]

I like that idea - we could even have the subset of lectures in the same repository, maybe a folder called book_mini or something. Then it could also link to the larger folder and explain all the other cool stuff that is in there

[jstac]

I've started it here: https://github.com/ExecutableBookProject/quantecon-mini-example

I liked the idea of putting it in the same repo but having two projects in one repo makes publishing via GH pages a bit more awkward, so in the end I put it in a separate one. I'll tidy it up today and note when it's done.

[jstac]

HTML version is now published: https://executablebookproject.github.io/quantecon-mini-example/docs/index.html

[choldgraf]

Can I make a tiny request and ask that we mention percetually-flat colormaps in the "learn more" section? That is a lot of "jet"-like colors I see on that page :-)

Apologies if I am being an annoying colormap person here, but I have strong opinions about colormaps :-D

[akhmerov]

The example looks really nice!

Isn't the doubled toc a bug though:

[choldgraf]

@akhmerov yep, thanks for the catch! It's because there is a toctree in the book content

[jstac]

Hi @choldgraf , just saw your comment about jet colormaps. I'm happy to be educated on this front. @najuzilu or @Harveyt47, please fix if you have time.

[choldgraf]

I'd check out https://bids.github.io/colormap/ for background on jet and how it led to viridis as the default in matplotlib, or https://jakevdp.github.io/blog/2014/10/16/how-bad-is-your-colormap/ from jake vdp, or the blog post that I linked above (https://predictablynoisy.com/makeitpop-intro) as overviews of the problem with jet. Basically, Jet is not a good colormap to use because people's perception of the difference between two colors in Jet does not match the actual difference in data that results in those two colors. AKA, you have non-linear jumps in perceived differences in color, for what is a linear change in data.

[najuzilu]

@choldgraf is there any/better way to show the toctree structure in index.md without having it display on the sidebar twice?

[choldgraf]

I'm not sure, its a question of whatever the limitations in sphinx are. Does anybody know if it's possible to define a toctree just for show on a page but without replicating it's structure? (under the hood, the CLI is adding toctrees to the content pages)

[rossbar]

The toctree directive has a :hidden: argument for that purpose - I haven't tried it in with the EBP tools though.

[najuzilu]

@rossbar it looks like this hides the directives under index.md

[choldgraf]

Yeah in a sense we want the reverse of hidden, which is to show a preexisting toctree for a page only