Use a submodule for notebooks folder

[arokem]

Wouldn't it be nice if the notebooks folder was a git submodule? And:

1. The summary file was somehow automatically generated from that folders "index.ipynb"
2. The requirements file was generated from the submodules requirements file.

[choldgraf]

Hmmm, so you think you could expand on that à little bit ? I am intrigued but I don't think I get the whole picture!

[arokem]

The use-case I have in mind are repos that are binder ready (i.e. have an index file, a requirements file, and a pile of notebooks), and that you'd like to provide both as such (e.g., on a jupyterhub), but also separately in this textbook format. Right now, unless I am missing something, you'd have to manage two separate repositories. I think that the auto-generation would also be nice for dealing with legacy repositories that are already set up for binder, but would require some work to get into this format.

[choldgraf]

ah ya, that makes more sense...I think the trick is what configuration to expose to people, and what to "automate". E.g., I have this little helper script:

https://github.com/choldgraf/textbooks-with-jupyter/blob/master/scripts/generate_summary_from_folders.py

that generates the summary file from a folder hierarchy, though it's a bit clunky since it just uses alphanumeric values to sort the files. Maybe that's fine? Would love to chat about how this could be done gracefully

[TomDonoghue]

Related to this idea (I think), but a slightly different approach - because I also had pre-existing repos of notebooks I wanted add a site to, without reformatting the repo, I just copy them from a remote and build from that. This doesn't do all the way in terms of having a submodule (that may well be a better solution, I just know less about it).

What I like about this, is that the notebooks can be more flatly organized in a repo, so if you want to scroll through them on Github, you don't have to click through on the textbook version of the repo. The textbook can then be hosted from either a 'gh-pages' branch, or a separate repo.

My make file looks something like this:

textbook:

	# Copy & build materials to create textbook from, and then clean up
	python scripts/clean.py
        rm -rf content/materials
	git clone --depth 1 https://github.com/COGS18/materials content/materials
	rm content/materials/README.md
	python scripts/generate_book.py
	rm -rf content/materials

I bring this up, because it is my main approach on a few repos now, and it might be worth mentioning in docs that it is easy to add a website on top of an existing repo of notebooks, without changing the original (simplest: add gh-page branch).

[choldgraf]

Another option would be to convert the generate_book.py scripts into a light-weight command-line function that takes as inputs a path to where the content is as well as kwargs for the various parameters (that would normally be stored in the _config.yml file). Then, you could run that function on whatever path-to-notebooks you want (even if they're in a different repository)

[choldgraf]

@arokem do you think that your concerns in this issue could be resolved by making this project a cookiecutter? (see #87 )

[arokem]

Yep! You might consider optionally pointing to a requirements file as part of the cookie-cutter process, to pull that in and put it in the right place.

[arokem]

Or even ask people to specify (one by one?) what their dependencies are and auto-generate the requirements file!

[choldgraf]

we've got a CLI to generate new books now! (along with a mild degree of TOC auto-generation), so gonna close this