Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Programmatically create markdown and jupyter notebook from a share dictionary with questions and answers #89

Merged
merged 16 commits into from Feb 10, 2020

Conversation

SebastianoF
Copy link
Contributor

@SebastianoF SebastianoF commented Nov 23, 2019

Over the past months I had found numpy 100 very handy for learning, and as a source of information.

At the same time it is not ideally implemented. If a question or answer needs an updated, this must be done manually in all files involved.

To solve this issue I had embedded headers, questions, hints and answer in a python file as strings and dictionaries and created a method to automatically generate the jupyter notebooks and the markdown files.

Changing the source dictionaries and stings would change the output files.

E.g: if a question needs update, just update it under -data_source.py- source/questions.ktx and then re-create all the files with the updated questions with:

python generators.py

Also, there is no more need to have several jupyter notebooks, since hints and answers can be queried programmatically with hint(n) and answer(n) from within the jupyter notebook.

A jupyter notebook where to query a random question (flashcard style) had been added too.

Pleaese note: binder and link in the readme may need to be updated, to keep the binder version in sync with the current repo!

After merging the repo would look like: https://github.com/SebastianoF/numpy-100/tree/dev

@SebastianoF SebastianoF changed the title Dev Programmatically create markdown and jupyter notebook from a share dictionary with questions and answers Nov 23, 2019
@rougier
Copy link
Owner

rougier commented Dec 1, 2019

Amazing work! Thanks. Do you think we could have a regular md file as the main source for generating others ? I'm not too fan of heaving actual contents in a Python file?

@SebastianoF
Copy link
Contributor Author

SebastianoF commented Dec 20, 2019

Welcome and sorry for the late answer.

Uhm... it is certainly possible to use markdown as source, although I do not think it would be optimal to have a markup language for data serialisation. This would imply to have the need of marshalling the .md into a structure like the one in the .py and the unmarshalling in back into the jupyter notebook, or other files that we want (restructuredText, HTML, docuwiki, latex...?).

What about storing the data into a json file? It would be already serialized, and ready to become anything we like in few lines of code (including markdown).

The structure would remain simple and would contain:

  • the data_source.json
  • the conversion module to deserialise the json in any format we like (for now md and jupyter notebook, it won't be terribly complicated to add any more format at that point)
  • a python script generating the output files.
  • the automatically created output file (added to the repo too)

@rougier
Copy link
Owner

rougier commented Dec 27, 2019

json might be a better option, yes. Also, it would be good to nice to keep the original file names because I suspect there are some people that link directly to this or that version. Ideally, you could only add the json file as well as the generator and left other files (more or less) untouched.

@SebastianoF
Copy link
Contributor Author

It seems there is a limitation in how to the newlines are parsed in the json, which make json not ideal as a source format for python code, whose text and indent are critical.
https://stackoverflow.com/questions/2392766/multiline-strings-in-json
Looking into other solutions. Suggestions are welcome in the meantime!

@SebastianoF
Copy link
Contributor Author

The most direct solution I could find, alternative to the json format, was to define a "text file with keys", that can be human readable (and modifiable), preserves the indentation, and can be seamlessly parsed into python dictionaries. This is then parsed into markdown, and jupyter notebook.

Not sure is the best idea, and suggestions are always welcome!

@SebastianoF
Copy link
Contributor Author

(also I had moved to same filenames as the master, although I would not advocate for starting names of executable files with numbers and mixing camelcase and snake conventions).

import numpy as np
```
If you find an error or think you've a better way to solve some of them, feel
free to open an issue at <https://github.com/rougier/numpy-100>File automatically generated. See the documentation to update questions/answers/hints programmatically.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a trailing "File automatically generated" here

@rougier
Copy link
Owner

rougier commented Jan 18, 2020

I've merged two small PRs, that might be the reason for the conflict. Sorry for that.

@SebastianoF
Copy link
Contributor Author

No problem!
I had manually updated the changes from #90 in the source file and re-created the .md and jupyter notebooks.

Hopefully this PR would make the next updates more straightforward, as only one text file will need to be changed.

@rougier
Copy link
Owner

rougier commented Feb 10, 2020

Thanks. I'm merging. There are some PR pending, I'll redirect them to the new format.

@rougier rougier merged commit 281ec48 into rougier:master Feb 10, 2020
@rougier
Copy link
Owner

rougier commented Feb 10, 2020

Just forgot. Maybe we now need a CONTRIBUTE.md to explain what to modify, what do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants