diff --git a/README.md b/README.md index b5453a6e..9c387a85 100644 --- a/README.md +++ b/README.md @@ -67,9 +67,10 @@ used in the main NumPy documentation has two reasons: You may notice our content is in markdown format (`.md` files). We review and host notebooks in the [MyST-NB](https://myst-nb.readthedocs.io/) format. We -accept both Jupyter notebooks (`.ipynb`) and MyST-NB notebooks (`.md`). If you want -to sync your `.ipynb` to your `.md` file follow the [pairing -tutorial](content/pairing.md). +accept both Jupyter notebooks (`.ipynb`) and MyST-NB notebooks (`.md`). +If you want to author `MyST` notebooks in jupyterlab, check out the +[jupyterlab_myst](https://mystmd.org/guide/quickstart-jupyter-lab-myst) extension! + ### Adding your own tutorials diff --git a/content/pairing.md b/content/pairing.md deleted file mode 100644 index 322fb462..00000000 --- a/content/pairing.md +++ /dev/null @@ -1,247 +0,0 @@ ---- -jupytext: - formats: ipynb,md:myst - text_representation: - extension: .md - format_name: myst - format_version: 0.13 - jupytext_version: 1.11.1 -kernelspec: - display_name: Python 3 - language: python - name: python3 ---- - -# Pairing Jupyter notebooks and MyST-NB - -## What you'll do -This guide will keep a Jupyter notebook synced _or paired_ between -`.ipynb` and `.md`. - -## What you'll learn -- The difference between Jupyter's json format and MyST-NB's markdown - format -- The benefits and drawbacks of json and markdown -- How to keep `.ipynb` and `.md` files in sync - -## What you'll need -- [Jupyter](https://jupyter.org/) -- [Jupytext](https://jupytext.readthedocs.io/en/latest/index.html) - ---- -## Background - -The [NumPy tutorials](https://github.com/numpy/numpy-tutorials) are -reviewed and executed as [MyST-NB](https://myst-nb.readthedocs.io/) -notebooks. Content is easier to review in this markdown format. You can -keep your `.ipynb` in sync with the content on NumPy tutorials. The -NumPy tutorials use -[Jupytext](https://jupytext.readthedocs.io/en/latest/index.html) to -convert your `.ipynb` file to [MyST -Markdown](https://jupytext.readthedocs.io/en/latest/formats-markdown.html) -format. - -Jupyter notebooks are stored on your disk in a -[json format](https://nbformat.readthedocs.io/en/latest/format_description.html). The json format is -very powerful and allows you to store almost any input and output that -Python libraries can create. The drawback is that it is hard to see and compare changes made in the notebook file when reviewing pull requests, because this means the reviewers are looking only at the raw json files. - -MyST-NB notebooks are stored on your disk in a -[markdown](https://en.wikipedia.org/wiki/Markdown) format. The markdown -format is a lightweight markup language. Its key design goal is -[_readability_](https://daringfireball.net/projects/markdown/syntax#philosophy). -The drawback is that markdown can only store the inputs of your code. -Each time you open the notebook, you must execute the inputs to see the -output. - -> __Note:__ You should use [common mark](https://commonmark.org) -> markdown cells. Jupyter only renders common mark markdown, but MyST-NB -> supports a variety of restructured text directives. These Sphinx -> markdown directives will render when NumPy tutorials are built into a -> static website, but they will show up as raw code when you open in -> Jupyter locally or on [Binder](https://mybinder.org). - -Consider these two versions of the same __Simple notebook example__. You -have three things in the notebooks: - -1. A markdown cell that explains the code - ```This code calculates 2+2 and prints the output.``` -2. A code cell that shows the code - ```python - x = 2 + 2 - print('x = ', x) - ``` -3. The output of the code cell - ```python - x = 4 - ``` ---- -__
Simple notebook example
__ -This code calculates 2+2 and prints the output. - -```{code-cell} -x = 2 + 2 -print("x = ", x) -``` - ---- - -Here are the two Simple notebook example raw inputs side-by-side: - - - - - - - - - - - -
json .ipynbMyST-NB .md
- -```json -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This code calculates 2+2 and prints the output" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "x = 4\n" - ] - } - ], - "source": [ - "x = 2 + 2\n", - "print('x = ', x)" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.8.3" - } - }, - "nbformat": 4, - "nbformat_minor": 4 -} -``` - - - -```` ---- -jupytext: - formats: ipynb,md:myst - text_representation: - extension: .md - format_name: myst - format_version: 0.12 - jupytext_version: 1.6.0 -kernelspec: - display_name: Python 3 - language: python - name: python3 ---- - -This code calculates 2+2 and prints the output - -```{code-cell} ipython3 -x = 2 + 2 -print('x = ', x) -``` -```` -
- -The MyST-NB `.md` is much shorter, but it does not save the output `4`. - - -## Pair your notebook files `.ipynb` and `.md` - -When you submit a Jupyter notebook to NumPy tutorials, we (the reviewers) will convert -it to a MyST-NB format. You can also submit the MyST-NB `.md` in your -pull request. -To keep the `.ipynb` and `.md` in sync--_or paired_--you need -[Jupytext](https://jupytext.readthedocs.io/en/latest/index.html). - -Install `jupytext` using: - -``` -pip install jupytext -``` -or -``` -conda install jupytext -c conda-forge -``` - -Once installed, start your `jupyter lab` or `jupyter notebook` -session in the browser. When launching `jupyter lab` it will ask you to rebuild -to include the Jupytext extension. - -You can pair the two formats in the classic Jupyter, Jupyter Lab, -or the command line: - -```{admonition} **1. Classic Jupyter Jupytext pairing** -:class: toggle - -![Animation showing pairing with Jupyter classic](_static/01-classic.gif) -``` - -```{admonition} **2. JupyterLab Jupytext pairing** -:class: toggle - -![Animation showing pairing with JupyterLab](_static/02-jupyterlab.gif) -``` - -````{admonition} **3. Command line Jupytext pairing** -:class: toggle - -```sh -jupytext --set-formats ipynb,myst notebook.ipynb -``` - -Then, update either the MyST markdown or notebook file: - -```sh -jupytext --sync notebook.ipynb -``` -```` - -> __Note:__ With Jupytext installed, the classic Jupyter interface will -> automatically open MyST files as notebooks. In JupyterLab, you can -> right-click and choose "Open With -> Notebook" to open as a notebook. -> The outputs of your code cells are only saved in the `.ipynb` file. - -## Wrapping up - -In this tutorial, you saw the json `.ipynb` and MyST-NB `.md` raw code -to create Jupyter notebooks. You can use both formats to create -tutorials. Now you can work in either a simple text editor like VIM -or emacs or continue building notebooks in your browser. Jupytext can -handle pairing to keep your work in sync. diff --git a/content/tutorial-svd.md b/content/tutorial-svd.md index c414cafe..d2c0adb6 100644 --- a/content/tutorial-svd.md +++ b/content/tutorial-svd.md @@ -43,7 +43,18 @@ from scipy.datasets import face img = face() ``` -**Note**: If you prefer, you can use your own image as you work through this tutorial. In order to transform your image into a NumPy array that can be manipulated, you can use the `imread` function from the [matplotlib.pyplot](https://matplotlib.org/api/_as_gen/matplotlib.pyplot.html#module-matplotlib.pyplot) submodule. Alternatively, you can use the [imageio.imread](https://imageio.readthedocs.io/en/stable/_autosummary/imageio.v3.imread.html) function from the `imageio` library. Be aware that if you use your own image, you'll likely need to adapt the steps below. For more information on how images are treated when converted to NumPy arrays, see [A crash course on NumPy for images](https://scikit-image.org/docs/stable/user_guide/numpy_images.html) from the `scikit-image` documentation. +```{note} +If you prefer, you can use your own image as you work through this tutorial. +In order to transform your image into a NumPy array that can be manipulated, you +can use the `imread` function from the +[matplotlib.pyplot](https://matplotlib.org/api/_as_gen/matplotlib.pyplot.html#module-matplotlib.pyplot) submodule. +Alternatively, you can use the +[imageio.imread](https://imageio.readthedocs.io/en/stable/_autosummary/imageio.v3.imread.html) +function from the `imageio` library. +Be aware that if you use your own image, you'll likely need to adapt the steps below. +For more information on how images are treated when converted to NumPy arrays, +see [A crash course on NumPy for images](https://scikit-image.org/docs/stable/user_guide/numpy_images.html) from the `scikit-image` documentation. +``` +++ @@ -105,7 +116,13 @@ Since we are going to perform linear algebra operations on this data, it might b img_array = img / 255 ``` -This operation, dividing an array by a scalar, works because of NumPy's [broadcasting rules](https://numpy.org/devdocs/user/theory.broadcasting.html#array-broadcasting-in-numpy). (Note that in real-world applications, it would be better to use, for example, the [img_as_float](https://scikit-image.org/docs/stable/api/skimage.html#skimage.img_as_float) utility function from `scikit-image`). +This operation, dividing an array by a scalar, works because of NumPy's [broadcasting rules](https://numpy.org/devdocs/user/theory.broadcasting.html#array-broadcasting-in-numpy). + +```{tip} +In real-world applications, it may be better to use, for example, the +[img_as_float](https://scikit-image.org/docs/stable/api/skimage.html#skimage.img_as_float) +utility function from `scikit-image`. +``` You can check that the above works by doing some tests; for example, inquiring about maximum and minimum values for this array: @@ -134,7 +151,19 @@ It is possible to use methods from linear algebra to approximate an existing set +++ -**Note**: We will use NumPy's linear algebra module, [numpy.linalg](https://numpy.org/devdocs/reference/routines.linalg.html#module-numpy.linalg), to perform the operations in this tutorial. Most of the linear algebra functions in this module can also be found in [scipy.linalg](https://docs.scipy.org/doc/scipy/reference/linalg.html#module-scipy.linalg), and users are encouraged to use the [scipy](https://docs.scipy.org/doc/scipy/reference/index.html#module-scipy) module for real-world applications. However, some functions in the [scipy.linalg](https://docs.scipy.org/doc/scipy/reference/linalg.html#module-scipy.linalg) module, such as the SVD function, only support 2D arrays. For more information on this, check the [scipy.linalg page](https://docs.scipy.org/doc/scipy/tutorial/linalg.html). +```{note} +We will use NumPy's linear algebra module, +[numpy.linalg](https://numpy.org/devdocs/reference/routines.linalg.html#module-numpy.linalg), +to perform the operations in this tutorial. +Most of the linear algebra functions in this module can also be found in +[scipy.linalg](https://docs.scipy.org/doc/scipy/reference/linalg.html#module-scipy.linalg), +and users are encouraged to use the [scipy](https://docs.scipy.org/doc/scipy/reference/index.html#module-scipy) +module for real-world applications. +However, some functions in the +[scipy.linalg](https://docs.scipy.org/doc/scipy/reference/linalg.html#module-scipy.linalg) +module, such as the SVD function, only support 2D arrays. +For more information on this, check the [scipy.linalg page](https://docs.scipy.org/doc/scipy/tutorial/linalg.html). +``` +++ @@ -177,7 +206,11 @@ import numpy as np U, s, Vt = np.linalg.svd(img_gray) ``` -**Note** If you are using your own image, this command might take a while to run, depending on the size of your image and your hardware. Don't worry, this is normal! The SVD can be a pretty intensive computation. +```{note} +If you are using your own image, this command might take a while to run, +depending on the size of your image and your hardware. +Don't worry, this is normal! The SVD can be a pretty intensive computation. +``` +++ @@ -188,9 +221,10 @@ U.shape, s.shape, Vt.shape ``` Note that `s` has a particular shape: it has only one dimension. This means that some linear algebra functions that expect 2d arrays might not work. For example, from the theory, one might expect `s` and `Vt` to be -compatible for multiplication. However, this is not true as `s` does not have a second axis. Executing +compatible for multiplication. However, this is not true as `s` does not have a second axis: -```python +```{code-cell} +:tags: [raises-exception] s @ Vt ``` @@ -331,7 +365,12 @@ plt.imshow(np.transpose(reconstructed, (1, 2, 0))) plt.show() ``` -In fact, `imshow` peforms this clipping under-the-hood, so if you skip the first line in the previous code cell, you might see a warning message saying `"Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers)."` +```{note} +In fact, `imshow` peforms this clipping under-the-hood, so if you skip the first +line in the previous code cell, you might see a warning message saying +`"Clipping input data to the valid range for imshow with RGB data ([0..1] for +floats or [0..255] for integers)."` +``` Now, to do the approximation, we must choose only the first `k` singular values for each color channel. This can be done using the following syntax: @@ -351,7 +390,7 @@ approx_img.shape which is not the right shape for showing the image. Finally, reordering the axes back to our original shape of `(768, 1024, 3)`, we can see our approximation: ```{code-cell} -plt.imshow(np.transpose(approx_img, (1, 2, 0))) +plt.imshow(np.transpose(np.clip(approx_img, 0, 1), (1, 2, 0))) plt.show() ``` diff --git a/ignore_testing b/ignore_testing index a72bcd51..e5b7e003 100644 --- a/ignore_testing +++ b/ignore_testing @@ -1,4 +1,3 @@ content/tutorial-deep-reinforcement-learning-with-pong-from-pixels.md -content/pairing.md content/tutorial-style-guide.md content/tutorial-nlp-from-scratch.md diff --git a/site/contributing.md b/site/contributing.md index a4287490..9fd31d4b 100644 --- a/site/contributing.md +++ b/site/contributing.md @@ -41,8 +41,8 @@ used in the main NumPy documentation has two reasons: You may notice our content is in markdown format (`.md` files). We review and host notebooks in the [MyST-NB](https://myst-nb.readthedocs.io/) format. We accept both Jupyter notebooks (`.ipynb`) and MyST-NB notebooks (`.md`). -If you want to sync your `.ipynb` to your `.md` file follow the [pairing -tutorial](content/pairing). +If you want to author `MyST` notebooks in jupyterlab, check out the +[jupyterlab_myst](https://mystmd.org/guide/quickstart-jupyter-lab-myst) extension! ## Adding your own tutorials