DOC: Add an interactive shell powered by JupyterLite to the website #47428

jtpio · 2022-06-20T11:49:21Z

This allows for easily trying pandas in a web browser without installing anything.

Since the default kernel is based on Pyodide, it also includes a couple of other libraries by default such as matplotlib.

Other popular libraries have also adopted JupyterLite to power their documentation with an interactive shell:

numpy: https://numpy.org/
sympy: https://www.sympy.org/en/shell.html

closes Add interactive terminal to pandas website #46682
~~Tests added and passed if fixing a bug or adding a new feature~~
~~All code checks passed.~~
~~Added type annotations to new arguments/methods/functions.~~
Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

TODO

Embed the JupyterLite REPL application
Create a separate repo for the JupyterLite deployment with a csv file and other custom settings
Add example code that can be copied and pasted
Move the demo repo to the pandas-dev organization

jtpio · 2022-06-20T13:52:37Z

Example of reading the iris.csv file:

jtpio · 2022-06-20T14:11:07Z

web/pandas/getting_started.md

+without installing anything on your computer:
+
+<iframe
+  src="https://jtpio.github.io/pandas-repl/repl/index.html?toolbar=1&kernel=python&code=import pandas as pd"


This will have to be updated after the repo is transferred to pandas-dev organization: https://github.com/jtpio/pandas-repl

Can the content be directly transferred and initialized in the pandas repo? We've been trying to minimize the number of auxiliary repos and our web deployment is done within this repo already.

Usually it's simpler to keep the content in a separate repo because the JupyterLite website is then deployed to GitHub Pages automatically when new changes are merged.

But if there is already a CI workflow deploying the pandas website on push, then this could also be integrated there.

But if there is already a CI workflow deploying the pandas website on push, then this could also be integrated there.

Looks like this might be relevant:

pandas/.github/workflows/docbuild-and-upload.yml

Lines 60 to 62 in 3364f9a

- name: Upload web

run: rsync -az --delete --exclude='pandas-docs' --exclude='docs' web/build/ docs@${{ secrets.server_ip }}:/usr/share/nginx/pandas

if: github.event_name == 'push' && github.ref == 'refs/heads/main'

In that case the content currently in https://github.com/jtpio/pandas-repl could be placed under web/lite for instance.

And then there could be an additional step running on CI to build the JupyterLite assets with jupyter lite build to web/build/lite, which would then be copied by the existing rsync command?

Yeah that workflow I think would be preferred

+1 with this approach. I'd keep the assets for the terminal in web/interactive_terminal or similar name.I think it's clearer. When building, web/build/lite or whatever it's standard is fine. If you copy files to that directory, the last steps of the website workflow should already take care of deploying it.

Also fine to keep things simple here, and create a DataFrame from the constructor first. We can discuss in a separate issue PR about the exact data and examples to use. Probably better to not mix the technical work with JupyterLite and the content of the terminal in the same discussion I think.

An example as simple as next would work for me:

df = pd.DataFrame({'num_legs': [2, 4], 'num_wings': [2, 0]}, index=['falcon', 'dog'])

Sounds good 👍

jtpio · 2022-06-20T14:34:19Z

[ ] Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

It's not a new feature to pandas itself, but could be interesting to get it into the changelog somewhere for visibility?

jtpio · 2022-06-20T14:38:07Z

Marking as ready for review since the basic functionality should now be working. Happy to iterate more in this PR or follow-up PRs based on feedback.

As mentioned in the top comment, the repo used for the JupyterLite deployment should be moved to the pandas-dev organization on GitHub.

cc a couple of folks involved in the original issue: #46682

@datapythonista @bennaaym @psychemedia @hamedgibago

Thanks!

jtpio · 2022-06-20T15:14:47Z

Added some example code above the REPL that folks can copy paste:

psychemedia · 2022-06-20T16:47:22Z

Would it be useful to also cover other pandas read/write methods, to test file type reading, if nothing else and also surface filetypes etc. that may not be handled correctly?

There is a full list available here: https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html

jtpio · 2022-06-20T17:42:36Z

@psychemedia have you tried to see if they work?

I think the main reason for having pd.read_csv() work in the REPL was to be able to load a dataset with a familiar API, and then manipulate it with pandas.

psychemedia · 2022-06-20T19:39:23Z

I do have a test notebook (cribbed from pandas docs), but I can't even get read_csv() to work with https://jupyterlite.readthedocs.io/en/latest/_static/lab/index.html yet on data/iris.csv atm (I was assuming that demo was the latest one? I tried reloading in browser dev tools which generally seems to clear cache etc. but that made no difference.) Will try to have a look in a day or two: currently buried in meeting a teaching marking deadline.

jtpio · 2022-06-20T20:59:09Z

Thanks @psychemedia for sharing the example notebook, this is useful. It looks like there is an issue saving the json file:

Mind opening an issue on the JupyterLite repo? Looks like this is an issue that should be fixed there, and is not strictly blocking for this PR since it can be addressed separately.

EDIT: tracked in jupyterlite/jupyterlite#682

datapythonista

Looks great. I added few suggestions, but the general idea looks perfect. Thanks for working on this!

.github/workflows/docbuild-and-upload.yml

web/pandas/interactive_terminal/README.md

web/pandas_web.py

web/pandas/getting_started.md

web/pandas/interactive_terminal/README.md

jtpio · 2022-06-22T10:06:42Z

Many thanks @datapythonista for the review and feedback!

The comments should have been addressed now.

jtpio · 2022-06-22T11:02:10Z

For those who would like to try it out, you can download the built website here: https://github.com/pandas-dev/pandas/actions/runs/2541517660

And then start a local server with python -m http.server:

datapythonista

Thanks @jtpio really nice. lgtm

Only thing, not sure if I'd delete the iris csv, since now it's not used, and if we decide to use a csv later, we may probably try to find one with more dtypes than iris. But no big deal.

jtpio · 2022-06-22T12:08:02Z

not sure if I'd delete the iris csv, since now it's not used,

Good catch. Happy to push a commit to remove it, so we keep things as simple as possible for now.

jtpio · 2022-06-22T12:15:07Z

Happy to push a commit to remove it,

Done in 7d75312

mroeschke · 2022-06-23T21:15:34Z

Very cool! Thanks @jtpio

…andas-dev#47428) * Add an interactive shell powered by JupyterLite * Update to the dedicated JupyterLite deployment * Add example code * Move build files to the pandas repo * Build the jupyterlite website * Load relative terminal * Update example code * Update wording * Fix trailing spaces * Move build dependencies to the top-level environment.yml * Move to `web/interactive_terminal` * Remove example code * Add note about the loading time * Update instructions in the README * Update build command on CI * Fix typo in .gitignore * Lint environment.yml * Remove unused import * Undo unrelated changes in environment.yml * Fix pre-commit check * Remove unused csv file

jtpio mentioned this pull request Jun 20, 2022

Add interactive terminal to pandas website #46682

Closed

jtpio commented Jun 20, 2022

View reviewed changes

jtpio changed the title ~~Add an interactive shell powered by JupyterLite to the website~~ ENH: Add an interactive shell powered by JupyterLite to the website Jun 20, 2022

jtpio changed the title ~~ENH: Add an interactive shell powered by JupyterLite to the website~~ DOC: Add an interactive shell powered by JupyterLite to the website Jun 20, 2022

jtpio marked this pull request as ready for review June 20, 2022 14:34

jtpio mentioned this pull request Jun 21, 2022

Some pandas read / write methods fail with some file types jupyterlite/jupyterlite#682

Open

datapythonista reviewed Jun 22, 2022

View reviewed changes

jtpio added 14 commits June 22, 2022 11:56

Add an interactive shell powered by JupyterLite

0ef26b8

Update to the dedicated JupyterLite deployment

4809746

Add example code

789d16f

Move build files to the pandas repo

8262c1c

Build the jupyterlite website

4b5fcf9

Load relative terminal

53b5efc

Update example code

6039fb4

Update wording

2241dac

Fix trailing spaces

a2059e9

Move build dependencies to the top-level environment.yml

e995588

Move to web/interactive_terminal

60e01ec

Remove example code

f51a015

Add note about the loading time

52e92ca

Update instructions in the README

05f73dc

jtpio force-pushed the jupyterlite branch from 3ee8edc to 05f73dc Compare June 22, 2022 09:57

jtpio added 5 commits June 22, 2022 11:57

Update build command on CI

b92eb5d

Fix typo in .gitignore

058a0f2

Lint environment.yml

9e60858

Remove unused import

276431f

Undo unrelated changes in environment.yml

491b354

Fix pre-commit check

967e7fb

datapythonista approved these changes Jun 22, 2022

View reviewed changes

Remove unused csv file

7d75312

mroeschke approved these changes Jun 23, 2022

View reviewed changes

mroeschke added Web pandas website Docs labels Jun 23, 2022

mroeschke added this to the 1.5 milestone Jun 23, 2022

mroeschke merged commit 2c947e0 into pandas-dev:main Jun 23, 2022

jtpio deleted the jupyterlite branch June 24, 2022 07:25

agriyakhetarpal mentioned this pull request Mar 18, 2024

ENH: out-of-tree Pyodide builds in CI for pandas #57891

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DOC: Add an interactive shell powered by JupyterLite to the website #47428

DOC: Add an interactive shell powered by JupyterLite to the website #47428

jtpio commented Jun 20, 2022 •

edited

Loading

jtpio commented Jun 20, 2022

jtpio Jun 20, 2022

mroeschke Jun 21, 2022

jtpio Jun 21, 2022

jtpio Jun 21, 2022

jtpio Jun 21, 2022 •

edited

Loading

mroeschke Jun 21, 2022

datapythonista Jun 22, 2022

jtpio Jun 22, 2022 •

edited

Loading

jtpio commented Jun 20, 2022

jtpio commented Jun 20, 2022

jtpio commented Jun 20, 2022

psychemedia commented Jun 20, 2022 •

edited

Loading

jtpio commented Jun 20, 2022

psychemedia commented Jun 20, 2022

jtpio commented Jun 20, 2022 •

edited

Loading

datapythonista left a comment

jtpio commented Jun 22, 2022

jtpio commented Jun 22, 2022

datapythonista left a comment

jtpio commented Jun 22, 2022

jtpio commented Jun 22, 2022

mroeschke commented Jun 23, 2022

	- name: Upload web
	run: rsync -az --delete --exclude='pandas-docs' --exclude='docs' web/build/ docs@${{ secrets.server_ip }}:/usr/share/nginx/pandas
	if: github.event_name == 'push' && github.ref == 'refs/heads/main'

DOC: Add an interactive shell powered by JupyterLite to the website #47428

DOC: Add an interactive shell powered by JupyterLite to the website #47428

Conversation

jtpio commented Jun 20, 2022 • edited Loading

TODO

jtpio commented Jun 20, 2022

jtpio Jun 20, 2022

Choose a reason for hiding this comment

mroeschke Jun 21, 2022

Choose a reason for hiding this comment

jtpio Jun 21, 2022

Choose a reason for hiding this comment

jtpio Jun 21, 2022

Choose a reason for hiding this comment

jtpio Jun 21, 2022 • edited Loading

Choose a reason for hiding this comment

mroeschke Jun 21, 2022

Choose a reason for hiding this comment

datapythonista Jun 22, 2022

Choose a reason for hiding this comment

jtpio Jun 22, 2022 • edited Loading

Choose a reason for hiding this comment

jtpio commented Jun 20, 2022

jtpio commented Jun 20, 2022

jtpio commented Jun 20, 2022

psychemedia commented Jun 20, 2022 • edited Loading

jtpio commented Jun 20, 2022

psychemedia commented Jun 20, 2022

jtpio commented Jun 20, 2022 • edited Loading

datapythonista left a comment

Choose a reason for hiding this comment

jtpio commented Jun 22, 2022

jtpio commented Jun 22, 2022

datapythonista left a comment

Choose a reason for hiding this comment

jtpio commented Jun 22, 2022

jtpio commented Jun 22, 2022

mroeschke commented Jun 23, 2022

jtpio commented Jun 20, 2022 •

edited

Loading

jtpio Jun 21, 2022 •

edited

Loading

jtpio Jun 22, 2022 •

edited

Loading

psychemedia commented Jun 20, 2022 •

edited

Loading

jtpio commented Jun 20, 2022 •

edited

Loading