Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Include custom extension to the jupyterlite #1366

Open
delongZhai opened this issue Apr 7, 2024 · 10 comments
Open

Include custom extension to the jupyterlite #1366

delongZhai opened this issue Apr 7, 2024 · 10 comments
Labels
documentation Improvements or additions to documentation

Comments

@delongZhai
Copy link

Hello, thank you for your work. I was trying to add some flavor with experimentation. However, I'm having some trouble of following through.

Problem

I build a front end extension that simply post message to parent when there's an output.

I got a question about installing it in my Jupyter lite assets. I tried to run pip install. and lab extension install. But that was not working. I also look at updating the Jupyter-lite.json, but there was no luck there. Can you please let me know what part I was missing? Thank you

I ran the following command

To install the extension, execute:

pip install jupyter-repl-msgbridge

To include the extension into labextension, execute:

jupyter labextension install jupyter-repl-msgbridge
jupyter labextension list
JupyterLab v4.0.13
/Library/Frameworks/Python.framework/Versions/3.12/share/jupyter/labextensions
        @jupyterlite/xeus-sqlite-kernel v0.2.0 enabled OK (python, jupyterlite_xeus_sqlite)
        jupyterlab_pygments v0.3.0 enabled OK (python, jupyterlab_pygments)
        jupyterlab-night v0.4.6 enabled  X (python, jupyterlab_night)
        bqplot v0.5.44 enabled  X (python, bqplot)
        ipycanvas v0.13.1 enabled OK
        jupyter-matplotlib v0.11.3 enabled OK
        jupyter-leaflet v0.18.2 enabled OK
        @jupyterlite/xeus-lua-kernel v0.3.0 enabled OK (python, jupyterlite_xeus_lua)
        ipyevents v2.0.1 enabled OK
        jupyterlab-plotly v5.20.0 enabled  X
        @jupyter-notebook/lab-extension v7.0.8 enabled OK
        @jupyterlab/geojson-extension v3.4.0 enabled OK (python, jupyterlab-geojson)
        @jupyterlab/fasta-extension v3.3.0 enabled OK (python, jupyterlab-fasta)
        @jupyter-widgets/jupyterlab-manager v5.0.10 enabled OK (python, jupyterlab_widgets)
        @jupyterlite/pyodide-kernel-extension v0.2.3 enabled OK (python, jupyterlite_pyodide_kernel)
        @jupyterlite/p5-kernel-extension v0.1.0 enabled OK (python, jupyterlite_p5_kernel)
        @jupyterlite/javascript-kernel-extension v0.2.3 enabled OK
        @timkpaine/jupyterlab_miami_nights v0.4.2 enabled OK (python, jupyterlab_miami_nights)


   The following extensions are outdated:
        jupyterlab-night
        bqplot
        jupyterlab-plotly
        
   Consider checking if an update is available for these packages.

Other labextensions (built into JupyterLab)
   app dir: /Library/Frameworks/Python.framework/Versions/3.12/share/jupyter/lab
        jupyter-repl-msgbridge v0.2.8 enabled OK

Here is my repo. https://github.com/ZDL-LLC/jupyter-repl-msgbridge
and here is the package I published. https://pypi.org/project/jupyter-repl-msgbridge/

Thank you again

Suggested Improvement
@delongZhai delongZhai added the documentation Improvements or additions to documentation label Apr 7, 2024
@jtpio
Copy link
Member

jtpio commented Apr 8, 2024 •
edited

@delongZhai Just to double check: you also need to run the jupyter lite build command to make sure the extension is correctly picked up.

Also your extension looks similar to what is described in this guide, so having a look there in the documentation might also help: https://jupyterlite.readthedocs.io/en/stable/howto/configure/advanced/iframe.html

@delongZhai
Copy link
Author

@delongZhai Just to double check: you also need to run the jupyter lite build command to make sure the extension is correctly picked up.

hello @jtpio, following out prior discussion, I ran that script, and I don't see the extension being included

jupyter lite build
static:jupyter-lite.json
.  pre_status:static:jupyter-lite.json
    tarball:         jupyterlite-app-0.2.3.tgz 9MB
    output:          /Users/delongzhai/Projects/scripts/_output
    lite dir:        /Users/delongzhai/Projects/scripts
    apps:            
    sourcemaps:      True
    unused packages: True
archive:archive
contents:contents
icons:icons
jupyterlite-pyodide-kernel-pyodide:pyodide
lite:jupyter-lite.json
mimetypes:jupyter-lite.json
serve:contents
settings:overrides
translation:translation
.  status:archive:archive
[LiteBuildApp] No archive (yet): scripts-jupyterlite.tgz
.  status:contents:contents
    contents: 0 files
.  status:icons:icons
    favicon files: 7 files
.  status:jupyterlite-pyodide-kernel-pyodide:pyodide
     URL: None
 archive: []
   cache: 0 files
   local: 0 files
.  status:lite:jupyter-lite.json
[LiteBuildApp]     jupyter-lite.(json|ipynb): 1 files
.  status:mimetypes:jupyter-lite.json
    filetypes:         26 
.  status:serve:contents
    url: http://127.0.0.1:8000/
    server: tornado
    headers:
.  status:settings:overrides
    overrides.json: 0
.  status:translation:translation
    translation files: 4 files
static:output_dir
-- pre_init:static:output_dir
static:unpack
-- init:static:unpack
federated_extensions:copy:ext:@jupyterlite/xeus-sqlite-kernel
federated_extensions:copy:ext:jupyterlab_pygments
federated_extensions:copy:ext:jupyterlab-night
federated_extensions:copy:ext:bqplot
federated_extensions:copy:ext:ipycanvas
federated_extensions:copy:ext:jupyter-matplotlib
federated_extensions:copy:ext:jupyter-leaflet
federated_extensions:copy:ext:@jupyterlite/xeus-lua-kernel
federated_extensions:copy:ext:ipyevents
federated_extensions:copy:ext:jupyterlab-plotly
federated_extensions:copy:ext:@jupyter-notebook/lab-extension
federated_extensions:copy:ext:@jupyterlab/geojson-extension
federated_extensions:copy:ext:@jupyterlab/fasta-extension
federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager
federated_extensions:copy:ext:@jupyterlite/pyodide-kernel-extension
federated_extensions:copy:ext:@jupyterlite/p5-kernel-extension
federated_extensions:copy:ext:@jupyterlite/javascript-kernel-extension
federated_extensions:copy:ext:@timkpaine/jupyterlab_miami_nights
-- pre_build:federated_extensions:copy:ext:@jupyterlite/xeus-sqlite-kernel
-- pre_build:federated_extensions:copy:ext:jupyterlab_pygments
-- pre_build:federated_extensions:copy:ext:jupyterlab-night
-- pre_build:federated_extensions:copy:ext:bqplot
-- pre_build:federated_extensions:copy:ext:ipycanvas
-- pre_build:federated_extensions:copy:ext:jupyter-matplotlib
-- pre_build:federated_extensions:copy:ext:jupyter-leaflet
-- pre_build:federated_extensions:copy:ext:@jupyterlite/xeus-lua-kernel
-- pre_build:federated_extensions:copy:ext:ipyevents
-- pre_build:federated_extensions:copy:ext:jupyterlab-plotly
-- pre_build:federated_extensions:copy:ext:@jupyter-notebook/lab-extension
-- pre_build:federated_extensions:copy:ext:@jupyterlab/geojson-extension
-- pre_build:federated_extensions:copy:ext:@jupyterlab/fasta-extension
-- pre_build:federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager
-- pre_build:federated_extensions:copy:ext:@jupyterlite/pyodide-kernel-extension
-- pre_build:federated_extensions:copy:ext:@jupyterlite/p5-kernel-extension
-- pre_build:federated_extensions:copy:ext:@jupyterlite/javascript-kernel-extension
-- pre_build:federated_extensions:copy:ext:@timkpaine/jupyterlab_miami_nights
icons:copy
lite:patch:repl/jupyter-lite.json
translation:copy
.  build:icons:copy
-- build:lite:patch:repl/jupyter-lite.json
-- build:translation:copy
federated_extensions:patch
federated_extensions:copy:theme:jupyterlab-night
federated_extensions:copy:theme:@timkpaine/jupyterlab_miami_nights
federated_extensions:settings
jupyterlite-pyodide-kernel-piplite:patch
mimetypes:patch
-- post_build:federated_extensions:patch
-- post_build:federated_extensions:copy:theme:jupyterlab-night
-- post_build:federated_extensions:copy:theme:@timkpaine/jupyterlab_miami_nights
-- post_build:federated_extensions:settings
.  post_build:jupyterlite-pyodide-kernel-piplite:patch
-- post_build:mimetypes:patch

@delongZhai
Copy link
Author

And for this step, is the purpose here to install all the custom package here and run jlpm run build?

if so, do you have a sample package.json that you can share? thanks

@jtpio
Copy link
Member

jtpio commented Apr 9, 2024 •
edited

According to the logs, it looks like it picked up some extensions (federated_extensions:copy:ext).

Is your extension installed in the same environment as other extensions? Also you may want to use jupyter labextension list to double check it's picked up correctly.

@delongZhai
Copy link
Author

hey @jtpio ,

what I see from the output is that it is included, however, not listed as other extensions.

jupyter labextension list
JupyterLab v4.0.13
/Library/Frameworks/Python.framework/Versions/3.12/share/jupyter/labextensions
        @jupyterlite/xeus-sqlite-kernel v0.2.0 enabled OK (python, jupyterlite_xeus_sqlite)
        jupyterlab_pygments v0.3.0 enabled OK (python, jupyterlab_pygments)
        jupyterlab-night v0.4.6 enabled  X (python, jupyterlab_night)
        bqplot v0.5.44 enabled  X (python, bqplot)
        ipycanvas v0.13.1 enabled OK
        jupyter-matplotlib v0.11.3 enabled OK
        jupyter-leaflet v0.18.2 enabled OK
        @jupyterlite/xeus-lua-kernel v0.3.0 enabled OK (python, jupyterlite_xeus_lua)
        ipyevents v2.0.1 enabled OK
        jupyterlab-plotly v5.20.0 enabled  X
        @jupyter-notebook/lab-extension v7.0.8 enabled OK
        @jupyterlab/geojson-extension v3.4.0 enabled OK (python, jupyterlab-geojson)
        @jupyterlab/fasta-extension v3.3.0 enabled OK (python, jupyterlab-fasta)
        @jupyter-widgets/jupyterlab-manager v5.0.10 enabled OK (python, jupyterlab_widgets)
        @jupyterlite/pyodide-kernel-extension v0.2.3 enabled OK (python, jupyterlite_pyodide_kernel)
        @jupyterlite/p5-kernel-extension v0.1.0 enabled OK (python, jupyterlite_p5_kernel)
        @jupyterlite/javascript-kernel-extension v0.2.3 enabled OK
        @timkpaine/jupyterlab_miami_nights v0.4.2 enabled OK (python, jupyterlab_miami_nights)


   The following extensions are outdated:
        jupyterlab-night
        bqplot
        jupyterlab-plotly
        
   Consider checking if an update is available for these packages.

Other labextensions (built into JupyterLab)
   app dir: /Library/Frameworks/Python.framework/Versions/3.12/share/jupyter/lab
        jupyter-repl-msgbridge v0.2.8 enabled OK

What I ran is in my fork of jupyter/demo. I didn't do the following steps, but aren't these for testing:

Move to a directory where the extension should be tested and run the build of this extension for JupyterLite:

mkdir examples
cd examples
jupyter lite build --output-dir lite
The following lines show that the extension has been correctly built:

...
federated_extensions:copy:ext:jupyterlab-iframe-bridge-example
.  pre_build:federated_extensions:copy:ext:jupyterlab-iframe-bridge-example
...
A jupyterlab-iframe-bridge-example/examples/lite/ directory containing everything needed for JupyterLite to work is created (notice the presence of our extension in the extensions subdirectory).

@jtpio
Copy link
Member

jtpio commented Apr 10, 2024

To include the extension into labextension, execute:

jupyter labextension install jupyter-repl-msgbridge

Re-reading the top message, this step should not be necessary. This used to be the way for installing extensions, but now the preferred way is to install extensions with pip.

Would you be able to try again in a fresh new environment?

@delongZhai
Copy link
Author

delongZhai commented Apr 10, 2024 •
edited

@jtpio , So I did the following steps:

(base) delongzhai@Delongs-MacBook-Pro scripts % conda env list
# conda environments:
#
base                  *  /opt/miniconda3
jupyterlab-ext           /opt/miniconda3/envs/jupyterlab-ext
scripts_env              /opt/miniconda3/envs/scripts_env
(base) delongzhai@Delongs-MacBook-Pro scripts % conda env remove -n scripts_env

Remove all packages in environment /opt/miniconda3/envs/scripts_env:

Everything found within the environment (/opt/miniconda3/envs/scripts_env), including any conda environment configurations and any non-conda files, will be deleted. Do you wish to continue?
 (y/[n])? y
(base) delongzhai@Delongs-MacBook-Pro scripts % conda create -n scripts_env
Channels:
 - defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /opt/miniconda3/envs/scripts_env



Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate scripts_env
#
# To deactivate an active environment, use
#
#     $ conda deactivate
(base) delongzhai@Delongs-MacBook-Pro scripts % conda activate scripts_env
(scripts_env) delongzhai@Delongs-MacBook-Pro scripts % python3 -m pip install -r requirements.txt
(scripts_env) delongzhai@Delongs-MacBook-Pro scripts % jupyter labextension list
JupyterLab v4.0.13
/Library/Frameworks/Python.framework/Versions/3.12/share/jupyter/labextensions
        @jupyterlite/xeus-sqlite-kernel v0.2.0 enabled OK (python, jupyterlite_xeus_sqlite)
        jupyterlab_pygments v0.3.0 enabled OK (python, jupyterlab_pygments)
        jupyterlab-night v0.4.6 enabled  X (python, jupyterlab_night)
        bqplot v0.5.44 enabled  X (python, bqplot)
        ipycanvas v0.13.1 enabled OK
        jupyter-matplotlib v0.11.3 enabled OK
        jupyter-leaflet v0.18.2 enabled OK
        @jupyterlite/xeus-lua-kernel v0.3.0 enabled OK (python, jupyterlite_xeus_lua)
        ipyevents v2.0.1 enabled OK
        jupyterlab-plotly v5.20.0 enabled  X
        @jupyter-notebook/lab-extension v7.0.8 enabled OK
        @jupyterlab/geojson-extension v3.4.0 enabled OK (python, jupyterlab-geojson)
        @jupyterlab/fasta-extension v3.3.0 enabled OK (python, jupyterlab-fasta)
        @jupyter-widgets/jupyterlab-manager v5.0.10 enabled OK (python, jupyterlab_widgets)
        @jupyterlite/pyodide-kernel-extension v0.2.3 enabled OK (python, jupyterlite_pyodide_kernel)
        @jupyterlite/p5-kernel-extension v0.1.0 enabled OK (python, jupyterlite_p5_kernel)
        @jupyterlite/javascript-kernel-extension v0.2.3 enabled OK
        @timkpaine/jupyterlab_miami_nights v0.4.2 enabled OK (python, jupyterlab_miami_nights)


   The following extensions are outdated:
        jupyterlab-night
        bqplot
        jupyterlab-plotly
        
   Consider checking if an update is available for these packages.

Other labextensions (built into JupyterLab)
   app dir: /Library/Frameworks/Python.framework/Versions/3.12/share/jupyter/lab
        jupyter-repl-msgbridge v0.2.8 enabled OK
(scripts_env) delongzhai@Delongs-MacBook-Pro scripts % jupyter lite build       
static:jupyter-lite.json
.  pre_status:static:jupyter-lite.json
    tarball:         jupyterlite-app-0.2.3.tgz 9MB
    output:          /Users/delongzhai/Projects/scripts/_output
    lite dir:        /Users/delongzhai/Projects/scripts
    apps:            
    sourcemaps:      True
    unused packages: True
archive:archive
contents:contents
icons:icons
jupyterlite-pyodide-kernel-pyodide:pyodide
lite:jupyter-lite.json
mimetypes:jupyter-lite.json
serve:contents
settings:overrides
translation:translation
.  status:archive:archive
[LiteBuildApp] No archive (yet): scripts-jupyterlite.tgz
.  status:contents:contents
    contents: 0 files
.  status:icons:icons
    favicon files: 7 files
.  status:jupyterlite-pyodide-kernel-pyodide:pyodide
     URL: None
 archive: []
   cache: 0 files
   local: 0 files
.  status:lite:jupyter-lite.json
[LiteBuildApp]     jupyter-lite.(json|ipynb): 1 files
.  status:mimetypes:jupyter-lite.json
    filetypes:         26 
.  status:serve:contents
    url: http://127.0.0.1:8000/
    server: tornado
    headers:
.  status:settings:overrides
    overrides.json: 0
.  status:translation:translation
    translation files: 4 files
static:output_dir
-- pre_init:static:output_dir
static:unpack
-- init:static:unpack
federated_extensions:copy:ext:@jupyterlite/xeus-sqlite-kernel
federated_extensions:copy:ext:jupyterlab_pygments
federated_extensions:copy:ext:jupyterlab-night
federated_extensions:copy:ext:bqplot
federated_extensions:copy:ext:ipycanvas
federated_extensions:copy:ext:jupyter-matplotlib
federated_extensions:copy:ext:jupyter-leaflet
federated_extensions:copy:ext:@jupyterlite/xeus-lua-kernel
federated_extensions:copy:ext:ipyevents
federated_extensions:copy:ext:jupyterlab-plotly
federated_extensions:copy:ext:@jupyter-notebook/lab-extension
federated_extensions:copy:ext:@jupyterlab/geojson-extension
federated_extensions:copy:ext:@jupyterlab/fasta-extension
federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager
federated_extensions:copy:ext:@jupyterlite/pyodide-kernel-extension
federated_extensions:copy:ext:@jupyterlite/p5-kernel-extension
federated_extensions:copy:ext:@jupyterlite/javascript-kernel-extension
federated_extensions:copy:ext:@timkpaine/jupyterlab_miami_nights
-- pre_build:federated_extensions:copy:ext:@jupyterlite/xeus-sqlite-kernel
-- pre_build:federated_extensions:copy:ext:jupyterlab_pygments
-- pre_build:federated_extensions:copy:ext:jupyterlab-night
-- pre_build:federated_extensions:copy:ext:bqplot
-- pre_build:federated_extensions:copy:ext:ipycanvas
-- pre_build:federated_extensions:copy:ext:jupyter-matplotlib
-- pre_build:federated_extensions:copy:ext:jupyter-leaflet
-- pre_build:federated_extensions:copy:ext:@jupyterlite/xeus-lua-kernel
-- pre_build:federated_extensions:copy:ext:ipyevents
-- pre_build:federated_extensions:copy:ext:jupyterlab-plotly
-- pre_build:federated_extensions:copy:ext:@jupyter-notebook/lab-extension
-- pre_build:federated_extensions:copy:ext:@jupyterlab/geojson-extension
-- pre_build:federated_extensions:copy:ext:@jupyterlab/fasta-extension
-- pre_build:federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager
-- pre_build:federated_extensions:copy:ext:@jupyterlite/pyodide-kernel-extension
-- pre_build:federated_extensions:copy:ext:@jupyterlite/p5-kernel-extension
-- pre_build:federated_extensions:copy:ext:@jupyterlite/javascript-kernel-extension
-- pre_build:federated_extensions:copy:ext:@timkpaine/jupyterlab_miami_nights
icons:copy
lite:patch:repl/jupyter-lite.json
translation:copy
.  build:icons:copy
-- build:lite:patch:repl/jupyter-lite.json
-- build:translation:copy
federated_extensions:patch
federated_extensions:copy:theme:jupyterlab-night
federated_extensions:copy:theme:@timkpaine/jupyterlab_miami_nights
federated_extensions:settings
jupyterlite-pyodide-kernel-piplite:patch
mimetypes:patch
-- post_build:federated_extensions:patch
-- post_build:federated_extensions:copy:theme:jupyterlab-night
-- post_build:federated_extensions:copy:theme:@timkpaine/jupyterlab_miami_nights
-- post_build:federated_extensions:settings
.  post_build:jupyterlite-pyodide-kernel-piplite:patch
-- post_build:mimetypes:patch

And with that, I still don't see the jupyter-repl-msgbridge being included

@dwootton
Copy link

dwootton commented Jun 6, 2024

+1 Also running into this issue!

@martinRenou
Copy link
Member

martinRenou commented Jun 6, 2024 •
edited

Looking at the jupyterlite logs, it seems your extension is not being picked up:

federated_extensions:copy:ext:@jupyterlite/xeus-sqlite-kernel
federated_extensions:copy:ext:jupyterlab_pygments
federated_extensions:copy:ext:jupyterlab-night
federated_extensions:copy:ext:bqplot
federated_extensions:copy:ext:ipycanvas
federated_extensions:copy:ext:jupyter-matplotlib
federated_extensions:copy:ext:jupyter-leaflet
federated_extensions:copy:ext:@jupyterlite/xeus-lua-kernel
federated_extensions:copy:ext:ipyevents
federated_extensions:copy:ext:jupyterlab-plotly
federated_extensions:copy:ext:@jupyter-notebook/lab-extension
federated_extensions:copy:ext:@jupyterlab/geojson-extension
federated_extensions:copy:ext:@jupyterlab/fasta-extension
federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager
federated_extensions:copy:ext:@jupyterlite/pyodide-kernel-extension
federated_extensions:copy:ext:@jupyterlite/p5-kernel-extension
federated_extensions:copy:ext:@jupyterlite/javascript-kernel-extension
federated_extensions:copy:ext:@timkpaine/jupyterlab_miami_nights

Could your jupyterlite installation be in another environment than your jupyterlab installation (the one where you installed jupyter-repl-msgbridge)?

Side note: if using jupyterlite-xeus, you can simply provide an environment.yml file at the place where you build your jupyterlite site and specify as a dependency (requires at least conda, micromamba or mamba installed):

name: xeus-lite-wasm
channels:
  - https://repo.mamba.pm/emscripten-forge
  - conda-forge
dependencies:
  - xeus-sqlite
  - xeus-python
  - pip: 
    - jupyter-repl-msgbridge

That will automatically pick up your labextension.

@dwootton
Copy link

dwootton commented Jun 6, 2024 •
edited

FWIW– I just realized my extensions package.json had a different jupyterlab config than the extensions that were getting picked up. When I updated it, the jupyterlite build started to pick it up.
now my top level jupyterlab config in package.json looks like:

"jupyterlab": { "extension": true, "outputDir": "notebook-to-url-lib", "_build": { "load": "./index.js", "extension": "./index.js" } },

and it now is getting loaded appropriately in my deployed jupyterlite!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jtpio @delongZhai @martinRenou @dwootton