Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to JupyterLab 4 and Notebook 7 #1019

Merged
merged 102 commits into from Sep 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
5487b92
Update JupyterLab and Lumino
jtpio May 31, 2023
96ce2df
Integrity
jtpio Aug 21, 2023
4f60392
Fix docs build
jtpio Aug 21, 2023
7910e5c
Cleanups
jtpio Aug 23, 2023
b1646fc
Lint
jtpio Aug 23, 2023
cef7a25
Lint
jtpio Aug 23, 2023
f6bbf07
try fix webpack build
jtpio Aug 23, 2023
46c52e2
fix sw name
jtpio Aug 23, 2023
1acfec0
debug webpack
jtpio Aug 23, 2023
57ddf99
wip building all apps (nb7, repl)
jtpio Aug 25, 2023
1715454
Build the repl app
jtpio Aug 30, 2023
89e3f35
Small cleanup
jtpio Aug 31, 2023
07820db
Built tree
jtpio Aug 31, 2023
f71b17a
Build notebooks
jtpio Aug 31, 2023
63ac96b
Add missing top bar plugin
jtpio Aug 31, 2023
c13c83f
Build edit
jtpio Aug 31, 2023
249d25d
Build consoles
jtpio Aug 31, 2023
fa992c9
Fix resolutions
jtpio Aug 31, 2023
33e4b90
Try fix dodo
jtpio Aug 31, 2023
f088999
Fix markdown rendering
jtpio Aug 31, 2023
27d4a4b
Fix markedparser
jtpio Aug 31, 2023
5b35398
Small fixes
jtpio Sep 1, 2023
c3df092
Lint
jtpio Sep 1, 2023
dee6375
Remove drawio extension
jtpio Sep 1, 2023
9bb0d95
Lint
jtpio Sep 1, 2023
6075bdd
Add opener plugin
jtpio Sep 1, 2023
617c268
Clean up playwright config
jtpio Sep 1, 2023
17e39e1
Fix docs URLs
jtpio Sep 1, 2023
31d5d83
Undo change in jupyter-lite.json
jtpio Sep 1, 2023
dbfd6dc
Fix resolutions
jtpio Sep 1, 2023
d93aa6c
Some ui tests fixes
jtpio Sep 1, 2023
57784cc
Update playwright config
jtpio Sep 1, 2023
166c4d0
More ui tests fixes
jtpio Sep 1, 2023
552f99c
Add workflow to update snapshots
jtpio Sep 1, 2023
fda7b47
More ui tests updates
jtpio Sep 1, 2023
adec31f
Lint
jtpio Sep 1, 2023
997b4f2
Update copy shareable link logic
jtpio Sep 4, 2023
f546b22
Bump galata
jtpio Sep 4, 2023
4de75b5
Update to the latest patch releases
jtpio Sep 4, 2023
86d01d7
Remove custom `waitForApplication`
jtpio Sep 4, 2023
4749349
Remove TODO
jtpio Sep 4, 2023
7def1ca
Add retries
jtpio Sep 5, 2023
50b2b0b
Trigger snapshot update
jtpio Sep 5, 2023
2460cd9
Try more triggers
jtpio Sep 5, 2023
fb0ae2a
Fix workflow
jtpio Sep 5, 2023
dca09f3
Undo test changes
jtpio Sep 5, 2023
c7efe96
Add missing setting editor to tree
jtpio Sep 5, 2023
cca11f0
Fix some plugins
jtpio Sep 5, 2023
f67823d
Sync demo env
jtpio Sep 6, 2023
2e67f84
Fix file browser CSS
jtpio Sep 6, 2023
c25a4e4
Minor webpack tweaks
jtpio Sep 6, 2023
f8ba8b3
Align dependencies
jtpio Sep 6, 2023
b241109
Update react, set same resolutions as lab
jtpio Sep 6, 2023
f9300a2
Try micromamba action on windows CI
jtpio Sep 6, 2023
3f1d51e
use newer action
jtpio Sep 6, 2023
ccfff4f
debug windows
jtpio Sep 6, 2023
d07f207
debug
jtpio Sep 6, 2023
dd5a97f
more dodo debug
jtpio Sep 6, 2023
76ec1f0
fix script name
jtpio Sep 6, 2023
8bc3b90
Enable playwright trace
jtpio Sep 7, 2023
97fd053
Use sys.executable to build UI tests
jtpio Sep 7, 2023
8f5099c
debug doit
jtpio Sep 7, 2023
3a7b288
Revert "Use sys.executable to build UI tests"
jtpio Sep 7, 2023
dc28d33
debug doit
jtpio Sep 7, 2023
cd14a26
more doit debugging
jtpio Sep 7, 2023
41f7d78
debug dodoooo
jtpio Sep 7, 2023
37a09c6
debug doit
jtpio Sep 7, 2023
b6fb004
Recreate snapshots on failure
jtpio Sep 7, 2023
94cf098
Update test theme
jtpio Sep 7, 2023
15de5b9
Set viewport size for embed tests
jtpio Sep 7, 2023
48876e7
lint
jtpio Sep 7, 2023
c18aa3e
Re-add create dir helper
jtpio Sep 7, 2023
f78c29d
Update Chromium snapshots
jtpio Sep 7, 2023
a7559b6
Use miami nights theme
jtpio Sep 7, 2023
e0bc35c
Fix link
jtpio Sep 7, 2023
9f665e1
fix link
jtpio Sep 7, 2023
dce169b
Revert back to darcula for now
jtpio Sep 7, 2023
d9febe3
Add tmp workaround for galata in firefox
jtpio Sep 8, 2023
b71725d
Update firefox snapshots
jtpio Sep 8, 2023
f7981fb
Fix sharing from notebook interface
jtpio Sep 8, 2023
955a374
Bump jupyterlab and notebook patch versions
jtpio Sep 8, 2023
1597da5
Lint
jtpio Sep 8, 2023
100bb4e
remove todos in update snapshot workflow
jtpio Sep 8, 2023
e76ed36
Fix contributing guide
jtpio Sep 8, 2023
caeb6c5
Try fix redirect issue
jtpio Sep 8, 2023
60120ec
Update packages/kernel/src/kernels.ts
jtpio Sep 11, 2023
75aab4c
Set trailingComma to es5 for now
jtpio Sep 11, 2023
ebc6fb7
Remove unneeded publicpath files
jtpio Sep 11, 2023
4210387
Link to webrtc-provider issue for lab 4 compat
jtpio Sep 11, 2023
97d08dd
Use protocol name from services
jtpio Sep 11, 2023
76813bd
Remove `jupyterlab-tour` for now
jtpio Sep 11, 2023
c48f110
Add link to jupyterlab-tour issue
jtpio Sep 11, 2023
fbc2477
Add link to rtc issue
jtpio Sep 11, 2023
8eb430c
Update readme screenshots
jtpio Sep 11, 2023
6dd5905
Add notebook to the list of federated extensions
jtpio Sep 11, 2023
07d0552
Remove notebook from jlab_core for now
jtpio Sep 12, 2023
3bf4d6d
Re-enable webpack cache
jtpio Sep 12, 2023
68255b4
Revert "Add notebook to the list of federated extensions"
jtpio Sep 12, 2023
0ee201a
Update kernelspy version
jtpio Sep 12, 2023
1457271
Add support for listing kernels
jtpio Sep 12, 2023
4155195
Disable announcements in lab
jtpio Sep 12, 2023
aed578b
Point to stable docs and demo in the README
jtpio Sep 13, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
29 changes: 16 additions & 13 deletions .binder/environment.yml
Expand Up @@ -14,7 +14,7 @@ dependencies:
# demo stuff
- ipywidgets >=8.0.4,<9
- jupyter-server-proxy
- retrolab >=0.3.21,<0.4
- notebook >=7.0.3,<8
# extra docs tools
- sphinx-autobuild
# avoid 200mb mkl
Expand All @@ -24,18 +24,21 @@ dependencies:
# rewriting `anaconda.org` links to `conda-forge`'s GitHub releases
### FEDERATED EXTENSIONS ###
- ipycytoscape
- jupyterlab_miami_nights
- jupyterlab_miami_nights >=0.4.0
- jupyterlab_widgets >=3.0.4,<4.0.0
- jupyterlab-drawio
- jupyterlab-fasta >=3,<4
- jupyterlab-geojson >=3,<4
- jupyterlab-github
- jupyterlab-kernelspy
- jupyterlab-markup
- jupyterlab-myst
- jupyterlab-tour >=3.1.0
- jupyterlab-webrtc-docprovider >=0.1.1
- theme-darcula >=3,<4
- jupyterlab-github >=4.0.0
- jupyterlab-kernelspy >=4.0.0
- jupyterlab-myst >=2.0.2
# TODO: re-add after JupyterLab 4 compatibility
bollwyvl marked this conversation as resolved.
Show resolved Hide resolved
# https://github.com/jupyterlab-contrib/jupyterlab-tour/issues/76
# - jupyterlab-tour >=3.1.0
# TODO: re-add after JupyterLab 4 compatibility
bollwyvl marked this conversation as resolved.
Show resolved Hide resolved
# https://github.com/jupyterlite/jupyterlab-webrtc-docprovider/issues/16
# - jupyterlab-webrtc-docprovider >=0.1.1
- jupyterlab-night
- jupyterlite-pyodide-kernel >=0.1.1
# if a package is pinned with == `doit check` will ensure the version is pinned
# in `docs/_build/_static/files/*.ipynb`
- bqplot
Expand All @@ -50,21 +53,21 @@ dependencies:
# the chunk below gets copied from `../docs/environment.yml` by `doit env`
### DOCS ENV ###
# runtimes
- nodejs >=18,<19
- nodejs >=20,<21
- python >=3.10,<3.11
# build
- doit >=0.34,<1
- hatch >=1.6.3,<2
- jsonschema >=3
- jupyter-server-mathjax >=0.2.3
- jupyterlab >=3.5.3,<3.6
- jupyterlab >=4.0.5,<5.0
- notebook >=7.0.1,<8.0
- pip
- pkginfo
- python-libarchive-c
- requests_cache
- twine
- wheel
- yarn <2
# cli
- wheel
# docs
Expand Down
4 changes: 2 additions & 2 deletions .binder/jupyterlite-proxy.json
Expand Up @@ -2,14 +2,14 @@
"ServerProxy": {
"servers": {
"jupyterlite-py": {
"command": ["yarn", "serve:py", "{port}"],
"command": ["jlpm", "serve:py", "{port}"],
"launcher_entry": {
"icon_path": "docs/_static/icon.svg",
"title": "JupyterLite (py)"
}
},
"jupyterlite-js": {
"command": ["yarn", "serve"],
"command": ["jlpm", "serve"],
"launcher_entry": {
"icon_path": "docs/_static/icon.svg",
"title": "JupyterLite (js)"
Expand Down
2 changes: 2 additions & 0 deletions .eslintignore
Expand Up @@ -3,6 +3,8 @@ lint-staged.config.js
scripts/serve.js

.yarn-packages
.yarn
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use the config to not use junk up the root with pnp? Or not use the pnp features at all?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't .pnp already ignored on the line below?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point would be to remove all the added .yarn and .pnp entries from the .gitignores and never see them on disk

  • are we even using pnp?
    • let's not create .pnp/.yarn at all, if possible
    • we're using node_modules
  • if we do need to create it for nx or lerna or whatever to be happy,
    • can we put it in build/.(pnp|yarn) or something

As I've been exploring these new "features," (still haven't found what value the new yarn adds, other than an incompatible lockfile) I've ended up with some patterns like this to improve the cacheability of node_modules and reduce the extreme amount of noise from e.g. jlpm:

enableInlineBuilds: false
enableTelemetry: false
httpTimeout: 60000
nodeLinker: node-modules
npmRegistryServer: https://registry.npmjs.org/
installStatePath: ./build/.cache/yarn/install-state.gz
cacheFolder: ./build/.cache/yarn/cache
logFilters:
  - code: YN0006
    level: discard
  - code: YN0002
    level: discard
  - code: YN0007
    level: discard
  - code: YN0013
    level: discard
  - code: YN0019
    level: discard
  - code: YN0008
    level: discard

As well as some env vars:

      "NX_CACHE_DIRECTORY": f"{P.ROOT}/build/.cache/nx",
      "NX_PROJECT_GRAPH_CACHE_DIRECTORY": f"{P.ROOT}/build/.cache/nx",
      "YARN_CACHE_FOLDER": f"${P.ROOT}/build/.cache/yarn"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason we should have this configuration here in the JupyterLite repo and not upstream in the JupyterLab repo?

If not, should we open an issue about it in the JupyterLab repo? (or maybe there is already one)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, dunno... this is purely from a "digging out from under a mountain of extensions" perspective, again, haven't really been tracking what upstream is doing with the new yarn stuff.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, and this could also go into package.json (unlike prettierignore, for some reason, buh)

.pnp*
node_modules
**/build
**/lib
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/build.yml
Expand Up @@ -202,8 +202,8 @@ jobs:
run: doit dev:py:jupyterlite-core
- name: Build the JavaScript kernel
run: |
yarn --frozen-lockfile
yarn run build:js-kernel
jlpm
jlpm run build:js-kernel
- name: Docs (app)
run: doit docs:app:build
- name: Docs (app archive)
Expand Down
65 changes: 65 additions & 0 deletions .github/workflows/playwright-update.yml
@@ -0,0 +1,65 @@
name: Update Playwright Snapshots

on:
issue_comment:
types: [created, edited]

permissions:
contents: write
pull-requests: write

jobs:
update-snapshots:
if:
${{ github.event.issue.pull_request && contains(github.event.comment.body, 'update
playwright snapshots') }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
browser: [firefox, chromium]

steps:
- name: Checkout
uses: actions/checkout@v3

- name: Checkout the branch from the PR that triggered the job
run: |
# PR branch remote must be checked out using https URL
git config --global hub.protocol https

hub pr checkout ${{ github.event.issue.number }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Base Setup
uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1

- name: Build
uses: ./.github/actions/build-dist

- name: Install the package
run: |
cd dist
python -m pip install -vv jupyterlite_core*.whl

# disable git hooks
git config core.hooksPath no-hooks

- name: Install the test dependencies
run: |
cd ui-tests
jlpm
jlpm build
jlpm playwright install

- name: Update snapshots
uses: jupyterlab/maintainer-tools/.github/actions/update-snapshots@v1
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
npm_client: jlpm
test_folder: ui-tests
start_server_script: 'null'
update_script: test:update --browser ${{ matrix.browser }}
env:
DEBUG: pw:webserver
17 changes: 10 additions & 7 deletions .github/workflows/ui-tests.yml
Expand Up @@ -16,14 +16,15 @@ jobs:

- name: Install JupyterLite
run: |
python3 -m pip install importlib_metadata 'doit >=0.34,<1' jupyter_core jupyterlab~=3.2
python3 -m pip install importlib_metadata 'doit >=0.34,<1' jupyter_core jupyterlab~=4.0
python3 -m pip install --find-links dist --no-index jupyterlite_core

- name: Build JupyterLite
run: |
cd ui-tests
jlpm
# Build the JupyterLite website
yarn build
jlpm build

- name: Upload the JupyterLite website
uses: actions/upload-artifact@v3
Expand Down Expand Up @@ -58,14 +59,16 @@ jobs:

- name: Install dependencies and browser
run: |
# Install JupyterLab to get jlpm
python -m pip install jupyterlab~=4.0
cd ui-tests
yarn --frozen-lockfile
yarn playwright install
jlpm
jlpm playwright install

- name: Test
run: |
cd ui-tests
yarn run test --browser ${{ matrix.browser }}
jlpm run test --browser ${{ matrix.browser }}

- name: Upload Playwright Test assets
if: always()
Expand All @@ -88,9 +91,9 @@ jobs:
run: |
cd ui-tests
# remove previous snapshots from other browser
yarn run clean:snapshots
jlpm run clean:snapshots
# generate new snapshots
yarn run test:update --browser ${{ matrix.browser }}
jlpm run test:update --browser ${{ matrix.browser }}

- name: Upload updated snapshots
if: failure()
Expand Down
13 changes: 6 additions & 7 deletions .github/workflows/win-dev.yml
Expand Up @@ -12,6 +12,7 @@ env:
CACHE_EPOCH: 5
CI: 0
WIN_DEV_IN_CI: 1
PYTHONUTF8: 1

jobs:
win-dev:
Expand All @@ -26,12 +27,13 @@ jobs:
git config --global core.eol lf
- name: Checkout
uses: actions/checkout@v3
- name: Conda
uses: conda-incubator/setup-miniconda@v2
- uses: mamba-org/setup-micromamba@v1
with:
micromamba-version: '1.4.9-0'
environment-file: .binder/environment.yml
miniforge-variant: Mambaforge
use-mamba: true
init-shell: cmd.exe
cache-environment: true
post-cleanup: 'all'
- name: Cache (node_modules)
uses: actions/cache@v3
id: cache-node-modules
Expand All @@ -49,6 +51,3 @@ jobs:
run: doit dev
- name: Lint
run: doit lint || doit lint || doit lint
- name: Move conda cache
run: |-
move C:\Users\runneradmin\conda_pkgs_dir C:\Users\runneradmin\not_conda_pkgs_dir
21 changes: 17 additions & 4 deletions .gitignore
Expand Up @@ -2,6 +2,8 @@
lib/
node_modules/
.yarn-packages/
.yarn/
.pnp*
*.egg-info/
.ipynb_checkpoints
*.tsbuildinfo
Expand Down Expand Up @@ -128,12 +130,13 @@ app/extensions
app/service-worker-*.js

# generated html
app/index.app.template.html
app/lab/index.html
app/repl/index.html
app/retro/consoles/index.html
app/retro/edit/index.html
app/retro/notebooks/index.html
app/retro/tree/index.html
app/consoles/index.html
app/edit/index.html
app/notebooks/index.html
app/tree/index.html

*.log

Expand All @@ -155,3 +158,13 @@ _output
ui-tests/playwright-report
ui-tests/test-results
ui-tests/ui-tests-app

# yarn >=2.x local files
.yarn/*
.pnp.*
ui-tests/.yarn/*
ui-tests/.pnp.*

# may be generated by jupyter lite build when in dev mode
app/api
app/static
2 changes: 2 additions & 0 deletions .prettierignore
Expand Up @@ -5,6 +5,8 @@

.ipynb_checkpoints/
.yarn-packages
.yarn
.pnp*
build
docs/_build
node_modules
Expand Down
3 changes: 2 additions & 1 deletion .prettierrc
@@ -1,5 +1,6 @@
{
"singleQuote": true,
"printWidth": 88,
"proseWrap": "always"
"proseWrap": "always",
"trailingComma": "es5"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, on the "clean up the root" tip, could move these few lines into package.json.

}
6 changes: 0 additions & 6 deletions .yarnrc

This file was deleted.

6 changes: 6 additions & 0 deletions .yarnrc.yml
@@ -0,0 +1,6 @@
enableImmutableInstalls: false
enableInlineBuilds: false
enableTelemetry: false
httpTimeout: 60000
nodeLinker: node-modules
npmRegistryServer: 'https://registry.yarnpkg.com'