How to install JupyterLab extensions

[ellisonbg]

Problem

I am working with @jweill-aws to improve the retrolab shell to handle the L and R areas. Wondering if there is a way to install JupyterLab extensions that use the L/R area into retrolab - such as the table of contents, running panel, debugger, etc.

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[JasonWeill]

Related issue: jupyter/notebook#6403

I've been working on adding left and right areas to RetroLab. I spent some time earlier today trying to bring the toc-extension and toc directories over from JupyterLab, but when I pulled in all the dependencies required to build that code, a single changelist included over 500 files. In addition, the extension expects to be added to an ILabShell, whereas RetroLab uses an IRetroShell interface. In addition, TableOfContents isn't properly recognized as a Widget because "Type 'import("/Users/jweill/git/retrolab/packages/apputils/node_modules/@lumino/widgets/types/layout").Layout | null' is not assignable to type 'import("/Users/jweill/git/retrolab/node_modules/@lumino/widgets/types/layout").Layout | null'."

Although we have had a previous discussion about merging RetroLab into the JupyterLab simple interface (#257) the work I've been doing for jupyter/notebook#6403 presumes that RetroLab will continue to exist on its own. The work I've done so far suggests that we might need data structures to bridge ILabShell with IRetroShell in order to use JupyterLab extensions like toc with both.

My work-in-progress branch for jupyter/notebook#6403: https://github.com/jweill-aws/retrolab/tree/left-right-widgets

[jtpio]

Thanks @ellisonbg @jweill-aws.

As a first step we'll need to add support for the left and right areas in the RetroShell:

retrolab/packages/application/src/shell.ts

Line 200 in 2a96903

export type Area = 'main' | 'top' | 'menu';

Maybe we can use a StackedPanel without the TabBar to place the widgets in it.

Additional upstream dependencies and extensions are added to the package.json here:

retrolab/app/package.json

Lines 94 to 131 in 2a96903

	"dependencies": {
	"@jupyterlab/application-extension": "^3.2.0",
	"@jupyterlab/apputils-extension": "^3.2.0",
	"@jupyterlab/celltags": "^3.2.0",
	"@jupyterlab/codemirror-extension": "^3.2.0",
	"@jupyterlab/completer-extension": "^3.2.0",
	"@jupyterlab/console-extension": "^3.2.0",
	"@jupyterlab/coreutils": "~5.2.0",
	"@jupyterlab/docmanager-extension": "^3.2.0",
	"@jupyterlab/docprovider-extension": "^3.2.0",
	"@jupyterlab/filebrowser-extension": "^3.2.0",
	"@jupyterlab/fileeditor-extension": "^3.2.0",
	"@jupyterlab/hub-extension": "^3.2.0",
	"@jupyterlab/javascript-extension": "^3.2.0",
	"@jupyterlab/json-extension": "^3.2.0",
	"@jupyterlab/mainmenu-extension": "^3.2.0",
	"@jupyterlab/mathjax2-extension": "^3.2.0",
	"@jupyterlab/notebook-extension": "^3.2.0",
	"@jupyterlab/pdf-extension": "^3.2.0",
	"@jupyterlab/rendermime-extension": "^3.2.0",
	"@jupyterlab/running-extension": "^3.2.0",
	"@jupyterlab/shortcuts-extension": "^3.2.0",
	"@jupyterlab/terminal-extension": "^3.2.0",
	"@jupyterlab/theme-dark-extension": "^3.2.0",
	"@jupyterlab/theme-light-extension": "^3.2.0",
	"@jupyterlab/tooltip-extension": "^3.2.0",
	"@jupyterlab/translation-extension": "^3.2.0",
	"@jupyterlab/vega5-extension": "^3.2.0",
	"@retrolab/application": "^0.3.12",
	"@retrolab/application-extension": "^0.3.12",
	"@retrolab/console-extension": "^0.3.12",
	"@retrolab/docmanager-extension": "^0.3.12",
	"@retrolab/help-extension": "^0.3.12",
	"@retrolab/notebook-extension": "^0.3.12",
	"@retrolab/terminal-extension": "^0.3.12",
	"@retrolab/tree-extension": "^0.3.12",
	"@retrolab/ui-components": "^0.3.12"
	},

And listed here:

retrolab/app/package.json

Lines 158 to 229 in 2a96903

	"extensions": [
	"@retrolab/application-extension",
	"@retrolab/console-extension",
	"@retrolab/docmanager-extension",
	"@retrolab/help-extension",
	"@retrolab/notebook-extension",
	"@retrolab/terminal-extension",
	"@retrolab/tree-extension",
	"@jupyterlab/application-extension",
	"@jupyterlab/apputils-extension",
	"@jupyterlab/codemirror-extension",
	"@jupyterlab/completer-extension",
	"@jupyterlab/console-extension",
	"@jupyterlab/docmanager-extension",
	"@jupyterlab/filebrowser-extension",
	"@jupyterlab/fileeditor-extension",
	"@jupyterlab/hub-extension",
	"@jupyterlab/mainmenu-extension",
	"@jupyterlab/mathjax2-extension",
	"@jupyterlab/notebook-extension",
	"@jupyterlab/rendermime-extension",
	"@jupyterlab/running-extension",
	"@jupyterlab/shortcuts-extension",
	"@jupyterlab/terminal-extension",
	"@jupyterlab/theme-dark-extension",
	"@jupyterlab/theme-light-extension",
	"@jupyterlab/tooltip-extension",
	"@jupyterlab/translation-extension"
	],
	"singletonPackages": [
	"@jupyterlab/application",
	"@jupyterlab/apputils",
	"@jupyterlab/celltags",
	"@jupyterlab/codeeditor",
	"@jupyterlab/completer",
	"@jupyterlab/console",
	"@jupyterlab/coreutils",
	"@jupyterlab/docmanager",
	"@jupyterlab/docprovider",
	"@jupyterlab/filebrowser",
	"@jupyterlab/fileeditor",
	"@jupyterlab/mainmenu",
	"@jupyterlab/notebook",
	"@jupyterlab/observables",
	"@jupyterlab/outputarea",
	"@jupyterlab/rendermime",
	"@jupyterlab/rendermime-interfaces",
	"@jupyterlab/services",
	"@jupyterlab/settingregistry",
	"@jupyterlab/shared-models",
	"@jupyterlab/statedb",
	"@jupyterlab/statusbar",
	"@jupyterlab/terminal",
	"@jupyterlab/tooltip",
	"@jupyterlab/translation",
	"@jupyterlab/ui-components",
	"@lumino/algorithm",
	"@lumino/application",
	"@lumino/commands",
	"@lumino/coreutils",
	"@lumino/disposable",
	"@lumino/domutils",
	"@lumino/dragdrop",
	"@lumino/messaging",
	"@lumino/properties",
	"@lumino/signaling",
	"@lumino/virtualdom",
	"@lumino/widgets",
	"react",
	"react-dom",
	"yjs"
	],

And loaded here:

retrolab/app/index.js

Lines 156 to 212 in 2a96903

	switch (page) {
	case 'tree': {
	baseMods = baseMods.concat([
	require('@jupyterlab/filebrowser-extension').default.filter(({ id }) =>
	[
	'@jupyterlab/filebrowser-extension:browser',
	'@jupyterlab/filebrowser-extension:download',
	'@jupyterlab/filebrowser-extension:file-upload-status',
	'@jupyterlab/filebrowser-extension:open-with',
	'@jupyterlab/filebrowser-extension:share-file'
	].includes(id)
	),
	require('@retrolab/tree-extension'),
	require('@jupyterlab/running-extension')
	]);
	break;
	}
	case 'notebooks': {
	baseMods = baseMods.concat([
	require('@jupyterlab/completer-extension').default.filter(({ id }) =>
	['@jupyterlab/completer-extension:notebooks'].includes(id)
	),
	require('@jupyterlab/tooltip-extension').default.filter(({ id }) =>
	[
	'@jupyterlab/tooltip-extension:manager',
	'@jupyterlab/tooltip-extension:notebooks'
	].includes(id)
	)
	]);
	break;
	}
	case 'consoles': {
	baseMods = baseMods.concat([
	require('@jupyterlab/completer-extension').default.filter(({ id }) =>
	['@jupyterlab/completer-extension:consoles'].includes(id)
	),
	require('@jupyterlab/tooltip-extension').default.filter(({ id }) =>
	[
	'@jupyterlab/tooltip-extension:manager',
	'@jupyterlab/tooltip-extension:consoles'
	].includes(id)
	)
	]);
	break;
	}
	case 'edit': {
	baseMods = baseMods.concat([
	require('@jupyterlab/completer-extension').default.filter(({ id }) =>
	['@jupyterlab/completer-extension:files'].includes(id)
	),
	require('@jupyterlab/filebrowser-extension').default.filter(({ id }) =>
	['@jupyterlab/filebrowser-extension:browser'].includes(id)
	)
	]);
	break;
	}
	}

So for @jupyterlab/toc-extension the change should normally be pretty minimal. The extensions in https://github.com/jupyterlab/retrolab/tree/main/packages are the ones specific to RetroLab.

Here is an example PR that added @jupyterlab/hub-extension to RetroLab: #148

[tonyfast]

are these left and right panels going to make significant visual changes relative to the classic interface? as we sunrise notebook, we've talked about having nearly the same interface as classic.

i think folks would love to use these tools. i could imagine the new features showing up over time. some stories i could imagine:

• i'm a user looking to write code and narrative in a classic notebook, i expect to see the form of the classic interface, if i don't the illusion is up
• i'm a user that writes markdown with headings in it, i'm alerted of the table of contents feature then i start outlining my ideas as a table of contents
• i'm a user writing code and throw an exception, the debugger alerts me of its existence.

if we are talking about adding the edges of retrolab, is the status bar a feature we'd like to have? while working in retro i find myself looking for a status bar that isn't there. it's a good feature of jupyterlab.

there is definitely precedence for these features. nbcontrib has a table of contents extensions that seems to dock on a side or float. they also add a navigate menu item that seems practical. there is definitely precen

[jtpio]

are these left and right panels going to make significant visual changes relative to the classic interface? as we sunrise notebook, we've talked about having nearly the same interface as classic.

I think we should make them as invisible as possible to normal users wanting the most vanilla classic experience. Just like it is the case now. Hidden by default should be the default.

As hinted in https://github.com/jupyterlab/retrolab/issues/268#issuecomment-960622997, we should check the current behavior in classic when some nbextensions are installed (for example toc2).

if we are talking about adding the edges of retrolab, is the status bar a feature we'd like to have?

Probably, but this should be hidden by default too. Otherwise we might end up re-implementing JupyterLab.

[JasonWeill]

@jtpio Thanks again for your detailed replies! I have a change in progress on my development branch to add the toc extension. As you had suggested, it uses a StackedPanel and no TabBar. I ran into significant compilation problems after incorporating toc-extension by copying the contents of packages/toc-extension over from jupyterlab to retrolab. For example, toc-extension expects the shell to be an instance of ILabShell, but RetroLab instead uses IRetroShell.

After modifying the toc-extension code to use the appropriate shell interface, I found an error because of conflicts between versons of @lumino/widgets. See below. I can try to remove the added packages and instead modify the package.json file as you stated in your comment on Saturday. Thanks again for your help!

Argument of type 'TableOfContents' is not assignable to parameter of type 'Widget'.
  The types of 'title.owner.layout' are incompatible between these types.
  Type 'import("/Users/jweill/git/retrolab/packages/apputils/node_modules/@lumino/widgets/types/layout").Layout | null' is not assignable to type 'import("/Users/jweill/git/retrolab/node_modules/@lumino/widgets/types/layout").Layout | null'.
  Type 'import("/Users/jweill/git/retrolab/packages/apputils/node_modules/@lumino/widgets/types/layout").Layout' is not assignable to type 'import("/Users/jweill/git/retrolab/node_modules/@lumino/widgets/types/layout").Layout'.
  Types of property 'removeWidget' are incompatible.
  Type '(widget: import("/Users/jweill/git/retrolab/packages/apputils/node_modules/@lumino/widgets/types/widget").Widget) => void' is not assignable to type '(widget: import("/Users/jweill/git/retrolab/node_modules/@lumino/widgets/types/widget").Widget) => void'.
  Types of parameters 'widget' and 'widget' are incompatible.
  Type 'Widget' is missing the following properties from type 'Widget': hiddenMode, _hiddenMode

[jtpio]

I ran into significant compilation problems after incorporating toc-extension by copying the contents of packages/toc-extension over from jupyterlab to retrolab. For example, toc-extension expects the shell to be an instance of ILabShell, but RetroLab instead uses IRetroShell.

We don't need to copy the toc-extension package to the retrolab repo. Instead we use the approach mentioned in #272 (comment) to incorporate it as an external dependency, so it can be reused as is.

At startup the @jupyterlab/toc-extension extension will add the widget to the left area: https://github.com/jupyterlab/jupyterlab/blob/114159269a76c4a8d4c1e725e882d7c8a7791869/packages/toc-extension/src/index.ts#L95

For example, toc-extension expects the shell to be an instance of ILabShell, but RetroLab instead uses IRetroShell.

Ah right, I thought it was adding the widget via app.shell already. So from here we have two options:

1. make a PR to jupyterlab to add the toc via app.shell instead of labShell. So labShell.add(toc, 'left', { rank: 400 }); becomes app.shell.add(toc, 'left', { rank: 400 });. We could then make the ILabShell dependency optional in the upstream @jupyterlab/toc:plugin: https://github.com/jupyterlab/jupyterlab/blob/114159269a76c4a8d4c1e725e882d7c8a7791869/packages/toc-extension/src/index.ts#L232
2. we start making RetroShell a LabShell (as discussed in Merge RetroLab with the JupyterLab Simple Interface? #257 (comment))

It would have been nice to start with option 1, so we only add what is needed. Instead of taking the lab shell and trimming / hiding what we don't want.

I found an error because of conflicts between versons of @lumino/widgets

Probably JupyterLab is using more recent versions of the Lumino packages. We can update them in RetroLab with by running this command:

jlpm run update:dependency --regex @lumino/ latest

[jtpio]

1. make a PR to jupyterlab to add the toc via app.shell instead of labShell

For reference there were a couple of upstream PRs to make some of the plugins more easily reusable in RetroLab. For example:

• Make the filebrowser plugins more reusable jupyterlab#9667
• Make some dependencies optional for the code console plugin jupyterlab#9467
• Make ITreePathUpdater optional in file browser plugin jupyterlab#9442
• Make ILabShell optional in the filebrowser factory plugin jupyterlab#9439

[JasonWeill]

Thanks again! I'm working on removing ILabShell from packages/toc-extension/src/index.ts, replaces references to labShell with app.shell, but this line now indicates an error because currentChanged is not a property of IShell:
https://github.com/jupyterlab/jupyterlab/blob/114159269a76c4a8d4c1e725e882d7c8a7791869/packages/toc-extension/src/index.ts#L192-L193

I'll see if I can do a little more refactoring. Should I open an issue in @jupyterlab/jupyterlab to capture this work?

[jtpio]

this line now indicates an error because currentChanged is not a property of IShell:

Right. So usually we make ILabShell an optional dependency (instead of requires). And guard the access to currentChanged with if (labShell) {}. This pattern is heavily used across the jupyterlab code base.

[JasonWeill]

Created issue and PR (jupyterlab/jupyterlab#11420) mentioned above to modify the toc extension for compatibility with RetroLab.

[jtpio]

Also if we want to move forward with this, we can temporarily vendor the code in the toc-extension: https://github.com/jupyterlab/jupyterlab/blob/master/packages/toc-extension/src/index.ts.

And add it to https://github.com/jupyterlab/retrolab/blob/main/packages/application-extension/src/index.ts for now to avoid creating a new package since it's temporary anyway.

Until a new release of the upstream @jupyterlab includes the change in jupyterlab/jupyterlab#11420. Then we can remove the vendored code and use the approach mentioned in #272 (comment).

That way we can still make progress and start opening a draft PR.

[JasonWeill]

Closing in favor of jupyter/notebook#6326.