Update to jupyterlab 3.1.0rc2

[bollwyvl]

References

• fixes Upgrade to JupyterLab 3.1.0rc2 #282
• RTD
• binder

Code changes

• bump baseline verisons
• resolve yarn, environment
• check package versions vs imports in check
• handle the context plugin

  • didn't have to do anything 🎉

• reorganize non-kernel/shell patches/mocks while investigating demo fails

  • each patch/mock is now wrapped in a try, and no longer brings down the whole runtime if one fails

• retro stuff??

User-facing changes

thing	screenshot
see context menu

Backwards-incompatible changes

• ???

[bollwyvl]

Looks like the altair example is failing on restart-and-run-all....

/lib/python3.8/site-packages/pandas/compat/__init__.py:117: UserWarning: Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError.
  warnings.warn(msg)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-1454662d1682> in <module>
----> 1 import altair as alt
      2 import pandas as pd
      3 
      4 source = pd.DataFrame({
      5     'a': ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I'],

/lib/python3.8/site-packages/altair/__init__.py in <module>
      2 __version__ = "4.1.0"
      3 
----> 4 from .vegalite import *
      5 from . import examples
      6 

/lib/python3.8/site-packages/altair/vegalite/__init__.py in <module>
      1 # flake8: noqa
----> 2 from .v4 import *

/lib/python3.8/site-packages/altair/vegalite/v4/__init__.py in <module>
      1 # flake8: noqa
----> 2 from .schema import *
      3 from .api import *
      4 
      5 from ...datasets import list_datasets, load_dataset

/lib/python3.8/site-packages/altair/vegalite/v4/schema/__init__.py in <module>
      1 # flake8: noqa
----> 2 from .core import *
      3 from .channels import *
      4 SCHEMA_VERSION = 'v4.8.1'
      5 SCHEMA_URL = 'https://vega.github.io/schema/vega-lite/v4.8.1.json'

/lib/python3.8/site-packages/altair/vegalite/v4/schema/core.py in <module>
      2 # tools/generate_schema_wrapper.py. Do not modify directly.
      3 
----> 4 from altair.utils.schemapi import SchemaBase, Undefined, _subclasses
      5 
      6 import pkgutil

/lib/python3.8/site-packages/altair/utils/__init__.py in <module>
     12 )
     13 from .html import spec_to_html
---> 14 from .plugin_registry import PluginRegistry
     15 from .deprecation import AltairDeprecationWarning
     16 

/lib/python3.8/site-packages/altair/utils/plugin_registry.py in <module>
      3 
      4 import entrypoints
----> 5 from toolz import curry
      6 
      7 

/lib/python3.8/site-packages/toolz/__init__.py in <module>
     20 from . import curried, sandbox
     21 
---> 22 functoolz._sigs.create_signature_registry()
     23 
     24 from ._version import get_versions

/lib/python3.8/site-packages/toolz/_signatures.py in create_signature_registry(module_info, signatures)
    664             if hasattr(module, name):
    665                 new_sigs = tuple(expand_sig(sig) for sig in sigs)
--> 666                 signatures[getattr(module, name)] = new_sigs
    667 
    668 

TypeError: unhashable type: 'pyodide.JsProxy'

[bollwyvl]

Plotly also failing:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-2-0f083345bb1a> in <module>
      4 fig.add_trace(go.Bar(y=[1, 4, 3, 2]))
      5 fig.update_layout(title = 'Hello Figure')
----> 6 fig.show()

/lib/python3.8/site-packages/plotly/basedatatypes.py in show(self, *args, **kwargs)
   3396         import plotly.io as pio
   3397 
-> 3398         return pio.show(self, *args, **kwargs)
   3399 
   3400     def to_json(self, *args, **kwargs):

/lib/python3.8/site-packages/_plotly_utils/importers.py in __getattr__(import_name)
     34             rel_module = ".".join(rel_path_parts[:-1])
     35             class_name = import_name
---> 36             class_module = importlib.import_module(rel_module, parent_name)
     37             return getattr(class_module, class_name)
     38 

/lib/python3.8/importlib/__init__.py in import_module(name, package)
    125                 break
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 
    129 

/lib/python3.8/importlib/_bootstrap.py in _gcd_import(name, package, level)

/lib/python3.8/importlib/_bootstrap.py in _find_and_load(name, import_)

/lib/python3.8/importlib/_bootstrap.py in _find_and_load_unlocked(name, import_)

/lib/python3.8/importlib/_bootstrap.py in _load_unlocked(spec)

/lib/python3.8/importlib/_bootstrap_external.py in exec_module(self, module)

/lib/python3.8/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

/lib/python3.8/site-packages/plotly/io/_renderers.py in <module>
     10 from plotly import optional_imports
     11 
---> 12 from plotly.io._base_renderers import (
     13     MimetypeRenderer,
     14     ExternalRenderer,

/lib/python3.8/site-packages/plotly/io/_base_renderers.py in <module>
     10 from plotly import utils, optional_imports
     11 from plotly.io import to_json, to_image, write_image, write_html
---> 12 from plotly.io._orca import ensure_server
     13 from plotly.io._utils import plotly_cdn_url
     14 from plotly.offline.offline import _get_jconfig, get_plotlyjs

/lib/python3.8/site-packages/plotly/io/_orca.py in <module>
     13 from pathlib import Path
     14 
---> 15 import tenacity
     16 from six import string_types
     17 

/lib/python3.8/site-packages/tenacity/__init__.py in <module>
    513 
    514 if tornado:
--> 515     from tenacity.tornadoweb import TornadoRetrying

/lib/python3.8/site-packages/tenacity/tornadoweb.py in <module>
     21 from tenacity import RetryCallState
     22 
---> 23 from tornado import gen
     24 
     25 if typing.TYPE_CHECKING:

ImportError: cannot import name 'gen' from 'tornado' (unknown location)

[bollwyvl]

fasta looks hard broken, both the document renderer and the mime renderer

Uncaught TypeError: t.on is not a function
    resetSeqs 681.80b3439aec1db6dfd35f.js:2
    e 681.80b3439aec1db6dfd35f.js:2
    resetSeqs 681.80b3439aec1db6dfd35f.js:2
    i 681.80b3439aec1db6dfd35f.js:2
    _loadSeqs 681.80b3439aec1db6dfd35f.js:2
    initialize 681.80b3439aec1db6dfd35f.js:2
    a 681.80b3439aec1db6dfd35f.js:2
    i 681.80b3439aec1db6dfd35f.js:2
    i 681.80b3439aec1db6dfd35f.js:2

[bollwyvl]

While some demos are broken, I don't think it's worth blocking the merge on those, in this case... they probably just each need separate issues.

[bollwyvl]

mmm... the plotly fix... doesn't. still looking into it...

[bollwyvl]

Plotly fix appears to work locally... looking into altair...

[bollwyvl]

The altair fail is from overloading builtins.input with a JsProxy... investigating...

[bollwyvl]

Looks like both plotly and altair are back to working... no further changes planned, as fasta's been on the ropes for a while...

[jtpio]

Thanks!!

The fasta standalone file has been an issue for a while and could be tracked separately yet. However the renderer in the notebook should work (#284 and jupyterlab/jupyterlab#10675 for reference, not a blocker for this PR).

[jtpio]

retro stuff??

We need to add the new @jupyterlab/application-extension:context-menu plugin to the build:

jupyterlite/app/retro/index.template.js

Line 51 in 9680e3e

let mods = [

I'll push a commit for this.

[jtpio]

We need to add the new @jupyterlab/application-extension:context-menu plugin to the build:

Hmm looks like 28d31bc is not enough. Looking into doing the update in Retro first: jupyterlab/retrolab#186

[jtpio]

Looks like both plotly and altair are back to working

Were those failures related to new versions of the libraries?

[bollwyvl]

Looks like both plotly and altair are back to working Were those failures related to new versions of the libraries?

Nah, more transient deps. I assume people just hadn't been reviewing every notebook every pr. Due to mocking tornado, we have to mock... More tornado. Because plotly might want to call home. Or something. Uses tenacity. Altair was broken by overloading builtins.input with a pyodide.JsProxy. this offended toolz.

[bollwyvl]

The various above transient dep issues are part of why #45 is so important... four years later, all the jyve demos still work (to the extent that they every worked 🤣).

A full archival-grade workflow will eventually need to figure out how to capture everything it possibly can and put it together in the deployable folder/archive... this will require kernel-specific knowledge, of course, but i think we're getting there. Getting to pyodide + whateverdide as built (and deduplicated) by conda-forge will be needed for the "real" fix, but i think we can get there with pyolite and multi-indexing, patched micropip in the near term.

investing in the JS kernels, we can likely figure out a prefetching strategy for es modules... i trust "hot" js packages probably the least in the whole stack.

[jtpio]

Hmm looks like 28d31bc is not enough

137a88a should fix it for retro.

[jtpio]

The various above transient dep issues are part of why #45 is so important... four years later, all the jyve demos still work (to the extent that they every worked rofl).

Fully agree 👍 Also just for the simple case of having everything easily working offline or in flight mode.

[bollwyvl]

I didna look at any of the RTC stuff... what are we expecting to have actually been fixed?

[bollwyvl]

Actually, RTC (and FASTA 🤯 ) lookin fine:

[bollwyvl]

Hoisted some observed errors to #286... not worth blocking here.

[jtpio]

and FASTA 🤯

That is weird, as this seems to be reproducible upstream (jupyterlab/jupyterlab#10675)

[bollwyvl]

seems to be reproducible

best kind of 🐛

[jtpio]

Thanks!

[jtpio]

As mentioned above, let's track the outstanding issue in separate issues (some are already opened).