Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade jupyterlab to 1.2.6 #499

Merged
merged 2 commits into from Feb 1, 2020
Merged

Upgrade jupyterlab to 1.2.6 #499

merged 2 commits into from Feb 1, 2020

Conversation

lwbayes
Copy link
Contributor

@lwbayes lwbayes commented Feb 1, 2020

This PR upgrades Jupyterlab to the latest version of 1.2.6 as of Feb 1st 2020. Build and integration tests passes locally.

  • jupyterlab-hubextension removed because this has been integrated to Jupyterlab 1.0.0+

  • Added --minimize=False option to Jupyterlab installation to avoid flaky build on low resource CI pipelines

  • Add / update documentation

  • Add tests

@lwbayes
Copy link
Contributor Author

lwbayes commented Feb 1, 2020

srv/src/integration-tests/test_install.py:143: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

group = 'jupyterhub-admins', path = '/opt/tljh/state/jupyterhub.sqlite-journal'

    def permissions_test(group, path, *, readable=None, writable=None, dirs_only=False):
        """Run a permissions test on all files in a path path"""
        # start a subprocess and become nobody:group in the process
        pool = ProcessPoolExecutor(1)
        pool.submit(setgroup, group)
    
        def access(path, flag):
            """Run access test in subproccess as nobody:group"""
            return pool.submit(os.access, path, flag).result()
    
        total_tested = 0
    
        failures = []
    
        # walk the directory and check permissions
        for root, dirs, files in os.walk(path):
            to_test = dirs
            if not dirs_only:
                to_test += files
            total_tested += len(to_test)
            for name in to_test:
                path = os.path.join(root, name)
                if os.path.islink(path):
                    # skip links
                    continue
>               st = os.lstat(path)
E               FileNotFoundError: [Errno 2] No such file or directory: '/opt/tljh/state/jupyterhub.sqlite-journal'

Getting this error in the CI build. I feel like this is a race-condition/flaky build problem: Some concurrent running test process resulted in a temp sqlite file that existed at list folder time but have disappeared at permission check time.

@betatim
Copy link
Member

betatim commented Feb 1, 2020

It feels weird to add a special option to the installer just to get it to pass on the CI. However there is jupyterlab/jupyterlab#7180 which also does this and (found via that issue) the https://github.com/jupyter/docker-stacks/ images also use this option.

With multiple lab extensions you can use something like the following to only build things once instead of for each extension you install:

jupyter labextension install jupyterlab-plotly@1.5.0 --no-build
jupyter labextension install plotlywidget@1.5.0 --no-build
# Added --minimize=False per https://github.com/jupyterlab/jupyterlab/issues/7180
jupyter lab build --minimize=False --dev-build=False

@betatim
Copy link
Member

betatim commented Feb 1, 2020

I don't think we run our test suite in parallel so the flakeyness must come from somewhere else :-/

@lwbayes
Copy link
Contributor Author

lwbayes commented Feb 1, 2020

It feels weird to add a special option to the installer just to get it to pass on the CI. However there is jupyterlab/jupyterlab#7180 which also does this and (found via that issue) the https://github.com/jupyter/docker-stacks/ images also use this option.

With multiple lab extensions you can use something like the following to only build things once instead of for each extension you install:

jupyter labextension install jupyterlab-plotly@1.5.0 --no-build
jupyter labextension install plotlywidget@1.5.0 --no-build
# Added --minimize=False per https://github.com/jupyterlab/jupyterlab/issues/7180
jupyter lab build --minimize=False --dev-build=False

Yep, I agree it is weird to do this. The main culprit is how much memory Webpack minify uses, and it has hang a t3.small machine on AWS (2GB memory) when I tested TLJH. So in a way it is solving a problem of deploying TLJH on a small test machine. But adding an adhoc option like this look ugly :(

The suggested method is good, and is actually how I tend to build lab extensions myself. Will update to use this method.

@betatim
Copy link
Member

betatim commented Feb 1, 2020

Nice work, merging!

@betatim betatim merged commit 5a4f47b into jupyterhub:master Feb 1, 2020
@lwbayes lwbayes deleted the upgrade-jupyterlab-to-1.2.6 branch February 2, 2020 22:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants