Switch to hatch backend

[blink1073]

References

cf jupyterlab/frontends-team-compass#146

Code changes

Updates python packaging infrastructure.

LSP packages and related plugins are explicitly removed from examples app and federated extensions.
The skip lsp/status request in logger for examples has been removed.

User-facing changes

None

Backwards-incompatible changes

None

Steps

• Move hatch_jupyter_builder to jupyterlab org.
• Get hatch_jupyter_builder on conda-forge.
• Write a helper script similar to the extension upgrade script to migrate packages.
  • The script will use hatch init --new to do the heavy lifting
  • It will also move flake8 config from setup.cfg to a new .flake8 file
  • It will migrate jupyter-packaging config in pyproject.toml to equivalent hatch_jupyter_builder config
  • It will look for npm_builder config in setup.py and convert it to editable-build-kwargs config for hatch_jupyter_builder
  • It will look for ignore patterns in .gitignore that are in the package folder and suggest them as artifacts config for inclusion (commented out with a note)
  • Make that script available as part of hatch_jupyter_builder. It should operate in a temporary virtual environment so it doesn't add any requirements to hatch_jupyter_builder. See docs
• Use that script to migrate extension-cookiecutter-ts
• Test this PR against a federated extension made from the migrated cookiecutter extension. Also make sure new extensions work with existing versions of jupyterlab - need a shim setup.py for older JupyterLabs.
• Prepare a PR for extension-cookiecutter-js
• Prepare a PR for widget-ts-cookiecutter
• Get this PR passing
• Draft a blog post about the switch and how to migrate existing extensions
• Merge all open PRs and publish the blog post
• Deprecate Jupyter-Packaging, pointing to the blog post

[jupyterlab-probot]

Thanks for making a pull request to jupyterlab!
To try out this branch on binder, follow this link:

[blink1073]

@fcollonval I'm not super familiar with the Galata tests, any idea what is going wrong there?

[blink1073]

I think it might have been due to differences in files including in the sdists. I have one more issue to work though - the .js.map files are getting included in the data folder, reported as pypa/hatch#265.

[blink1073]

WIP migration script

[jtpio]

This looks like it would fix the issue noticed in jupyter/notebook#6425 (comment) 👍

[blink1073]

I ended up adding a setup.py shim to projects to avoid needing to backport fixes

[blink1073]

Update: it looks like this approach is going to work. I've tested ipywidgets and an extension made from the cookiecutter PR against both this branch and 3.4.2.

Next steps as I see them are to:

• Create draft PRs to update extension-cookiecutter-js and widget-ts-cookiecutter
• Draft a blog post about the switch and how to migrate existing extensions
• Merge the cookiecutter PRs and publish the blog post

[trungleduc]

For the examples a skip condition was implemented:

jupyterlab/examples/example.spec.ts

Line 21 in a650dd8

if (msg.type() === 'error' && !url.includes('lsp/status')) {

I don't know why we are being it in browser_check now. Probably the request is done faster than before or we got lucky before and Playwright was successful before the request was made.

We should be able to remove this check after adding jupyter-lsp to the dependency list

[fcollonval]

Thanks Trung - let's see if the CI is happy. Then I'll clean up the exceptions for lsp/status request.

[trungleduc]

The issue with the schema file is that the build process builds the frontend packages before installing the python dependencies. So the schema file is generated without jupyter-lsp installed.

[jtpio]

CI is now passing.

[fcollonval]

@jtpio would you mind having a look? It is good to go for me. But as I did some changes I'll would appreciate another review.

[jtpio]

Just checked the releaser built assets and they look good.

However Binder fails to build: https://mybinder.org/v2/gh/blink1073/jupyterlab/use-hatch?urlpath=lab-dev

> @jupyterlab/repo-top@0.0.1 integrity /home/jovyan
> node scripts/ensure-buildutils.js && node buildutils/lib/ensure-repo.js

Error: Command failed: hatchling version
/bin/sh: 1: hatchling: not found

    at checkExecSyncError (child_process.js:790:11)
    at Object.execSync (child_process.js:863:15)
    at run (/home/jovyan/buildutils/lib/utils.js:242:43)
    at Object.getPythonVersion (/home/jovyan/buildutils/lib/utils.js:188:19)
    at ensureIntegrity (/home/jovyan/buildutils/lib/ensure-repo.js:774:24)
    at processTicksAndRejections (internal/process/task_queues.js:95:5) {
  status: 127,
  signal: null,
  output: [
    null,
    <Buffer >,
    <Buffer 2f 62 69 6e 2f 73 68 3a 20 31 3a 20 68 61 74 63 68 6c 69 6e 67 3a 20 6e 6f 74 20 66 6f 75 6e 64 0a>
  ],
  pid: 800,
  stdout: <Buffer >,
  stderr: <Buffer 2f 62 69 6e 2f 73 68 3a 20 31 3a 20 68 61 74 63 68 6c 69 6e 67 3a 20 6e 6f 74 20 66 6f 75 6e 64 0a>
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @jupyterlab/repo-top@0.0.1 integrity: `node scripts/ensure-buildutils.js && node buildutils/lib/ensure-repo.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @jupyterlab/repo-top@0.0.1 integrity script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/jovyan/.npm/_logs/2022-08-08T09_05_21_505Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @jupyterlab/repo-top@0.0.1 build:dev: `npm run integrity && npm run build:all && cd dev_mode && npm run build`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @jupyterlab/repo-top@0.0.1 build:dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/jovyan/.npm/_logs/2022-08-08T09_05_21_542Z-debug.log
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Removing intermediate container 3938375ec3e9
The command '/bin/sh -c ./binder/postBuild' returned a non-zero code: 1Built image, launching...

[fcollonval]

I pushed a fix for the binder configuration.

[fcollonval]

Thanks a lot @blink1073

[jtpio]

I pushed a fix for the binder configuration.

Nice thanks @fcollonval 👍

[jtpio]

Looks good, thanks all!

[jtpio]

We can make a new 4.0 pre-release right after merging this, since there are also other (rather big) changes that have not been released yet.