Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging Refactor #3270

Merged
merged 24 commits into from Dec 4, 2017
Merged

Packaging Refactor #3270

merged 24 commits into from Dec 4, 2017

Conversation

@blink1073
Copy link
Member

@blink1073 blink1073 commented Nov 21, 2017

  • Separates core and dev modes. They do not share file paths and are now semantically different. The jupyterlab/ directory is no longer part of the yarn workspaces, so we don't need our previous egg_info workaround due to the symlinks. This also makes it much clearer what the assets are and who they belong to (e.g. there is no more package.app.json).
  • Uses an updated setupbase.py from jupyter-packaging to ensure build assets. We always ensure a core mode, and ensure that the core mode matches the current version before publishing.
  • If installing in develop mode, we run the equivalent of jlpm install. We also automatically ensure that the application is built before running jupyter lab --dev-mode.
  • Adds caching of the miniconda and python packages, but starting with a fresh conda environment each time
  • Improves lab extension CLI and fixes some bugs as well: Fixes #3264. Fixes #3231. Fixes #3227. Fixes #3279.
@blink1073 blink1073 added this to the Beta milestone Nov 21, 2017
@blink1073 blink1073 changed the title [WIP] Packaging Refactor Packaging Refactor Nov 22, 2017
@blink1073 blink1073 changed the title Packaging Refactor [WIP] Packaging Refactor Nov 22, 2017
@blink1073
Copy link
Member Author

@blink1073 blink1073 commented Nov 22, 2017

Back to wip because we appear to have an issue with wheel building.

@blink1073 blink1073 changed the title [WIP] Packaging Refactor Packaging Refactor Nov 23, 2017
@blink1073
Copy link
Member Author

@blink1073 blink1073 commented Nov 23, 2017

The issue was that we were looking for the data_files before they had been built. We now specify them declaratively so we can fetch them after the node build step.

CONTRIBUTING.md Outdated
* The `jlpm` command is a JupyterLab-provided, locked version of the [yarn](https://yarnpkg.com/en/). If you have `yarn` installed already, you can use
the `yarn` command when developing, and it will use the local version of `yarn`
in `jupyterlab/yarn.js` when run in the repository or a built application
* The `jlpm` command is a JupyterLab-provided, locked version of the [jlpm](https://yarnpkg.com/en/). If you have `jlpm` installed already, you can use
Copy link
Member

@ian-r-rose ian-r-rose Nov 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an overly-aggressive find/replace for yarn-->jlpm

@@ -37,5 +37,5 @@ inquirer.prompt(questions).then(answers => {
data.name = name;
data.description = description;
utils.writePackageData(jsonPath, data);
utils.run('jlpm run integrity');
utils.run('npm run integrity');
Copy link
Member

@ian-r-rose ian-r-rose Nov 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you comment on the places where npm must still be used? I have found that confusing (it looks like it is most of the scripts in buildutils?).



// Update the jlpm.lock file.
utils.run('jlpm', { cwd: staging });
Copy link
Member

@ian-r-rose ian-r-rose Nov 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But not this one.

@@ -4,9 +4,9 @@ JupyterLab extensions add functionality to the JupyterLab application.
They can provide new file viewer types, launcher activities, and new Notebook
output renderers for example.

### Installing Node.js and npm
### Installing Node.j
Copy link
Member

@ian-r-rose ian-r-rose Nov 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: Node.j --> Node.js

@@ -139,21 +137,20 @@
}


if 'setuptools' in sys.modules:
setup_args.update(setuptools_args)
# Because of this we do not need a MANIFEST.in
Copy link
Member

@vidartf vidartf Nov 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then the file can (and should?) be removed?

Copy link
Member Author

@blink1073 blink1073 Nov 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is removed in this PR.

Copy link
Member

@vidartf vidartf Dec 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I must have missed it in the changed files overview.

@blink1073 blink1073 force-pushed the jupyter-packaging branch 2 times, most recently from 5cd208b to 4de766c Nov 29, 2017
@blink1073 blink1073 mentioned this pull request Dec 1, 2017
@blink1073 blink1073 removed this from the Beta milestone Dec 1, 2017
@blink1073 blink1073 added this to the 0.30 milestone Dec 1, 2017
@blink1073 blink1073 force-pushed the jupyter-packaging branch from 74181f9 to 456bc58 Dec 1, 2017
blink1073 added 2 commits Dec 2, 2017
Finish packaging updates

wip refactor packaging, core, and dev modes

wip packaging updates

reorg

wip packaging setup

wip packaging

cleanup

Move mock packages out of jupyterlab dir

Always use yarn for developers

wip update dev and core mode

clean up commands and extension

cleanup

update docs

Update .yarnrc

update ci scripts

Update ensure repo

fix integrity check

Update gitignore

Switch back to jlpm

wip

Update scripts

cleanup

cleanup

cleanup

cleanup

no need for egg-info hack anymore

clean up handling of mock packages

update selenium check

Refactor travis

Cleanup

reorder matrix

Fix handling of link and unlinking extensions

Fix handling of postcss

Cache the miniconda dir

fix travis yml syntax

fix travis yml syntax

Update group names

fix miniconda handling

Update docs

Fix handling of miniconda path

reorder path

tweak karma settings

cleanup

Remove manifest in favor of package_data

fix path addition

more setup cleanup

Add staging data to package data

Make sure our data_files work

add comment

use glob2 to get proper files

defer using glob2

cleanup

Inline handling of ** globs

Add a node_modules blocker

use fresh environments on every build

fix removal of conda envs

use env create for requirements file

ensure selenium is installed

Fix jupyterlab#3231

front load the longer job

Fix jupyterlab#3227

Uniform handling of error messages for extension commands

Let process errors propagate

Fix #jupyterlab#3264

Fix validation and add tests

Fix property name

fix handling of app_dir

Remove pdb from py.test so builds don't time out

Use npm to run scripts in the re-usable build utils

Clean up build:update

Fix glob handling

Use declarative specs for data files and package files

Add detection of extension module shadowing

Fix package_data_spec

Add workaround for git clean behavior on windows

remove debug statement

Clean up setupbase

Address comments

Remove outdated file check

glob handling cleanup

Incorporate jupyter-packaging jupyterlab#20

clean up setup.py

Make sure build:src works

Reinstate build:src

Fix spelling

wip add test manager

fix pip install .

Clean up selenium check implementation and temp usage

Invert requirements

clean up handling of launcher versions in script

fix version spec
@blink1073 blink1073 force-pushed the jupyter-packaging branch from 456bc58 to dcf3204 Dec 2, 2017
@blink1073 blink1073 mentioned this pull request Dec 2, 2017
@blink1073 blink1073 mentioned this pull request Dec 3, 2017
8 tasks
afshin
afshin approved these changes Dec 4, 2017
Copy link
Member

@afshin afshin left a comment

Awesome, let's roll. Thanks!

@afshin afshin merged commit 7cb1900 into jupyterlab:master Dec 4, 2017
2 checks passed
@blink1073 blink1073 deleted the jupyter-packaging branch Dec 6, 2017
@lock lock bot locked as resolved and limited conversation to collaborators Aug 9, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.