Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Extension Overhaul #2023

Closed
wants to merge 47 commits into from
Closed

Conversation

@blink1073
Copy link
Member

@blink1073 blink1073 commented Apr 4, 2017

Fixes #2011.

Status:
We have a build script that extracts the dependency assets from a node package into a static output tree in packages/extension-builder. We have a packages/main that will be used to create the main assets of the application.

@blink1073 blink1073 added this to the Beta milestone Apr 4, 2017
@blink1073
Copy link
Member Author

@blink1073 blink1073 commented Apr 5, 2017

I looked at read-installed but it does not (yet) traverse to parent directories. We may need to fork/PR this repo if it turns out that our solution is brittle.

@blink1073
Copy link
Member Author

@blink1073 blink1073 commented Apr 5, 2017

More progress. I took the build directory into a clean folder outside of jupyterlab, installed webpack and loaders, and was able to build the resulting file tree. The entry point file will be templated to include the desired modules.

module.exports = {
  entry:  './lib/jupyterlab/src/main',
  output: {
    path: __dirname + '/build',
    filename: '[name].bundle.js',
  },
  module: {
    rules: [
      { test: /\.css$/, use: ['style-loader', 'css-loader'] },
      { test: /\.json$/, use: 'json-loader' },
      { test: /\.html$/, use: 'file-loader' },
      { test: /\.(jpg|png|gif)$/, use: 'file-loader' },
      { test: /\.js.map$/, use: 'file-loader' },
      { test: /\.woff2(\?v=\d+\.\d+\.\d+)?$/, use: 'url-loader?limit=10000&mimetype=application/font-woff' },
      { test: /\.woff(\?v=\d+\.\d+\.\d+)?$/, use: 'url-loader?limit=10000&mimetype=application/font-woff' },
      { test: /\.ttf(\?v=\d+\.\d+\.\d+)?$/, use: 'url-loader?limit=10000&mimetype=application/octet-stream' },
      { test: /\.eot(\?v=\d+\.\d+\.\d+)?$/, use: 'file-loader' },
      { test: /\.svg(\?v=\d+\.\d+\.\d+)?$/, use: 'url-loader?limit=10000&mimetype=image/svg+xml' }
    ],
  },
  node: {
    fs: 'empty'
  },
  bail: true,
  devtool: 'source-map'
}

@blink1073
Copy link
Member Author

@blink1073 blink1073 commented Apr 5, 2017

I tried running npm install and yarn install without an internet connection and neither one handled it well. npm install tries for several seconds before resorting to local files. yarn install works offline when you have a local cache, but not without. We could pre-populate that cache. For now, npm dedupe is working.

@blink1073
Copy link
Member Author

@blink1073 blink1073 commented Apr 5, 2017

Current status: with a bit of hand-jamming, the main application loads.

@blink1073
Copy link
Member Author

@blink1073 blink1073 commented Apr 5, 2017

Build is now automated, but only for the main application. It takes 32 seconds to run the whole process, which:

  • builds the main extension files (~5sec)
  • bundles the main extension (~1sec)
  • copies files (~4sec)
  • runs npm install (~9sec)
  • runs webpack (~10sec)

@blink1073 blink1073 force-pushed the extension-bundler branch from 5b65955 to 3c742de Apr 5, 2017
@blink1073
Copy link
Member Author

@blink1073 blink1073 commented Apr 7, 2017

Hit a temporary roadblock in that npm dedupe does not work properly when trying to combine trees from separate npm installs. Instead we should be using a local npm cache directory, which stores data as (for example):

~/.npm/@phosphor/widgets/0.1.3/package/package.json
~/.npm/@phosphor/widgets/0.1.3/package.tgz

We can set the desired cache folder using the cache key in .npmrc.

https://docs.npmjs.com/cli/cache#details
https://docs.npmjs.com/misc/config#cache

@blink1073
Copy link
Member Author

@blink1073 blink1073 commented Apr 7, 2017

local cache is working as expected

@blink1073
Copy link
Member Author

@blink1073 blink1073 commented Apr 7, 2017

(need to populate it programmatically next).

@blink1073
Copy link
Member Author

@blink1073 blink1073 commented Apr 7, 2017

Stopping here for the week. Some notes:

  • The cache tarballs can't include any of the npm metadata, so we need to used existing cached ones or create them using a cleansed staging area. We can differentiate a local package by whether it has the npm metadata (minimally a dist field).
  • We can't install a cache tarball directly (the ones produced by npm pack have a different structure), but we can run npm pack on only the top level package and use that in our templated package.json file as file:./foo.tgz.
  • I tried the above using an npm packed file and the system level package cache and it works, so this is promising.

@blink1073
Copy link
Member Author

@blink1073 blink1073 commented Apr 11, 2017

Closing in favor of the new approach outlined here.

@blink1073 blink1073 closed this Apr 11, 2017
@blink1073 blink1073 mentioned this pull request Apr 21, 2017
@lock lock bot locked as resolved and limited conversation to collaborators Aug 10, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

2 participants